justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267118.1@dunegpschedd01.fnal.gov

Jobsub ID267118.1@dunegpschedd01.fnal.gov
Workflow ID11113
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 23:40:43
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 01:28:01
From worker nodeHostnamedunegli-8035303-0-fnpc23117.fnal.gov
cpuinfoAMD EPYC 9354 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 23:42:13
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T191007Z_gen_001937_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6337s)
CPU time1h (6281s = 99%)
Max RSS bytes3112501248 (2968 MiB)
Outputting started2025-12-12 01:27:51
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11113/1/001/analysisLowE_20251211T234214Z.root
Finished2025-12-12 01:28:01
Saved logsjustin-logs:267118.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

, 
Output 2: 0.0452439, 0.231967, 0.689481, 0.0333079, 
Output 3: 0.949434, 0.0333617, 0.00704676, 0.0101575, 
Output 4: 0.800737, 0.153931, 0.0366157, 0.00871644, 
Output 5: 0.665863, 0.241235, 0.0745465, 0.0183557, 
Output 6: 0.0125749, 0.0342318, 0.0495048, 0.903688, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 19370 at 12-Dec-2025 01:17:26 UTC
Error: A CaloHitList is empty
Boundary wire vector sizes: 7104, 7009, 6865
minwire 0: 22
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1080, 1579
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.112105, 
Output 1: 0.0865948, 0.068474, 0.0917862, 0.753145, 
Output 2: 0.0646523, 0.193357, 0.71091, 0.0310799, 
Output 3: 0.977611, 0.00779916, 0.00151122, 0.0130782, 
Output 4: 0.694104, 0.282316, 0.0202746, 0.00330516, 
Output 5: 0.376675, 0.464255, 0.125157, 0.0339133, 
Output 6: 0.0189538, 0.0477977, 0.0624681, 0.87078, 

12-Dec-2025 01:27:22 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 01:27:22 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/36/3e/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T191007Z_gen_001937_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              579.176       624.52        676.403       629.492       35.2693        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00121501    0.00659577     0.0188555    0.00421218    0.00562525       10     
reco:pandora:StandardPandora                            578.215       623.499       675.303       628.545       35.0946        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00122491    0.00870278     0.0638813    0.00206079     0.0184671       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000988619   0.00255583    0.00878868    0.00130876    0.00236623       10     
reco:pandoracalo:Calorimetry                          0.000193904   0.00495582     0.0166574    0.00125608    0.00620698       10     
reco:pandorapid:Chi2ParticleID                        4.5078e-05    0.000198933   0.000591337   0.000148626   0.000175284      10     
reco:cvnmap:CVNMapper                                  0.012878      0.0174337     0.0267253     0.0172587    0.00372369       10     
reco:cvneva:CVNEvaluator                               0.546269      0.694303       1.56681      0.589556      0.294175        10     
reco:energyrecnumu:EnergyReco                          0.0552612     0.0660206     0.0943055     0.0606034     0.0117828       10     
reco:energyrecnue:EnergyReco                          0.00125184    0.00474782    0.00724516    0.00437363    0.00163546       10     
reco:energyrecnc:EnergyReco                           0.00390351    0.00495621    0.00711757    0.00434407    0.00117633       10     
[art]:TriggerResults:TriggerResultInserter             9.735e-06    2.01214e-05   9.5976e-05    1.11565e-05   2.53782e-05      10     
end_path:out1:RootOutput                               5.118e-06    8.9597e-06    3.2279e-05    5.9345e-06    7.86381e-06      10     
end_path:out1:RootOutput(write)                        0.176832      0.209783      0.304528      0.190995      0.0403157       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21787.2 MB
  Peak resident set size usage (VmHWM): 3112.5 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  40
Added hit list length: 40 | Max was set to: -3.40282e+38
True Neutron Energy: 939.567
True Electron Energy: 40.6302
Completed MCParticle Loop
True Neutrino Energy: 44.4297
 Size of all hits: 40
hit initial : 43.6869 | hit correction : 1.40888
hit initial : 114.073 | hit correction : 1.4096
hit initial : 300.871 | hit correction : 1.40955
hit initial : 244.08 | hit correction : 1.40957
hit initial : 244.512 | hit correction : 1.40964
hit initial : 335.609 | hit correction : 1.40967
hit initial : 303.288 | hit correction : 1.40972
hit initial : 222.815 | hit correction : 1.40976
hit initial : 180.719 | hit correction : 1.40969
hit initial : 186.693 | hit correction : 1.40961
hit initial : 168.282 | hit correction : 1.40953
hit initial : 168.951 | hit correction : 1.40942
hit initial : 184.063 | hit correction : 1.40928
hit initial : 180.482 | hit correction : 1.40915
hit initial : 159.248 | hit correction : 1.40907
hit initial : 161.936 | hit correction : 1.40902
hit initial : 165.883 | hit correction : 1.40898
hit initial : 103.964 | hit correction : 1.40894
 *** Total Charge : 4889.66
 Looking at view: 0 | Number of hits in view: 18
hit initial : 96.6214 | hit correction : 1.4097
hit initial : 109.418 | hit correction : 1.40978
hit initial : 219.918 | hit correction : 1.40971
hit initial : 97.5944 | hit correction : 1.40973
hit initial : 217.541 | hit correction : 1.40961
hit initial : 350.597 | hit correction : 1.40964
hit initial : 101.611 | hit correction : 1.40968
hit initial : 365.408 | hit correction : 1.4096
hit initial : 108.208 | hit correction : 1.40964
hit initial : 121.468 | hit correction : 1.40981
hit initial : 124.404 | hit correction : 1.40958
hit initial : 183.119 | hit correction : 1.40946
hit initial : 130.187 | hit correction : 1.40951
hit initial : 138.795 | hit correction : 1.40944
hit initial : 141.602 | hit correction : 1.40934
hit initial : 130.695 | hit correction : 1.40921
hit initial : 138.667 | hit correction : 1.40913
hit initial : 133.006 | hit correction : 1.40901
hit initial : 120.928 | hit correction : 1.40906
hit initial : 134.882 | hit correction : 1.40899
hit initial : 127.746 | hit correction : 1.40897
hit initial : 94.9097 | hit correction : 1.40886
 *** Total Charge : 4774.3
 Looking at view: 2 | Number of hits in view: 22
Begin processing the 7th record. run: 5026 subRun: 0 event: 19367 at 12-Dec-2025 01:27:42 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 17.8369
Completed MCParticle Loop
True Neutrino Energy: 33.3146
 Size of all hits: 0
Begin processing the 8th record. run: 5026 subRun: 0 event: 19368 at 12-Dec-2025 01:27:44 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 13.9558
Completed MCParticle Loop
True Neutrino Energy: 19.345
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 19369 at 12-Dec-2025 01:27:46 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 18.1108
Completed MCParticle Loop
True Neutrino Energy: 21.8554
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 19370 at 12-Dec-2025 01:27:48 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.571
True Electron Energy: 1.31793
Completed MCParticle Loop
True Neutrino Energy: 4.60141
 Size of all hits: 0
12-Dec-2025 01:27:50 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0445554     0.061222      0.118099      0.0481957      0.02356        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000318914   0.00181849     0.0143082    0.000393982   0.00416484       10     
end_path:ana:AnalyzeEvents         0.0440426     0.0592313     0.116925      0.0476349     0.0233864       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19473 MB
  Peak resident set size usage (VmHWM): 1532.7 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:01:02 UTC       justIN version: 01.06.00