justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261354.1@dunegpschedd02.fnal.gov

Jobsub ID261354.1@dunegpschedd02.fnal.gov
Workflow ID11112
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 23:30:43
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 01:36:43
From worker nodeHostnamedunegli-7875986-0-fnpc22004.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 23:31:01
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T013113Z_gen_003093_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time2h (7529s)
CPU time2h (7230s = 96%)
Max RSS bytes2929422336 (2793 MiB)
Outputting started2025-12-12 01:36:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11112/1/001/analysisLowE_20251211T233103Z.root
Finished2025-12-12 01:36:43
Saved logsjustin-logs:261354.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

2, 0.162561, 0.790089, 0.0165394, 
Output 3: 0.964778, 0.0282556, 0.00296354, 0.00400322, 
Output 4: 0.395173, 0.500477, 0.0934277, 0.0109219, 
Output 5: 0.279136, 0.439292, 0.23199, 0.0495821, 
Output 6: 0.047203, 0.119762, 0.11452, 0.718515, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 30930 at 12-Dec-2025 01:25:04 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6615, 6518, 6558
minwire 0: 32
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 41, 540
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.141904, 
Output 1: 0.0967859, 0.100441, 0.221639, 0.581134, 
Output 2: 0.120804, 0.223887, 0.621369, 0.0339393, 
Output 3: 0.968167, 0.0251503, 0.00221972, 0.00446242, 
Output 4: 0.322553, 0.628218, 0.0454469, 0.00378173, 
Output 5: 0.579045, 0.381596, 0.0326648, 0.00669387, 
Output 6: 0.0242045, 0.0812078, 0.086863, 0.807725, 

12-Dec-2025 01:35:59 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 01:35:59 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/fa/e0/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T013113Z_gen_003093_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              630.67        742.24        907.142       732.727       105.631        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00165718    0.00793108     0.0250196    0.00406324     0.0074925       10     
reco:pandora:StandardPandora                            629.429       740.855       905.832       731.448       105.358        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00161912     0.0185475     0.0762318    0.00821722     0.0231764       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00163319    0.00469268     0.0122009    0.00265011    0.00365598       10     
reco:pandoracalo:Calorimetry                          0.000254403   0.00564818     0.0167939    0.00175651    0.00663324       10     
reco:pandorapid:Chi2ParticleID                        5.7098e-05    0.000268208   0.00076884    0.000214633   0.000210321      10     
reco:cvnmap:CVNMapper                                  0.0170838     0.0325008     0.0726439     0.0219744     0.0204085       10     
reco:cvneva:CVNEvaluator                               0.738841      0.938942       2.37561       0.76053      0.480196        10     
reco:energyrecnumu:EnergyReco                         0.00302044     0.0467186     0.0850319     0.057596      0.0276534       10     
reco:energyrecnue:EnergyReco                          0.000794759   0.00341112    0.00568206    0.00471644    0.00199974       10     
reco:energyrecnc:EnergyReco                           0.00461732     0.0397762     0.199386     0.00514072     0.061282        10     
[art]:TriggerResults:TriggerResultInserter            1.0029e-05    2.02705e-05   4.2872e-05    1.4883e-05    1.1286e-05       10     
end_path:out1:RootOutput                               5.05e-06     9.3541e-06     2.149e-05    7.9605e-06    4.82045e-06      10     
end_path:out1:RootOutput(write)                        0.226495      0.286162      0.444513      0.231281      0.0793907       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21686 MB
  Peak resident set size usage (VmHWM): 2929.42 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 170.367 | hit correction : 1.09595
hit initial : 74.0203 | hit correction : 1.09603
 *** Total Charge : 5598.11
 Looking at view: 2 | Number of hits in view: 24
Begin processing the 7th record. run: 5027 subRun: 0 event: 30927 at 12-Dec-2025 01:36:22 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 941.649
True Neutron Energy: 940.039
True Electron Energy: 33.3083
Completed MCParticle Loop
True Neutrino Energy: 50.595
 Size of all hits: 0
Begin processing the 8th record. run: 5027 subRun: 0 event: 30928 at 12-Dec-2025 01:36:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  39
Added hit list length: 39 | Max was set to: -3.40282e+38
True Electron Energy: 34.2962
Completed MCParticle Loop
True Neutrino Energy: 51.6021
 Size of all hits: 39
hit initial : 43.4448 | hit correction : 1.00706
hit initial : 101.284 | hit correction : 1.00703
hit initial : 59.4179 | hit correction : 1.01618
hit initial : 71.3129 | hit correction : 1.01616
hit initial : 21.6272 | hit correction : 1.01054
hit initial : 100.744 | hit correction : 1.01054
hit initial : 168.241 | hit correction : 1.01052
hit initial : 343.346 | hit correction : 1.01057
hit initial : 397.289 | hit correction : 1.01069
hit initial : 273.452 | hit correction : 1.01101
hit initial : 269.162 | hit correction : 1.01141
hit initial : 379.613 | hit correction : 1.01193
hit initial : 340.154 | hit correction : 1.01253
hit initial : 336.821 | hit correction : 1.01313
hit initial : 340.137 | hit correction : 1.01346
hit initial : 275.097 | hit correction : 1.01386
hit initial : 115.939 | hit correction : 1.01401
 *** Total Charge : 3680.46
 Looking at view: 0 | Number of hits in view: 17
hit initial : 309.435 | hit correction : 1.01067
hit initial : 14.4651 | hit correction : 1.01327
hit initial : 136.274 | hit correction : 1.01056
hit initial : 48.7284 | hit correction : 1.01615
hit initial : 85.8907 | hit correction : 1.01617
hit initial : 876.958 | hit correction : 1.01151
hit initial : 227.414 | hit correction : 1.01221
hit initial : 476.123 | hit correction : 1.01304
hit initial : 372.73 | hit correction : 1.0135
hit initial : 249.486 | hit correction : 1.01392
hit initial : 71.3371 | hit correction : 1.01402
 *** Total Charge : 2904.53
 Looking at view: 1 | Number of hits in view: 11
hit initial : 363.569 | hit correction : 1.01227
hit initial : 400.838 | hit correction : 1.01162
hit initial : 251.107 | hit correction : 1.01114
hit initial : 221.041 | hit correction : 1.01089
hit initial : 189.467 | hit correction : 1.01069
hit initial : 212.209 | hit correction : 1.01056
hit initial : 265.571 | hit correction : 1.01054
hit initial : 42.93 | hit correction : 1.01568
hit initial : 378.621 | hit correction : 1.01291
hit initial : 450.966 | hit correction : 1.01341
hit initial : 380.836 | hit correction : 1.01395
 *** Total Charge : 3195.46
 Looking at view: 2 | Number of hits in view: 11
Begin processing the 9th record. run: 5027 subRun: 0 event: 30929 at 12-Dec-2025 01:36:27 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 8.70842
Completed MCParticle Loop
True Neutrino Energy: 25.0654
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 30930 at 12-Dec-2025 01:36:28 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.571
True Electron Energy: 4.20476
Completed MCParticle Loop
True Neutrino Energy: 9.58286
 Size of all hits: 0
12-Dec-2025 01:36:30 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0470335     0.0620793     0.0857832     0.0605225     0.0120484       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000380181   0.000949609   0.00413857    0.000451166   0.00111072       10     
end_path:ana:AnalyzeEvents         0.0464687     0.0609152     0.0848052     0.0598821     0.0116383       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19474.7 MB
  Peak resident set size usage (VmHWM): 1534.37 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:00:05 UTC       justIN version: 01.06.00