justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266933.0@dunegpschedd01.fnal.gov

Jobsub ID266933.0@dunegpschedd01.fnal.gov
Workflow ID11114
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 17:52:23
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 00:56:06
From worker nodeHostnamedunegli-8029109-0-fnpc19104.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 17:54:13
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T214936Z_gen_004409_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time7h (25301s)
CPU time6h (25122s = 99%)
Max RSS bytes2713473024 (2587 MiB)
Outputting started2025-12-12 00:55:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11114/1/001/analysisLowE_20251211T175415Z.root
Finished2025-12-12 00:56:06
Saved logsjustin-logs:266933.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

failure: 1093, 1592
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0779487, 
Output 1: 0.0643308, 0.144885, 0.0838329, 0.706951, 
Output 2: 0.0463533, 0.256842, 0.656082, 0.0407229, 
Output 3: 0.966165, 0.0197587, 0.00415615, 0.00991972, 
Output 4: 0.930373, 0.0583918, 0.00884098, 0.00239404, 
Output 5: 0.664369, 0.216504, 0.0893871, 0.0297393, 
Output 6: 0.0188513, 0.0402561, 0.0514683, 0.889424, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 44090 at 12-Dec-2025 00:19:11 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6668, 6577, 6741
minwire 0: 15
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.103554, 
Output 1: 0.0628121, 0.0798246, 0.188565, 0.668798, 
Output 2: 0.104944, 0.22144, 0.644293, 0.0293227, 
Output 3: 0.95469, 0.0373369, 0.0033783, 0.00459516, 
Output 4: 0.326247, 0.622475, 0.0476678, 0.00361031, 
Output 5: 0.535786, 0.423932, 0.0349518, 0.00533072, 
Output 6: 0.0269979, 0.0864653, 0.0856082, 0.800929, 

12-Dec-2025 00:55:17 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 00:55:17 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e1/18/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T214936Z_gen_004409_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2160.6        2512.23       2825.41       2526.75       222.562        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00288086     0.0102371     0.0250983    0.00940755    0.00560378       10     
reco:pandora:StandardPandora                            2158.83       2510.17       2823.37       2524.95       222.691        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000420744    0.0179456     0.122386     0.00332887     0.0355327       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000388613   0.00510464      0.01655     0.00313448    0.00506475       10     
reco:pandoracalo:Calorimetry                          0.000210802    0.010503      0.0377748    0.00250443     0.0136772       10     
reco:pandorapid:Chi2ParticleID                        7.8991e-05    0.000270779   0.000948159   0.000108171   0.000282496      10     
reco:cvnmap:CVNMapper                                  0.043733      0.0505723     0.0798197     0.0475029     0.0100742       10     
reco:cvneva:CVNEvaluator                               0.924833       1.4367        3.16239       1.35581      0.607943        10     
reco:energyrecnumu:EnergyReco                         0.00579812     0.0800776     0.126031      0.0951448     0.0375046       10     
reco:energyrecnue:EnergyReco                            0.00138     0.00682638     0.0117737    0.00737356    0.00376657       10     
reco:energyrecnc:EnergyReco                           0.00676297     0.0337892     0.158826     0.00929399     0.0514931       10     
[art]:TriggerResults:TriggerResultInserter             1.653e-05    2.97775e-05   8.8211e-05    2.5651e-05    2.02696e-05      10     
end_path:out1:RootOutput                               7.26e-06     1.20861e-05    3.339e-05     9.46e-06     7.53137e-06      10     
end_path:out1:RootOutput(write)                        0.303787      0.399491      0.464051      0.420632      0.0604182       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21691.2 MB
  Peak resident set size usage (VmHWM): 2713.47 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
True Neutron Energy: 939.821
True Electron Energy: 9.97351
Completed MCParticle Loop
True Neutrino Energy: 25.4451
 Size of all hits: 0
Begin processing the 7th record. run: 5026 subRun: 0 event: 44087 at 12-Dec-2025 00:55:44 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 944.211
True Neutron Energy: 941.536
True Neutron Energy: 939.765
True Electron Energy: 9.6804
Completed MCParticle Loop
True Neutrino Energy: 31.6371
 Size of all hits: 0
Begin processing the 8th record. run: 5026 subRun: 0 event: 44088 at 12-Dec-2025 00:55:46 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 19.0749
Completed MCParticle Loop
True Neutrino Energy: 24.454
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 44089 at 12-Dec-2025 00:55:49 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  36
Added hit list length: 36 | Max was set to: -3.40282e+38
This pfp has size:  18
True Electron Energy: 35.047
Completed MCParticle Loop
True Neutrino Energy: 52.2835
 Size of all hits: 36
hit initial : 104.444 | hit correction : 1.3433
hit initial : 254.902 | hit correction : 1.34309
hit initial : 366.86 | hit correction : 1.34279
hit initial : 419.823 | hit correction : 1.34247
 *** Total Charge : 1538.87
 Looking at view: 0 | Number of hits in view: 4
hit initial : 146.141 | hit correction : 1.34038
hit initial : 284.162 | hit correction : 1.34165
hit initial : 82.3031 | hit correction : 1.34048
hit initial : 207.248 | hit correction : 1.34191
hit initial : 199.625 | hit correction : 1.34219
hit initial : 197.286 | hit correction : 1.34237
hit initial : 146.622 | hit correction : 1.34318
hit initial : 43.1265 | hit correction : 1.34341
hit initial : 21.7933 | hit correction : 1.34376
hit initial : 27.9331 | hit correction : 1.34311
hit initial : 50.1672 | hit correction : 1.34311
hit initial : 247.119 | hit correction : 1.34298
hit initial : 196.88 | hit correction : 1.34288
hit initial : 166.861 | hit correction : 1.34275
hit initial : 180.479 | hit correction : 1.34255
 *** Total Charge : 2950
 Looking at view: 1 | Number of hits in view: 15
hit initial : 15.8055 | hit correction : 1.34278
hit initial : 162.167 | hit correction : 1.34263
hit initial : 142.938 | hit correction : 1.34201
hit initial : 603.556 | hit correction : 1.34212
hit initial : 241.123 | hit correction : 1.34177
hit initial : 212.341 | hit correction : 1.34204
hit initial : 264.606 | hit correction : 1.34228
hit initial : 252.516 | hit correction : 1.34252
hit initial : 18.8034 | hit correction : 1.34337
hit initial : 36.683 | hit correction : 1.3426
hit initial : 24.4054 | hit correction : 1.34373
hit initial : 18.3034 | hit correction : 1.34267
hit initial : 38.3074 | hit correction : 1.34256
hit initial : 53.421 | hit correction : 1.3426
hit initial : 181.253 | hit correction : 1.34322
hit initial : 227.637 | hit correction : 1.34303
hit initial : 202.902 | hit correction : 1.34278
 *** Total Charge : 3620.18
 Looking at view: 2 | Number of hits in view: 17
Begin processing the 10th record. run: 5026 subRun: 0 event: 44090 at 12-Dec-2025 00:55:51 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.573
True Electron Energy: 0.920672
Completed MCParticle Loop
True Neutrino Energy: 4.20413
 Size of all hits: 0
12-Dec-2025 00:55:54 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0658308     0.0711767     0.0827454     0.0681427    0.00596477       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000402103   0.000490189   0.000652786   0.000492094   7.66163e-05      10     
end_path:ana:AnalyzeEvents         0.065136      0.0704818     0.0821791     0.0675124    0.00596542       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.5 MB
  Peak resident set size usage (VmHWM): 1530.59 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:58:56 UTC       justIN version: 01.06.00