justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 259587.2@dunegpschedd02.fnal.gov

Jobsub ID259587.2@dunegpschedd02.fnal.gov
Workflow ID11086
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-09 12:55:44
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-10 20:11:53
From worker nodeHostnamedunegli-7867684-0-fnpc18010.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 12:21:08
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T233820Z_gen_004802_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time7h (28189s)
CPU time7h (26897s = 95%)
Max RSS bytes3070164992 (2927 MiB)
Outputting started2025-12-10 20:10:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11086/1/001/analysisLowE_20251210T122109Z.root
Finished2025-12-10 20:11:53
Saved logsjustin-logs:259587.2-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

TUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6508, 6550, 6558
minwire 0: 50
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2125, 2624
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0692795, 
Output 1: 0.0522796, 0.0533257, 0.077398, 0.816997, 
Output 2: 0.0594345, 0.17975, 0.728971, 0.0318446, 
Output 3: 0.968443, 0.0133004, 0.00328044, 0.0149757, 
Output 4: 0.845953, 0.133653, 0.0158304, 0.00456291, 
Output 5: 0.597167, 0.307754, 0.0733252, 0.0217532, 
Output 6: 0.0135483, 0.0427624, 0.0540418, 0.889648, 

10-Dec-2025 20:08:53 UTC  Closed output file "matt_reco2.root"
10-Dec-2025 20:08:53 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/09/2f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T233820Z_gen_004802_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1938.1        2792.31       4565.54       2450.35       844.674        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00210964     0.0115778     0.0221037     0.0114936    0.00517001       10     
reco:pandora:StandardPandora                            1935.59       2774.87       4488.35       2445.79       821.269        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00336132     0.033145      0.167032      0.0114751     0.047224        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000765824   0.00762485     0.0237125     0.0064549    0.00648024       10     
reco:pandoracalo:Calorimetry                          0.000391027    0.0156773     0.0870017    0.000913207    0.0267224       10     
reco:pandorapid:Chi2ParticleID                        0.000115839   0.00046772    0.00305114    0.000198274   0.000862243      10     
reco:cvnmap:CVNMapper                                  0.0400294     0.0791767     0.126724       0.0808       0.0253182       10     
reco:cvneva:CVNEvaluator                                1.88009       2.73757       4.87317       2.19525       1.0211         10     
reco:energyrecnumu:EnergyReco                          0.0100707      13.9379       73.3466      0.125976       27.7706        10     
reco:energyrecnue:EnergyReco                          0.00163097     0.0101143     0.0287189    0.00309588     0.0107326       10     
reco:energyrecnc:EnergyReco                            0.0100988     0.0512109     0.127472      0.0201849     0.0445548       10     
[art]:TriggerResults:TriggerResultInserter            2.4106e-05    4.89975e-05   0.000117191    3.443e-05    3.00096e-05      10     
end_path:out1:RootOutput                              1.0901e-05    2.67794e-05   7.9229e-05    1.8034e-05    2.01164e-05      10     
end_path:out1:RootOutput(write)                         0.40584       0.55382      0.824236      0.462713      0.152124        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21784.8 MB
  Peak resident set size usage (VmHWM): 3070.16 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 50.1804 | hit correction : 1.31108
hit initial : 297.41 | hit correction : 1.31117
hit initial : 164.264 | hit correction : 1.31171
hit initial : 118.083 | hit correction : 1.31107
hit initial : 54.2019 | hit correction : 1.31036
hit initial : 410.073 | hit correction : 1.31094
hit initial : 25.9708 | hit correction : 1.30975
hit initial : 18.5804 | hit correction : 1.30926
hit initial : 177.564 | hit correction : 1.30931
hit initial : 135.824 | hit correction : 1.30932
hit initial : 177.425 | hit correction : 1.31058
hit initial : 138.15 | hit correction : 1.31055
hit initial : 32.2357 | hit correction : 1.31051
hit initial : 32.1127 | hit correction : 1.30817
hit initial : 27.4209 | hit correction : 1.30696
hit initial : 16.943 | hit correction : 1.30702
hit initial : 191.972 | hit correction : 1.31054
hit initial : 194.542 | hit correction : 1.31019
hit initial : 322.315 | hit correction : 1.3101
hit initial : 170.454 | hit correction : 1.30995
hit initial : 170.718 | hit correction : 1.30968
hit initial : 287.849 | hit correction : 1.30954
hit initial : 244.215 | hit correction : 1.30945
hit initial : 285.585 | hit correction : 1.30939
hit initial : 215.49 | hit correction : 1.30921
hit initial : 160.067 | hit correction : 1.30908
hit initial : 148.725 | hit correction : 1.30892
hit initial : 154.681 | hit correction : 1.30879
hit initial : 152.286 | hit correction : 1.30867
hit initial : 170.189 | hit correction : 1.30851
hit initial : 201.044 | hit correction : 1.30837
hit initial : 173.434 | hit correction : 1.30822
hit initial : 165.507 | hit correction : 1.30808
hit initial : 168.793 | hit correction : 1.3079
hit initial : 184.237 | hit correction : 1.30767
hit initial : 118.378 | hit correction : 1.30753
 *** Total Charge : 7539.06
 Looking at view: 2 | Number of hits in view: 36
Begin processing the 9th record. run: 5026 subRun: 0 event: 48019 at 10-Dec-2025 20:10:40 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  21
Added hit list length: 21 | Max was set to: -3.40282e+38
True Electron Energy: 16.403
Completed MCParticle Loop
True Neutrino Energy: 22.5996
 Size of all hits: 21
hit initial : 222.655 | hit correction : 1.18344
hit initial : 498.475 | hit correction : 1.18344
hit initial : 49.232 | hit correction : 1.18425
hit initial : 597.617 | hit correction : 1.18332
hit initial : 107.267 | hit correction : 1.18411
hit initial : 103.404 | hit correction : 1.18314
hit initial : 77.6558 | hit correction : 1.18243
hit initial : 64.7866 | hit correction : 1.18539
 *** Total Charge : 2036.87
 Looking at view: 0 | Number of hits in view: 8
hit initial : 65.013 | hit correction : 1.18254
hit initial : 80.7255 | hit correction : 1.18243
hit initial : 101.393 | hit correction : 1.18423
hit initial : 47.36 | hit correction : 1.1842
hit initial : 308.126 | hit correction : 1.18343
hit initial : 643.817 | hit correction : 1.1834
hit initial : 363.402 | hit correction : 1.18334
 *** Total Charge : 1905.06
 Looking at view: 1 | Number of hits in view: 7
hit initial : 154.922 | hit correction : 1.18256
hit initial : 159.081 | hit correction : 1.18418
hit initial : 31.814 | hit correction : 1.18325
hit initial : 457.716 | hit correction : 1.18344
hit initial : 808.17 | hit correction : 1.18333
hit initial : 87.4781 | hit correction : 1.18342
 *** Total Charge : 2010.77
 Looking at view: 2 | Number of hits in view: 6
Begin processing the 10th record. run: 5026 subRun: 0 event: 48020 at 10-Dec-2025 20:10:48 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.57
True Electron Energy: 6.73019
Completed MCParticle Loop
True Neutrino Energy: 12.491
 Size of all hits: 0
10-Dec-2025 20:10:55 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.123433      0.445654       1.60535      0.317764       0.4096         10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000716241    0.0750178     0.207984      0.0259885     0.0884638       10     
end_path:ana:AnalyzeEvents         0.122165      0.370214       1.39928      0.307391      0.353878        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.3 MB
  Peak resident set size usage (VmHWM): 1535.5 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:26:56 UTC       justIN version: 01.06.00