justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268011.1@dunegpschedd01.fnal.gov

Jobsub ID268011.1@dunegpschedd01.fnal.gov
Workflow ID11175
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 11:34:34
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 21:20:51
From worker nodeHostnamedunegli-7890499-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 11:35:26
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T032159Z_gen_004996_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time9h (35107s)
CPU time9h (34547s = 98%)
Max RSS bytes2799312896 (2669 MiB)
Outputting started2025-12-13 21:20:34
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11175/1/001/analysisLowE_20251213T113528Z.root
Finished2025-12-13 21:20:51
Saved logsjustin-logs:268011.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

/exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoracontent/LArMonitoring/EventValidationBaseAlgorithm.cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6796, 6692, 6649
minwire 0: 40
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1725, 2224
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0989147, 
Output 1: 0.162288, 0.326333, 0.26913, 0.242249, 
Output 2: 0.0272432, 0.0667034, 0.898702, 0.00735121, 
Output 3: 0.959253, 0.0369896, 0.00175926, 0.00199812, 
Output 4: 0.721201, 0.241183, 0.028717, 0.00889851, 
Output 5: 0.298252, 0.392888, 0.206389, 0.102472, 
Output 6: 0.14033, 0.273218, 0.134363, 0.45209, 

13-Dec-2025 21:19:14 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 21:19:14 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/da/42/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T032159Z_gen_004996_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3150.01       3475.34       3972.66       3388.1        267.582        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00103316    0.00369919     0.0158063    0.00123961    0.00474408       10     
reco:pandora:StandardPandora                            3145.71       3470.28       3968.19       3381.82       267.784        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000427135    0.0443674     0.110755     0.00994469     0.0497385       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000457692   0.00712521     0.0420419    0.00234239     0.0121223       10     
reco:pandoracalo:Calorimetry                          0.000231065    0.0180806     0.088107     0.00157239     0.0319134       10     
reco:pandorapid:Chi2ParticleID                         8.468e-05    0.000592125   0.00443951    0.000165347    0.0012833       10     
reco:cvnmap:CVNMapper                                  0.0730661     0.0962492      0.12006      0.0915686     0.0149154       10     
reco:cvneva:CVNEvaluator                                2.93598       4.12294       7.23187       3.48736       1.29048        10     
reco:energyrecnumu:EnergyReco                          0.109856      0.131377      0.152091      0.134447      0.0117158       10     
reco:energyrecnue:EnergyReco                          0.00251735    0.00956862     0.0173774    0.00964338    0.00647406       10     
reco:energyrecnc:EnergyReco                            0.0143681     0.0155922     0.0172357     0.0154645    0.000912087      10     
[art]:TriggerResults:TriggerResultInserter            2.1751e-05    4.22896e-05   0.000156995   3.0402e-05    3.85072e-05      10     
end_path:out1:RootOutput                               8.896e-06    1.4822e-05     4.777e-05    1.14065e-05   1.11275e-05      10     
end_path:out1:RootOutput(write)                        0.576962      0.608099       0.67756      0.594607      0.0322185       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21577.7 MB
  Peak resident set size usage (VmHWM): 2799.31 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 117.561 | hit correction : 1.28412
hit initial : 150.342 | hit correction : 1.28399
hit initial : 135.407 | hit correction : 1.28391
hit initial : 133.581 | hit correction : 1.28386
hit initial : 113.83 | hit correction : 1.28384
hit initial : 166.723 | hit correction : 1.28386
hit initial : 113.56 | hit correction : 1.28387
 *** Total Charge : 3130.93
 Looking at view: 2 | Number of hits in view: 21
Begin processing the 6th record. run: 5027 subRun: 0 event: 49956 at 13-Dec-2025 21:20:14 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  10
Added hit list length: 10 | Max was set to: -3.40282e+38
This pfp has size:  6
True Neutron Energy: 939.566
True Electron Energy: 7.88194
Completed MCParticle Loop
True Neutrino Energy: 11.1655
 Size of all hits: 10
hit initial : 323.948 | hit correction : 1.02336
hit initial : 255.058 | hit correction : 1.02334
hit initial : 199.511 | hit correction : 1.02266
hit initial : 107.409 | hit correction : 1.02264
 *** Total Charge : 906.401
 Looking at view: 0 | Number of hits in view: 4
hit initial : 288.617 | hit correction : 1.0234
hit initial : 249.875 | hit correction : 1.02332
hit initial : 137.387 | hit correction : 1.02267
hit initial : 197.171 | hit correction : 1.02265
 *** Total Charge : 893.212
 Looking at view: 1 | Number of hits in view: 4
hit initial : 805.155 | hit correction : 1.02328
hit initial : 210.524 | hit correction : 1.0225
 *** Total Charge : 1039.16
 Looking at view: 2 | Number of hits in view: 2
Begin processing the 7th record. run: 5027 subRun: 0 event: 49957 at 13-Dec-2025 21:20:18 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 7.0455
Completed MCParticle Loop
True Neutrino Energy: 12.8059
 Size of all hits: 0
Begin processing the 8th record. run: 5027 subRun: 0 event: 49958 at 13-Dec-2025 21:20:22 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 48.1042
Completed MCParticle Loop
True Neutrino Energy: 53.5045
 Size of all hits: 0
Begin processing the 9th record. run: 5027 subRun: 0 event: 49959 at 13-Dec-2025 21:20:26 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  8
Added hit list length: 8 | Max was set to: -3.40282e+38
This pfp has size:  8
True Electron Energy: 8.14823
Completed MCParticle Loop
True Neutrino Energy: 13.529
 Size of all hits: 8
hit initial : 33.6841 | hit correction : 1.22576
hit initial : 110.381 | hit correction : 1.2258
hit initial : 60.806 | hit correction : 1.22581
 *** Total Charge : 251.131
 Looking at view: 0 | Number of hits in view: 3
hit initial : 48.5001 | hit correction : 1.2258
hit initial : 105.207 | hit correction : 1.22579
hit initial : 39.2458 | hit correction : 1.22579
 *** Total Charge : 236.521
 Looking at view: 1 | Number of hits in view: 3
hit initial : 36.4315 | hit correction : 1.22574
hit initial : 145.768 | hit correction : 1.2258
 *** Total Charge : 223.338
 Looking at view: 2 | Number of hits in view: 2
Begin processing the 10th record. run: 5027 subRun: 0 event: 49960 at 13-Dec-2025 21:20:29 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Neutron Energy: 939.569
True Electron Energy: 21.3168
Completed MCParticle Loop
True Neutrino Energy: 24.6083
 Size of all hits: 0
13-Dec-2025 21:20:34 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0921947     0.199897      0.799624      0.108791      0.212963        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00057307    0.00848361     0.0783817    0.000732114    0.0232997       10     
end_path:ana:AnalyzeEvents         0.0913616     0.191094      0.720453      0.107723      0.191124        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19474.4 MB
  Peak resident set size usage (VmHWM): 1535.58 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:22:29 UTC       justIN version: 01.06.00