justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261461.1@dunegpschedd02.fnal.gov

Jobsub ID261461.1@dunegpschedd02.fnal.gov
Workflow ID11158
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-12 04:12:59
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 06:28:23
From worker nodeHostnamedunegli-7890304-0-fnpc22033.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 00:48:30
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T224331Z_gen_000509_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (20379s)
CPU time5h (20059s = 98%)
Max RSS bytes2783674368 (2654 MiB)
Outputting started2025-12-13 06:28:10
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11158/1/001/analysisLowE_20251213T004832Z.root
Finished2025-12-13 06:28:23
Saved logsjustin-logs:261461.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ALIZED
    in function: Run
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoracontent/LArMonitoring/EventValidationBaseAlgorithm.cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6693, 6630, 6678
minwire 0: 49
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 3, 502
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.123259, 
Output 1: 0.116464, 0.11659, 0.235696, 0.53125, 
Output 2: 0.0567704, 0.111488, 0.811865, 0.0198759, 
Output 3: 0.9721, 0.0228691, 0.00184256, 0.00318832, 
Output 4: 0.507956, 0.443337, 0.0420448, 0.00666292, 
Output 5: 0.360802, 0.41555, 0.161998, 0.0616498, 
Output 6: 0.0813919, 0.177241, 0.119702, 0.621665, 

13-Dec-2025 06:27:40 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 06:27:40 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/58/ee/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T224331Z_gen_000509_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1082.98       2020.33       2522.45       2192.45       463.288        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00149209    0.00585831     0.0215974    0.00256284    0.00657989       10     
reco:pandora:StandardPandora                            1081.86       2017.99       2519.43       2190.5        462.729        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000324752    0.0752716     0.192891      0.102708      0.063071        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00029116     0.0135969     0.0971568    0.00297368     0.0280962       10     
reco:pandoracalo:Calorimetry                          0.000177214   0.000896246   0.00272165    0.000496402   0.000774512      10     
reco:pandorapid:Chi2ParticleID                        5.3661e-05    0.000307494    0.001013     0.000138781   0.000316886      10     
reco:cvnmap:CVNMapper                                  0.0615852     0.118249      0.235493      0.0970806     0.0498639       10     
reco:cvneva:CVNEvaluator                               0.766311       1.56526       4.66959       1.27859       1.07726        10     
reco:energyrecnumu:EnergyReco                          0.0608088      0.11117       0.17625      0.116074      0.042015        10     
reco:energyrecnue:EnergyReco                          0.00160018    0.00579008     0.016342     0.00503673     0.0043211       10     
reco:energyrecnc:EnergyReco                           0.00513279     0.0100923     0.0196392    0.00802282    0.00512384       10     
[art]:TriggerResults:TriggerResultInserter            1.2744e-05    5.08299e-05   0.000176213   2.68405e-05   4.7399e-05       10     
end_path:out1:RootOutput                               8.747e-06    2.11758e-05   6.0895e-05    1.58845e-05   1.46505e-05      10     
end_path:out1:RootOutput(write)                        0.219635      0.434014      0.783167      0.446617      0.181848        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21568.5 MB
  Peak resident set size usage (VmHWM): 2783.67 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
This pfp has size:  5
True Electron Energy: 9.12751
Completed MCParticle Loop
True Neutrino Energy: 12.8571
 Size of all hits: 13
hit initial : 35.732 | hit correction : 1.28946
hit initial : 64.0097 | hit correction : 1.28885
hit initial : 350.314 | hit correction : 1.28894
hit initial : 346.783 | hit correction : 1.28863
hit initial : 61.2883 | hit correction : 1.28847
 *** Total Charge : 1105.95
 Looking at view: 0 | Number of hits in view: 5
hit initial : 190.17 | hit correction : 1.28883
hit initial : 444.446 | hit correction : 1.28879
hit initial : 198.97 | hit correction : 1.28874
 *** Total Charge : 1074.32
 Looking at view: 1 | Number of hits in view: 3
hit initial : 59.8512 | hit correction : 1.28807
hit initial : 41.6013 | hit correction : 1.28952
hit initial : 336.157 | hit correction : 1.28911
hit initial : 29.8025 | hit correction : 1.28867
hit initial : 356.491 | hit correction : 1.28858
 *** Total Charge : 1061.85
 Looking at view: 2 | Number of hits in view: 5
Begin processing the 9th record. run: 5027 subRun: 0 event: 5089 at 13-Dec-2025 06:28:06 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  32
Added hit list length: 32 | Max was set to: -3.40282e+38
This pfp has size:  7
This pfp has size:  6
True Electron Energy: 20.9686
Completed MCParticle Loop
True Neutrino Energy: 24.704
 Size of all hits: 32
hit initial : 31.9699 | hit correction : 1.24645
hit initial : 156.5 | hit correction : 1.24648
hit initial : 259.886 | hit correction : 1.2465
hit initial : 211.227 | hit correction : 1.24649
hit initial : 260.372 | hit correction : 1.24624
hit initial : 373.025 | hit correction : 1.24567
hit initial : 55.6263 | hit correction : 1.24565
hit initial : 347.266 | hit correction : 1.24496
hit initial : 312.363 | hit correction : 1.24427
hit initial : 190.312 | hit correction : 1.24383
hit initial : 44.0552 | hit correction : 1.24359
 *** Total Charge : 2793.1
 Looking at view: 0 | Number of hits in view: 11
hit initial : 97.917 | hit correction : 1.2465
hit initial : 288.144 | hit correction : 1.24649
hit initial : 209.52 | hit correction : 1.24648
hit initial : 135.267 | hit correction : 1.24636
hit initial : 27.2064 | hit correction : 1.24555
hit initial : 122.569 | hit correction : 1.24574
hit initial : 251.088 | hit correction : 1.24563
hit initial : 303.487 | hit correction : 1.24466
hit initial : 172.996 | hit correction : 1.24447
hit initial : 150.321 | hit correction : 1.24381
hit initial : 118.917 | hit correction : 1.24389
 *** Total Charge : 2338.23
 Looking at view: 1 | Number of hits in view: 11
hit initial : 136.989 | hit correction : 1.24649
hit initial : 176.442 | hit correction : 1.24647
hit initial : 418.248 | hit correction : 1.24647
hit initial : 397.378 | hit correction : 1.2454
hit initial : 332.781 | hit correction : 1.24598
hit initial : 34.5397 | hit correction : 1.24549
hit initial : 192.998 | hit correction : 1.24471
hit initial : 171.36 | hit correction : 1.24432
hit initial : 186.133 | hit correction : 1.24396
hit initial : 145.979 | hit correction : 1.24362
 *** Total Charge : 2731.11
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 10th record. run: 5027 subRun: 0 event: 5090 at 13-Dec-2025 06:28:08 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Neutron Energy: 939.577
True Neutron Energy: 940.711
True Electron Energy: 35.8257
Completed MCParticle Loop
True Neutrino Energy: 53.1507
 Size of all hits: 0
13-Dec-2025 06:28:10 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0490021     0.0667908     0.0866889     0.0640088     0.0111316       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000449167   0.000656278   0.00115415    0.000554606   0.000246334      10     
end_path:ana:AnalyzeEvents         0.0483504     0.065946      0.0854028     0.0630573     0.0110353       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19473.4 MB
  Peak resident set size usage (VmHWM): 1533.42 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 13:29:33 UTC       justIN version: 01.06.00