justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262270.0@dunegpschedd02.fnal.gov

Jobsub ID262270.0@dunegpschedd02.fnal.gov
Workflow ID11176
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 16:58:50
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 22:04:16
From worker nodeHostnamedunegli-7890613-0-fnpc19133.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 17:00:53
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003152Z_gen_002456_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (18191s)
CPU time4h (17940s = 98%)
Max RSS bytes2835447808 (2704 MiB)
Outputting started2025-12-13 22:04:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11176/1/001/analysisLowE_20251213T170054Z.root
Finished2025-12-13 22:04:16
Saved logsjustin-logs:262270.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ent/larpandoracontent/LArMonitoring/EventValidationBaseAlgorithm.cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6980, 6911, 6832
minwire 0: 15
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.154538, 
Output 1: 0.0905317, 0.183959, 0.190217, 0.535293, 
Output 2: 0.0614628, 0.241122, 0.669434, 0.0279815, 
Output 3: 0.953383, 0.0368289, 0.00427486, 0.00551333, 
Output 4: 0.543351, 0.389928, 0.0589059, 0.0078156, 
Output 5: 0.38086, 0.490564, 0.109733, 0.0188426, 
Output 6: 0.0401872, 0.0928995, 0.0980389, 0.768874, 

13-Dec-2025 22:03:21 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 22:03:21 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/b4/34/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003152Z_gen_002456_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1537.6        1799.65       2140.03       1724.69       191.916        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000851197   0.00443693     0.0224002    0.000995668   0.00716907       10     
reco:pandora:StandardPandora                            1535.67       1797.47       2138.42       1722.95       191.528        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00789457     0.0447297     0.0982245     0.0378633     0.0226505       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00125272    0.00746735     0.0333777    0.00546439    0.00880386       10     
reco:pandoracalo:Calorimetry                          0.000286972   0.00496102     0.0355689    0.00056373     0.0104138       10     
reco:pandorapid:Chi2ParticleID                        8.2521e-05    0.00038405    0.00272685    0.000104511   0.000783197      10     
reco:cvnmap:CVNMapper                                  0.0466122     0.0832602     0.218329      0.0713754     0.0473099       10     
reco:cvneva:CVNEvaluator                               0.993352       1.57527       4.07021       1.42127      0.849327        10     
reco:energyrecnumu:EnergyReco                          0.0754908     0.0961845     0.129585      0.0982186     0.0148318       10     
reco:energyrecnue:EnergyReco                          0.00125143     0.0023127    0.00786246    0.00135572    0.00199827       10     
reco:energyrecnc:EnergyReco                            0.0076068    0.00897368     0.013619     0.00798917    0.00201831       10     
[art]:TriggerResults:TriggerResultInserter             1.421e-05    2.60111e-05   0.000113561   1.6805e-05    2.92239e-05      10     
end_path:out1:RootOutput                               9.38e-06     1.45361e-05   4.0301e-05     1.069e-05    9.19985e-06      10     
end_path:out1:RootOutput(write)                        0.299458      0.345153      0.506605       0.31707      0.0664028       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21574.3 MB
  Peak resident set size usage (VmHWM): 2835.45 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 *** Total Charge : 1474.16
 Looking at view: 0 | Number of hits in view: 10
hit initial : 105.759 | hit correction : 1.16088
hit initial : 254.901 | hit correction : 1.16105
hit initial : 126.973 | hit correction : 1.16048
hit initial : 110.554 | hit correction : 1.16042
hit initial : 87.6628 | hit correction : 1.16124
hit initial : 187.343 | hit correction : 1.16042
hit initial : 220.615 | hit correction : 1.16052
hit initial : 166.445 | hit correction : 1.16064
hit initial : 83.5551 | hit correction : 1.16072
 *** Total Charge : 1559.75
 Looking at view: 1 | Number of hits in view: 9
hit initial : 224.192 | hit correction : 1.16101
hit initial : 357.62 | hit correction : 1.16046
hit initial : 276.198 | hit correction : 1.16126
hit initial : 764.234 | hit correction : 1.16055
hit initial : 32.3347 | hit correction : 1.16067
 *** Total Charge : 1920.49
 Looking at view: 2 | Number of hits in view: 5
Begin processing the 10th record. run: 5027 subRun: 0 event: 24560 at 13-Dec-2025 22:04:01 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  46
Added hit list length: 46 | Max was set to: -3.40282e+38
True Neutron Energy: 939.572
True Electron Energy: 26.4801
Completed MCParticle Loop
True Neutrino Energy: 32.7073
 Size of all hits: 46
hit initial : 78.3595 | hit correction : 1.18079
hit initial : 146.565 | hit correction : 1.18078
hit initial : 69.856 | hit correction : 1.18073
hit initial : 94.0547 | hit correction : 1.16918
hit initial : 195.374 | hit correction : 1.16913
hit initial : 203.951 | hit correction : 1.16902
hit initial : 206.75 | hit correction : 1.16885
hit initial : 211.658 | hit correction : 1.16861
hit initial : 224.076 | hit correction : 1.16834
hit initial : 274.477 | hit correction : 1.16792
hit initial : 331.963 | hit correction : 1.16728
hit initial : 320.713 | hit correction : 1.16668
hit initial : 26.3834 | hit correction : 1.16663
hit initial : 361.531 | hit correction : 1.16581
hit initial : 332.781 | hit correction : 1.16521
hit initial : 104.489 | hit correction : 1.16507
 *** Total Charge : 3719.66
 Looking at view: 0 | Number of hits in view: 16
hit initial : 70.8522 | hit correction : 1.18077
hit initial : 168.23 | hit correction : 1.18078
hit initial : 72.244 | hit correction : 1.18078
hit initial : 104.164 | hit correction : 1.16913
hit initial : 309.282 | hit correction : 1.16905
hit initial : 577.903 | hit correction : 1.16859
hit initial : 144.685 | hit correction : 1.16849
hit initial : 161.268 | hit correction : 1.16729
hit initial : 168.825 | hit correction : 1.16657
hit initial : 147.793 | hit correction : 1.16667
hit initial : 187.861 | hit correction : 1.1655
hit initial : 292.99 | hit correction : 1.16512
hit initial : 66.7312 | hit correction : 1.16486
 *** Total Charge : 2891.02
 Looking at view: 1 | Number of hits in view: 13
hit initial : 41.3743 | hit correction : 1.18088
hit initial : 208.205 | hit correction : 1.18079
hit initial : 60.8278 | hit correction : 1.18064
hit initial : 143.196 | hit correction : 1.16919
hit initial : 278.127 | hit correction : 1.16907
hit initial : 290.477 | hit correction : 1.16887
hit initial : 168.714 | hit correction : 1.16861
hit initial : 135.624 | hit correction : 1.16841
hit initial : 130.824 | hit correction : 1.16825
hit initial : 191.529 | hit correction : 1.16797
hit initial : 239.399 | hit correction : 1.16752
hit initial : 293.794 | hit correction : 1.167
hit initial : 283.592 | hit correction : 1.16655
hit initial : 210.375 | hit correction : 1.16597
hit initial : 225.707 | hit correction : 1.16547
hit initial : 226.041 | hit correction : 1.16492
hit initial : 54.7614 | hit correction : 1.16465
 *** Total Charge : 3719.41
 Looking at view: 2 | Number of hits in view: 17
13-Dec-2025 22:04:04 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0775906     0.0940468     0.114109      0.0935139    0.00974734       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000402844   0.000542719   0.00109102    0.000477844   0.000189372      10     
end_path:ana:AnalyzeEvents         0.0770339     0.0933144     0.112494      0.0928558    0.00953265       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19481 MB
  Peak resident set size usage (VmHWM): 1540.99 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:23:22 UTC       justIN version: 01.06.00