justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265370.1@dunegpschedd01.fnal.gov

Jobsub ID265370.1@dunegpschedd01.fnal.gov
Workflow ID11086
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-09 12:47:44
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-10 19:19:55
From worker nodeHostnamedunegli-8024622-0-fnpc9059.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 11:58:30
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T225933Z_gen_001025_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time7h (26439s)
CPU time7h (25621s = 96%)
Max RSS bytes3061817344 (2919 MiB)
Outputting started2025-12-10 19:19:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11086/1/001/analysisLowE_20251210T115831Z.root
Finished2025-12-10 19:19:55
Saved logsjustin-logs:265370.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 7026, 6815, 6775
minwire 0: 40
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2308, 2807
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.129798, 
Output 1: 0.0539754, 0.15823, 0.17763, 0.610165, 
Output 2: 0.0692437, 0.218941, 0.676208, 0.0356064, 
Output 3: 0.961405, 0.0231811, 0.00434777, 0.0110663, 
Output 4: 0.475825, 0.474244, 0.0452178, 0.00471341, 
Output 5: 0.454722, 0.43165, 0.0931263, 0.0205023, 
Output 6: 0.0305319, 0.0658411, 0.0741628, 0.829464, 

10-Dec-2025 19:17:03 UTC  Closed output file "matt_reco2.root"
10-Dec-2025 19:17:03 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/c2/9b/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T225933Z_gen_001025_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1468.55       2563.39       3662.8        2777.63       692.646        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00147543     0.0145471     0.0540425     0.0124074     0.0139774       10     
reco:pandora:StandardPandora                            1466.79       2560.38        3660         2774.09       692.401        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000599068    0.0450499     0.0949952     0.026576      0.0366796       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000786684    0.0366132     0.141694      0.0108753     0.0439857       10     
reco:pandoracalo:Calorimetry                          0.000294028    0.0010694    0.00289163    0.000709827   0.000889952      10     
reco:pandorapid:Chi2ParticleID                        8.1793e-05    0.000257907   0.00124684    0.00014264    0.000335262      10     
reco:cvnmap:CVNMapper                                  0.0384876     0.102515      0.205695      0.0989635     0.0439998       10     
reco:cvneva:CVNEvaluator                                1.05868       1.89115       3.12565       1.70103      0.706557        10     
reco:energyrecnumu:EnergyReco                           0.00457      0.053602      0.159497      0.0151042     0.0606278       10     
reco:energyrecnue:EnergyReco                           0.0010251    0.00299814     0.010996     0.00210971    0.00276384       10     
reco:energyrecnc:EnergyReco                            0.0101889     0.0890089     0.201583      0.0940082      0.05979        10     
[art]:TriggerResults:TriggerResultInserter            1.6754e-05    3.37341e-05   6.7903e-05     2.793e-05    1.55122e-05      10     
end_path:out1:RootOutput                               3.072e-06    1.18771e-05   2.1067e-05    1.16355e-05   4.89596e-06      10     
end_path:out1:RootOutput(write)                         0.43198      0.765978       1.91187       0.58373       0.42804        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12128.8 MB
  Peak resident set size usage (VmHWM): 3061.82 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 189.348 | hit correction : 1.28854
 *** Total Charge : 2430.54
 Looking at view: 1 | Number of hits in view: 13
hit initial : 69.5044 | hit correction : 1.28827
hit initial : 30.8858 | hit correction : 1.29018
hit initial : 129.868 | hit correction : 1.29023
hit initial : 173.543 | hit correction : 1.29037
hit initial : 116.155 | hit correction : 1.29038
hit initial : 61.3962 | hit correction : 1.28903
hit initial : 329.688 | hit correction : 1.28908
hit initial : 348.564 | hit correction : 1.28918
hit initial : 310.921 | hit correction : 1.28943
hit initial : 396.586 | hit correction : 1.29
hit initial : 248.574 | hit correction : 1.29057
hit initial : 465.484 | hit correction : 1.28877
hit initial : 59.908 | hit correction : 1.2888
hit initial : 291.869 | hit correction : 1.28861
hit initial : 290.459 | hit correction : 1.28834
hit initial : 257.332 | hit correction : 1.28832
 *** Total Charge : 4616.52
 Looking at view: 2 | Number of hits in view: 16
Begin processing the 8th record. run: 5027 subRun: 0 event: 10248 at 10-Dec-2025 19:18:42 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 13.8937
Completed MCParticle Loop
True Neutrino Energy: 19.4076
 Size of all hits: 0
Begin processing the 9th record. run: 5027 subRun: 0 event: 10249 at 10-Dec-2025 19:18:50 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  31
Added hit list length: 31 | Max was set to: -3.40282e+38
True Electron Energy: 25.7759
Completed MCParticle Loop
True Neutrino Energy: 64.7999
 Size of all hits: 31
hit initial : 112.297 | hit correction : 1.16158
hit initial : 206.014 | hit correction : 1.16158
hit initial : 252.925 | hit correction : 1.1616
hit initial : 207.191 | hit correction : 1.16159
hit initial : 68.9656 | hit correction : 1.16156
hit initial : 31.8044 | hit correction : 1.16156
hit initial : 176.235 | hit correction : 1.16161
hit initial : 281.085 | hit correction : 1.16173
hit initial : 265.674 | hit correction : 1.16198
hit initial : 164.133 | hit correction : 1.1621
hit initial : 38.7868 | hit correction : 1.16214
 *** Total Charge : 2097.04
 Looking at view: 0 | Number of hits in view: 11
hit initial : 260.083 | hit correction : 1.16158
hit initial : 411.2 | hit correction : 1.16159
hit initial : 61.5707 | hit correction : 1.1629
hit initial : 37.2019 | hit correction : 1.16289
hit initial : 149.025 | hit correction : 1.16183
hit initial : 140.99 | hit correction : 1.16159
hit initial : 460.372 | hit correction : 1.16182
hit initial : 302.12 | hit correction : 1.16188
 *** Total Charge : 2117.43
 Looking at view: 1 | Number of hits in view: 8
hit initial : 151.001 | hit correction : 1.16158
hit initial : 171.433 | hit correction : 1.16155
hit initial : 143.811 | hit correction : 1.16159
hit initial : 120.047 | hit correction : 1.16164
hit initial : 137.542 | hit correction : 1.16161
hit initial : 114.62 | hit correction : 1.16153
hit initial : 190.58 | hit correction : 1.16157
hit initial : 158.312 | hit correction : 1.16163
hit initial : 158.336 | hit correction : 1.1618
hit initial : 158.196 | hit correction : 1.16198
hit initial : 229.815 | hit correction : 1.16213
hit initial : 26.5984 | hit correction : 1.16216
 *** Total Charge : 2044.97
 Looking at view: 2 | Number of hits in view: 12
Begin processing the 10th record. run: 5027 subRun: 0 event: 10250 at 10-Dec-2025 19:18:58 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.575
True Electron Energy: 1.09145
Completed MCParticle Loop
True Neutrino Energy: 4.37506
 Size of all hits: 0
10-Dec-2025 19:19:05 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0868262     0.279394       0.7383        0.25856      0.171777        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000679999    0.0260833     0.0905575    0.000949836    0.0336821       10     
end_path:ana:AnalyzeEvents         0.0856521     0.252981      0.702588      0.221982      0.161172        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9818.54 MB
  Peak resident set size usage (VmHWM): 1544.02 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:29:01 UTC       justIN version: 01.06.00