justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266043.8@dunegpschedd01.fnal.gov

Jobsub ID266043.8@dunegpschedd01.fnal.gov
Workflow ID11087
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-10 13:59:01
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-11 17:28:36
From worker nodeHostnamedunegli-7867869-0-fnpc9076.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 08:39:18
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213556Z_gen_004323_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (31734s)
CPU time8h (31355s = 98%)
Max RSS bytes3036823552 (2896 MiB)
Outputting started2025-12-11 17:28:13
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11087/1/001/analysisLowE_20251211T083920Z.root
Finished2025-12-11 17:28:36
Saved logsjustin-logs:266043.8-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

_LowE_Build/srcs/larpandoracontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6670, 6668, 6759
minwire 0: 36
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1997, 2496
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0668878, 
Output 1: 0.0325277, 0.00799236, 0.026462, 0.933018, 
Output 2: 0.1656, 0.384027, 0.394322, 0.0560513, 
Output 3: 0.945729, 0.030163, 0.0073195, 0.0167882, 
Output 4: 0.852876, 0.134556, 0.0107281, 0.00184012, 
Output 5: 0.261587, 0.705348, 0.031632, 0.00143279, 
Output 6: 0.0095103, 0.0481198, 0.06702, 0.87535, 

11-Dec-2025 17:26:45 UTC  Closed output file "matt_reco2.root"
11-Dec-2025 17:26:45 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/f4/da/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213556Z_gen_004323_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2260.19       3137.76       3964.41       3167.16       431.602        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00336143     0.0166052     0.0354426     0.0162744    0.00833187       10     
reco:pandora:StandardPandora                            2257.4        3134.2        3961.1        3164.21       430.855        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00057208     0.0595216      0.26638      0.0232862     0.0771869       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000421335    0.0156798     0.0735716    0.00782246     0.0226116       10     
reco:pandoracalo:Calorimetry                          0.000253097   0.00855916     0.0464155    0.00071012     0.0144046       10     
reco:pandorapid:Chi2ParticleID                        8.5857e-05    0.000635172   0.00318036    0.000132114   0.00102906       10     
reco:cvnmap:CVNMapper                                  0.0751993     0.0958308     0.134551      0.0908095     0.0189169       10     
reco:cvneva:CVNEvaluator                                1.85704       2.55997       7.39774       2.03576       1.6158         10     
reco:energyrecnumu:EnergyReco                         0.00789947     0.0987807     0.324177      0.0784207     0.0997831       10     
reco:energyrecnue:EnergyReco                          0.00238084    0.00804573     0.0196813    0.00371195    0.00648404       10     
reco:energyrecnc:EnergyReco                            0.0129302     0.0832278     0.200622      0.0684984     0.0711638       10     
[art]:TriggerResults:TriggerResultInserter            2.9763e-05    5.82857e-05   0.000236742   4.07145e-05   5.97669e-05      10     
end_path:out1:RootOutput                               5.576e-06    1.84501e-05   6.9897e-05    1.33655e-05   1.73422e-05      10     
end_path:out1:RootOutput(write)                        0.410447      0.616677      0.851095      0.613259      0.100666        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12071.6 MB
  Peak resident set size usage (VmHWM): 3036.82 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
This pfp has size:  19
Added hit list length: 19 | Max was set to: -3.40282e+38
True Electron Energy: 55.0628
Completed MCParticle Loop
True Neutrino Energy: 61.3379
 Size of all hits: 19
hit initial : 146.602 | hit correction : 1.37662
hit initial : 136.996 | hit correction : 1.37559
hit initial : 162.137 | hit correction : 1.37679
hit initial : 41.5431 | hit correction : 1.37559
hit initial : 189.4 | hit correction : 1.37711
hit initial : 113.992 | hit correction : 1.37808
hit initial : 122.159 | hit correction : 1.37738
hit initial : 179.059 | hit correction : 1.37808
hit initial : 48.5539 | hit correction : 1.37816
 *** Total Charge : 1570.49
 Looking at view: 1 | Number of hits in view: 9
hit initial : 39.8124 | hit correction : 1.37726
hit initial : 411.252 | hit correction : 1.37655
hit initial : 424.47 | hit correction : 1.37649
hit initial : 100.164 | hit correction : 1.3781
hit initial : 307.734 | hit correction : 1.37802
hit initial : 28.422 | hit correction : 1.37811
hit initial : 291.702 | hit correction : 1.37709
hit initial : 351.945 | hit correction : 1.37667
hit initial : 975.957 | hit correction : 1.37683
hit initial : 1078.86 | hit correction : 1.37692
 *** Total Charge : 5521.94
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 9th record. run: 5026 subRun: 0 event: 43229 at 11-Dec-2025 17:28:01 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  30
Added hit list length: 30 | Max was set to: -3.40282e+38
True Electron Energy: 21.2552
Completed MCParticle Loop
True Neutrino Energy: 26.6341
 Size of all hits: 30
hit initial : 106.414 | hit correction : 1.44007
hit initial : 78.9538 | hit correction : 1.44254
hit initial : 214.727 | hit correction : 1.44254
hit initial : 395.9 | hit correction : 1.44241
hit initial : 121.531 | hit correction : 1.44239
hit initial : 65.5583 | hit correction : 1.44088
hit initial : 47.0645 | hit correction : 1.44089
hit initial : 121.625 | hit correction : 1.44085
hit initial : 279.27 | hit correction : 1.44051
 *** Total Charge : 2063.05
 Looking at view: 0 | Number of hits in view: 9
hit initial : 52.2378 | hit correction : 1.44083
hit initial : 27.5462 | hit correction : 1.4425
hit initial : 116.721 | hit correction : 1.44257
hit initial : 261.007 | hit correction : 1.44256
hit initial : 341.134 | hit correction : 1.44241
hit initial : 115.388 | hit correction : 1.44235
hit initial : 244.036 | hit correction : 1.44129
hit initial : 72.8779 | hit correction : 1.44006
hit initial : 335.69 | hit correction : 1.44078
hit initial : 231.711 | hit correction : 1.44035
 *** Total Charge : 2592.46
 Looking at view: 1 | Number of hits in view: 10
hit initial : 41.5167 | hit correction : 1.44247
hit initial : 166.469 | hit correction : 1.44258
hit initial : 74.679 | hit correction : 1.44084
hit initial : 134.24 | hit correction : 1.44279
hit initial : 31.7163 | hit correction : 1.44082
hit initial : 173.457 | hit correction : 1.4425
hit initial : 257.773 | hit correction : 1.4423
hit initial : 163.372 | hit correction : 1.44119
hit initial : 51.1845 | hit correction : 1.44222
hit initial : 567.921 | hit correction : 1.4407
hit initial : 150.676 | hit correction : 1.44056
 *** Total Charge : 2613.54
 Looking at view: 2 | Number of hits in view: 11
Begin processing the 10th record. run: 5026 subRun: 0 event: 43230 at 11-Dec-2025 17:28:06 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.569
True Electron Energy: 14.0845
Completed MCParticle Loop
True Neutrino Energy: 17.3797
 Size of all hits: 0
11-Dec-2025 17:28:12 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.118985      0.172437      0.232018      0.171693      0.028477        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00068034    0.000939369   0.00159079    0.000849362   0.000268252      10     
end_path:ana:AnalyzeEvents         0.117495      0.171225      0.229593      0.170728      0.028295        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9809.57 MB
  Peak resident set size usage (VmHWM): 1536.11 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:28:46 UTC       justIN version: 01.06.00