justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264365.8@dunegpschedd01.fnal.gov

Jobsub ID264365.8@dunegpschedd01.fnal.gov
Workflow ID11030
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-06 11:05:54
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-06 19:19:49
From worker nodeHostnamedunegli-7832544-0-fnpc7304.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 11:06:29
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003108Z_gen_002430_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (29583s)
CPU time8h (29085s = 98%)
Max RSS bytes3103813632 (2960 MiB)
Outputting started2025-12-06 19:19:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11030/1/001/analysisLowE_20251206T110630Z.root
Finished2025-12-06 19:19:49
Saved logsjustin-logs:264365.8-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 7030, 6942, 6782
minwire 0: 44
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.107409, 
Output 1: 0.0364383, 0.0462815, 0.10031, 0.81697, 
Output 2: 0.0463774, 0.280183, 0.639191, 0.034248, 
Output 3: 0.949145, 0.0231073, 0.00720468, 0.0205434, 
Output 4: 0.28166, 0.618567, 0.0932743, 0.00649866, 
Output 5: 0.698811, 0.251817, 0.0418881, 0.0074838, 
Output 6: 0.00654634, 0.0214965, 0.0406045, 0.931353, 

06-Dec-2025 19:18:06 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 19:18:06 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/9d/d1/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003108Z_gen_002430_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2571.7        2926.91       3259.16       2950.33       216.793        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00199832     0.016948      0.0502712     0.0130229     0.0130177       10     
reco:pandora:StandardPandora                            2569.38       2923.41       3255.72       2945.92       216.685        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00132735     0.0422752     0.168285      0.0289846     0.0482003       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000389559   0.00954005     0.0212182    0.00809558    0.00637484       10     
reco:pandoracalo:Calorimetry                          0.000222515    0.0150747     0.0913413    0.00219447     0.0280421       10     
reco:pandorapid:Chi2ParticleID                        7.1947e-05    0.000549965   0.00300881    0.000298939   0.000825214      10     
reco:cvnmap:CVNMapper                                  0.0543167     0.0881252     0.149585      0.0799096     0.0313122       10     
reco:cvneva:CVNEvaluator                                1.38905       2.42141       5.21786       2.23343      0.981955        10     
reco:energyrecnumu:EnergyReco                          0.0144246     0.127907      0.234141      0.140429      0.0606021       10     
reco:energyrecnue:EnergyReco                          0.00227166    0.00681653     0.0160424    0.00352328    0.00551774       10     
reco:energyrecnc:EnergyReco                            0.0140601     0.0431501     0.155825      0.016246      0.0544903       10     
[art]:TriggerResults:TriggerResultInserter            3.0044e-05    4.70366e-05   0.00010646    4.09535e-05   2.04515e-05      10     
end_path:out1:RootOutput                              1.1067e-05    1.92462e-05   3.5805e-05    1.57835e-05   7.53653e-06      10     
end_path:out1:RootOutput(write)                        0.534917      0.722459      0.856531      0.726271      0.0957758       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9967.2 MB
  Peak resident set size usage (VmHWM): 3103.81 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 17.7429 | hit correction : 1.22595
hit initial : 48.2645 | hit correction : 1.22601
hit initial : 45.102 | hit correction : 1.23038
hit initial : 80.7015 | hit correction : 1.23026
hit initial : 303.117 | hit correction : 1.23035
hit initial : 94.5371 | hit correction : 1.23064
hit initial : 65.9902 | hit correction : 1.23138
hit initial : 92.4443 | hit correction : 1.23136
hit initial : 166.748 | hit correction : 1.23184
hit initial : 240.431 | hit correction : 1.23177
hit initial : 170.227 | hit correction : 1.23186
hit initial : 46.8606 | hit correction : 1.23179
hit initial : 338.219 | hit correction : 1.23196
hit initial : 32.2869 | hit correction : 1.23298
hit initial : 217.65 | hit correction : 1.233
hit initial : 227.285 | hit correction : 1.23312
hit initial : 198.961 | hit correction : 1.23328
hit initial : 224.882 | hit correction : 1.23344
hit initial : 191.657 | hit correction : 1.23355
hit initial : 198.553 | hit correction : 1.23366
hit initial : 218.524 | hit correction : 1.23379
hit initial : 230.037 | hit correction : 1.23393
hit initial : 129.249 | hit correction : 1.23407
 *** Total Charge : 6111.1
 Looking at view: 2 | Number of hits in view: 34
Begin processing the 9th record. run: 5027 subRun: 0 event: 24299 at 06-Dec-2025 19:19:21 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  35
Added hit list length: 35 | Max was set to: -3.40282e+38
This pfp has size:  21
True Electron Energy: 52.2218
Completed MCParticle Loop
True Neutrino Energy: 67.8719
 Size of all hits: 35
hit initial : 39.336 | hit correction : 1.2157
hit initial : 86.3339 | hit correction : 1.20627
hit initial : 220.928 | hit correction : 1.20627
hit initial : 129.845 | hit correction : 1.2063
hit initial : 123.39 | hit correction : 1.21277
hit initial : 290.04 | hit correction : 1.21277
hit initial : 157.275 | hit correction : 1.21275
hit initial : 615.946 | hit correction : 1.21091
hit initial : 523.13 | hit correction : 1.21072
hit initial : 402.074 | hit correction : 1.21082
hit initial : 389.91 | hit correction : 1.21027
hit initial : 257.347 | hit correction : 1.20998
hit initial : 258.062 | hit correction : 1.21068
hit initial : 121.762 | hit correction : 1.21076
hit initial : 263.502 | hit correction : 1.21069
 *** Total Charge : 4695.43
 Looking at view: 0 | Number of hits in view: 15
hit initial : 50.8677 | hit correction : 1.20984
hit initial : 115.192 | hit correction : 1.21278
hit initial : 254.833 | hit correction : 1.21276
hit initial : 163.935 | hit correction : 1.21276
hit initial : 22.2194 | hit correction : 1.21086
hit initial : 135.528 | hit correction : 1.21076
hit initial : 236.137 | hit correction : 1.21067
hit initial : 218.984 | hit correction : 1.21066
hit initial : 175.482 | hit correction : 1.21067
hit initial : 184.413 | hit correction : 1.21077
hit initial : 198.92 | hit correction : 1.21102
hit initial : 202.011 | hit correction : 1.21111
hit initial : 191.919 | hit correction : 1.21101
hit initial : 182.22 | hit correction : 1.21086
hit initial : 184.325 | hit correction : 1.21071
hit initial : 190.71 | hit correction : 1.21054
hit initial : 209.236 | hit correction : 1.21038
hit initial : 193.256 | hit correction : 1.21018
hit initial : 180.778 | hit correction : 1.20999
hit initial : 146.773 | hit correction : 1.20988
 *** Total Charge : 4162.93
 Looking at view: 1 | Number of hits in view: 20
Begin processing the 10th record. run: 5027 subRun: 0 event: 24300 at 06-Dec-2025 19:19:26 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.574
True Electron Energy: 10.3441
Completed MCParticle Loop
True Neutrino Energy: 15.7277
 Size of all hits: 0
06-Dec-2025 19:19:32 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.089529      0.239707      0.841645      0.181804      0.203392        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000555808   0.000785831   0.00149308    0.000722531   0.000253496      10     
end_path:ana:AnalyzeEvents         0.0885063     0.238696      0.839533      0.180981      0.203036        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 7662.62 MB
  Peak resident set size usage (VmHWM): 1536.97 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:10:33 UTC       justIN version: 01.06.00