justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265772.1@dunegpschedd01.fnal.gov

Jobsub ID265772.1@dunegpschedd01.fnal.gov
Workflow ID11086
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-10 04:06:32
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-10 19:07:13
From worker nodeHostnamedunegli-8024417-0-fnpc22028.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 15:54:19
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T020911Z_gen_003658_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time3h (11539s)
CPU time3h (11366s = 98%)
Max RSS bytes3064287232 (2922 MiB)
Outputting started2025-12-10 19:06:38
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11086/1/001/analysisLowE_20251210T155420Z.root
Finished2025-12-10 19:07:13
Saved logsjustin-logs:265772.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tex/DlVertexingAlgorithm.cc line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6869, 6820, 6755
minwire 0: 20
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.102468, 
Output 1: 0.0649163, 0.0401815, 0.0544622, 0.84044, 
Output 2: 0.120388, 0.272678, 0.580501, 0.0264334, 
Output 3: 0.983171, 0.0128947, 0.000620127, 0.00331462, 
Output 4: 0.516721, 0.446203, 0.0334314, 0.00364442, 
Output 5: 0.495566, 0.450053, 0.0457409, 0.00863972, 
Output 6: 0.0150548, 0.0500648, 0.0718331, 0.863047, 

10-Dec-2025 19:05:40 UTC  Closed output file "matt_reco2.root"
10-Dec-2025 19:05:40 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/81/5c/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T020911Z_gen_003658_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1001.6        1139.05       1250.68       1126.42       73.0648        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00228724     0.0115861     0.0354965    0.00942109    0.00848419       10     
reco:pandora:StandardPandora                            1000.51       1137.46       1248.97       1124.95       73.2001        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00045569     0.0234239     0.129779     0.00744912     0.0377394       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000329511   0.00636322     0.0291816    0.00460811    0.00809379       10     
reco:pandoracalo:Calorimetry                          0.000154412    0.0140153     0.0582864    0.00148373     0.0225103       10     
reco:pandorapid:Chi2ParticleID                        6.2729e-05    0.000323309   0.00117061    0.000112783   0.000376236      10     
reco:cvnmap:CVNMapper                                  0.0248931     0.0495802     0.0877828     0.0449748     0.0201213       10     
reco:cvneva:CVNEvaluator                               0.751288       1.11675       3.13483      0.797444      0.692555        10     
reco:energyrecnumu:EnergyReco                         0.00166244     0.0471673     0.116117      0.0562234     0.036286        10     
reco:energyrecnue:EnergyReco                          0.000848141   0.00384819     0.0090553    0.00379145    0.00280968       10     
reco:energyrecnc:EnergyReco                           0.00501088     0.0316031     0.0849273    0.00742747     0.0319848       10     
[art]:TriggerResults:TriggerResultInserter            1.5008e-05    4.0869e-05    0.000205618   2.3094e-05    5.51648e-05      10     
end_path:out1:RootOutput                               9.588e-06    1.70381e-05   6.1947e-05    1.2223e-05    1.50406e-05      10     
end_path:out1:RootOutput(write)                        0.248629      0.286719      0.422758      0.256167      0.0625555       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21748.7 MB
  Peak resident set size usage (VmHWM): 3064.29 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Begin processing the 7th record. run: 5027 subRun: 0 event: 36577 at 10-Dec-2025 19:06:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  47
Added hit list length: 47 | Max was set to: -3.40282e+38
True Electron Energy: 43.4196
Completed MCParticle Loop
True Neutrino Energy: 49.6234
 Size of all hits: 47
hit initial : 65.3705 | hit correction : 1.01324
hit initial : 73.9174 | hit correction : 1.01323
hit initial : 87.8666 | hit correction : 1.01562
hit initial : 40.0053 | hit correction : 1.01661
hit initial : 275.336 | hit correction : 1.01556
hit initial : 64.8844 | hit correction : 1.0166
hit initial : 354.096 | hit correction : 1.01538
hit initial : 326.076 | hit correction : 1.01507
hit initial : 305.516 | hit correction : 1.01469
hit initial : 277.479 | hit correction : 1.01436
hit initial : 56.4408 | hit correction : 1.0111
hit initial : 224 | hit correction : 1.01413
hit initial : 114.675 | hit correction : 1.01335
hit initial : 262.29 | hit correction : 1.01106
hit initial : 168.521 | hit correction : 1.01386
hit initial : 164.823 | hit correction : 1.01317
hit initial : 33.2642 | hit correction : 1.01254
hit initial : 29.7468 | hit correction : 1.01175
hit initial : 200.841 | hit correction : 1.01093
hit initial : 67.2574 | hit correction : 1.01255
hit initial : 117.641 | hit correction : 1.01174
hit initial : 216.499 | hit correction : 1.01094
hit initial : 58.5648 | hit correction : 1.01109
hit initial : 91.0378 | hit correction : 1.01061
hit initial : 140.885 | hit correction : 1.01109
hit initial : 257.529 | hit correction : 1.01059
hit initial : 52.3706 | hit correction : 1.01057
 *** Total Charge : 4181.75
 Looking at view: 0 | Number of hits in view: 27
hit initial : 69.9754 | hit correction : 1.01659
hit initial : 45.1628 | hit correction : 1.0156
hit initial : 40.0188 | hit correction : 1.01658
hit initial : 435.268 | hit correction : 1.01438
hit initial : 102.329 | hit correction : 1.01421
hit initial : 16.1618 | hit correction : 1.01182
hit initial : 38.2235 | hit correction : 1.01554
hit initial : 104.758 | hit correction : 1.01436
hit initial : 62.8797 | hit correction : 1.0126
hit initial : 71.6154 | hit correction : 1.01177
hit initial : 35.3149 | hit correction : 1.0111
hit initial : 92.0721 | hit correction : 1.01061
hit initial : 37.1107 | hit correction : 1.0117
hit initial : 193.375 | hit correction : 1.01103
hit initial : 171.359 | hit correction : 1.01133
hit initial : 297.711 | hit correction : 1.01093
hit initial : 313.709 | hit correction : 1.01101
hit initial : 95.6598 | hit correction : 1.01512
hit initial : 255.234 | hit correction : 1.01544
hit initial : 203.064 | hit correction : 1.01552
 *** Total Charge : 2716.29
 Looking at view: 1 | Number of hits in view: 20
Begin processing the 8th record. run: 5027 subRun: 0 event: 36578 at 10-Dec-2025 19:06:28 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 8.57595
Completed MCParticle Loop
True Neutrino Energy: 11.8635
 Size of all hits: 0
Begin processing the 9th record. run: 5027 subRun: 0 event: 36579 at 10-Dec-2025 19:06:31 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 58.0083
Completed MCParticle Loop
True Neutrino Energy: 61.3509
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 36580 at 10-Dec-2025 19:06:34 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.58
True Electron Energy: 21.9455
Completed MCParticle Loop
True Neutrino Energy: 27.3268
 Size of all hits: 0
10-Dec-2025 19:06:38 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0755135     0.0991649     0.117224      0.102917      0.0125269       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000594432   0.000960174   0.00182308    0.000968769   0.000338552      10     
end_path:ana:AnalyzeEvents         0.0744506     0.0978561      0.11626      0.101805      0.0124539       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.9 MB
  Peak resident set size usage (VmHWM): 1534.71 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:28:46 UTC       justIN version: 01.06.00