justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 260271.0@dunegpschedd02.fnal.gov

Jobsub ID260271.0@dunegpschedd02.fnal.gov
Workflow ID11090
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-10 14:07:00
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-11 08:56:34
From worker nodeHostnamedunegli-7872446-0-fnpc23026.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 05:45:38
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T021115Z_gen_001171_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time3h (11404s)
CPU time3h (10929s = 95%)
Max RSS bytes3063738368 (2921 MiB)
Outputting started2025-12-11 08:55:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11090/1/001/analysisLowE_20251211T054540Z.root
Finished2025-12-11 08:56:34
Saved logsjustin-logs:260271.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gorithm.cc line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6601, 6551, 6686
minwire 0: 68
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1919, 2418
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0801341, 
Output 1: 0.0693385, 0.12913, 0.137159, 0.664373, 
Output 2: 0.0767479, 0.318285, 0.578812, 0.0261547, 
Output 3: 0.946449, 0.0446381, 0.00415585, 0.0047572, 
Output 4: 0.444467, 0.495636, 0.0555534, 0.00434385, 
Output 5: 0.740365, 0.242286, 0.0151732, 0.00217652, 
Output 6: 0.0197677, 0.0729463, 0.0813173, 0.825969, 

11-Dec-2025 08:54:47 UTC  Closed output file "matt_reco2.root"
11-Dec-2025 08:54:47 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/23/38/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T021115Z_gen_001171_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              747.584       1122.8        1797.27       820.34        444.103        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00364309    0.00852034     0.0182259    0.00796041    0.00383498       10     
reco:pandora:StandardPandora                            746.41        1121.17       1795.9        819.036       443.549        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00274122     0.0246549     0.185434     0.00429357     0.054042        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000672546   0.00643262     0.0310047    0.00153472    0.00919623       10     
reco:pandoracalo:Calorimetry                          0.000243936    0.0298307     0.100058      0.0252783     0.0317758       10     
reco:pandorapid:Chi2ParticleID                         6.289e-05    0.000253543    0.0016541    9.6565e-05    0.000467472      10     
reco:cvnmap:CVNMapper                                  0.0253907     0.0483601     0.0774236     0.0411021     0.0185321       10     
reco:cvneva:CVNEvaluator                                0.74613       1.09051       3.20708      0.853242       0.71237        10     
reco:energyrecnumu:EnergyReco                         0.00281301     0.061585      0.111552      0.0618943     0.0266125       10     
reco:energyrecnue:EnergyReco                          0.000844605   0.00457416    0.00679938    0.00549131    0.00222314       10     
reco:energyrecnc:EnergyReco                           0.00514927     0.0266375     0.142328     0.00589842     0.0425735       10     
[art]:TriggerResults:TriggerResultInserter            1.3125e-05    5.31689e-05   0.000344068   1.95175e-05   9.72411e-05      10     
end_path:out1:RootOutput                               9.127e-06    2.17395e-05   9.8849e-05    1.24385e-05   2.58737e-05      10     
end_path:out1:RootOutput(write)                        0.234636       0.33109      0.657571      0.246071      0.160356        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21782.2 MB
  Peak resident set size usage (VmHWM): 3063.74 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 Size of all hits: 0
Begin processing the 6th record. run: 5027 subRun: 0 event: 11706 at 11-Dec-2025 08:55:28 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.571
True Electron Energy: 5.94188
Completed MCParticle Loop
True Neutrino Energy: 12.1265
 Size of all hits: 0
Begin processing the 7th record. run: 5027 subRun: 0 event: 11707 at 11-Dec-2025 08:55:31 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 2.80924
Completed MCParticle Loop
True Neutrino Energy: 6.09313
 Size of all hits: 0
Begin processing the 8th record. run: 5027 subRun: 0 event: 11708 at 11-Dec-2025 08:55:33 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  36
Added hit list length: 36 | Max was set to: -3.40282e+38
True Electron Energy: 25.3458
Completed MCParticle Loop
True Neutrino Energy: 29.0998
 Size of all hits: 36
hit initial : 25.6698 | hit correction : 1.05723
hit initial : 54.2349 | hit correction : 1.05718
hit initial : 52.1026 | hit correction : 1.0621
hit initial : 159.091 | hit correction : 1.06208
hit initial : 121.685 | hit correction : 1.06049
hit initial : 52.9939 | hit correction : 1.06002
hit initial : 117.018 | hit correction : 1.06199
hit initial : 625.958 | hit correction : 1.06058
hit initial : 190.497 | hit correction : 1.06065
hit initial : 261.224 | hit correction : 1.05995
hit initial : 462.749 | hit correction : 1.05978
hit initial : 196.268 | hit correction : 1.05971
 *** Total Charge : 2459.48
 Looking at view: 0 | Number of hits in view: 12
hit initial : 82.4005 | hit correction : 1.06124
hit initial : 83.233 | hit correction : 1.06207
hit initial : 187.264 | hit correction : 1.06136
hit initial : 117.467 | hit correction : 1.06076
hit initial : 155.482 | hit correction : 1.06209
hit initial : 183.367 | hit correction : 1.06063
hit initial : 81.9258 | hit correction : 1.06012
hit initial : 60.1619 | hit correction : 1.06211
hit initial : 679.304 | hit correction : 1.06009
hit initial : 42.4359 | hit correction : 1.06076
hit initial : 28.7647 | hit correction : 1.06076
hit initial : 663.717 | hit correction : 1.05984
hit initial : 221.818 | hit correction : 1.05972
 *** Total Charge : 2743.73
 Looking at view: 1 | Number of hits in view: 13
hit initial : 58.5207 | hit correction : 1.06212
hit initial : 279.821 | hit correction : 1.06161
hit initial : 213.94 | hit correction : 1.06211
hit initial : 333.935 | hit correction : 1.06098
hit initial : 231.806 | hit correction : 1.06062
hit initial : 364.235 | hit correction : 1.06033
hit initial : 225.51 | hit correction : 1.06004
hit initial : 441.155 | hit correction : 1.05962
hit initial : 227.35 | hit correction : 1.05993
hit initial : 133.027 | hit correction : 1.05986
hit initial : 117.402 | hit correction : 1.05969
 *** Total Charge : 2785.69
 Looking at view: 2 | Number of hits in view: 11
Begin processing the 9th record. run: 5027 subRun: 0 event: 11709 at 11-Dec-2025 08:55:36 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 6.3243
Completed MCParticle Loop
True Neutrino Energy: 12.0851
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 11710 at 11-Dec-2025 08:55:39 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.568
True Electron Energy: 2.75973
Completed MCParticle Loop
True Neutrino Energy: 6.48444
 Size of all hits: 0
11-Dec-2025 08:55:42 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0561407     0.084099      0.127844      0.0767131     0.0191811       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000362694    0.0036642     0.016983     0.00060143    0.00605405       10     
end_path:ana:AnalyzeEvents         0.0556478     0.0802248     0.127103      0.0753333     0.0180287       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.5 MB
  Peak resident set size usage (VmHWM): 1534.59 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:29:02 UTC       justIN version: 01.06.00