justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254694.0@dunegpschedd01.fnal.gov

Jobsub ID254694.0@dunegpschedd01.fnal.gov
Workflow ID10411
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-21 00:03:45
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-21 09:21:40
From worker nodeHostnamedunegli-7833571-0-fnpc22019.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 07:09:01
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T221256Z_gen_001603_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time2h (7946s)
CPU time2h (7893s = 99%)
Max RSS bytes3053490176 (2912 MiB)
Outputting started2025-11-21 09:21:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/10411/1/001/analysisLowE_20251121T070901Z.root
Finished2025-11-21 09:21:40
Saved logsjustin-logs:254694.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

T_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6674, 6571, 6553
minwire 0: 73
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2248, 2747
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0866261, 
Output 1: 0.0687529, 0.0723064, 0.164185, 0.694755, 
Output 2: 0.0501431, 0.116659, 0.819585, 0.0136129, 
Output 3: 0.974303, 0.018186, 0.00189036, 0.00562006, 
Output 4: 0.379382, 0.525293, 0.0810654, 0.0142597, 
Output 5: 0.623268, 0.312338, 0.0495604, 0.0148327, 
Output 6: 0.0343347, 0.0836729, 0.0794547, 0.802538, 

21-Nov-2025 09:20:58 UTC  Closed output file "matt_reco2.root"
21-Nov-2025 09:20:58 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/5a/8f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T221256Z_gen_001603_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              726.067       785.595       1031.59       759.426       84.3167        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00174715     0.0053708    0.00882744    0.00636928    0.00288299       10     
reco:pandora:StandardPandora                            725.09        784.349       1030.54       757.367       84.3767        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000428081    0.0324497     0.125303      0.0315722     0.034991        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000322721   0.00276344    0.00796697     0.0014822    0.00231864       10     
reco:pandoracalo:Calorimetry                          0.000170593   0.00125824    0.00768723    0.000292128   0.00219931       10     
reco:pandorapid:Chi2ParticleID                        5.2168e-05    0.00019296    0.000655812   9.72545e-05   0.000211827      10     
reco:cvnmap:CVNMapper                                  0.0238156     0.031409      0.0467483     0.0296763    0.00653196       10     
reco:cvneva:CVNEvaluator                               0.667127      0.904134       2.28358      0.684894      0.473737        10     
reco:energyrecnumu:EnergyReco                         0.00143797     0.033455      0.0920396     0.0244918     0.0313601       10     
reco:energyrecnue:EnergyReco                          0.000756273   0.00212681    0.00568176    0.000972522   0.00189944       10     
reco:energyrecnc:EnergyReco                           0.00445074     0.0294178     0.0633797     0.0197691     0.0258313       10     
[art]:TriggerResults:TriggerResultInserter            1.0109e-05    2.28313e-05   8.0322e-05     1.297e-05    2.2104e-05       10     
end_path:out1:RootOutput                               2.805e-06    9.0092e-06    2.8514e-05     7.68e-06     7.26916e-06      10     
end_path:out1:RootOutput(write)                        0.193758      0.203262      0.247392      0.196222      0.0163905       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21755.9 MB
  Peak resident set size usage (VmHWM): 3053.49 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Begin processing the 9th record. run: 3799 subRun: 0 event: 16029 at 21-Nov-2025 09:21:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  58
Added hit list length: 58 | Max was set to: -3.40282e+38
True Neutron Energy: 939.719
True Electron Energy: 45.1434
Completed MCParticle Loop
True Neutrino Energy: 54.1407
 Size of all hits: 58
hit initial : 132.055 | hit correction : 1.23227
hit initial : 99.5573 | hit correction : 1.23125
hit initial : 82.557 | hit correction : 1.23119
hit initial : 174.124 | hit correction : 1.23837
hit initial : 464.734 | hit correction : 1.23828
hit initial : 255.786 | hit correction : 1.23538
hit initial : 184.101 | hit correction : 1.23647
hit initial : 99.2054 | hit correction : 1.23789
hit initial : 39.8152 | hit correction : 1.23656
hit initial : 51.8697 | hit correction : 1.24111
hit initial : 446.449 | hit correction : 1.23806
hit initial : 265.796 | hit correction : 1.23633
hit initial : 362.552 | hit correction : 1.23541
hit initial : 52.7815 | hit correction : 1.23517
hit initial : 101.857 | hit correction : 1.2325
hit initial : 224.158 | hit correction : 1.23241
hit initial : 241.702 | hit correction : 1.23122
 *** Total Charge : 4051.91
 Looking at view: 0 | Number of hits in view: 17
hit initial : 57.7033 | hit correction : 1.23116
hit initial : 47.3915 | hit correction : 1.24113
hit initial : 34.6509 | hit correction : 1.24113
hit initial : 138.901 | hit correction : 1.23838
hit initial : 59.7517 | hit correction : 1.23841
hit initial : 200.275 | hit correction : 1.23827
hit initial : 214.019 | hit correction : 1.23834
hit initial : 402.41 | hit correction : 1.23807
hit initial : 218.008 | hit correction : 1.23785
hit initial : 30.2788 | hit correction : 1.23668
hit initial : 278.748 | hit correction : 1.23642
hit initial : 147.51 | hit correction : 1.23633
hit initial : 190.339 | hit correction : 1.23617
hit initial : 311.185 | hit correction : 1.2354
hit initial : 196.482 | hit correction : 1.2352
hit initial : 53.2454 | hit correction : 1.23254
hit initial : 199.784 | hit correction : 1.23248
hit initial : 347.002 | hit correction : 1.23199
hit initial : 244.637 | hit correction : 1.2313
 *** Total Charge : 4167.44
 Looking at view: 1 | Number of hits in view: 19
hit initial : 37.8096 | hit correction : 1.23816
hit initial : 23.3895 | hit correction : 1.24116
hit initial : 42.5746 | hit correction : 1.24114
hit initial : 86.0892 | hit correction : 1.23848
hit initial : 183.377 | hit correction : 1.23842
hit initial : 28.1011 | hit correction : 1.23842
hit initial : 181.886 | hit correction : 1.23838
hit initial : 135.321 | hit correction : 1.23824
hit initial : 127.184 | hit correction : 1.238
hit initial : 226.975 | hit correction : 1.23786
hit initial : 471.856 | hit correction : 1.23782
hit initial : 64.0382 | hit correction : 1.23662
hit initial : 213.037 | hit correction : 1.23652
hit initial : 683.563 | hit correction : 1.23618
hit initial : 230.131 | hit correction : 1.23548
hit initial : 347.427 | hit correction : 1.2352
hit initial : 67.9503 | hit correction : 1.23262
hit initial : 44.0803 | hit correction : 1.23247
hit initial : 296.071 | hit correction : 1.23242
hit initial : 233.797 | hit correction : 1.2318
hit initial : 252.737 | hit correction : 1.23109
hit initial : 38.0575 | hit correction : 1.23089
 *** Total Charge : 4962.53
 Looking at view: 2 | Number of hits in view: 22
Begin processing the 10th record. run: 3799 subRun: 0 event: 16030 at 21-Nov-2025 09:21:25 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.568
True Electron Energy: 3.83435
Completed MCParticle Loop
True Neutrino Energy: 11.6195
 Size of all hits: 0
21-Nov-2025 09:21:27 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0471684     0.0591226     0.0805844     0.0568484    0.00934466       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000445704   0.000606984   0.00157639    0.000513813   0.000325643      10     
end_path:ana:AnalyzeEvents         0.0465788     0.0583093     0.0799137     0.0553997    0.00935569       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.9 MB
  Peak resident set size usage (VmHWM): 1536.91 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-09 03:28:28 UTC       justIN version: 01.06.00