justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266043.17@dunegpschedd01.fnal.gov

Jobsub ID266043.17@dunegpschedd01.fnal.gov
Workflow ID11087
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-10 13:59:01
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-11 17:31:46
From worker nodeHostnamedunegli-7867869-0-fnpc9076.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 08:39:40
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T180136Z_gen_001294_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (31907s)
CPU time8h (31528s = 98%)
Max RSS bytes3041316864 (2900 MiB)
Outputting started2025-12-11 17:31:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11087/1/001/analysisLowE_20251211T083941Z.root
Finished2025-12-11 17:31:46
Saved logsjustin-logs:266043.17-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ailure in algorithm Alg0004, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
PandoraContentApi::GetList(*this, m_inputVertexListName, pVertexList) throw STATUS_CODE_NOT_INITIALIZED
    in function: GetHitRegion
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 624
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6754, 6696, 6659
minwire 0: 21
minwire 1: 1
minwire 2: 2
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.106768, 
Output 1: 0.0725667, 0.0501177, 0.132791, 0.744525, 
Output 2: 0.0839913, 0.20871, 0.679465, 0.0278336, 
Output 3: 0.968374, 0.026033, 0.00190055, 0.00369255, 
Output 4: 0.283811, 0.654104, 0.0578723, 0.00421302, 
Output 5: 0.621807, 0.316454, 0.0493792, 0.0123597, 
Output 6: 0.0261881, 0.0775624, 0.0841749, 0.812075, 

11-Dec-2025 17:30:29 UTC  Closed output file "matt_reco2.root"
11-Dec-2025 17:30:29 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/76/38/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T180136Z_gen_001294_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2184.2        3153.78       4026.83       3085.29       480.504        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00745118     0.0163459     0.0328908     0.0157117    0.00642498       10     
reco:pandora:StandardPandora                            2182.37       3144.4        3956.88       3083.34       468.314        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00824826     0.074619      0.303219      0.0465172     0.0832023       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00149385     0.0146857     0.0538534    0.00684619     0.0188343       10     
reco:pandoracalo:Calorimetry                          0.000227631   0.00563731     0.0385619    0.000989801    0.0113416       10     
reco:pandorapid:Chi2ParticleID                        6.9657e-05    0.000576406   0.00415175    0.000193153   0.00119315       10     
reco:cvnmap:CVNMapper                                  0.0572187     0.0941323     0.144553      0.0962111     0.0244761       10     
reco:cvneva:CVNEvaluator                                1.15491       2.33464       7.19523       1.99766       1.6623         10     
reco:energyrecnumu:EnergyReco                         0.00750319     0.054746      0.135039      0.0217766     0.0490005       10     
reco:energyrecnue:EnergyReco                          0.00183105    0.00404105     0.0138671    0.00325253    0.00332695       10     
reco:energyrecnc:EnergyReco                           0.00989126      6.21342       61.3592       0.1215        18.382         10     
[art]:TriggerResults:TriggerResultInserter            2.3776e-05    5.92356e-05   0.000274468   3.6967e-05    7.21188e-05      10     
end_path:out1:RootOutput                               8.83e-06     2.32286e-05   0.000114824   1.37535e-05   3.06269e-05      10     
end_path:out1:RootOutput(write)                        0.388222      0.561584       0.75046      0.596001      0.113961        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12120.9 MB
  Peak resident set size usage (VmHWM): 3041.32 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 182.925 | hit correction : 1.06807
hit initial : 197.06 | hit correction : 1.06835
hit initial : 328.888 | hit correction : 1.06873
hit initial : 244.895 | hit correction : 1.06876
hit initial : 128.758 | hit correction : 1.06878
hit initial : 124.769 | hit correction : 1.06882
hit initial : 138.046 | hit correction : 1.06885
hit initial : 197.075 | hit correction : 1.06885
hit initial : 234.013 | hit correction : 1.06884
hit initial : 235.298 | hit correction : 1.06891
hit initial : 390.599 | hit correction : 1.06896
hit initial : 181.32 | hit correction : 1.06899
hit initial : 128.159 | hit correction : 1.06901
hit initial : 136.624 | hit correction : 1.06903
hit initial : 136.435 | hit correction : 1.06904
hit initial : 145.997 | hit correction : 1.06905
hit initial : 145.951 | hit correction : 1.06909
hit initial : 150.466 | hit correction : 1.06913
hit initial : 61.8191 | hit correction : 1.06917
hit initial : 46.1126 | hit correction : 1.06956
 *** Total Charge : 4919.52
 Looking at view: 2 | Number of hits in view: 26
Begin processing the 8th record. run: 5026 subRun: 0 event: 12938 at 11-Dec-2025 17:31:16 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  30
Added hit list length: 30 | Max was set to: -3.40282e+38
True Electron Energy: 19.711
Completed MCParticle Loop
True Neutrino Energy: 23.45
 Size of all hits: 30
hit initial : 23.9114 | hit correction : 1.04236
hit initial : 27.7776 | hit correction : 1.04546
hit initial : 73.2076 | hit correction : 1.04549
hit initial : 26.7296 | hit correction : 1.04551
hit initial : 97.3133 | hit correction : 1.04374
hit initial : 375.121 | hit correction : 1.04213
hit initial : 198.002 | hit correction : 1.04226
hit initial : 376.276 | hit correction : 1.04211
hit initial : 341.869 | hit correction : 1.04247
hit initial : 419.103 | hit correction : 1.04304
hit initial : 373.546 | hit correction : 1.04357
 *** Total Charge : 2432.78
 Looking at view: 0 | Number of hits in view: 11
hit initial : 49.749 | hit correction : 1.0439
hit initial : 62.412 | hit correction : 1.04548
hit initial : 211.016 | hit correction : 1.04378
hit initial : 39.9866 | hit correction : 1.04549
hit initial : 110.672 | hit correction : 1.04231
hit initial : 225.538 | hit correction : 1.04219
hit initial : 255.446 | hit correction : 1.04205
hit initial : 259.079 | hit correction : 1.04208
hit initial : 308.319 | hit correction : 1.04246
hit initial : 343.742 | hit correction : 1.04291
hit initial : 338.512 | hit correction : 1.04338
 *** Total Charge : 2298.92
 Looking at view: 1 | Number of hits in view: 11
hit initial : 61.1952 | hit correction : 1.04548
hit initial : 205.169 | hit correction : 1.0439
hit initial : 46.0489 | hit correction : 1.04549
hit initial : 337.529 | hit correction : 1.04201
hit initial : 391.526 | hit correction : 1.04218
hit initial : 326.259 | hit correction : 1.04228
hit initial : 165.514 | hit correction : 1.04266
hit initial : 749.503 | hit correction : 1.04315
 *** Total Charge : 2380.51
 Looking at view: 2 | Number of hits in view: 8
Begin processing the 9th record. run: 5026 subRun: 0 event: 12939 at 11-Dec-2025 17:31:20 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 9.48324
Completed MCParticle Loop
True Neutrino Energy: 24.9576
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 12940 at 11-Dec-2025 17:31:23 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.567
True Electron Energy: 1.0785
Completed MCParticle Loop
True Neutrino Energy: 4.80263
 Size of all hits: 0
11-Dec-2025 17:31:27 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0753531     0.103491      0.164583      0.0912686     0.0277547       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000444826   0.000616189   0.00113621    0.000570309   0.000188132      10     
end_path:ana:AnalyzeEvents         0.0746389     0.102676      0.163714      0.0905774     0.0275866       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9811.75 MB
  Peak resident set size usage (VmHWM): 1538.03 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:30:08 UTC       justIN version: 01.06.00