justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266242.0@dunegpschedd01.fnal.gov

Jobsub ID266242.0@dunegpschedd01.fnal.gov
Workflow ID11111
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-10 19:01:15
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-11 11:32:12
From worker nodeHostnamedunegli-7867971-0-fnpc22033.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 09:00:41
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213104Z_gen_004098_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time2h (9081s)
CPU time2h (8957s = 98%)
Max RSS bytes2925666304 (2790 MiB)
Outputting started2025-12-11 11:32:02
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11111/1/001/analysisLowE_20251211T090042Z.root
Finished2025-12-11 11:32:12
Saved logsjustin-logs:266242.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

y: 
Output 0: 0.149607, 
Output 1: 0.0626601, 0.109829, 0.236251, 0.59126, 
Output 2: 0.0527337, 0.164325, 0.757064, 0.0258769, 
Output 3: 0.957173, 0.0320903, 0.00469321, 0.00604377, 
Output 4: 0.419355, 0.50765, 0.0652666, 0.00772841, 
Output 5: 0.335307, 0.489904, 0.146059, 0.0287298, 
Output 6: 0.0298006, 0.0817637, 0.0859742, 0.802462, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 40980 at 11-Dec-2025 11:20:19 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6875, 6774, 6700
minwire 0: 45
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.106849, 
Output 1: 0.0663752, 0.0660624, 0.153101, 0.714462, 
Output 2: 0.090682, 0.127197, 0.76282, 0.0193008, 
Output 3: 0.965312, 0.0306236, 0.00142467, 0.00263941, 
Output 4: 0.304706, 0.608019, 0.0760408, 0.0112342, 
Output 5: 0.488704, 0.389047, 0.0893772, 0.0328717, 
Output 6: 0.0705746, 0.160954, 0.123962, 0.644509, 

11-Dec-2025 11:31:30 UTC  Closed output file "matt_reco2.root"
11-Dec-2025 11:31:30 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/29/e3/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213104Z_gen_004098_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              658.136       893.936       1089.42       929.959       154.274        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00166559    0.00774075     0.0173989    0.00589333    0.00507363       10     
reco:pandora:StandardPandora                             657.1        892.609       1087.96       928.804       154.073        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000311627    0.0163988     0.0625986    0.00456854     0.0203376       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000236946   0.00240847    0.00952815    0.00102885    0.00292941       10     
reco:pandoracalo:Calorimetry                          0.000155342   0.00397679     0.0209667    0.000450599   0.00630059       10     
reco:pandorapid:Chi2ParticleID                        4.7589e-05    0.000163786   0.000908553   7.57375e-05   0.000250804      10     
reco:cvnmap:CVNMapper                                  0.0290104     0.0390174     0.0677954     0.035882      0.0108193       10     
reco:cvneva:CVNEvaluator                                0.69752      0.921767       2.0467        0.76835       0.38985        10     
reco:energyrecnumu:EnergyReco                         0.00252959     0.0505852     0.0817756     0.063734      0.0281604       10     
reco:energyrecnue:EnergyReco                          0.00080624    0.00332602    0.00770688    0.00314334    0.00234297       10     
reco:energyrecnc:EnergyReco                           0.00475239     0.0260902     0.105631     0.00658542     0.0334869       10     
[art]:TriggerResults:TriggerResultInserter            1.1181e-05    3.1787e-05    0.000134604   1.6977e-05    3.59359e-05      10     
end_path:out1:RootOutput                               4.128e-06    1.30746e-05   3.9565e-05    1.09305e-05   9.88075e-06      10     
end_path:out1:RootOutput(write)                        0.198263      0.254197      0.362183      0.219988      0.0642054       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21688.9 MB
  Peak resident set size usage (VmHWM): 2925.67 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 186.644 | hit correction : 1.15196
hit initial : 21.921 | hit correction : 1.15211
hit initial : 246.998 | hit correction : 1.15171
 *** Total Charge : 1543.71
 Looking at view: 0 | Number of hits in view: 6
hit initial : 118.803 | hit correction : 1.15207
hit initial : 142.508 | hit correction : 1.15161
hit initial : 32.4005 | hit correction : 1.152
hit initial : 63.9431 | hit correction : 1.1512
hit initial : 127.08 | hit correction : 1.15117
hit initial : 43.5925 | hit correction : 1.15116
hit initial : 305.539 | hit correction : 1.15172
hit initial : 467.58 | hit correction : 1.15172
 *** Total Charge : 1498.81
 Looking at view: 1 | Number of hits in view: 8
hit initial : 23.8136 | hit correction : 1.15207
hit initial : 166.767 | hit correction : 1.15195
hit initial : 142.512 | hit correction : 1.15156
hit initial : 128.711 | hit correction : 1.1515
hit initial : 119.234 | hit correction : 1.15154
hit initial : 249.66 | hit correction : 1.15185
hit initial : 147.968 | hit correction : 1.1521
hit initial : 92.2769 | hit correction : 1.15123
hit initial : 161.006 | hit correction : 1.15116
hit initial : 139.108 | hit correction : 1.15169
 *** Total Charge : 1578.99
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 8th record. run: 5026 subRun: 0 event: 40978 at 11-Dec-2025 11:31:56 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 1.50196
Completed MCParticle Loop
True Neutrino Energy: 5.2262
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 40979 at 11-Dec-2025 11:31:58 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  25
Added hit list length: 25 | Max was set to: -3.40282e+38
True Electron Energy: 26.4471
Completed MCParticle Loop
True Neutrino Energy: 33.5639
 Size of all hits: 25
hit initial : 100.055 | hit correction : 1.32291
hit initial : 402.403 | hit correction : 1.32286
hit initial : 586.36 | hit correction : 1.32272
hit initial : 233.431 | hit correction : 1.32255
 *** Total Charge : 1749
 Looking at view: 0 | Number of hits in view: 4
hit initial : 63.7417 | hit correction : 1.31877
hit initial : 110.033 | hit correction : 1.31878
hit initial : 93.3533 | hit correction : 1.32233
hit initial : 250.083 | hit correction : 1.32249
hit initial : 141.426 | hit correction : 1.32295
hit initial : 337.129 | hit correction : 1.32289
hit initial : 483.054 | hit correction : 1.32277
 *** Total Charge : 1955.4
 Looking at view: 1 | Number of hits in view: 7
hit initial : 53.3505 | hit correction : 1.32221
hit initial : 124.258 | hit correction : 1.32231
hit initial : 115.959 | hit correction : 1.32245
hit initial : 334.038 | hit correction : 1.32302
hit initial : 160.358 | hit correction : 1.32361
hit initial : 90.7984 | hit correction : 1.32477
hit initial : 187.701 | hit correction : 1.324
hit initial : 24.7119 | hit correction : 1.32285
hit initial : 335.653 | hit correction : 1.32454
hit initial : 30.8214 | hit correction : 1.32445
hit initial : 164.678 | hit correction : 1.32291
hit initial : 293.801 | hit correction : 1.32288
hit initial : 221.619 | hit correction : 1.3227
hit initial : 129.467 | hit correction : 1.32259
 *** Total Charge : 3000.17
 Looking at view: 2 | Number of hits in view: 14
Begin processing the 10th record. run: 5026 subRun: 0 event: 40980 at 11-Dec-2025 11:32:00 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.592
True Electron Energy: 4.07593
Completed MCParticle Loop
True Neutrino Energy: 10.2614
 Size of all hits: 0
11-Dec-2025 11:32:02 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0467597     0.0593788     0.0749413     0.0580251    0.00961474       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000319803   0.000498423   0.00103368    0.000457528   0.000189377      10     
end_path:ana:AnalyzeEvents         0.0462229     0.0586958     0.0733514     0.0574592    0.00943781       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19479.5 MB
  Peak resident set size usage (VmHWM): 1535.69 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:00:10 UTC       justIN version: 01.06.00