justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267910.1@dunegpschedd01.fnal.gov

Jobsub ID267910.1@dunegpschedd01.fnal.gov
Workflow ID11168
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 08:12:24
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 20:33:40
From worker nodeHostnamedunegli-7890498-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 08:14:41
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T192147Z_gen_002182_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time12h (44320s)
CPU time11h (41603s = 93%)
Max RSS bytes2836676608 (2705 MiB)
Outputting started2025-12-13 20:33:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11168/1/001/analysisLowE_20251213T081445Z.root
Finished2025-12-13 20:33:40
Saved logsjustin-logs:267910.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

raModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 20:32:06 UTC run: 5026 subRun: 0 event: 21820
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 20:32:06 UTC run: 5026 subRun: 0 event: 21820
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6806, 6528, 6572
minwire 0: 34
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2287, 2786
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.118287, 
Output 1: 0.0978414, 0.0530366, 0.0998992, 0.749223, 
Output 2: 0.0402591, 0.171375, 0.766602, 0.0217632, 
Output 3: 0.970004, 0.00714601, 0.00179672, 0.0210533, 
Output 4: 0.483341, 0.47045, 0.0411429, 0.00506621, 
Output 5: 0.400522, 0.435837, 0.129689, 0.0339507, 
Output 6: 0.0142889, 0.0286875, 0.045482, 0.911542, 

13-Dec-2025 20:32:10 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 20:32:10 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/c2/5d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T192147Z_gen_002182_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2924.21       4387.12       5648.48       4467.87       880.914        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00122697     0.0040664     0.0213885    0.00208725    0.00583681       10     
reco:pandora:StandardPandora                            2921.37       4329.38       5581.72       4398.94       860.284        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000710427    0.0770675     0.198011      0.0549503     0.0770865       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000718993    0.0226778     0.0894374    0.00913808      0.03325        10     
reco:pandoracalo:Calorimetry                          0.000417586   0.00309785     0.0220988    0.00102929    0.00634658       10     
reco:pandorapid:Chi2ParticleID                        0.000109075   0.00093383     0.0076023    0.000188275    0.0022237       10     
reco:cvnmap:CVNMapper                                  0.0952164     0.203573      0.438125      0.156386      0.107218        10     
reco:cvneva:CVNEvaluator                                2.15175       3.74395       12.3937       2.34436       2.98102        10     
reco:energyrecnumu:EnergyReco                          0.0969634      52.987        71.3694       65.3752       26.5423        10     
reco:energyrecnue:EnergyReco                          0.00195969    0.00726178     0.0210605    0.00355567    0.00642335       10     
reco:energyrecnc:EnergyReco                            0.0109985     0.0156308     0.025343      0.0138323    0.00439621       10     
[art]:TriggerResults:TriggerResultInserter            2.4827e-05    7.48902e-05   0.000342034   4.5331e-05    9.0619e-05       10     
end_path:out1:RootOutput                              1.2042e-05    2.78553e-05   0.000105879   1.99325e-05   2.63361e-05      10     
end_path:out1:RootOutput(write)                        0.419692      0.668364       1.22431      0.611143      0.247988        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21570.8 MB
  Peak resident set size usage (VmHWM): 2836.68 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 208.614 | hit correction : 1.17748
hit initial : 320.987 | hit correction : 1.17735
hit initial : 408.069 | hit correction : 1.17728
hit initial : 315.537 | hit correction : 1.17726
hit initial : 241.134 | hit correction : 1.17717
hit initial : 311.118 | hit correction : 1.17709
hit initial : 141.064 | hit correction : 1.17656
hit initial : 503.271 | hit correction : 1.17686
hit initial : 294.716 | hit correction : 1.1769
hit initial : 192.384 | hit correction : 1.17687
hit initial : 175.206 | hit correction : 1.17677
hit initial : 169.558 | hit correction : 1.17664
hit initial : 181.347 | hit correction : 1.17649
hit initial : 174.914 | hit correction : 1.17637
 *** Total Charge : 7471.51
 Looking at view: 1 | Number of hits in view: 30
hit initial : 101.149 | hit correction : 1.17611
hit initial : 137.41 | hit correction : 1.18133
hit initial : 212.925 | hit correction : 1.18128
hit initial : 150.156 | hit correction : 1.18103
hit initial : 103.328 | hit correction : 1.18089
hit initial : 34.173 | hit correction : 1.1764
hit initial : 247.277 | hit correction : 1.17643
hit initial : 26.5994 | hit correction : 1.17652
hit initial : 37.1493 | hit correction : 1.17876
hit initial : 25.8556 | hit correction : 1.17873
hit initial : 1120.26 | hit correction : 1.17725
hit initial : 939.75 | hit correction : 1.17737
hit initial : 29.8078 | hit correction : 1.17792
hit initial : 28.3758 | hit correction : 1.1779
hit initial : 25.5633 | hit correction : 1.17756
hit initial : 665.548 | hit correction : 1.17747
hit initial : 367.759 | hit correction : 1.17728
hit initial : 287.834 | hit correction : 1.17714
hit initial : 400.18 | hit correction : 1.17708
hit initial : 556.618 | hit correction : 1.17687
hit initial : 306.194 | hit correction : 1.17685
hit initial : 210.512 | hit correction : 1.17657
hit initial : 248.811 | hit correction : 1.17669
hit initial : 240.18 | hit correction : 1.17648
 *** Total Charge : 7657.65
 Looking at view: 2 | Number of hits in view: 24
Begin processing the 9th record. run: 5026 subRun: 0 event: 21819 at 13-Dec-2025 20:33:12 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  4
Added hit list length: 4 | Max was set to: -3.40282e+38
True Electron Energy: 50.1617
Completed MCParticle Loop
True Neutrino Energy: 53.9209
 Size of all hits: 4
hit initial : 46.1417 | hit correction : 1.00646
 *** Total Charge : 46.4399
 Looking at view: 0 | Number of hits in view: 1
hit initial : 31.6164 | hit correction : 1.00647
hit initial : 31.3278 | hit correction : 1.00647
 *** Total Charge : 63.3515
 Looking at view: 1 | Number of hits in view: 2
hit initial : 77.9586 | hit correction : 1.00647
 *** Total Charge : 78.463
 Looking at view: 2 | Number of hits in view: 1
Begin processing the 10th record. run: 5026 subRun: 0 event: 21820 at 13-Dec-2025 20:33:16 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  4
This pfp has size:  5
Added hit list length: 5 | Max was set to: -3.40282e+38
True Neutron Energy: 939.575
True Electron Energy: 30.8521
Completed MCParticle Loop
True Neutrino Energy: 37.0786
 Size of all hits: 5
hit initial : 41.942 | hit correction : 1.0171
 *** Total Charge : 42.6592
 Looking at view: 0 | Number of hits in view: 1
hit initial : 33.8807 | hit correction : 1.01707
 *** Total Charge : 34.4589
 Looking at view: 1 | Number of hits in view: 1
hit initial : 36.7083 | hit correction : 1.01431
hit initial : 51.529 | hit correction : 1.0171
hit initial : 18.5573 | hit correction : 1.01704
 *** Total Charge : 108.518
 Looking at view: 2 | Number of hits in view: 3
13-Dec-2025 20:33:21 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0992718     0.217408       1.13442      0.116824      0.305871        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000601323   0.00834534     0.0750139    0.000723232    0.0222316       10     
end_path:ana:AnalyzeEvents         0.0984295     0.208749       1.05857      0.115903      0.283498        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19471.1 MB
  Peak resident set size usage (VmHWM): 1532.7 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:05:53 UTC       justIN version: 01.06.00