Jobsub ID 261944.1@dunegpschedd02.fnal.gov

Jobsub ID	261944.1@dunegpschedd02.fnal.gov
Workflow ID	11171
Stage ID	1
User name	osbiston@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-12-13 05:26:16
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-12-13 17:04:17
From worker node	Hostname	dunegli-7890610-0-fnpc9038.fnal.gov
	cpuinfo	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-12-13 05:27:16
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T014338Z_gen_000863_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	11h (41783s)
	CPU time	11h (41323s = 98%)
	Max RSS bytes	2805362688 (2675 MiB)
Outputting started	2025-12-13 17:03:39
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11171/1/001/analysisLowE_20251213T052717Z.root
Finished	2025-12-13 17:04:17
Saved logs	justin-logs:261944.1-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

event: 8630
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 17:01:35 UTC run: 5027 subRun: 0 event: 8630
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6757, 6693, 6795
minwire 0: 12
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.116091, 
Output 1: 0.0639848, 0.0715511, 0.211271, 0.653193, 
Output 2: 0.0235965, 0.0869479, 0.877219, 0.0122361, 
Output 3: 0.965104, 0.0242323, 0.00444634, 0.00621726, 
Output 4: 0.523444, 0.368104, 0.0880909, 0.0203616, 
Output 5: 0.242542, 0.447593, 0.253719, 0.0561466, 
Output 6: 0.0560044, 0.104491, 0.0875584, 0.751946, 

13-Dec-2025 17:01:40 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 17:01:40 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/67/05/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T014338Z_gen_000863_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3737.18       4140.03       4369.16       4186.77       197.524        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00195579    0.00640252     0.0194958    0.00282342     0.0062251       10     
reco:pandora:StandardPandora                            3733.43       4135.95       4366.44       4183.25       198.916        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000718227    0.148092      0.193832      0.163373       0.0558         10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000696211    0.016873      0.0827647    0.00942977     0.0228461       10     
reco:pandoracalo:Calorimetry                          0.000490767   0.00377534     0.0152688    0.00148843    0.00441993       10     
reco:pandorapid:Chi2ParticleID                        0.000146122   0.000872476   0.00598853    0.000327854   0.00170678       10     
reco:cvnmap:CVNMapper                                  0.124629      0.153097      0.185169       0.15495      0.0176767       10     
reco:cvneva:CVNEvaluator                                1.57506       2.89341       9.04779       2.37131       2.07034        10     
reco:energyrecnumu:EnergyReco                          0.125748      0.161044      0.248436      0.154493      0.0312857       10     
reco:energyrecnue:EnergyReco                          0.00361882    0.00594897     0.0156505    0.00425612    0.00346345       10     
reco:energyrecnc:EnergyReco                            0.0145781     0.0166424     0.0232456     0.0156534    0.00259142       10     
[art]:TriggerResults:TriggerResultInserter            3.8966e-05    6.24767e-05   0.000172083   5.13105e-05   3.70479e-05      10     
end_path:out1:RootOutput                              1.2935e-05    2.08096e-05   4.8488e-05    1.57555e-05   1.10558e-05      10     
end_path:out1:RootOutput(write)                        0.446685      0.675878       0.98483      0.662739      0.128746        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11911.3 MB
  Peak resident set size usage (VmHWM): 2805.36 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 143.951 | hit correction : 1.29023
hit initial : 66.8622 | hit correction : 1.29015
hit initial : 206.194 | hit correction : 1.28987
hit initial : 134.287 | hit correction : 1.28906
hit initial : 76.7944 | hit correction : 1.289
hit initial : 35.2952 | hit correction : 1.2889
 *** Total Charge : 6710.29
 Looking at view: 0 | Number of hits in view: 31
hit initial : 39.0757 | hit correction : 1.33174
hit initial : 155.283 | hit correction : 1.33165
hit initial : 182.768 | hit correction : 1.33151
hit initial : 61.4439 | hit correction : 1.33143
hit initial : 16.0535 | hit correction : 1.31006
hit initial : 43.2287 | hit correction : 1.31006
hit initial : 20.3147 | hit correction : 1.31007
hit initial : 92.5389 | hit correction : 1.30421
hit initial : 322.846 | hit correction : 1.30419
hit initial : 341.374 | hit correction : 1.30413
hit initial : 210.062 | hit correction : 1.30385
hit initial : 190.737 | hit correction : 1.30334
hit initial : 198.182 | hit correction : 1.3029
hit initial : 196.997 | hit correction : 1.30264
hit initial : 221.119 | hit correction : 1.30238
hit initial : 365.521 | hit correction : 1.30196
hit initial : 459.972 | hit correction : 1.30177
hit initial : 73.6536 | hit correction : 1.30374
hit initial : 358.313 | hit correction : 1.30161
hit initial : 111.773 | hit correction : 1.30372
hit initial : 143.169 | hit correction : 1.3015
hit initial : 46.8304 | hit correction : 1.30368
hit initial : 30.0144 | hit correction : 1.30085
hit initial : 70.305 | hit correction : 1.30085
hit initial : 30.2393 | hit correction : 1.30085
hit initial : 76.5961 | hit correction : 1.29185
hit initial : 291.881 | hit correction : 1.29182
hit initial : 332.014 | hit correction : 1.29175
hit initial : 247.013 | hit correction : 1.29141
hit initial : 267.701 | hit correction : 1.29076
hit initial : 291.861 | hit correction : 1.29029
hit initial : 257.77 | hit correction : 1.28993
hit initial : 279.164 | hit correction : 1.28945
hit initial : 273.819 | hit correction : 1.28908
hit initial : 121.258 | hit correction : 1.28901
 *** Total Charge : 8348.28
 Looking at view: 1 | Number of hits in view: 35
hit initial : 36.8361 | hit correction : 1.33194
hit initial : 196.891 | hit correction : 1.33164
hit initial : 183.029 | hit correction : 1.33144
hit initial : 25.0718 | hit correction : 1.31004
hit initial : 61.9695 | hit correction : 1.31006
hit initial : 276.072 | hit correction : 1.30419
hit initial : 175.48 | hit correction : 1.30398
hit initial : 342.556 | hit correction : 1.30415
hit initial : 362.83 | hit correction : 1.3037
hit initial : 176.922 | hit correction : 1.303
hit initial : 165.356 | hit correction : 1.30272
hit initial : 146.318 | hit correction : 1.30259
hit initial : 152.568 | hit correction : 1.30241
hit initial : 358.084 | hit correction : 1.30199
hit initial : 547.643 | hit correction : 1.30177
hit initial : 50.4559 | hit correction : 1.30377
hit initial : 340.952 | hit correction : 1.30159
hit initial : 131.471 | hit correction : 1.30372
hit initial : 131.35 | hit correction : 1.30145
hit initial : 40.4533 | hit correction : 1.30363
hit initial : 43.0556 | hit correction : 1.30087
hit initial : 82.497 | hit correction : 1.30084
hit initial : 249.921 | hit correction : 1.29186
hit initial : 96.4585 | hit correction : 1.29179
hit initial : 205.382 | hit correction : 1.29187
hit initial : 122.593 | hit correction : 1.29171
hit initial : 165.358 | hit correction : 1.29151
hit initial : 185.674 | hit correction : 1.29111
hit initial : 182.378 | hit correction : 1.29067
hit initial : 216.775 | hit correction : 1.29031
hit initial : 199.117 | hit correction : 1.29005
hit initial : 168.845 | hit correction : 1.28976
hit initial : 163.089 | hit correction : 1.28949
hit initial : 235.457 | hit correction : 1.28907
hit initial : 202.402 | hit correction : 1.28896
 *** Total Charge : 8348.83
 Looking at view: 2 | Number of hits in view: 35
13-Dec-2025 17:03:38 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.121578      0.204189      0.408028       0.18387      0.0747128       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000861751   0.00579213     0.0263574    0.00121071    0.00931056       10     
end_path:ana:AnalyzeEvents         0.120198      0.197987      0.384477      0.182286      0.0680851       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9811.75 MB
  Peak resident set size usage (VmHWM): 1540 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0