Jobsub ID 42010.1@dunegpschedd01.fnal.gov

Jobsub ID	42010.1@dunegpschedd01.fnal.gov
Workflow ID	2246
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-09-07 01:35:24
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-09-07 04:58:42
From worker node	Hostname	dunegli-6756886-0-fnpc22024.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-07 01:36:15
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250126T165231Z_gen_001098_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	3h (12114s)
	CPU time	3h (11968s = 98%)
	Max RSS bytes	3053539328 (2912 MiB)
Outputting started	2025-09-07 04:58:09
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/02246/1/001/mydetsimreco1reco2_20250907T013636Z.root
Finished	2025-09-07 04:58:42
Saved logs	justin-logs:42010.1-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

2025 03:11:01 UTC
07-Sep-2025 03:11:05 UTC  Opened output file with pattern "matt_reco1.root"
Begin processing the 2nd record. run: 4923 subRun: 0 event: 10972 at 07-Sep-2025 03:11:15 UTC
Begin processing the 3rd record. run: 4923 subRun: 0 event: 10973 at 07-Sep-2025 03:11:19 UTC
Begin processing the 4th record. run: 4923 subRun: 0 event: 10974 at 07-Sep-2025 03:11:23 UTC
Begin processing the 5th record. run: 4923 subRun: 0 event: 10975 at 07-Sep-2025 03:11:26 UTC
Begin processing the 6th record. run: 4923 subRun: 0 event: 10976 at 07-Sep-2025 03:11:30 UTC
Begin processing the 7th record. run: 4923 subRun: 0 event: 10977 at 07-Sep-2025 03:11:34 UTC
Begin processing the 8th record. run: 4923 subRun: 0 event: 10978 at 07-Sep-2025 03:11:37 UTC
Begin processing the 9th record. run: 4923 subRun: 0 event: 10979 at 07-Sep-2025 03:11:41 UTC
Begin processing the 10th record. run: 4923 subRun: 0 event: 10980 at 07-Sep-2025 03:11:44 UTC
07-Sep-2025 03:11:49 UTC  Closed output file "matt_reco1.root"
07-Sep-2025 03:11:49 UTC  Closed input file "matt_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      1.73696       1.79071       2.00161       1.77209      0.0714275       10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000402621   0.000442378   0.00053972    0.000417039   4.40894e-05      10     
reco:gaushit:GausHitFinder                     0.194489      0.201831      0.212534      0.200261     0.00534353       10     
reco:ophit10ppm:OpHitFinder                     1.45528       1.50367       1.70136       1.48764      0.0667528       10     
[art]:TriggerResults:TriggerResultInserter    1.5499e-05    2.36648e-05   7.8769e-05    1.8069e-05    1.84014e-05      10     
end_path:out1:RootOutput                       4.91e-06     9.4337e-06    4.6728e-05     5.19e-06     1.24365e-05      10     
end_path:out1:RootOutput(write)                0.0820995     0.0843696     0.0874931     0.0837006    0.00180206       10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20204.3 MB
  Peak resident set size usage (VmHWM): 2212.05 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 738, 1237
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.104062, 
Output 1: 0.048616, 0.0674317, 0.128977, 0.754975, 
Output 2: 0.0379198, 0.204907, 0.73491, 0.022263, 
Output 3: 0.96052, 0.0245067, 0.00442763, 0.0105456, 
Output 4: 0.242183, 0.64554, 0.104086, 0.0081911, 
Output 5: 0.67891, 0.249159, 0.0574533, 0.0144782, 
Output 6: 0.0134761, 0.0418801, 0.0606443, 0.884, 

Begin processing the 8th record. run: 4923 subRun: 0 event: 10978 at 07-Sep-2025 04:25:18 UTC
Boundary wire vector sizes: 6726, 6630, 6578
minwire 0: 3
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1825, 2324
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.115466, 
Output 1: 0.0538909, 0.0611499, 0.122425, 0.762534, 
Output 2: 0.0607897, 0.219596, 0.695911, 0.0237037, 
Output 3: 0.97093, 0.0231717, 0.00231359, 0.0035848, 
Output 4: 0.284072, 0.625496, 0.0837931, 0.00663921, 
Output 5: 0.47879, 0.43412, 0.0746255, 0.0124646, 
Output 6: 0.0340984, 0.109622, 0.106652, 0.749627, 

Begin processing the 9th record. run: 4923 subRun: 0 event: 10979 at 07-Sep-2025 04:36:07 UTC
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  07-Sep-2025 04:47:02 UTC run: 4923 subRun: 0 event: 10979
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6615, 6652, 6518
minwire 0: 28
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1821, 2320
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.123026, 
Output 1: 0.485539, 0.0765847, 0.172422, 0.265454, 
Output 2: 0.0364196, 0.0764043, 0.878573, 0.00860293, 
Output 3: 0.973835, 0.0236769, 0.000746361, 0.00174192, 
Output 4: 0.539667, 0.425903, 0.0292242, 0.00520619, 
Output 5: 0.211779, 0.398794, 0.251391, 0.138035, 
Output 6: 0.172064, 0.274944, 0.142016, 0.410977, 

Begin processing the 10th record. run: 4923 subRun: 0 event: 10980 at 07-Sep-2025 04:47:03 UTC
Boundary wire vector sizes: 6806, 6791, 6719
minwire 0: 13
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 650, 1149
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.129004, 
Output 1: 0.0512228, 0.233683, 0.189319, 0.525775, 
Output 2: 0.0480123, 0.295469, 0.626636, 0.0298827, 
Output 3: 0.952093, 0.0358177, 0.00558744, 0.00650164, 
Output 4: 0.340436, 0.571279, 0.0827848, 0.0054997, 
Output 5: 0.694113, 0.258145, 0.0401579, 0.00758456, 
Output 6: 0.0195751, 0.0597444, 0.0747363, 0.845944, 

07-Sep-2025 04:58:08 UTC  Closed output file "mydetsimreco1reco2_20250907T013636Z.root"
07-Sep-2025 04:58:08 UTC  Closed input file "matt_reco1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              611.811       632.474       662.738       633.806       17.7702        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00044933    0.00356206    0.00504392    0.00419586    0.00143328       10     
reco:pandora:StandardPandora                            610.493       631.366       661.738       632.822       17.8886        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.0004817     0.0118387     0.0507847    0.00379894     0.0147705       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000773021   0.00352318     0.0133682     0.0019226    0.00367752       10     
reco:pandoracalo:Calorimetry                          0.000177185   0.00160436     0.0118537    0.000247908   0.00347359       10     
reco:pandorapid:Chi2ParticleID                         5.282e-05    0.000152119   0.000916422   6.81535e-05   0.000254894      10     
reco:cvnmap:CVNMapper                                  0.0175627     0.0217628     0.0431477     0.0185822    0.00744326       10     
reco:cvneva:CVNEvaluator                               0.670462      0.794846       1.73573      0.700561      0.313929        10     
reco:energyrecnumu:EnergyReco                         0.00245879     0.0195219     0.0733767    0.00578658     0.0219398       10     
reco:energyrecnue:EnergyReco                          0.00084145     0.0018204    0.00518482    0.000913025   0.00166891       10     
reco:energyrecnc:EnergyReco                           0.00499123     0.0213815     0.033087      0.0313125     0.0133141       10     
[art]:TriggerResults:TriggerResultInserter            1.0851e-05    1.77495e-05   5.8801e-05    1.33355e-05   1.37618e-05      10     
end_path:out1:RootOutput                               8.937e-06    1.26969e-05   3.4215e-05    1.03045e-05   7.22065e-06      10     
end_path:out1:RootOutput(write)                        0.214204      0.227578      0.256542      0.227665      0.0112309       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21760.1 MB
  Peak resident set size usage (VmHWM): 3053.54 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0