Jobsub ID 268617.8@dunegpschedd02.fnal.gov

Jobsub ID	268617.8@dunegpschedd02.fnal.gov
Workflow ID	11715
Stage ID	1
User name	jierans@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-11 08:20:36
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2026-01-11 09:09:53
From worker node	Hostname	dunegli-8142297-0-fnpc23027.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-11 09:06:49
Input files	fardet-vd:fardet-vd_mc_le_mc_2024a_workflow0000003785_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000350_lim000025_mergeLar_20250204T141136_20250610T120555Z_reco2.root
Jobscript	Exit code	0
	Real time	2m (174s)
	CPU time	1m (86s = 49%)
	Max RSS bytes	615268352 (586 MiB)
Outputting started	2026-01-11 09:09:44
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/jierans/vd_1x8x14_nuecc_with_pds/ntuple/fnal/11715/1/001/triggerAna_11715_2026-01-11T_090700Z.ntuple.root
Finished	2026-01-11 09:09:53
Saved logs	justin-logs:268617.8-dunegpschedd02.fnal.gov.logs.tgz
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

veMakerPDS: Generated 161 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 157 failed to backtrack. 0%.
Begin processing the 2477th record. run: 3785 subRun: 0 event: 49377 at 11-Jan-2026 09:09:42 UTC
TriggerPrimitiveMakerPDS: found 170 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 156 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 156 failed to backtrack. 1.28205%.
Begin processing the 2478th record. run: 3785 subRun: 0 event: 49378 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 127 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 125 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 124 failed to backtrack. 0.806452%.
Begin processing the 2479th record. run: 3785 subRun: 0 event: 49379 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 147 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 120 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 120 failed to backtrack. 3.33333%.
Begin processing the 2480th record. run: 3785 subRun: 0 event: 49380 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 116 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 81 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 78 failed to backtrack. 0%.
Begin processing the 2481st record. run: 3785 subRun: 0 event: 49381 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 233 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 249 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 7 / 246 failed to backtrack. 2.84553%.
Begin processing the 2482nd record. run: 3785 subRun: 0 event: 49382 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 177 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 150 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 148 failed to backtrack. 2.7027%.
Begin processing the 2483rd record. run: 3785 subRun: 0 event: 49383 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 202 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 203 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 203 failed to backtrack. 2.46305%.
Begin processing the 2484th record. run: 3785 subRun: 0 event: 49384 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 172 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 147 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 145 failed to backtrack. 3.44828%.
Begin processing the 2485th record. run: 3785 subRun: 0 event: 49385 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 189 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 191 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 189 failed to backtrack. 1.5873%.
Begin processing the 2486th record. run: 3785 subRun: 0 event: 49386 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 172 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 131 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 131 failed to backtrack. 2.29008%.
Begin processing the 2487th record. run: 3785 subRun: 0 event: 49387 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 191 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 163 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 160 failed to backtrack. 1.875%.
Begin processing the 2488th record. run: 3785 subRun: 0 event: 49388 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 126 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 80 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 80 failed to backtrack. 3.75%.
Begin processing the 2489th record. run: 3785 subRun: 0 event: 49389 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 211 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 147 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 142 failed to backtrack. 2.11268%.
Begin processing the 2490th record. run: 3785 subRun: 0 event: 49390 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 156 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 145 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 9 / 142 failed to backtrack. 6.33803%.
Begin processing the 2491st record. run: 3785 subRun: 0 event: 49391 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 120 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 107 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 106 failed to backtrack. 4.71698%.
Begin processing the 2492nd record. run: 3785 subRun: 0 event: 49392 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 204 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 202 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 200 failed to backtrack. 1.5%.
Begin processing the 2493rd record. run: 3785 subRun: 0 event: 49393 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 132 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 114 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 113 failed to backtrack. 4.42478%.
Begin processing the 2494th record. run: 3785 subRun: 0 event: 49394 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 161 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 153 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 153 failed to backtrack. 1.96078%.
Begin processing the 2495th record. run: 3785 subRun: 0 event: 49395 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 145 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 103 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 101 failed to backtrack. 1.9802%.
Begin processing the 2496th record. run: 3785 subRun: 0 event: 49396 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 293 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 321 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 320 failed to backtrack. 1.5625%.
Begin processing the 2497th record. run: 3785 subRun: 0 event: 49397 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 46 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 30 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 29 failed to backtrack. 0%.
Begin processing the 2498th record. run: 3785 subRun: 0 event: 49398 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 33 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 12 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 11 failed to backtrack. 0%.
Begin processing the 2499th record. run: 3785 subRun: 0 event: 49399 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 271 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 223 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 6 / 221 failed to backtrack. 2.71493%.
Begin processing the 2500th record. run: 3785 subRun: 0 event: 49400 at 11-Jan-2026 09:09:43 UTC
TriggerPrimitiveMakerPDS: found 141 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 139 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 137 failed to backtrack. 0.729927%.
11-Jan-2026 09:09:44 UTC  Closed output file "tpg_11715_2026-01-11T_090700Z.root"
11-Jan-2026 09:09:44 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a3/a2/fardet-vd_mc_le_mc_2024a_workflow0000003785_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000350_lim000025_mergeLar_20250204T141136_20250610T120555Z_reco2.root"

===================================================================================================================================================
TimeTracker printout (sec)                                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================================
Full event                                                        0.0026403     0.0115062      3.20295     0.00574784     0.0935646      2500    
---------------------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                           0.000137624   0.000543914    0.0194431    0.000428429   0.000703625     2500    
makers:tpmakerPDSLessSimpleThreshold:TriggerPrimitiveMakerPDS    0.00080986    0.00253698     0.0333258     0.0020425    0.00228597      2500    
[art]:TriggerResults:TriggerResultInserter                        6.492e-06    1.31137e-05   0.000162992   1.2484e-05    5.84645e-06     2500    
end_path:out1:RootOutput                                          9.61e-07     2.79367e-06   0.000126351    2.795e-06    2.89876e-06     2500    
end_path:triggerAna:TriggerAnaTree                               0.000849886   0.00268203     0.045165      0.0019822    0.00255698      2500    
end_path:out1:RootOutput(write)                                  0.000252112   0.000769304    0.0505067    0.000650685   0.00143052      2500    
===================================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1314.26 MB
  Peak resident set size usage (VmHWM): 623.616 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0