Jobsub ID 280590.0@dunegpschedd01.fnal.gov

Jobsub ID	280590.0@dunegpschedd01.fnal.gov
Workflow ID	12031
Stage ID	1
User name	jierans@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-18 06:51:25
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2026-01-18 07:52:50
From worker node	Hostname	dunegli-8199662-0-fnpc23046.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-18 07:42:42
Input files	fardet-vd:fardet-vd_mc_le_mc_2024a_list-4252_input.list_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000325_lim000025_mergeLar_20241220T190718_20250610T121128Z_reco2.root
Jobscript	Exit code	0
	Real time	10m (601s)
	CPU time	1m (74s = 12%)
	Max RSS bytes	615141376 (586 MiB)
Outputting started	2026-01-18 07:52:43
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/jierans/vd_1x8x14_nuecc_with_pds/ntuple/fnal/12031/1/001/triggerAna_12031_2026-01-18T_074249Z.ntuple.root
Finished	2026-01-18 07:52:50
Saved logs	justin-logs:280590.0-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

kerPDS: Generated 251 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 244 failed to backtrack. 0.819672%.
Begin processing the 2477th record. run: 4252 subRun: 0 event: 34977 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 57 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 45 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 43 failed to backtrack. 0%.
Begin processing the 2478th record. run: 4252 subRun: 0 event: 34978 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 260 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 233 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 227 failed to backtrack. 1.32159%.
Begin processing the 2479th record. run: 4252 subRun: 0 event: 34979 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 122 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 129 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 125 failed to backtrack. 2.4%.
Begin processing the 2480th record. run: 4252 subRun: 0 event: 34980 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 61 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 41 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 39 failed to backtrack. 2.5641%.
Begin processing the 2481st record. run: 4252 subRun: 0 event: 34981 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 79 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 59 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 56 failed to backtrack. 1.78571%.
Begin processing the 2482nd record. run: 4252 subRun: 0 event: 34982 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 132 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 131 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 131 failed to backtrack. 1.52672%.
Begin processing the 2483rd record. run: 4252 subRun: 0 event: 34983 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 71 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 59 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 57 failed to backtrack. 5.26316%.
Begin processing the 2484th record. run: 4252 subRun: 0 event: 34984 at 18-Jan-2026 07:52:39 UTC
TriggerPrimitiveMakerPDS: found 165 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 127 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 126 failed to backtrack. 3.96825%.
Begin processing the 2485th record. run: 4252 subRun: 0 event: 34985 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 242 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 216 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 6 / 209 failed to backtrack. 2.87081%.
Begin processing the 2486th record. run: 4252 subRun: 0 event: 34986 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 139 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 106 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 104 failed to backtrack. 4.80769%.
Begin processing the 2487th record. run: 4252 subRun: 0 event: 34987 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 206 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 188 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 184 failed to backtrack. 2.71739%.
Begin processing the 2488th record. run: 4252 subRun: 0 event: 34988 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 183 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 165 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 158 failed to backtrack. 2.53165%.
Begin processing the 2489th record. run: 4252 subRun: 0 event: 34989 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 104 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 83 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 82 failed to backtrack. 6.09756%.
Begin processing the 2490th record. run: 4252 subRun: 0 event: 34990 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 33 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 18 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 18 failed to backtrack. 5.55556%.
Begin processing the 2491st record. run: 4252 subRun: 0 event: 34991 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 36 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 24 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 23 failed to backtrack. 4.34783%.
Begin processing the 2492nd record. run: 4252 subRun: 0 event: 34992 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 211 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 215 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 205 failed to backtrack. 1.95122%.
Begin processing the 2493rd record. run: 4252 subRun: 0 event: 34993 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 339 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 315 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 6 / 314 failed to backtrack. 1.91083%.
Begin processing the 2494th record. run: 4252 subRun: 0 event: 34994 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 161 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 135 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 129 failed to backtrack. 3.87597%.
Begin processing the 2495th record. run: 4252 subRun: 0 event: 34995 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 144 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 122 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 120 failed to backtrack. 0.833333%.
Begin processing the 2496th record. run: 4252 subRun: 0 event: 34996 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 105 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 94 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 90 failed to backtrack. 3.33333%.
Begin processing the 2497th record. run: 4252 subRun: 0 event: 34997 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 129 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 119 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 112 failed to backtrack. 4.46429%.
Begin processing the 2498th record. run: 4252 subRun: 0 event: 34998 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 174 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 148 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 147 failed to backtrack. 1.36054%.
Begin processing the 2499th record. run: 4252 subRun: 0 event: 34999 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 139 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 100 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 91 failed to backtrack. 3.2967%.
Begin processing the 2500th record. run: 4252 subRun: 0 event: 35000 at 18-Jan-2026 07:52:40 UTC
TriggerPrimitiveMakerPDS: found 80 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 67 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 65 failed to backtrack. 0%.
18-Jan-2026 07:52:41 UTC  Closed output file "tpg_12031_2026-01-18T_074249Z.root"
18-Jan-2026 07:52:41 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/b6/03/fardet-vd_mc_le_mc_2024a_list-4252_input.list_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000325_lim000025_mergeLar_20241220T190718_20250610T121128Z_reco2.root"

===================================================================================================================================================
TimeTracker printout (sec)                                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================================
Full event                                                        0.0026421     0.0376435      1.11736       0.01669      0.0801324      2500    
---------------------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                           0.000130811   0.00226056     0.0852723    0.000450154   0.00582183      2500    
makers:tpmakerPDSLessSimpleThreshold:TriggerPrimitiveMakerPDS    0.000845362    0.0170321     0.323159     0.00689859     0.0290859      2500    
[art]:TriggerResults:TriggerResultInserter                        6.131e-06    1.41999e-05   0.000142493   1.29345e-05   7.64825e-06     2500    
end_path:out1:RootOutput                                          9.02e-07     3.30218e-06   6.5415e-05     2.986e-06    2.23428e-06     2500    
end_path:triggerAna:TriggerAnaTree                               0.000709802   0.00717269     0.248886     0.00208741     0.0213315      2500    
end_path:out1:RootOutput(write)                                  0.00026625    0.00454912      0.25598     0.000629377    0.016454       2500    
===================================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1314.24 MB
  Peak resident set size usage (VmHWM): 623.87 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0