Jobsub ID 280590.3@dunegpschedd01.fnal.gov

Jobsub ID	280590.3@dunegpschedd01.fnal.gov
Workflow ID	12031
Stage ID	1
User name	jierans@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-18 06:51:25
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2026-01-18 07:45:51
From worker node	Hostname	dunegli-8356189-0-fnpc23031.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-18 07:43:08
Input files	fardet-vd:fardet-vd_mc_le_mc_2024a_list-4276_input.list_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241220T203832_20250610T121005Z_reco2.root
Jobscript	Exit code	0
	Real time	2m (151s)
	CPU time	1m (95s = 62%)
	Max RSS bytes	618418176 (589 MiB)
Outputting started	2026-01-18 07:45:40
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/jierans/vd_1x8x14_nuecc_with_pds/ntuple/fnal/12031/1/001/triggerAna_12031_2026-01-18T_074319Z.ntuple.root
Finished	2026-01-18 07:45:51
Saved logs	justin-logs:280590.3-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

rPDS: Generated 10 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 8 failed to backtrack. 25%.
Begin processing the 2477th record. run: 4276 subRun: 0 event: 42477 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 272 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 242 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 7 / 235 failed to backtrack. 2.97872%.
Begin processing the 2478th record. run: 4276 subRun: 0 event: 42478 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 54 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 45 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 43 failed to backtrack. 4.65116%.
Begin processing the 2479th record. run: 4276 subRun: 0 event: 42479 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 125 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 98 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 97 failed to backtrack. 1.03093%.
Begin processing the 2480th record. run: 4276 subRun: 0 event: 42480 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 67 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 52 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 49 failed to backtrack. 4.08163%.
Begin processing the 2481st record. run: 4276 subRun: 0 event: 42481 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 127 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 113 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 6 / 110 failed to backtrack. 5.45455%.
Begin processing the 2482nd record. run: 4276 subRun: 0 event: 42482 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 110 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 86 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 82 failed to backtrack. 2.43902%.
Begin processing the 2483rd record. run: 4276 subRun: 0 event: 42483 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 204 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 202 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 6 / 195 failed to backtrack. 3.07692%.
Begin processing the 2484th record. run: 4276 subRun: 0 event: 42484 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 149 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 122 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 116 failed to backtrack. 2.58621%.
Begin processing the 2485th record. run: 4276 subRun: 0 event: 42485 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 110 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 83 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 83 failed to backtrack. 0%.
Begin processing the 2486th record. run: 4276 subRun: 0 event: 42486 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 225 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 235 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 7 / 230 failed to backtrack. 3.04348%.
Begin processing the 2487th record. run: 4276 subRun: 0 event: 42487 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 117 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 128 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 125 failed to backtrack. 2.4%.
Begin processing the 2488th record. run: 4276 subRun: 0 event: 42488 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 109 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 101 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 97 failed to backtrack. 3.09278%.
Begin processing the 2489th record. run: 4276 subRun: 0 event: 42489 at 18-Jan-2026 07:45:38 UTC
TriggerPrimitiveMakerPDS: found 180 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 155 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 151 failed to backtrack. 2.64901%.
Begin processing the 2490th record. run: 4276 subRun: 0 event: 42490 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 179 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 184 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 180 failed to backtrack. 0.555556%.
Begin processing the 2491st record. run: 4276 subRun: 0 event: 42491 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 117 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 103 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 103 failed to backtrack. 1.94175%.
Begin processing the 2492nd record. run: 4276 subRun: 0 event: 42492 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 171 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 150 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 146 failed to backtrack. 1.36986%.
Begin processing the 2493rd record. run: 4276 subRun: 0 event: 42493 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 141 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 134 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 131 failed to backtrack. 0.763359%.
Begin processing the 2494th record. run: 4276 subRun: 0 event: 42494 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 161 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 152 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 147 failed to backtrack. 0.680272%.
Begin processing the 2495th record. run: 4276 subRun: 0 event: 42495 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 78 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 59 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 58 failed to backtrack. 1.72414%.
Begin processing the 2496th record. run: 4276 subRun: 0 event: 42496 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 160 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 148 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 142 failed to backtrack. 2.8169%.
Begin processing the 2497th record. run: 4276 subRun: 0 event: 42497 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 183 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 173 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 167 failed to backtrack. 2.99401%.
Begin processing the 2498th record. run: 4276 subRun: 0 event: 42498 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 283 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 244 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 237 failed to backtrack. 1.68776%.
Begin processing the 2499th record. run: 4276 subRun: 0 event: 42499 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 65 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 31 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 30 failed to backtrack. 10%.
Begin processing the 2500th record. run: 4276 subRun: 0 event: 42500 at 18-Jan-2026 07:45:39 UTC
TriggerPrimitiveMakerPDS: found 207 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 241 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 237 failed to backtrack. 2.1097%.
18-Jan-2026 07:45:39 UTC  Closed output file "tpg_12031_2026-01-18T_074319Z.root"
18-Jan-2026 07:45:39 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/6a/1d/fardet-vd_mc_le_mc_2024a_list-4276_input.list_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241220T203832_20250610T121005Z_reco2.root"

===================================================================================================================================================
TimeTracker printout (sec)                                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================================
Full event                                                       0.00321598    0.00888774     0.350732     0.00662829     0.011852       2500    
---------------------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                           0.000186667   0.000683844    0.0497108    0.000539222   0.00205367      2500    
makers:tpmakerPDSLessSimpleThreshold:TriggerPrimitiveMakerPDS    0.000813929   0.00287583     0.143231     0.00233731    0.00484069      2500    
[art]:TriggerResults:TriggerResultInserter                        8.106e-06    1.84657e-05   0.000257252   1.7283e-05    9.05357e-06     2500    
end_path:out1:RootOutput                                          1.333e-06    4.12477e-06   4.7782e-05     4.038e-06    2.41947e-06     2500    
end_path:triggerAna:TriggerAnaTree                               0.000943607   0.00306579     0.078745     0.00223479    0.00400239      2500    
end_path:out1:RootOutput(write)                                  0.000378174   0.00127201     0.0858191    0.000814414   0.00434873      2500    
===================================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1314.23 MB
  Peak resident set size usage (VmHWM): 624.275 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0