Jobsub ID 268617.7@dunegpschedd02.fnal.gov

Jobsub ID	268617.7@dunegpschedd02.fnal.gov
Workflow ID	11715
Stage ID	1
User name	jierans@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-11 08:20:36
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2026-01-11 09:08:56
From worker node	Hostname	dunegli-8298663-0-fnpc22020.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-11 09:06:09
Input files	fardet-vd:fardet-vd_mc_le_mc_2024a_list-4279_input.list_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000275_lim000025_mergeLar_20241220T211052_20250610T122346Z_reco2.root
Jobscript	Exit code	0
	Real time	2m (154s)
	CPU time	1m (65s = 42%)
	Max RSS bytes	620146688 (591 MiB)
Outputting started	2026-01-11 09:08:44
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/jierans/vd_1x8x14_nuecc_with_pds/ntuple/fnal/11715/1/001/triggerAna_11715_2026-01-11T_090624Z.ntuple.root
Finished	2026-01-11 09:08:56
Saved logs	justin-logs:268617.7-dunegpschedd02.fnal.gov.logs.tgz
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

kerPDS: Generated 53 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 53 failed to backtrack. 9.43396%.
Begin processing the 2477th record. run: 4279 subRun: 0 event: 29977 at 11-Jan-2026 09:08:42 UTC
TriggerPrimitiveMakerPDS: found 41 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 27 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 27 failed to backtrack. 3.7037%.
Begin processing the 2478th record. run: 4279 subRun: 0 event: 29978 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 194 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 179 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 177 failed to backtrack. 1.12994%.
Begin processing the 2479th record. run: 4279 subRun: 0 event: 29979 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 114 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 69 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 69 failed to backtrack. 0%.
Begin processing the 2480th record. run: 4279 subRun: 0 event: 29980 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 149 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 142 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 141 failed to backtrack. 3.5461%.
Begin processing the 2481st record. run: 4279 subRun: 0 event: 29981 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 14 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 2 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 0 / 2 failed to backtrack. 0%.
Begin processing the 2482nd record. run: 4279 subRun: 0 event: 29982 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 126 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 110 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 110 failed to backtrack. 1.81818%.
Begin processing the 2483rd record. run: 4279 subRun: 0 event: 29983 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 226 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 243 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 242 failed to backtrack. 1.65289%.
Begin processing the 2484th record. run: 4279 subRun: 0 event: 29984 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 157 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 116 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 115 failed to backtrack. 1.73913%.
Begin processing the 2485th record. run: 4279 subRun: 0 event: 29985 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 240 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 176 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 6 / 174 failed to backtrack. 3.44828%.
Begin processing the 2486th record. run: 4279 subRun: 0 event: 29986 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 123 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 89 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 86 failed to backtrack. 3.48837%.
Begin processing the 2487th record. run: 4279 subRun: 0 event: 29987 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 175 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 149 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 145 failed to backtrack. 1.37931%.
Begin processing the 2488th record. run: 4279 subRun: 0 event: 29988 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 62 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 35 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 34 failed to backtrack. 5.88235%.
Begin processing the 2489th record. run: 4279 subRun: 0 event: 29989 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 186 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 174 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 168 failed to backtrack. 2.38095%.
Begin processing the 2490th record. run: 4279 subRun: 0 event: 29990 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 235 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 230 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 9 / 228 failed to backtrack. 3.94737%.
Begin processing the 2491st record. run: 4279 subRun: 0 event: 29991 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 67 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 34 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 34 failed to backtrack. 2.94118%.
Begin processing the 2492nd record. run: 4279 subRun: 0 event: 29992 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 168 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 146 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 145 failed to backtrack. 2.06897%.
Begin processing the 2493rd record. run: 4279 subRun: 0 event: 29993 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 182 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 190 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 7 / 190 failed to backtrack. 3.68421%.
Begin processing the 2494th record. run: 4279 subRun: 0 event: 29994 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 167 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 154 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 153 failed to backtrack. 1.30719%.
Begin processing the 2495th record. run: 4279 subRun: 0 event: 29995 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 50 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 20 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 1 / 17 failed to backtrack. 5.88235%.
Begin processing the 2496th record. run: 4279 subRun: 0 event: 29996 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 195 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 218 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 214 failed to backtrack. 2.33645%.
Begin processing the 2497th record. run: 4279 subRun: 0 event: 29997 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 158 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 136 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 4 / 133 failed to backtrack. 3.00752%.
Begin processing the 2498th record. run: 4279 subRun: 0 event: 29998 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 48 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 16 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 2 / 15 failed to backtrack. 13.3333%.
Begin processing the 2499th record. run: 4279 subRun: 0 event: 29999 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 83 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 57 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 3 / 54 failed to backtrack. 5.55556%.
Begin processing the 2500th record. run: 4279 subRun: 0 event: 30000 at 11-Jan-2026 09:08:43 UTC
TriggerPrimitiveMakerPDS: found 222 OpDetWaveform objects.
TriggerPrimitiveMakerPDS: Generated 230 TPs.
For TP collection tpmakerPDSLessSimpleThreshold::TriggerSimAna, 5 / 228 failed to backtrack. 2.19298%.
11-Jan-2026 09:08:43 UTC  Closed output file "tpg_11715_2026-01-11T_090624Z.root"
11-Jan-2026 09:08:43 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/47/30/fardet-vd_mc_le_mc_2024a_list-4279_input.list_out1_standard_reco1_dunevd10kt_1x8x14_3view_30deg_hit-reconstructed_merged_skip000275_lim000025_mergeLar_20241220T211052_20250610T122346Z_reco2.root"

===================================================================================================================================================
TimeTracker printout (sec)                                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================================
Full event                                                       0.00215936    0.00976249     0.857252     0.00457974     0.0397599      2500    
---------------------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                           0.000116651   0.000617439    0.0170268    0.000395611   0.00135975      2500    
makers:tpmakerPDSLessSimpleThreshold:TriggerPrimitiveMakerPDS    0.000686666   0.00246422     0.0411241    0.00171328    0.00272821      2500    
[art]:TriggerResults:TriggerResultInserter                        5.891e-06    9.7841e-06    0.000255031    6.863e-06    9.89959e-06     2500    
end_path:out1:RootOutput                                          8.72e-07     2.04422e-06   1.8144e-05    1.2725e-06    1.72233e-06     2500    
end_path:triggerAna:TriggerAnaTree                               0.000732091   0.00278983      0.14527     0.00161531    0.00451075      2500    
end_path:out1:RootOutput(write)                                  0.000213132    0.0005798     0.0507691    0.000410058   0.00167538      2500    
===================================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1314.23 MB
  Peak resident set size usage (VmHWM): 621.912 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0