Jobsub ID 20437.24@dunegpschedd01.fnal.gov

Jobsub ID	20437.24@dunegpschedd01.fnal.gov
Workflow ID	249
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-02 11:31:14
Site	NL_SURFsara
Entry	DUNE_SurfSARA_arc02
Last heartbeat	2025-08-02 18:22:23
From worker node	Hostname	wn-la-14.gina.surf.nl
	cpuinfo	AMD EPYC 9754 128-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-02 11:32:38
Input files	monte-carlo-000249-008657
Jobscript	Exit code	0
	Real time	6h (24353s)
	CPU time	6h (24281s = 99%)
	Max RSS bytes	1988980736 (1896 MiB)
Outputting started	2025-08-02 18:18:32
Output files	fardet-hd:prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_reco.root
Finished	2025-08-02 18:22:23
Saved logs	justin-logs:20437.24-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

th regularization...
Begin: 270528
0 270519
Begin processing the 46th record. run: 249 subRun: 0 event: 432846 at 02-Aug-2025 20:18:21 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 23 XUs and 20 XVs -> 11 XUVs
C:0 T:1 42 XUs and 43 XVs -> 24 XUVs
C:0 T:2 20 XUs and 19 XVs -> 11 XUVs
C:0 T:3 8 XUs and 10 XVs -> 5 XUVs
C:0 T:4 10 XUs and 8 XVs -> 4 XUVs
C:0 T:5 5 XUs and 4 XVs -> 2 XUVs
C:0 T:6 5 XUs and 6 XVs -> 1 XUVs
C:0 T:7 6 XUs and 4 XVs -> 3 XUVs
C:0 T:8 3 XUs and 2 XVs -> 1 XUVs
C:0 T:9 2 XUs and 1 XVs -> 1 XUVs
C:0 T:10 0 XUs and 0 XVs -> 0 XUVs
C:0 T:11 13 XUs and 11 XVs -> 6 XUVs
C:0 T:12 2 XUs and 3 XVs -> 1 XUVs
C:0 T:13 8 XUs and 4 XVs -> 2 XUVs
C:0 T:14 0 XUs and 1 XVs -> 0 XUVs
C:0 T:15 11 XUs and 14 XVs -> 6 XUVs
C:0 T:16 13 XUs and 10 XVs -> 4 XUVs
C:0 T:17 5 XUs and 8 XVs -> 3 XUVs
C:0 T:18 7 XUs and 8 XVs -> 5 XUVs
C:0 T:19 1 XUs and 1 XVs -> 1 XUVs
C:0 T:20 21 XUs and 24 XVs -> 14 XUVs
C:0 T:21 37 XUs and 34 XVs -> 21 XUVs
C:0 T:22 21 XUs and 20 XVs -> 14 XUVs
C:0 T:23 30 XUs and 25 XVs -> 16 XUVs
156 XUVs total
119 collection wire objects
156 potential space points
Neighbour search...
340 tests to find 182 neighbours
Iterating with no regularization...
Begin: 501744
0 410555
1 409770
2 409750
Now with regularization...
Begin: 294964
0 294949
Begin processing the 47th record. run: 249 subRun: 0 event: 432847 at 02-Aug-2025 20:18:22 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 17 XUs and 17 XVs -> 10 XUVs
C:0 T:1 11 XUs and 14 XVs -> 4 XUVs
C:0 T:2 18 XUs and 19 XVs -> 8 XUVs
C:0 T:3 24 XUs and 26 XVs -> 15 XUVs
C:0 T:4 2 XUs and 2 XVs -> 0 XUVs
C:0 T:5 25 XUs and 18 XVs -> 12 XUVs
C:0 T:6 15 XUs and 13 XVs -> 9 XUVs
C:0 T:7 3 XUs and 3 XVs -> 2 XUVs
C:0 T:8 4 XUs and 4 XVs -> 1 XUVs
C:0 T:9 11 XUs and 15 XVs -> 7 XUVs
C:0 T:10 2 XUs and 6 XVs -> 2 XUVs
C:0 T:11 13 XUs and 16 XVs -> 9 XUVs
C:0 T:12 8 XUs and 5 XVs -> 3 XUVs
C:0 T:13 8 XUs and 8 XVs -> 5 XUVs
C:0 T:14 6 XUs and 7 XVs -> 6 XUVs
C:0 T:15 6 XUs and 6 XVs -> 2 XUVs
C:0 T:16 6 XUs and 8 XVs -> 4 XUVs
C:0 T:17 14 XUs and 15 XVs -> 10 XUVs
C:0 T:18 1 XUs and 1 XVs -> 0 XUVs
C:0 T:19 2 XUs and 2 XVs -> 1 XUVs
C:0 T:20 27 XUs and 32 XVs -> 16 XUVs
C:0 T:21 31 XUs and 36 XVs -> 18 XUVs
C:0 T:22 26 XUs and 23 XVs -> 11 XUVs
C:0 T:23 36 XUs and 38 XVs -> 18 XUVs
173 XUVs total
135 collection wire objects
173 potential space points
Neighbour search...
355 tests to find 182 neighbours
Iterating with no regularization...
Begin: 465522
0 386175
1 385300
2 385287
Now with regularization...
Begin: 268334
0 268324
Begin processing the 48th record. run: 249 subRun: 0 event: 432848 at 02-Aug-2025 20:18:22 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 20 XUs and 23 XVs -> 11 XUVs
C:0 T:1 29 XUs and 30 XVs -> 16 XUVs
C:0 T:2 10 XUs and 12 XVs -> 4 XUVs
C:0 T:3 13 XUs and 15 XVs -> 10 XUVs
C:0 T:4 2 XUs and 2 XVs -> 2 XUVs
C:0 T:5 2 XUs and 2 XVs -> 2 XUVs
C:0 T:6 3 XUs and 4 XVs -> 2 XUVs
C:0 T:7 15 XUs and 13 XVs -> 8 XUVs
C:0 T:8 3 XUs and 2 XVs -> 1 XUVs
C:0 T:9 10 XUs and 9 XVs -> 4 XUVs
C:0 T:10 10 XUs and 9 XVs -> 4 XUVs
C:0 T:11 9 XUs and 9 XVs -> 5 XUVs
C:0 T:12 6 XUs and 6 XVs -> 3 XUVs
C:0 T:13 0 XUs and 2 XVs -> 0 XUVs
C:0 T:14 3 XUs and 2 XVs -> 0 XUVs
C:0 T:15 2 XUs and 3 XVs -> 1 XUVs
C:0 T:16 6 XUs and 10 XVs -> 5 XUVs
C:0 T:17 1 XUs and 0 XVs -> 0 XUVs
C:0 T:18 1 XUs and 1 XVs -> 0 XUVs
C:0 T:19 4 XUs and 6 XVs -> 3 XUVs
C:0 T:20 10 XUs and 11 XVs -> 5 XUVs
C:0 T:21 11 XUs and 15 XVs -> 6 XUVs
C:0 T:22 24 XUs and 31 XVs -> 17 XUVs
C:0 T:23 35 XUs and 36 XVs -> 17 XUVs
126 XUVs total
103 collection wire objects
126 potential space points
Neighbour search...
228 tests to find 100 neighbours
Iterating with no regularization...
Begin: 350824
0 320117
1 320089
Now with regularization...
Begin: 233267
0 233263
Begin processing the 49th record. run: 249 subRun: 0 event: 432849 at 02-Aug-2025 20:18:23 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 19 XUs and 25 XVs -> 12 XUVs
C:0 T:1 35 XUs and 32 XVs -> 21 XUVs
C:0 T:2 8 XUs and 11 XVs -> 4 XUVs
C:0 T:3 20 XUs and 21 XVs -> 8 XUVs
C:0 T:4 4 XUs and 4 XVs -> 4 XUVs
C:0 T:5 3 XUs and 3 XVs -> 1 XUVs
C:0 T:6 4 XUs and 3 XVs -> 2 XUVs
C:0 T:7 6 XUs and 9 XVs -> 3 XUVs
C:0 T:8 3 XUs and 3 XVs -> 2 XUVs
C:0 T:9 7 XUs and 6 XVs -> 4 XUVs
C:0 T:10 3 XUs and 2 XVs -> 2 XUVs
C:0 T:11 1 XUs and 2 XVs -> 1 XUVs
C:0 T:12 5 XUs and 7 XVs -> 3 XUVs
C:0 T:13 5 XUs and 6 XVs -> 3 XUVs
C:0 T:14 5 XUs and 4 XVs -> 3 XUVs
C:0 T:15 2 XUs and 3 XVs -> 2 XUVs
C:0 T:16 10 XUs and 7 XVs -> 3 XUVs
C:0 T:17 4 XUs and 3 XVs -> 2 XUVs
C:0 T:18 6 XUs and 8 XVs -> 3 XUVs
C:0 T:19 9 XUs and 9 XVs -> 7 XUVs
C:0 T:20 23 XUs and 22 XVs -> 12 XUVs
C:0 T:21 15 XUs and 13 XVs -> 6 XUVs
C:0 T:22 10 XUs and 9 XVs -> 4 XUVs
C:0 T:23 13 XUs and 9 XVs -> 5 XUVs
117 XUVs total
91 collection wire objects
117 potential space points
Neighbour search...
231 tests to find 112 neighbours
Iterating with no regularization...
Begin: 388363
0 312386
1 312044
2 312017
Now with regularization...
Begin: 236783
0 236769
Begin processing the 50th record. run: 249 subRun: 0 event: 432850 at 02-Aug-2025 20:18:24 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 15 XUs and 9 XVs -> 5 XUVs
C:0 T:1 14 XUs and 16 XVs -> 8 XUVs
C:0 T:2 19 XUs and 17 XVs -> 10 XUVs
C:0 T:3 11 XUs and 11 XVs -> 8 XUVs
C:0 T:4 6 XUs and 6 XVs -> 2 XUVs
C:0 T:5 2 XUs and 1 XVs -> 1 XUVs
C:0 T:6 6 XUs and 7 XVs -> 4 XUVs
C:0 T:7 8 XUs and 8 XVs -> 4 XUVs
C:0 T:8 0 XUs and 0 XVs -> 0 XUVs
C:0 T:9 17 XUs and 20 XVs -> 12 XUVs
C:0 T:10 8 XUs and 5 XVs -> 5 XUVs
C:0 T:11 10 XUs and 6 XVs -> 6 XUVs
C:0 T:12 9 XUs and 10 XVs -> 6 XUVs
C:0 T:13 8 XUs and 8 XVs -> 7 XUVs
C:0 T:14 7 XUs and 7 XVs -> 3 XUVs
C:0 T:15 7 XUs and 10 XVs -> 5 XUVs
C:0 T:16 2 XUs and 2 XVs -> 1 XUVs
C:0 T:17 11 XUs and 11 XVs -> 7 XUVs
C:0 T:18 0 XUs and 0 XVs -> 0 XUVs
C:0 T:19 6 XUs and 8 XVs -> 5 XUVs
C:0 T:20 13 XUs and 16 XVs -> 8 XUVs
C:0 T:21 13 XUs and 15 XVs -> 10 XUVs
C:0 T:22 10 XUs and 10 XVs -> 8 XUVs
C:0 T:23 19 XUs and 19 XVs -> 10 XUVs
135 XUVs total
98 collection wire objects
135 potential space points
Neighbour search...
307 tests to find 170 neighbours
Iterating with no regularization...
Begin: 306950
0 239135
1 238307
2 238277
Now with regularization...
Begin: 154868
0 154855
02-Aug-2025 20:18:25 CEST  Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_reco.root"
02-Aug-2025 20:18:25 CEST  Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0503738     0.0544306     0.0751797     0.0539844    0.00372074       50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000348621   0.000480937   0.00100825    0.000467524   0.000104448      50     
reco:gaushit:GausHitFinder                     0.0381005     0.0405149     0.0621996     0.0401868    0.00326327       50     
reco:spsolve:SpacePointSolver                 0.00329165    0.00387652    0.00630465    0.00380414    0.000479835      50     
reco:hitfd:DisambigFromSpacePoints            0.00100682    0.00117404    0.00281537     0.0011319    0.000245783      50     
reco:rns:RandomNumberSaver                    2.0991e-05    2.6108e-05    0.000147561   2.2419e-05    1.78226e-05      50     
[art]:TriggerResults:TriggerResultInserter     9.825e-06    1.14604e-05   3.9419e-05    1.0866e-05    4.03974e-06      50     
end_path:out1:RootOutput                       4.376e-06    5.88198e-06   3.0175e-05    4.9925e-06    3.91883e-06      50     
end_path:out1:RootOutput(write)               0.00708993    0.00808947     0.0108202    0.00806366    0.000633371      50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1773.74 MB
  Peak resident set size usage (VmHWM): 890.008 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657.root 109521311
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4.root 1031565817
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim.root 2956871802
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_hist.root 12750
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_reco.root 2961888729
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_reco.root.ext.json 1578
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_reco.root.json 1821
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113247Z_gen_008657_supernova_g4_detsim_reco.root.temp.json 1821