justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20437.14@dunegpschedd01.fnal.gov

Jobsub ID20437.14@dunegpschedd01.fnal.gov
Workflow ID249
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-02 11:31:14
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-08-02 16:19:21
From worker nodeHostnamehnode60.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-02 11:32:37
Input filesmonte-carlo-000249-008549
JobscriptExit code0
Real time4h (17042s)
CPU time4h (16769s = 98%)
Max RSS bytes1977880576 (1886 MiB)
Outputting started2025-08-02 16:16:40
Output filesfardet-hd:prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_reco.root
Finished2025-08-02 16:19:21
Saved logsjustin-logs:20437.14-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

9
1 306510
Now with regularization...
Begin: 211953
0 211918
Begin processing the 46th record. run: 249 subRun: 0 event: 427446 at 02-Aug-2025 18:16:35 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 31 XUs and 30 XVs -> 16 XUVs
C:0 T:1 16 XUs and 19 XVs -> 11 XUVs
C:0 T:2 16 XUs and 17 XVs -> 9 XUVs
C:0 T:3 16 XUs and 12 XVs -> 9 XUVs
C:0 T:4 3 XUs and 2 XVs -> 2 XUVs
C:0 T:5 5 XUs and 2 XVs -> 2 XUVs
C:0 T:6 10 XUs and 11 XVs -> 5 XUVs
C:0 T:7 6 XUs and 5 XVs -> 4 XUVs
C:0 T:8 6 XUs and 6 XVs -> 3 XUVs
C:0 T:9 4 XUs and 3 XVs -> 2 XUVs
C:0 T:10 6 XUs and 6 XVs -> 3 XUVs
C:0 T:11 8 XUs and 10 XVs -> 7 XUVs
C:0 T:12 4 XUs and 6 XVs -> 4 XUVs
C:0 T:13 9 XUs and 7 XVs -> 5 XUVs
C:0 T:14 9 XUs and 10 XVs -> 6 XUVs
C:0 T:15 12 XUs and 8 XVs -> 6 XUVs
C:0 T:16 8 XUs and 6 XVs -> 3 XUVs
C:0 T:17 5 XUs and 7 XVs -> 4 XUVs
C:0 T:18 0 XUs and 0 XVs -> 0 XUVs
C:0 T:19 2 XUs and 2 XVs -> 1 XUVs
C:0 T:20 16 XUs and 15 XVs -> 9 XUVs
C:0 T:21 2 XUs and 5 XVs -> 1 XUVs
C:0 T:22 37 XUs and 45 XVs -> 27 XUVs
C:0 T:23 25 XUs and 23 XVs -> 16 XUVs
155 XUVs total
115 collection wire objects
155 potential space points
Neighbour search...
303 tests to find 148 neighbours
Iterating with no regularization...
Begin: 432956
0 368211
1 367857
Now with regularization...
Begin: 260241
0 260229
Begin processing the 47th record. run: 249 subRun: 0 event: 427447 at 02-Aug-2025 18:16:35 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 10 XUs and 11 XVs -> 5 XUVs
C:0 T:1 13 XUs and 14 XVs -> 6 XUVs
C:0 T:2 5 XUs and 5 XVs -> 3 XUVs
C:0 T:3 21 XUs and 17 XVs -> 13 XUVs
C:0 T:4 6 XUs and 3 XVs -> 2 XUVs
C:0 T:5 12 XUs and 11 XVs -> 8 XUVs
C:0 T:6 6 XUs and 6 XVs -> 3 XUVs
C:0 T:7 0 XUs and 2 XVs -> 0 XUVs
C:0 T:8 10 XUs and 12 XVs -> 4 XUVs
C:0 T:9 12 XUs and 12 XVs -> 9 XUVs
C:0 T:10 0 XUs and 0 XVs -> 0 XUVs
C:0 T:11 7 XUs and 6 XVs -> 3 XUVs
C:0 T:12 13 XUs and 13 XVs -> 9 XUVs
C:0 T:13 4 XUs and 4 XVs -> 3 XUVs
C:0 T:14 4 XUs and 3 XVs -> 3 XUVs
C:0 T:15 3 XUs and 2 XVs -> 2 XUVs
C:0 T:16 7 XUs and 7 XVs -> 5 XUVs
C:0 T:17 7 XUs and 9 XVs -> 6 XUVs
C:0 T:18 17 XUs and 15 XVs -> 10 XUVs
C:0 T:19 14 XUs and 16 XVs -> 8 XUVs
C:0 T:20 15 XUs and 16 XVs -> 7 XUVs
C:0 T:21 28 XUs and 27 XVs -> 15 XUVs
C:0 T:22 22 XUs and 20 XVs -> 14 XUVs
C:0 T:23 20 XUs and 19 XVs -> 12 XUVs
150 XUVs total
109 collection wire objects
150 potential space points
Neighbour search...
346 tests to find 196 neighbours
Iterating with no regularization...
Begin: 348214
0 274688
1 273707
2 273675
Now with regularization...
Begin: 174606
0 174590
Begin processing the 48th record. run: 249 subRun: 0 event: 427448 at 02-Aug-2025 18:16:36 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 18 XUs and 20 XVs -> 10 XUVs
C:0 T:1 21 XUs and 18 XVs -> 10 XUVs
C:0 T:2 14 XUs and 13 XVs -> 10 XUVs
C:0 T:3 19 XUs and 15 XVs -> 9 XUVs
C:0 T:4 9 XUs and 9 XVs -> 5 XUVs
C:0 T:5 7 XUs and 6 XVs -> 4 XUVs
C:0 T:6 7 XUs and 5 XVs -> 3 XUVs
C:0 T:7 7 XUs and 8 XVs -> 7 XUVs
C:0 T:8 17 XUs and 17 XVs -> 9 XUVs
C:0 T:9 6 XUs and 6 XVs -> 3 XUVs
C:0 T:10 7 XUs and 8 XVs -> 3 XUVs
C:0 T:11 0 XUs and 1 XVs -> 0 XUVs
C:0 T:12 6 XUs and 6 XVs -> 5 XUVs
C:0 T:13 6 XUs and 8 XVs -> 4 XUVs
C:0 T:14 0 XUs and 0 XVs -> 0 XUVs
C:0 T:15 10 XUs and 11 XVs -> 7 XUVs
C:0 T:16 12 XUs and 14 XVs -> 7 XUVs
C:0 T:17 9 XUs and 8 XVs -> 5 XUVs
C:0 T:18 3 XUs and 3 XVs -> 2 XUVs
C:0 T:19 6 XUs and 3 XVs -> 2 XUVs
C:0 T:20 9 XUs and 7 XVs -> 4 XUVs
C:0 T:21 28 XUs and 30 XVs -> 16 XUVs
C:0 T:22 14 XUs and 16 XVs -> 7 XUVs
C:0 T:23 10 XUs and 14 XVs -> 7 XUVs
139 XUVs total
107 collection wire objects
139 potential space points
Neighbour search...
251 tests to find 112 neighbours
Iterating with no regularization...
Begin: 476052
0 437389
1 437320
Now with regularization...
Begin: 343218
0 343214
Begin processing the 49th record. run: 249 subRun: 0 event: 427449 at 02-Aug-2025 18:16:36 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 32 XUs and 29 XVs -> 17 XUVs
C:0 T:1 15 XUs and 16 XVs -> 11 XUVs
C:0 T:2 32 XUs and 35 XVs -> 19 XUVs
C:0 T:3 19 XUs and 18 XVs -> 13 XUVs
C:0 T:4 9 XUs and 9 XVs -> 5 XUVs
C:0 T:5 6 XUs and 7 XVs -> 2 XUVs
C:0 T:6 6 XUs and 5 XVs -> 3 XUVs
C:0 T:7 3 XUs and 3 XVs -> 1 XUVs
C:0 T:8 4 XUs and 8 XVs -> 3 XUVs
C:0 T:9 14 XUs and 16 XVs -> 9 XUVs
C:0 T:10 8 XUs and 13 XVs -> 6 XUVs
C:0 T:11 1 XUs and 0 XVs -> 0 XUVs
C:0 T:12 3 XUs and 2 XVs -> 0 XUVs
C:0 T:13 8 XUs and 8 XVs -> 4 XUVs
C:0 T:14 5 XUs and 3 XVs -> 2 XUVs
C:0 T:15 3 XUs and 2 XVs -> 0 XUVs
C:0 T:16 6 XUs and 3 XVs -> 2 XUVs
C:0 T:17 9 XUs and 13 XVs -> 8 XUVs
C:0 T:18 6 XUs and 4 XVs -> 3 XUVs
C:0 T:19 14 XUs and 12 XVs -> 10 XUVs
C:0 T:20 29 XUs and 22 XVs -> 15 XUVs
C:0 T:21 27 XUs and 30 XVs -> 20 XUVs
C:0 T:22 13 XUs and 12 XVs -> 8 XUVs
C:0 T:23 15 XUs and 14 XVs -> 9 XUVs
170 XUVs total
126 collection wire objects
170 potential space points
Neighbour search...
348 tests to find 178 neighbours
Iterating with no regularization...
Begin: 403336
0 353684
1 353186
2 353172
Now with regularization...
Begin: 241482
0 241474
Begin processing the 50th record. run: 249 subRun: 0 event: 427450 at 02-Aug-2025 18:16:37 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 14 XUs and 15 XVs -> 7 XUVs
C:0 T:1 17 XUs and 17 XVs -> 10 XUVs
C:0 T:2 20 XUs and 16 XVs -> 9 XUVs
C:0 T:3 15 XUs and 15 XVs -> 7 XUVs
C:0 T:4 6 XUs and 8 XVs -> 4 XUVs
C:0 T:5 5 XUs and 7 XVs -> 4 XUVs
C:0 T:6 7 XUs and 7 XVs -> 4 XUVs
C:0 T:7 1 XUs and 2 XVs -> 1 XUVs
C:0 T:8 1 XUs and 1 XVs -> 0 XUVs
C:0 T:9 12 XUs and 9 XVs -> 5 XUVs
C:0 T:10 1 XUs and 2 XVs -> 0 XUVs
C:0 T:11 9 XUs and 9 XVs -> 4 XUVs
C:0 T:12 2 XUs and 2 XVs -> 1 XUVs
C:0 T:13 1 XUs and 1 XVs -> 0 XUVs
C:0 T:14 9 XUs and 8 XVs -> 6 XUVs
C:0 T:15 11 XUs and 11 XVs -> 7 XUVs
C:0 T:16 10 XUs and 9 XVs -> 6 XUVs
C:0 T:17 6 XUs and 5 XVs -> 4 XUVs
C:0 T:18 7 XUs and 13 XVs -> 6 XUVs
C:0 T:19 4 XUs and 4 XVs -> 2 XUVs
C:0 T:20 20 XUs and 17 XVs -> 11 XUVs
C:0 T:21 27 XUs and 27 XVs -> 17 XUVs
C:0 T:22 21 XUs and 16 XVs -> 12 XUVs
C:0 T:23 20 XUs and 21 XVs -> 9 XUVs
136 XUVs total
99 collection wire objects
136 potential space points
Neighbour search...
308 tests to find 164 neighbours
Iterating with no regularization...
Begin: 322332
0 291099
1 290966
Now with regularization...
Begin: 206951
0 206936
02-Aug-2025 18:16:38 CEST  Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_reco.root"
02-Aug-2025 18:16:38 CEST  Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0395033     0.0434027     0.0504115     0.0431326    0.00211594       50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000241521   0.000325309   0.00044762    0.000324079   4.47449e-05      50     
reco:gaushit:GausHitFinder                     0.0314379     0.0334298     0.0361848     0.0332606    0.00117219       50     
reco:spsolve:SpacePointSolver                 0.00210621    0.00269082     0.0048873    0.00263351    0.000409928      50     
reco:hitfd:DisambigFromSpacePoints            0.000724921   0.000899366   0.00185374    0.000887753   0.000153997      50     
reco:rns:RandomNumberSaver                    1.2001e-05    1.55219e-05   9.4614e-05    1.34555e-05   1.15777e-05      50     
[art]:TriggerResults:TriggerResultInserter     6.811e-06    7.98778e-06   2.5221e-05     7.615e-06    2.51657e-06      50     
end_path:out1:RootOutput                        2.3e-06     2.86014e-06   1.0971e-05     2.665e-06    1.19048e-06      50     
end_path:out1:RootOutput(write)               0.00479037     0.0058748     0.0076691    0.00587688    0.000499618      50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1773.76 MB
  Peak resident set size usage (VmHWM): 887.968 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549.root 109400984
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4.root 1030056444
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim.root 2956607431
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_hist.root 12750
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_reco.root 2961604559
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_reco.root.ext.json 1578
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_reco.root.json 1821
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113242Z_gen_008549_supernova_g4_detsim_reco.root.temp.json 1821
justIN time: 2025-08-05 02:44:12 UTC       justIN version: 01.04.00