justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20437.34@dunegpschedd01.fnal.gov

Jobsub ID20437.34@dunegpschedd01.fnal.gov
Workflow ID249
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-02 11:31:14
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-08-02 16:39:20
From worker nodeHostnamewn-da-01.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-02 11:32:06
Input filesmonte-carlo-000249-008253
JobscriptExit code0
Real time5h (18223s)
CPU time5h (18171s = 99%)
Max RSS bytes1981714432 (1889 MiB)
Outputting started2025-08-02 16:35:50
Output filesfardet-hd:prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_reco.root
Finished2025-08-02 16:39:20
Saved logsjustin-logs:20437.34-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ation...
Begin: 491915
0 419042
1 418968
Now with regularization...
Begin: 296663
0 296645
Begin processing the 46th record. run: 249 subRun: 0 event: 412646 at 02-Aug-2025 18:35:40 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 9 XUs and 7 XVs -> 4 XUVs
C:0 T:1 14 XUs and 13 XVs -> 8 XUVs
C:0 T:2 16 XUs and 15 XVs -> 7 XUVs
C:0 T:3 10 XUs and 12 XVs -> 6 XUVs
C:0 T:4 7 XUs and 4 XVs -> 3 XUVs
C:0 T:5 19 XUs and 19 XVs -> 11 XUVs
C:0 T:6 7 XUs and 9 XVs -> 5 XUVs
C:0 T:7 5 XUs and 6 XVs -> 3 XUVs
C:0 T:8 11 XUs and 16 XVs -> 8 XUVs
C:0 T:9 5 XUs and 3 XVs -> 1 XUVs
C:0 T:10 2 XUs and 2 XVs -> 1 XUVs
C:0 T:11 6 XUs and 4 XVs -> 4 XUVs
C:0 T:12 8 XUs and 2 XVs -> 1 XUVs
C:0 T:13 2 XUs and 2 XVs -> 2 XUVs
C:0 T:14 7 XUs and 7 XVs -> 4 XUVs
C:0 T:15 11 XUs and 13 XVs -> 6 XUVs
C:0 T:16 6 XUs and 8 XVs -> 5 XUVs
C:0 T:17 2 XUs and 3 XVs -> 2 XUVs
C:0 T:18 1 XUs and 0 XVs -> 0 XUVs
C:0 T:19 3 XUs and 2 XVs -> 2 XUVs
C:0 T:20 16 XUs and 12 XVs -> 8 XUVs
C:0 T:21 19 XUs and 21 XVs -> 14 XUVs
C:0 T:22 32 XUs and 30 XVs -> 22 XUVs
C:0 T:23 16 XUs and 16 XVs -> 7 XUVs
134 XUVs total
97 collection wire objects
134 potential space points
Neighbour search...
294 tests to find 160 neighbours
Iterating with no regularization...
Begin: 343781
0 313934
1 313825
Now with regularization...
Begin: 225680
0 225667
Begin processing the 47th record. run: 249 subRun: 0 event: 412647 at 02-Aug-2025 18:35:40 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 14 XUs and 13 XVs -> 9 XUVs
C:0 T:1 13 XUs and 15 XVs -> 7 XUVs
C:0 T:2 24 XUs and 22 XVs -> 13 XUVs
C:0 T:3 12 XUs and 14 XVs -> 5 XUVs
C:0 T:4 0 XUs and 0 XVs -> 0 XUVs
C:0 T:5 3 XUs and 2 XVs -> 2 XUVs
C:0 T:6 6 XUs and 7 XVs -> 4 XUVs
C:0 T:7 7 XUs and 10 XVs -> 4 XUVs
C:0 T:8 6 XUs and 8 XVs -> 3 XUVs
C:0 T:9 8 XUs and 9 XVs -> 4 XUVs
C:0 T:10 5 XUs and 6 XVs -> 2 XUVs
C:0 T:11 2 XUs and 3 XVs -> 1 XUVs
C:0 T:12 3 XUs and 3 XVs -> 1 XUVs
C:0 T:13 10 XUs and 6 XVs -> 4 XUVs
C:0 T:14 11 XUs and 10 XVs -> 7 XUVs
C:0 T:15 10 XUs and 9 XVs -> 4 XUVs
C:0 T:16 7 XUs and 7 XVs -> 4 XUVs
C:0 T:17 2 XUs and 2 XVs -> 0 XUVs
C:0 T:18 3 XUs and 3 XVs -> 2 XUVs
C:0 T:19 7 XUs and 6 XVs -> 4 XUVs
C:0 T:20 4 XUs and 5 XVs -> 3 XUVs
C:0 T:21 16 XUs and 14 XVs -> 8 XUVs
C:0 T:22 23 XUs and 20 XVs -> 15 XUVs
C:0 T:23 10 XUs and 12 XVs -> 7 XUVs
113 XUVs total
89 collection wire objects
113 potential space points
Neighbour search...
223 tests to find 110 neighbours
Iterating with no regularization...
Begin: 349707
0 303454
1 303443
Now with regularization...
Begin: 229261
0 229254
Begin processing the 48th record. run: 249 subRun: 0 event: 412648 at 02-Aug-2025 18:35:41 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 19 XUs and 19 XVs -> 10 XUVs
C:0 T:1 18 XUs and 18 XVs -> 9 XUVs
C:0 T:2 25 XUs and 22 XVs -> 15 XUVs
C:0 T:3 24 XUs and 17 XVs -> 13 XUVs
C:0 T:4 0 XUs and 2 XVs -> 0 XUVs
C:0 T:5 3 XUs and 4 XVs -> 2 XUVs
C:0 T:6 6 XUs and 7 XVs -> 4 XUVs
C:0 T:7 5 XUs and 5 XVs -> 2 XUVs
C:0 T:8 1 XUs and 1 XVs -> 0 XUVs
C:0 T:9 5 XUs and 4 XVs -> 4 XUVs
C:0 T:10 8 XUs and 8 XVs -> 6 XUVs
C:0 T:11 13 XUs and 18 XVs -> 9 XUVs
C:0 T:12 5 XUs and 8 XVs -> 3 XUVs
C:0 T:13 7 XUs and 4 XVs -> 3 XUVs
C:0 T:14 8 XUs and 7 XVs -> 3 XUVs
C:0 T:15 5 XUs and 6 XVs -> 3 XUVs
C:0 T:16 4 XUs and 1 XVs -> 0 XUVs
C:0 T:17 5 XUs and 3 XVs -> 1 XUVs
C:0 T:18 9 XUs and 7 XVs -> 5 XUVs
C:0 T:19 7 XUs and 7 XVs -> 2 XUVs
C:0 T:20 24 XUs and 25 XVs -> 12 XUVs
C:0 T:21 26 XUs and 26 XVs -> 13 XUVs
C:0 T:22 19 XUs and 16 XVs -> 10 XUVs
C:0 T:23 19 XUs and 18 XVs -> 10 XUVs
139 XUVs total
103 collection wire objects
139 potential space points
Neighbour search...
305 tests to find 160 neighbours
Iterating with no regularization...
Begin: 372021
0 319687
1 319077
2 319074
Now with regularization...
Begin: 230416
0 230408
Begin processing the 49th record. run: 249 subRun: 0 event: 412649 at 02-Aug-2025 18:35:42 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 8 XUs and 15 XVs -> 7 XUVs
C:0 T:1 16 XUs and 19 XVs -> 7 XUVs
C:0 T:2 21 XUs and 18 XVs -> 12 XUVs
C:0 T:3 14 XUs and 12 XVs -> 6 XUVs
C:0 T:4 3 XUs and 1 XVs -> 1 XUVs
C:0 T:5 7 XUs and 5 XVs -> 4 XUVs
C:0 T:6 5 XUs and 7 XVs -> 3 XUVs
C:0 T:7 0 XUs and 0 XVs -> 0 XUVs
C:0 T:8 5 XUs and 6 XVs -> 3 XUVs
C:0 T:9 10 XUs and 10 XVs -> 7 XUVs
C:0 T:10 5 XUs and 10 XVs -> 4 XUVs
C:0 T:11 18 XUs and 15 XVs -> 9 XUVs
C:0 T:12 6 XUs and 5 XVs -> 3 XUVs
C:0 T:13 4 XUs and 4 XVs -> 3 XUVs
C:0 T:14 5 XUs and 3 XVs -> 3 XUVs
C:0 T:15 4 XUs and 5 XVs -> 2 XUVs
C:0 T:16 4 XUs and 3 XVs -> 3 XUVs
C:0 T:17 6 XUs and 5 XVs -> 2 XUVs
C:0 T:18 2 XUs and 3 XVs -> 0 XUVs
C:0 T:19 7 XUs and 3 XVs -> 2 XUVs
C:0 T:20 0 XUs and 4 XVs -> 0 XUVs
C:0 T:21 16 XUs and 22 XVs -> 12 XUVs
C:0 T:22 26 XUs and 22 XVs -> 13 XUVs
C:0 T:23 14 XUs and 13 XVs -> 8 XUVs
114 XUVs total
89 collection wire objects
114 potential space points
Neighbour search...
226 tests to find 108 neighbours
Iterating with no regularization...
Begin: 267066
0 225344
1 225140
Now with regularization...
Begin: 149719
0 149708
Begin processing the 50th record. run: 249 subRun: 0 event: 412650 at 02-Aug-2025 18:35:42 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 35 XUs and 37 XVs -> 22 XUVs
C:0 T:1 7 XUs and 9 XVs -> 2 XUVs
C:0 T:2 40 XUs and 35 XVs -> 22 XUVs
C:0 T:3 17 XUs and 15 XVs -> 10 XUVs
C:0 T:4 2 XUs and 1 XVs -> 1 XUVs
C:0 T:5 9 XUs and 9 XVs -> 6 XUVs
C:0 T:6 4 XUs and 5 XVs -> 3 XUVs
C:0 T:7 7 XUs and 9 XVs -> 5 XUVs
C:0 T:8 8 XUs and 5 XVs -> 4 XUVs
C:0 T:9 6 XUs and 6 XVs -> 5 XUVs
C:0 T:10 0 XUs and 3 XVs -> 0 XUVs
C:0 T:11 4 XUs and 3 XVs -> 2 XUVs
C:0 T:12 9 XUs and 12 XVs -> 5 XUVs
C:0 T:13 10 XUs and 10 XVs -> 8 XUVs
C:0 T:14 6 XUs and 7 XVs -> 4 XUVs
C:0 T:15 4 XUs and 6 XVs -> 4 XUVs
C:0 T:16 8 XUs and 14 XVs -> 7 XUVs
C:0 T:17 5 XUs and 6 XVs -> 2 XUVs
C:0 T:18 14 XUs and 18 XVs -> 10 XUVs
C:0 T:19 1 XUs and 4 XVs -> 1 XUVs
C:0 T:20 13 XUs and 13 XVs -> 9 XUVs
C:0 T:21 13 XUs and 11 XVs -> 7 XUVs
C:0 T:22 6 XUs and 10 XVs -> 5 XUVs
C:0 T:23 27 XUs and 25 XVs -> 13 XUVs
157 XUVs total
121 collection wire objects
157 potential space points
Neighbour search...
329 tests to find 170 neighbours
Iterating with no regularization...
Begin: 449233
0 409324
1 408946
Now with regularization...
Begin: 308952
0 308914
02-Aug-2025 18:35:45 CEST  Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_reco.root"
02-Aug-2025 18:35:45 CEST  Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0433047     0.0485576     0.0588999     0.0482276    0.00231738       50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000288779   0.000380385   0.000734806   0.000373434   7.84327e-05      50     
reco:gaushit:GausHitFinder                     0.0342304     0.0373393     0.040541      0.0373229    0.00115456       50     
reco:spsolve:SpacePointSolver                 0.00235939    0.00308948    0.00551564    0.00306407    0.000459874      50     
reco:hitfd:DisambigFromSpacePoints            0.000830158   0.00104744    0.00229579    0.00101713    0.000200994      50     
reco:rns:RandomNumberSaver                    1.4978e-05    2.05734e-05   0.000134617   1.70975e-05   1.65849e-05      50     
[art]:TriggerResults:TriggerResultInserter     7.163e-06    9.19326e-06   2.9246e-05    8.3715e-06    3.36119e-06      50     
end_path:out1:RootOutput                       2.815e-06    3.76076e-06   1.2503e-05    3.5665e-06    1.31739e-06      50     
end_path:out1:RootOutput(write)               0.00527884    0.00645899    0.00920687    0.00648251    0.000584257      50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1773.71 MB
  Peak resident set size usage (VmHWM): 893.53 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253.root 109341612
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4.root 1028991940
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim.root 2950010539
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_hist.root 12750
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_reco.root 2955027848
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_reco.root.ext.json 1578
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_reco.root.json 1821
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113212Z_gen_008253_supernova_g4_detsim_reco.root.temp.json 1821
justIN time: 2025-08-05 00:45:31 UTC       justIN version: 01.04.00