justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20437.12@dunegpschedd01.fnal.gov

Jobsub ID20437.12@dunegpschedd01.fnal.gov
Workflow ID249
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-02 11:31:14
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-08-02 17:04:45
From worker nodeHostnametd810.pic.es
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-02 11:32:25
Input filesmonte-carlo-000249-008420
JobscriptExit code0
Real time5h (19833s)
CPU time5h (19304s = 97%)
Max RSS bytes1959120896 (1868 MiB)
Outputting started2025-08-02 17:03:00
Output filesfardet-hd:prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_reco.root
Finished2025-08-02 17:04:45
Saved logsjustin-logs:20437.12-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

92
0 279851
1 279591
Now with regularization...
Begin: 200575
0 200566
Begin processing the 46th record. run: 249 subRun: 0 event: 420996 at 02-Aug-2025 19:02:52 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 17 XUs and 20 XVs -> 11 XUVs
C:0 T:1 9 XUs and 9 XVs -> 3 XUVs
C:0 T:2 24 XUs and 23 XVs -> 14 XUVs
C:0 T:3 23 XUs and 24 XVs -> 12 XUVs
C:0 T:4 1 XUs and 1 XVs -> 0 XUVs
C:0 T:5 5 XUs and 6 XVs -> 4 XUVs
C:0 T:6 7 XUs and 7 XVs -> 4 XUVs
C:0 T:7 7 XUs and 5 XVs -> 3 XUVs
C:0 T:8 9 XUs and 11 XVs -> 7 XUVs
C:0 T:9 0 XUs and 1 XVs -> 0 XUVs
C:0 T:10 5 XUs and 3 XVs -> 1 XUVs
C:0 T:11 12 XUs and 16 XVs -> 9 XUVs
C:0 T:12 5 XUs and 6 XVs -> 2 XUVs
C:0 T:13 1 XUs and 0 XVs -> 0 XUVs
C:0 T:14 8 XUs and 10 XVs -> 7 XUVs
C:0 T:15 5 XUs and 8 XVs -> 5 XUVs
C:0 T:16 19 XUs and 19 XVs -> 12 XUVs
C:0 T:17 3 XUs and 2 XVs -> 2 XUVs
C:0 T:18 6 XUs and 7 XVs -> 4 XUVs
C:0 T:19 2 XUs and 2 XVs -> 1 XUVs
C:0 T:20 29 XUs and 24 XVs -> 15 XUVs
C:0 T:21 21 XUs and 20 XVs -> 10 XUVs
C:0 T:22 15 XUs and 15 XVs -> 8 XUVs
C:0 T:23 29 XUs and 31 XVs -> 22 XUVs
156 XUVs total
119 collection wire objects
156 potential space points
Neighbour search...
332 tests to find 172 neighbours
Iterating with no regularization...
Begin: 447586
0 388462
1 388261
Now with regularization...
Begin: 283882
0 283871
Begin processing the 47th record. run: 249 subRun: 0 event: 420997 at 02-Aug-2025 19:02:53 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 24 XUs and 23 XVs -> 16 XUVs
C:0 T:1 27 XUs and 31 XVs -> 20 XUVs
C:0 T:2 11 XUs and 11 XVs -> 7 XUVs
C:0 T:3 8 XUs and 7 XVs -> 5 XUVs
C:0 T:4 10 XUs and 7 XVs -> 4 XUVs
C:0 T:5 2 XUs and 1 XVs -> 0 XUVs
C:0 T:6 11 XUs and 8 XVs -> 7 XUVs
C:0 T:7 7 XUs and 6 XVs -> 4 XUVs
C:0 T:8 2 XUs and 2 XVs -> 1 XUVs
C:0 T:9 5 XUs and 5 XVs -> 4 XUVs
C:0 T:10 4 XUs and 4 XVs -> 1 XUVs
C:0 T:11 6 XUs and 5 XVs -> 3 XUVs
C:0 T:12 9 XUs and 7 XVs -> 4 XUVs
C:0 T:13 0 XUs and 0 XVs -> 0 XUVs
C:0 T:14 8 XUs and 11 XVs -> 6 XUVs
C:0 T:15 2 XUs and 4 XVs -> 2 XUVs
C:0 T:16 0 XUs and 0 XVs -> 0 XUVs
C:0 T:17 9 XUs and 7 XVs -> 5 XUVs
C:0 T:18 1 XUs and 2 XVs -> 0 XUVs
C:0 T:19 4 XUs and 4 XVs -> 3 XUVs
C:0 T:20 14 XUs and 15 XVs -> 6 XUVs
C:0 T:21 21 XUs and 21 XVs -> 9 XUVs
C:0 T:22 12 XUs and 14 XVs -> 5 XUVs
C:0 T:23 13 XUs and 19 XVs -> 9 XUVs
121 XUVs total
97 collection wire objects
121 potential space points
Neighbour search...
225 tests to find 104 neighbours
Iterating with no regularization...
Begin: 373256
0 332162
1 331724
2 331724
Now with regularization...
Begin: 248917
0 248914
Begin processing the 48th record. run: 249 subRun: 0 event: 420998 at 02-Aug-2025 19:02:54 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 11 XUs and 12 XVs -> 6 XUVs
C:0 T:1 29 XUs and 28 XVs -> 19 XUVs
C:0 T:2 19 XUs and 16 XVs -> 12 XUVs
C:0 T:3 20 XUs and 16 XVs -> 10 XUVs
C:0 T:4 11 XUs and 10 XVs -> 6 XUVs
C:0 T:5 9 XUs and 8 XVs -> 5 XUVs
C:0 T:6 10 XUs and 11 XVs -> 6 XUVs
C:0 T:7 9 XUs and 8 XVs -> 4 XUVs
C:0 T:8 10 XUs and 8 XVs -> 3 XUVs
C:0 T:9 2 XUs and 2 XVs -> 1 XUVs
C:0 T:10 6 XUs and 7 XVs -> 3 XUVs
C:0 T:11 19 XUs and 18 XVs -> 13 XUVs
C:0 T:12 6 XUs and 4 XVs -> 3 XUVs
C:0 T:13 1 XUs and 0 XVs -> 0 XUVs
C:0 T:14 6 XUs and 10 XVs -> 4 XUVs
C:0 T:15 0 XUs and 0 XVs -> 0 XUVs
C:0 T:16 0 XUs and 0 XVs -> 0 XUVs
C:0 T:17 6 XUs and 7 XVs -> 4 XUVs
C:0 T:18 5 XUs and 6 XVs -> 3 XUVs
C:0 T:19 8 XUs and 6 XVs -> 4 XUVs
C:0 T:20 10 XUs and 14 XVs -> 5 XUVs
C:0 T:21 14 XUs and 15 XVs -> 9 XUVs
C:0 T:22 15 XUs and 17 XVs -> 8 XUVs
C:0 T:23 19 XUs and 25 XVs -> 14 XUVs
142 XUVs total
106 collection wire objects
142 potential space points
Neighbour search...
276 tests to find 134 neighbours
Iterating with no regularization...
Begin: 415113
0 375219
1 375214
Now with regularization...
Begin: 286546
0 286540
Begin processing the 49th record. run: 249 subRun: 0 event: 420999 at 02-Aug-2025 19:02:54 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 35 XUs and 33 XVs -> 16 XUVs
C:0 T:1 24 XUs and 32 XVs -> 14 XUVs
C:0 T:2 21 XUs and 22 XVs -> 14 XUVs
C:0 T:3 17 XUs and 16 XVs -> 11 XUVs
C:0 T:4 4 XUs and 4 XVs -> 2 XUVs
C:0 T:5 6 XUs and 4 XVs -> 2 XUVs
C:0 T:6 6 XUs and 5 XVs -> 3 XUVs
C:0 T:7 5 XUs and 7 XVs -> 2 XUVs
C:0 T:8 9 XUs and 7 XVs -> 5 XUVs
C:0 T:9 8 XUs and 8 XVs -> 5 XUVs
C:0 T:10 9 XUs and 8 XVs -> 5 XUVs
C:0 T:11 4 XUs and 5 XVs -> 3 XUVs
C:0 T:12 9 XUs and 10 XVs -> 6 XUVs
C:0 T:13 12 XUs and 10 XVs -> 6 XUVs
C:0 T:14 12 XUs and 11 XVs -> 8 XUVs
C:0 T:15 8 XUs and 12 XVs -> 7 XUVs
C:0 T:16 3 XUs and 2 XVs -> 2 XUVs
C:0 T:17 6 XUs and 7 XVs -> 3 XUVs
C:0 T:18 1 XUs and 0 XVs -> 0 XUVs
C:0 T:19 1 XUs and 0 XVs -> 0 XUVs
C:0 T:20 21 XUs and 15 XVs -> 11 XUVs
C:0 T:21 21 XUs and 19 XVs -> 11 XUVs
C:0 T:22 14 XUs and 15 XVs -> 7 XUVs
C:0 T:23 22 XUs and 22 XVs -> 10 XUVs
153 XUVs total
117 collection wire objects
153 potential space points
Neighbour search...
309 tests to find 156 neighbours
Iterating with no regularization...
Begin: 412735
0 372367
1 372264
Now with regularization...
Begin: 266758
0 266752
Begin processing the 50th record. run: 249 subRun: 0 event: 421000 at 02-Aug-2025 19:02:55 CEST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 31 XUs and 24 XVs -> 17 XUVs
C:0 T:1 23 XUs and 23 XVs -> 11 XUVs
C:0 T:2 10 XUs and 16 XVs -> 7 XUVs
C:0 T:3 24 XUs and 26 XVs -> 14 XUVs
C:0 T:4 7 XUs and 6 XVs -> 5 XUVs
C:0 T:5 11 XUs and 12 XVs -> 8 XUVs
C:0 T:6 1 XUs and 1 XVs -> 1 XUVs
C:0 T:7 23 XUs and 22 XVs -> 15 XUVs
C:0 T:8 5 XUs and 6 XVs -> 2 XUVs
C:0 T:9 15 XUs and 11 XVs -> 6 XUVs
C:0 T:10 9 XUs and 8 XVs -> 4 XUVs
C:0 T:11 5 XUs and 5 XVs -> 3 XUVs
C:0 T:12 4 XUs and 6 XVs -> 3 XUVs
C:0 T:13 2 XUs and 5 XVs -> 2 XUVs
C:0 T:14 6 XUs and 7 XVs -> 4 XUVs
C:0 T:15 10 XUs and 13 XVs -> 8 XUVs
C:0 T:16 9 XUs and 11 XVs -> 8 XUVs
C:0 T:17 7 XUs and 6 XVs -> 2 XUVs
C:0 T:18 4 XUs and 4 XVs -> 3 XUVs
C:0 T:19 4 XUs and 4 XVs -> 0 XUVs
C:0 T:20 16 XUs and 13 XVs -> 8 XUVs
C:0 T:21 8 XUs and 9 XVs -> 4 XUVs
C:0 T:22 11 XUs and 12 XVs -> 6 XUVs
C:0 T:23 15 XUs and 13 XVs -> 9 XUVs
150 XUVs total
115 collection wire objects
150 potential space points
Neighbour search...
298 tests to find 148 neighbours
Iterating with no regularization...
Begin: 496623
0 441095
1 441016
Now with regularization...
Begin: 342376
0 342369
02-Aug-2025 19:02:56 CEST  Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_reco.root"
02-Aug-2025 19:02:56 CEST  Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0485649     0.0530733     0.062032      0.0529408    0.00245021       50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000278783   0.000360607   0.000734498   0.000353819   7.65245e-05      50     
reco:gaushit:GausHitFinder                     0.0386309     0.0409615     0.0435072     0.0410282    0.00107393       50     
reco:spsolve:SpacePointSolver                  0.0025734    0.00338098     0.0117339    0.00321302     0.0012701       50     
reco:hitfd:DisambigFromSpacePoints            0.000876769   0.00105053    0.00214785    0.00103832    0.000173771      50     
reco:rns:RandomNumberSaver                    1.3721e-05    1.70084e-05   0.000108351   1.51305e-05   1.3067e-05       50     
[art]:TriggerResults:TriggerResultInserter     7.33e-06     8.44712e-06   2.8221e-05     7.956e-06    2.85272e-06      50     
end_path:out1:RootOutput                       2.37e-06     2.85712e-06    1.139e-05     2.69e-06     1.23565e-06      50     
end_path:out1:RootOutput(write)               0.00567056    0.00711098    0.00876941    0.00704653    0.00059994       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1773.71 MB
  Peak resident set size usage (VmHWM): 882.549 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420.root 109389551
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4.root 1030826649
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim.root 2956919589
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_hist.root 12750
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_reco.root 2961883562
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_reco.root.ext.json 1578
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_reco.root.json 1821
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250802T113231Z_gen_008420_supernova_g4_detsim_reco.root.temp.json 1821
justIN time: 2025-08-05 02:44:27 UTC       justIN version: 01.04.00