Jobsub ID 214584.33@dunegpschedd01.fnal.gov
| Jobsub ID | 214584.33@dunegpschedd01.fnal.gov |
| Workflow ID | 8202 |
| Stage ID | 1 |
| User name | epennacc@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-09-29 12:22:17 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc03 |
| Last heartbeat | 2025-09-29 13:13:38 |
| From worker node | Hostname | wn-la-14.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2025-09-29 12:23:35 |
| Input files | monte-carlo-008202-008734
|
| Outputting started | 2025-09-29 13:09:13 |
| Output files | |
| Finished | 2025-09-29 13:50:41 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
1] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.62 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.62 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.61 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.6 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.59 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.58 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Drifter : 0.46 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.45 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Retagger : 0.44 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.42 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.41 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.41 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.37 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.37 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::DepoBagger : 0.37 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.37 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.37 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.35 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.34 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.34 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.34 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.33 sec
[15:05:12.591] I [ timer ] Timer: wcls::FrameSaver : 0.32 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::DepoSetFanout : 0.32 sec
[15:05:12.591] I [ timer ] Timer: WireCell::Gen::Reframer : 0.29 sec
[15:05:12.591] I [ timer ] Timer: wcls::SimDepoSource : 0.08 sec
[15:05:12.591] I [ timer ] Timer: Total node execution : 174.51000095903873 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
Begin processing the 10th record. run: 8202 subRun: 0 event: 87340 at 29-Sep-2025 15:05:28 CEST
SimDepoSource got 81852 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSource: ready with 81852 depos spanning: [-2245.38, 2244.75]us
Retagger: tagging trace set: daq with 30720 traces, 0 summary
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 16.04 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 15.92 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 14.88 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 14.86 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 14.72 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 14.58 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 14.46 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.62 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.44 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.28 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 9.46 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::DepoTransform : 5.94 sec
[15:08:46.768] I [ timer ] Timer: wcls::SimChannelSink : 1.19 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.83 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.79 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.79 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.77 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.76 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.76 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.76 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.76 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.76 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.75 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.72 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.71 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.7 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.68 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.68 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.67 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.67 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.65 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.65 sec
[15:08:46.768] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.61 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.61 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.6 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.59 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.56 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Drifter : 0.5 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.5 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.47 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.47 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.46 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.46 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Retagger : 0.42 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.41 sec
[15:08:46.769] I [ timer ] Timer: wcls::FrameSaver : 0.4 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.39 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.38 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.38 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.38 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.37 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::DepoSetFanout : 0.35 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.33 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::DepoBagger : 0.32 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.31 sec
[15:08:46.769] I [ timer ] Timer: WireCell::Gen::Reframer : 0.26 sec
[15:08:46.769] I [ timer ] Timer: wcls::SimDepoSource : 0.14 sec
[15:08:46.769] I [ timer ] Timer: Total node execution : 177.92000029981136 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
29-Sep-2025 15:09:03 CEST Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4_detsim.root"
29-Sep-2025 15:09:03 CEST Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 171.043 199.357 224.056 203.569 17.8856 10
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00132946 0.00406984 0.00531752 0.00435573 0.00115386 10
simulate:tpcrawdecoder:WireCellToolkit 162.135 188.038 212.133 191.596 16.9313 10
simulate:opdigi:OpDetDigitizerDUNE 1.88814 2.37498 2.63172 2.49562 0.261287 10
simulate:rns:RandomNumberSaver 2.8664e-05 5.49722e-05 0.000230345 3.2864e-05 5.88539e-05 10
[art]:TriggerResults:TriggerResultInserter 1.1057e-05 1.63818e-05 4.4217e-05 1.40605e-05 9.36835e-06 10
end_path:out1:RootOutput 3.946e-06 6.1401e-06 1.956e-05 4.8425e-06 4.489e-06 10
end_path:out1:RootOutput(write) 7.01491 8.9403 9.57527 9.39353 0.955961 10
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4964.27 MB
Peak resident set size usage (VmHWM): 3315.45 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734.root 22126037
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4.root 131087696
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4_detsim.root 2142962179
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4_detsim.root.ext.json 1575
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4_detsim.root.json 1841
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4_detsim.root.temp.json 1841
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250929T122345Z_gen_008734_supernova_g4_detsim_hist.root 519
