Jobsub ID 213626.178@dunegpschedd01.fnal.gov
| Jobsub ID | 213626.178@dunegpschedd01.fnal.gov |
| Workflow ID | 8146 |
| Stage ID | 1 |
| User name | epennacc@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-09-27 10:41:43 |
| Site | UK_Glasgow |
| Entry | CLAS12_T3_UK_ScotGrid_GLA_ce04_scitok |
| Last heartbeat | 2025-09-27 14:21:57 |
| From worker node | Hostname | wn-h17-005.beowulf.cluster |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-27 11:57:21 |
| Input files | monte-carlo-008146-006128
|
| Jobscript | Exit code | 0 |
| Real time | 1h (5599s) |
| CPU time | 1h (5424s = 96%) |
| Max RSS bytes | 3514343424 (3351 MiB) |
| Outputting started | 2025-09-27 13:30:41 |
| Output files | |
| Finished | 2025-09-27 14:21:57 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
WireCell::Gen::IncoherentAddNoise : 0.97 sec
[14:23:18.040] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.95 sec
[14:23:18.040] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.94 sec
[14:23:18.040] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.91 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.91 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.88 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.87 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.79 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::DepoBagger : 0.77 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.7 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.69 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.62 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.61 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.61 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.6 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.6 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.57 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.56 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.54 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.51 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Retagger : 0.5 sec
[14:23:18.041] I [ timer ] Timer: wcls::FrameSaver : 0.48 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.47 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::Reframer : 0.47 sec
[14:23:18.041] I [ timer ] Timer: WireCell::Gen::DepoSetFanout : 0.44 sec
[14:23:18.041] I [ timer ] Timer: wcls::SimDepoSource : 0.21 sec
[14:23:18.041] I [ timer ] Timer: Total node execution : 345.240001603961 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
Begin processing the 10th record. run: 8146 subRun: 0 event: 61280 at 27-Sep-2025 14:23:46 BST
SimDepoSource got 141654 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSource: ready with 141654 depos spanning: [-2245.97, 2245.54]us
Retagger: tagging trace set: daq with 30720 traces, 0 summary
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 31.09 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 29.98 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 29.83 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 28.16 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 27.76 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 24.96 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 24.87 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 22.82 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 22.76 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 20.22 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 18.29 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::DepoTransform : 17.82 sec
[14:30:03.828] I [ timer ] Timer: wcls::SimChannelSink : 4.07 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.41 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Drifter : 1.29 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 1.17 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.16 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.16 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.15 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 1.15 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.15 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.13 sec
[14:30:03.828] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.12 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.12 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.1 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 1.07 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.06 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Digitizer : 1.05 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 1.01 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.97 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.96 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.96 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.96 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.93 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.92 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.9 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.9 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.76 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.75 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.73 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.7 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::DepoBagger : 0.68 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.65 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.63 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.63 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.62 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.6 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Retagger : 0.59 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.59 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.56 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::Reframer : 0.55 sec
[14:30:03.829] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.54 sec
[14:30:03.830] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.52 sec
[14:30:03.830] I [ timer ] Timer: WireCell::Gen::DepoSetFanout : 0.52 sec
[14:30:03.830] I [ timer ] Timer: wcls::FrameSaver : 0.51 sec
[14:30:03.830] I [ timer ] Timer: WireCell::Gen::Reframer : 0.5 sec
[14:30:03.830] I [ timer ] Timer: wcls::SimDepoSource : 0.19 sec
[14:30:03.830] I [ timer ] Timer: Total node execution : 340.2499992251396 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
27-Sep-2025 14:30:30 BST Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4_detsim.root"
27-Sep-2025 14:30:30 BST Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 377.328 397.42 409.297 399.476 9.47838 10
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00112843 0.00488849 0.00736109 0.00493399 0.00169143 10
simulate:tpcrawdecoder:WireCellToolkit 355.027 375.567 385.257 378.665 9.09636 10
simulate:opdigi:OpDetDigitizerDUNE 4.182 4.42984 4.79823 4.43291 0.169471 10
simulate:rns:RandomNumberSaver 4.0271e-05 9.09676e-05 0.00038163 6.36735e-05 9.77259e-05 10
[art]:TriggerResults:TriggerResultInserter 1.9072e-05 3.35225e-05 8.3877e-05 3.1292e-05 1.763e-05 10
end_path:out1:RootOutput 6.512e-06 1.17154e-05 3.6142e-05 9.713e-06 8.28172e-06 10
end_path:out1:RootOutput(write) 15.6144 17.4174 19.2346 17.2938 1.1379 10
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5173.78 MB
Peak resident set size usage (VmHWM): 3514.34 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128.root 21990969
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4.root 180737152
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4_detsim.root 2352418953
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4_detsim.root.ext.json 1561
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4_detsim.root.json 1827
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4_detsim.root.temp.json 1827
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T115731Z_gen_006128_supernova_g4_detsim_hist.root 519
