Jobsub ID 208650.177@dunegpschedd02.fnal.gov
| Jobsub ID | 208650.177@dunegpschedd02.fnal.gov |
| Workflow ID | 8199 |
| Stage ID | 1 |
| User name | epennacc@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-09-28 21:10:33 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2025-09-28 22:02:50 |
| From worker node | Hostname | dunegli-7053642-0-fnpc17007.fnal.gov |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2025-09-28 21:21:01 |
| Input files | monte-carlo-008199-001659
|
| Outputting started | 2025-09-28 22:02:50 |
| Output files | |
| Finished | 2025-09-28 22:40:15 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
: WireCell::Gen::IncoherentAddNoise : 0.46 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.45 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.45 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.44 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.44 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.41 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.4 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.34 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::DepoBagger : 0.33 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.3 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.3 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.3 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.27 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.27 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.26 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.23 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.22 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.2 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.2 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.19 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::DepoSetFanout : 0.19 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.18 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Reframer : 0.17 sec
[21:59:55.999] I [ timer ] Timer: WireCell::Gen::Retagger : 0.14 sec
[21:59:55.999] I [ timer ] Timer: wcls::FrameSaver : 0.13 sec
[21:59:55.999] I [ timer ] Timer: wcls::SimDepoSource : 0.06 sec
[21:59:55.999] I [ timer ] Timer: Total node execution : 122.1200002245605 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
Begin processing the 10th record. run: 8199 subRun: 0 event: 16590 at 28-Sep-2025 22:00:08 UTC
SimDepoSource got 81448 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSource: ready with 81448 depos spanning: [-2244.39, 2245.07]us
Retagger: tagging trace set: daq with 30720 traces, 0 summary
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.72 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.59 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.32 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.24 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.21 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.12 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 10.03 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 9.57 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 9.43 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 9.41 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 6.11 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoTransform : 5.82 sec
[22:02:35.049] I [ timer ] Timer: wcls::SimChannelSink : 1.92 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Drifter : 0.73 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.72 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.71 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.7 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.69 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.69 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.66 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.63 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.63 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.61 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.61 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.61 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.59 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.58 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.57 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.55 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.54 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.52 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.48 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.47 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.47 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.45 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.44 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.43 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::IncoherentAddNoise : 0.42 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.38 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.33 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.32 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoBagger : 0.32 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.31 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.31 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.28 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.27 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.27 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::DepoSetFanout : 0.27 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.25 sec
[22:02:35.049] I [ timer ] Timer: wcls::FrameSaver : 0.24 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.24 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.23 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.23 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.22 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Reframer : 0.22 sec
[22:02:35.049] I [ timer ] Timer: WireCell::Gen::Retagger : 0.17 sec
[22:02:35.049] I [ timer ] Timer: wcls::SimDepoSource : 0.13 sec
[22:02:35.049] I [ timer ] Timer: Total node execution : 133.97999967634678 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
28-Sep-2025 22:02:48 UTC Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4_detsim.root"
28-Sep-2025 22:02:48 UTC Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 141.696 160.713 195.853 156.675 16.4401 10
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00129685 0.00767035 0.0568108 0.00220366 0.0163922 10
simulate:tpcrawdecoder:WireCellToolkit 131.665 148.435 181.127 146.939 14.0841 10
simulate:opdigi:OpDetDigitizerDUNE 1.49908 1.62564 2.17141 1.57079 0.184432 10
simulate:rns:RandomNumberSaver 3.4867e-05 6.12917e-05 0.00024803 4.0201e-05 6.24667e-05 10
[art]:TriggerResults:TriggerResultInserter 1.4875e-05 1.95206e-05 4.1333e-05 1.66935e-05 7.77934e-06 10
end_path:out1:RootOutput 5.544e-06 7.7856e-06 2.1129e-05 6.411e-06 4.49259e-06 10
end_path:out1:RootOutput(write) 7.92029 10.6441 25.3453 8.52257 5.06327 10
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4971.84 MB
Peak resident set size usage (VmHWM): 3320.03 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659.root 22086642
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4.root 131080726
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4_detsim.root 2143846324
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4_detsim.root.ext.json 1575
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4_detsim.root.json 1841
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4_detsim.root.temp.json 1841
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250928T212105Z_gen_001659_supernova_g4_detsim_hist.root 519
