Jobsub ID 281318.35@dunegpschedd01.fnal.gov
| Jobsub ID | 281318.35@dunegpschedd01.fnal.gov |
| Workflow ID | 12080 |
| Stage ID | 3 |
| User name | usher@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 28800 (8 hours) |
| Submitted time | 2026-01-18 19:24:11 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep208 |
| Last heartbeat | 2026-01-21 16:07:48 |
| From worker node | Hostname | heplnc157.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2026-01-18 21:13:35 |
| Input files | usertests:mpvmpr_monte-carlo-012080-001197_gen_g4.root
|
| Outputting started | 2026-01-18 23:04:12 |
| Output files | |
| Finished | 2026-01-21 16:54:39 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
nsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 40th record. run: 211197 subRun: 0 event: 40 at 18-Jan-2026 22:42:37 GMT
SimDepoSetSource got 63955 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 63955 depos spanning: [0.23584, 9.21237]us
Retagger: tagging trace set: wiener with 17920 traces, 17920 summary
Retagger: tagging trace set: gauss with 17920 traces, 0 summary
wclsFrameSaver: saving 17920 traces tagged "gauss"
FrameSaver: q=1.6823e+06 n=129338 tag=gauss
wclsFrameSaver: saving 17920 traces tagged "wiener"
FrameSaver: q=1.90839e+06 n=119961 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 41st record. run: 211197 subRun: 0 event: 41 at 18-Jan-2026 22:44:13 GMT
SimDepoSetSource got 111007 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 111007 depos spanning: [4.09477, 13.5599]us
Retagger: tagging trace set: wiener with 25600 traces, 25600 summary
Retagger: tagging trace set: gauss with 25600 traces, 0 summary
wclsFrameSaver: saving 25600 traces tagged "gauss"
FrameSaver: q=3.00201e+06 n=332206 tag=gauss
wclsFrameSaver: saving 25600 traces tagged "wiener"
FrameSaver: q=3.22857e+06 n=303398 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 42nd record. run: 211197 subRun: 0 event: 42 at 18-Jan-2026 22:46:53 GMT
SimDepoSetSource got 61341 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 61341 depos spanning: [0.085457, 8.89873]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=1.38562e+06 n=143137 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=1.49981e+06 n=132288 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 43rd record. run: 211197 subRun: 0 event: 43 at 18-Jan-2026 22:47:56 GMT
SimDepoSetSource got 98888 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 98888 depos spanning: [0.384526, 7.11818]us
Retagger: tagging trace set: wiener with 15360 traces, 15360 summary
Retagger: tagging trace set: gauss with 15360 traces, 0 summary
wclsFrameSaver: saving 15360 traces tagged "gauss"
FrameSaver: q=2.55608e+06 n=268140 tag=gauss
wclsFrameSaver: saving 15360 traces tagged "wiener"
FrameSaver: q=2.76021e+06 n=246410 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 44th record. run: 211197 subRun: 0 event: 44 at 18-Jan-2026 22:49:47 GMT
SimDepoSetSource got 66555 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 66555 depos spanning: [5.85897, 9.24181]us
Retagger: tagging trace set: wiener with 20480 traces, 20480 summary
Retagger: tagging trace set: gauss with 20480 traces, 0 summary
wclsFrameSaver: saving 20480 traces tagged "gauss"
FrameSaver: q=1.94572e+06 n=249861 tag=gauss
wclsFrameSaver: saving 20480 traces tagged "wiener"
FrameSaver: q=2.07749e+06 n=215099 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 45th record. run: 211197 subRun: 0 event: 45 at 18-Jan-2026 22:51:51 GMT
SimDepoSetSource got 217214 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 217214 depos spanning: [0.620363, 8.81545]us
Retagger: tagging trace set: wiener with 15360 traces, 15360 summary
Retagger: tagging trace set: gauss with 15360 traces, 0 summary
wclsFrameSaver: saving 15360 traces tagged "gauss"
FrameSaver: q=4.93277e+06 n=428923 tag=gauss
wclsFrameSaver: saving 15360 traces tagged "wiener"
FrameSaver: q=5.42195e+06 n=388557 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 46th record. run: 211197 subRun: 0 event: 46 at 18-Jan-2026 22:54:10 GMT
SimDepoSetSource got 134878 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 134878 depos spanning: [2.59521, 5.9993]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=2.7119e+06 n=221365 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=2.94738e+06 n=202126 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 47th record. run: 211197 subRun: 0 event: 47 at 18-Jan-2026 22:55:21 GMT
SimDepoSetSource got 217055 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 217055 depos spanning: [2.10725, 9.62571]us
Retagger: tagging trace set: wiener with 20480 traces, 20480 summary
Retagger: tagging trace set: gauss with 20480 traces, 0 summary
wclsFrameSaver: saving 20480 traces tagged "gauss"
FrameSaver: q=5.33595e+06 n=439313 tag=gauss
wclsFrameSaver: saving 20480 traces tagged "wiener"
FrameSaver: q=5.77579e+06 n=392579 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 48th record. run: 211197 subRun: 0 event: 48 at 18-Jan-2026 22:58:16 GMT
SimDepoSetSource got 84315 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 84315 depos spanning: [1.75708, 9.58266]us
Retagger: tagging trace set: wiener with 20480 traces, 20480 summary
Retagger: tagging trace set: gauss with 20480 traces, 0 summary
wclsFrameSaver: saving 20480 traces tagged "gauss"
FrameSaver: q=2.3555e+06 n=247034 tag=gauss
wclsFrameSaver: saving 20480 traces tagged "wiener"
FrameSaver: q=2.52831e+06 n=208838 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 49th record. run: 211197 subRun: 0 event: 49 at 18-Jan-2026 23:00:29 GMT
SimDepoSetSource got 113502 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 113502 depos spanning: [3.11011, 4.56487]us
Retagger: tagging trace set: wiener with 17920 traces, 17920 summary
Retagger: tagging trace set: gauss with 17920 traces, 0 summary
wclsFrameSaver: saving 17920 traces tagged "gauss"
FrameSaver: q=1.97494e+06 n=193909 tag=gauss
wclsFrameSaver: saving 17920 traces tagged "wiener"
FrameSaver: q=2.18224e+06 n=167822 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 50th record. run: 211197 subRun: 0 event: 50 at 18-Jan-2026 23:02:42 GMT
SimDepoSetSource got 101043 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 101043 depos spanning: [0.466152, 1.64535]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=2.18876e+06 n=143355 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=2.31058e+06 n=135138 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
18-Jan-2026 23:04:11 GMT Closed output file "mpvmpr_monte-carlo-012080-001197_gen_g4_detsim.root"
18-Jan-2026 23:04:11 GMT Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/98/b3/mpvmpr_monte-carlo-012080-001197_gen_g4.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 46.1302 130.519 287.452 122.11 55.9359 50
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000732536 0.00613796 0.0456712 0.00441216 0.00705236 50
simulate:tpcrawdecoder:WireCellToolkit 45.6378 128.349 282.914 120.182 54.9391 50
simulate:opdigi:OpDetDigitizerDUNE 0.207809 0.96612 2.30425 0.891109 0.532856 50
simulate:rns:RandomNumberSaver 2.7171e-05 5.10696e-05 0.000245351 4.44035e-05 2.98619e-05 50
[art]:TriggerResults:TriggerResultInserter 1.3124e-05 2.17557e-05 6.7717e-05 1.9692e-05 9.62849e-06 50
end_path:out1:RootOutput 3.125e-06 6.76138e-06 2.2943e-05 6.5125e-06 2.62511e-06 50
end_path:out1:RootOutput(write) 0.211333 1.1972 3.64401 0.979778 0.809735 50
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4439.57 MB
Peak resident set size usage (VmHWM): 2727.27 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 pltdune04 pltdune 2.6G Jan 18 23:04 mpvmpr_monte-carlo-012080-001197_gen_g4_detsim.root
Wrote processed DID: usertests:mpvmpr_monte-carlo-012080-001197_gen_g4.root
=== DETSIM job completed successfully ===