justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297770.5@dunegpschedd02.fnal.gov

Jobsub ID297770.5@dunegpschedd02.fnal.gov
Workflow ID12686
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 15:58:04
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2026-02-03 16:21:38
From worker nodeHostnamedune001-1829973.0-lcg2695.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 15:58:24
Input filesusertests:prod_monte-carlo-012686-000019_gen_g4.root
JobscriptExit code0
Real time21m (1304s)
CPU time20m (1233s = 94%)
Max RSS bytes1740865536 (1660 MiB)
Outputting started2026-02-03 16:20:09
Output files
Finished2026-02-03 16:21:38
Saved logsjustin-logs:297770.5-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

clsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=295848 n=45070 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=306557 n=43883 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 40th record. run: 210019 subRun: 0 event: 40 at 03-Feb-2026 16:16:05 UTC
SimDepoSetSource got 16109 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 16109 depos spanning: [0.853716, 0.870971]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=457357 n=70701 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=475827 n=68269 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 41st record. run: 210019 subRun: 0 event: 41 at 03-Feb-2026 16:16:37 UTC
SimDepoSetSource got 101 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 101 depos spanning: [0.946366, 0.946499]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=3671.66 n=1480 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=4836.63 n=1434 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 42nd record. run: 210019 subRun: 0 event: 42 at 03-Feb-2026 16:16:46 UTC
SimDepoSetSource got 538 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 538 depos spanning: [0.277609, 0.45604]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=23311.8 n=2797 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=27290.4 n=2572 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 43rd record. run: 210019 subRun: 0 event: 43 at 03-Feb-2026 16:16:56 UTC
SimDepoSetSource got 11115 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 11115 depos spanning: [0.0934351, 0.104478]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=323456 n=48893 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=347598 n=40184 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 44th record. run: 210019 subRun: 0 event: 44 at 03-Feb-2026 16:17:28 UTC
SimDepoSetSource got 25932 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 25932 depos spanning: [0.96412, 1.55854]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=752741 n=76993 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=775952 n=69420 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 45th record. run: 210019 subRun: 0 event: 45 at 03-Feb-2026 16:18:07 UTC
SimDepoSetSource got 9840 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 9840 depos spanning: [0.747547, 0.759178]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=192609 n=32557 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=199735 n=32003 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 46th record. run: 210019 subRun: 0 event: 46 at 03-Feb-2026 16:18:19 UTC
SimDepoSetSource got 11025 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 11025 depos spanning: [0.238589, 0.252221]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=341322 n=44941 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=363319 n=37631 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 47th record. run: 210019 subRun: 0 event: 47 at 03-Feb-2026 16:18:41 UTC
SimDepoSetSource got 18360 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 18360 depos spanning: [0.492104, 0.511807]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=527649 n=76812 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=546990 n=71222 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 48th record. run: 210019 subRun: 0 event: 48 at 03-Feb-2026 16:19:07 UTC
SimDepoSetSource got 1494 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 1494 depos spanning: [0.191757, 0.195372]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=35493.3 n=7285 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=37339.1 n=7095 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 49th record. run: 210019 subRun: 0 event: 49 at 03-Feb-2026 16:19:17 UTC
SimDepoSetSource got 15932 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 15932 depos spanning: [0.933309, 0.950429]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=453181 n=52228 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=464239 n=49267 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 50th record. run: 210019 subRun: 0 event: 50 at 03-Feb-2026 16:19:42 UTC
SimDepoSetSource got 18802 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 18802 depos spanning: [0.685835, 0.702277]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=518970 n=68788 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=537676 n=66491 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 16:20:07 UTC  Closed output file "prod_monte-carlo-012686-000019_gen_g4_detsim.root"
03-Feb-2026 16:20:07 UTC  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/usertests/1b/d0/prod_monte-carlo-012686-000019_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      9.23199       24.8374       64.525        23.889        12.5469        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000585672   0.00849506     0.104222     0.00234256     0.0193499       50     
simulate:tpcrawdecoder:WireCellToolkit          9.17593       24.5185       63.979        23.5569       12.4582        50     
simulate:opdigi:OpDetDigitizerDUNE             0.0106937      0.18352      0.391571      0.172604      0.0705519       50     
simulate:rns:RandomNumberSaver                2.7511e-05    4.77626e-05   0.000329517   4.0541e-05    4.33842e-05      50     
[art]:TriggerResults:TriggerResultInserter    1.2469e-05    2.03597e-05   7.1798e-05    1.93895e-05   9.62331e-06      50     
end_path:out1:RootOutput                       3.195e-06    7.4334e-06    3.2879e-05     6.72e-06     5.81121e-06      50     
end_path:out1:RootOutput(write)                0.0398558     0.126212       0.25026      0.119636      0.0497395       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3665.24 MB
  Peak resident set size usage (VmHWM): 1740.87 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 242M Feb  3 16:20 prod_monte-carlo-012686-000019_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-012686-000019_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 04:27:01 UTC       justIN version: 01.06.00