justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303330.11@dunegpschedd01.fnal.gov

Jobsub ID303330.11@dunegpschedd01.fnal.gov
Workflow ID12671
Stage ID3
User nameachatter@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 10:10:42
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod03
Last heartbeat2026-02-03 10:46:33
From worker nodeHostnamewm14.grid.hep.ph.ic.ac.uk
cpuinfoAMD EPYC 7742 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 10:11:32
Input filesusertests:prod_monte-carlo-012671-000011_gen_g4.root
JobscriptExit code0
Real time33m (2019s)
CPU time30m (1834s = 90%)
Max RSS bytes1660674048 (1583 MiB)
Outputting started2026-02-03 10:45:13
Output files
Finished2026-02-03 10:46:33
Saved logsjustin-logs:303330.11-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

auss"
FrameSaver: q=341544 n=46269 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=349797 n=44608 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 40th record. run: 210011 subRun: 0 event: 40 at 03-Feb-2026 10:37:33 GMT
SimDepoSetSource got 32353 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 32353 depos spanning: [0.420908, 0.455149]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=957180 n=106776 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=985668 n=101040 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 41st record. run: 210011 subRun: 0 event: 41 at 03-Feb-2026 10:38:22 GMT
SimDepoSetSource got 18293 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 18293 depos spanning: [0.886795, 0.907366]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=535287 n=51609 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=555609 n=48125 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 42nd record. run: 210011 subRun: 0 event: 42 at 03-Feb-2026 10:38:45 GMT
SimDepoSetSource got 1393 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 1393 depos spanning: [0.977198, 0.981858]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=32865 n=5197 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=35031.7 n=4833 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 43rd record. run: 210011 subRun: 0 event: 43 at 03-Feb-2026 10:39:01 GMT
SimDepoSetSource got 15908 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 15908 depos spanning: [0.468809, 0.663527]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=464463 n=58353 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=510738 n=46073 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 44th record. run: 210011 subRun: 0 event: 44 at 03-Feb-2026 10:40:12 GMT
SimDepoSetSource got 34185 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 34185 depos spanning: [0.577777, 0.613111]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=979628 n=99104 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=1.03279e+06 n=85729 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 45th record. run: 210011 subRun: 0 event: 45 at 03-Feb-2026 10:41:13 GMT
SimDepoSetSource got 13160 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 13160 depos spanning: [0.75754, 1.20463]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=376623 n=34095 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=414389 n=27687 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 46th record. run: 210011 subRun: 0 event: 46 at 03-Feb-2026 10:41:52 GMT
SimDepoSetSource got 23424 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 23424 depos spanning: [0.921263, 0.945566]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=672238 n=101421 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=691057 n=96662 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 47th record. run: 210011 subRun: 0 event: 47 at 03-Feb-2026 10:43:03 GMT
SimDepoSetSource got 3783 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3783 depos spanning: [0.891549, 0.928944]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=114709 n=21191 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=118717 n=19858 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 48th record. run: 210011 subRun: 0 event: 48 at 03-Feb-2026 10:43:37 GMT
SimDepoSetSource got 23978 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 23978 depos spanning: [0.776567, 0.799805]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=525373 n=76726 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=549560 n=72453 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 49th record. run: 210011 subRun: 0 event: 49 at 03-Feb-2026 10:44:34 GMT
SimDepoSetSource got 767 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 767 depos spanning: [0.957203, 0.959172]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=22929.8 n=2098 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=26524.2 n=1965 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 50th record. run: 210011 subRun: 0 event: 50 at 03-Feb-2026 10:44:51 GMT
SimDepoSetSource got 5219 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5219 depos spanning: [0.910791, 0.916807]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=146057 n=24872 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=150176 n=23723 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 10:45:12 GMT  Closed output file "prod_monte-carlo-012671-000011_gen_g4_detsim.root"
03-Feb-2026 10:45:12 GMT  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/a6/6b/prod_monte-carlo-012671-000011_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      15.7825       38.3711       93.5554       36.0967       19.0162        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000367052    0.0133884     0.272984     0.00196143     0.0423721       50     
simulate:tpcrawdecoder:WireCellToolkit          15.5329       37.8946       92.4993       35.5187       18.8324        50     
simulate:opdigi:OpDetDigitizerDUNE             0.0473336     0.235422      0.501555      0.226518      0.105802        50     
simulate:rns:RandomNumberSaver                2.2463e-05    6.12063e-05   0.000708927   4.4128e-05    9.56571e-05      50     
[art]:TriggerResults:TriggerResultInserter    1.2583e-05    1.86679e-05   6.2298e-05    1.7067e-05    6.85375e-06      50     
end_path:out1:RootOutput                       3.347e-06    6.05762e-06   2.7111e-05    4.4035e-06    3.54616e-06      50     
end_path:out1:RootOutput(write)                0.086102      0.226976      0.564435      0.188171      0.143385        50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3368.83 MB
  Peak resident set size usage (VmHWM): 1660.67 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r--. 1 job0003 job0003 209M Feb  3 10:45 prod_monte-carlo-012671-000011_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-012671-000011_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 04:47:36 UTC       justIN version: 01.06.00