justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297635.0@dunegpschedd02.fnal.gov

Jobsub ID297635.0@dunegpschedd02.fnal.gov
Workflow ID12672
Stage ID3
User namelmcmahon@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 10:58:45
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2026-02-03 11:23:36
From worker nodeHostnamehnode76.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 10:59:32
Input filesusertests:prod_monte-carlo-012672-000008_gen_g4.root
JobscriptExit code0
Real time23m (1388s)
CPU time16m (966s = 69%)
Max RSS bytes1792471040 (1709 MiB)
Outputting started2026-02-03 11:22:42
Output files
Finished2026-02-03 11:23:36
Saved logsjustin-logs:297635.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

rameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=168844 n=30124 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=173834 n=27518 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 40th record. run: 210008 subRun: 0 event: 40 at 03-Feb-2026 12:19:25 CET
SimDepoSetSource got 9310 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 9310 depos spanning: [0.868337, 0.876258]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=189330 n=30495 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=198361 n=28972 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 41st record. run: 210008 subRun: 0 event: 41 at 03-Feb-2026 12:19:51 CET
SimDepoSetSource got 4597 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 4597 depos spanning: [0.540986, 0.546821]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=125709 n=16150 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=128408 n=15036 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 42nd record. run: 210008 subRun: 0 event: 42 at 03-Feb-2026 12:20:01 CET
SimDepoSetSource got 209 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 209 depos spanning: [0.0119824, 0.0122249]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=6034.02 n=1767 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=6932.49 n=1725 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 43rd record. run: 210008 subRun: 0 event: 43 at 03-Feb-2026 12:20:08 CET
SimDepoSetSource got 13042 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 13042 depos spanning: [0.298764, 1.72477]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=406082 n=38502 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=420628 n=34160 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 44th record. run: 210008 subRun: 0 event: 44 at 03-Feb-2026 12:20:19 CET
SimDepoSetSource got 893 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 893 depos spanning: [0.299509, 0.3006]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=30807.6 n=5048 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=35974.5 n=4184 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 45th record. run: 210008 subRun: 0 event: 45 at 03-Feb-2026 12:20:27 CET
SimDepoSetSource got 22677 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 22677 depos spanning: [0.568098, 0.591767]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=696546 n=78103 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=767411 n=62850 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 46th record. run: 210008 subRun: 0 event: 46 at 03-Feb-2026 12:21:02 CET
SimDepoSetSource got 8298 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 8298 depos spanning: [0.602587, 0.926097]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=254283 n=39841 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=261616 n=38069 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 47th record. run: 210008 subRun: 0 event: 47 at 03-Feb-2026 12:21:19 CET
SimDepoSetSource got 19043 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 19043 depos spanning: [0.787544, 0.807718]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=483035 n=78533 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=503882 n=77377 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 48th record. run: 210008 subRun: 0 event: 48 at 03-Feb-2026 12:21:46 CET
SimDepoSetSource got 19982 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 19982 depos spanning: [0.865962, 0.889463]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=604550 n=66120 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=655702 n=54634 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 49th record. run: 210008 subRun: 0 event: 49 at 03-Feb-2026 12:22:12 CET
SimDepoSetSource got 3241 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3241 depos spanning: [0.0491921, 0.0551492]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=102093 n=8743 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=108951 n=7540 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 50th record. run: 210008 subRun: 0 event: 50 at 03-Feb-2026 12:22:21 CET
SimDepoSetSource got 12249 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 12249 depos spanning: [0.327521, 0.338171]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=303531 n=47981 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=315349 n=46570 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 12:22:41 CET  Closed output file "prod_monte-carlo-012672-000008_gen_g4_detsim.root"
03-Feb-2026 12:22:41 CET  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/usertests/53/47/prod_monte-carlo-012672-000008_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      7.54786       26.4191       260.17        16.8648       40.9334        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000277078   0.00541514     0.0420916    0.00136949     0.0115397       50     
simulate:tpcrawdecoder:WireCellToolkit          7.46266       26.2106       259.993       16.6613       40.9104        50     
simulate:opdigi:OpDetDigitizerDUNE             0.0269198      0.12164      0.238722      0.117406      0.0533348       50     
simulate:rns:RandomNumberSaver                 1.475e-05    3.05187e-05   0.000404491   1.93655e-05   5.41724e-05      50     
[art]:TriggerResults:TriggerResultInserter     7.63e-06     1.2232e-05    7.3613e-05    9.2305e-06    9.69329e-06      50     
end_path:out1:RootOutput                        1.9e-06     3.38446e-06   3.2941e-05     2.585e-06    4.28942e-06      50     
end_path:out1:RootOutput(write)                0.0320731     0.0810968     0.181182      0.0761937     0.0358437       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3551.11 MB
  Peak resident set size usage (VmHWM): 1792.47 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 195M Feb  3 12:22 prod_monte-carlo-012672-000008_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-012672-000008_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 05:43:12 UTC       justIN version: 01.06.00