justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271805.0@dunegpschedd02.fnal.gov

Jobsub ID271805.0@dunegpschedd02.fnal.gov
Workflow ID11876
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit28800 (8 hours)
Submitted time2026-01-14 03:40:06
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2026-01-14 03:53:28
From worker nodeHostnamewn-lb-13.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 03:41:12
Input filesusertests:prod_monte-carlo-011876-000002_gen_g4.root
JobscriptExit code0
Real time9m (572s)
CPU time9m (561s = 98%)
Max RSS bytes1423835136 (1357 MiB)
Outputting started2026-01-14 03:50:45
Output filesusertests:prod_monte-carlo-011876-000002_gen_g4_detsim.root
Finished2026-01-14 03:53:28
Saved logsjustin-logs:271805.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

"
FrameSaver: q=17626.1 n=4045 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=18810.4 n=3952 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 20000014 subRun: 0 event: 10 at 14-Jan-2026 04:46:14 CET
SimDepoSetSource got 14061 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14061 depos spanning: [0.329037, 0.41601]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=373098 n=59392 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=383261 n=58568 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 11th record. run: 20000014 subRun: 0 event: 11 at 14-Jan-2026 04:46:34 CET
SimDepoSetSource got 13070 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 13070 depos spanning: [0.417785, 0.428884]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=374579 n=29667 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=420159 n=25074 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 12th record. run: 20000014 subRun: 0 event: 12 at 14-Jan-2026 04:46:50 CET
SimDepoSetSource got 3480 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3480 depos spanning: [0.0231377, 0.0267075]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=90412.1 n=10005 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=95816.5 n=8301 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 13th record. run: 20000014 subRun: 0 event: 13 at 14-Jan-2026 04:47:06 CET
SimDepoSetSource got 22052 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 22052 depos spanning: [0.682337, 0.705547]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=595990 n=85142 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=648773 n=67014 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 14th record. run: 20000014 subRun: 0 event: 14 at 14-Jan-2026 04:47:55 CET
SimDepoSetSource got 12040 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 12040 depos spanning: [0.0170334, 0.029828]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=333455 n=55965 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=345813 n=49433 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 15th record. run: 20000014 subRun: 0 event: 15 at 14-Jan-2026 04:48:28 CET
SimDepoSetSource got 5331 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5331 depos spanning: [0.598716, 0.637471]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=145824 n=20755 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=151503 n=19706 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 16th record. run: 20000014 subRun: 0 event: 16 at 14-Jan-2026 04:48:56 CET
SimDepoSetSource got 7481 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 7481 depos spanning: [0.845104, 0.852302]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=170928 n=26676 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=178313 n=24390 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 17th record. run: 20000014 subRun: 0 event: 17 at 14-Jan-2026 04:49:12 CET
SimDepoSetSource got 2192 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2192 depos spanning: [0.670564, 0.673023]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=42367.8 n=6891 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=45598.5 n=5836 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 18th record. run: 20000014 subRun: 0 event: 18 at 14-Jan-2026 04:49:25 CET
SimDepoSetSource got 6009 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 6009 depos spanning: [0.0789526, 0.513002]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=203386 n=29380 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=219143 n=24205 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 19th record. run: 20000014 subRun: 0 event: 19 at 14-Jan-2026 04:49:50 CET
SimDepoSetSource got 2654 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2654 depos spanning: [0.0157019, 0.123861]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=98886.1 n=13431 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=110249 n=10911 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 20th record. run: 20000014 subRun: 0 event: 20 at 14-Jan-2026 04:50:15 CET
SimDepoSetSource got 9735 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 9735 depos spanning: [0.479747, 1.35514]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=244187 n=39207 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=254680 n=38761 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 04:50:44 CET  Closed output file "prod_monte-carlo-011876-000002_gen_g4_detsim.root"
14-Jan-2026 04:50:44 CET  Closed input file "root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/usertests/99/08/prod_monte-carlo-011876-000002_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      13.0197       24.8644       49.4069       24.9261       9.84726        20     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000383373   0.00117569    0.00331699    0.00111033    0.000730029      20     
simulate:tpcrawdecoder:WireCellToolkit          12.8729       24.6026       49.045        24.6398       9.78454        20     
simulate:opdigi:OpDetDigitizerDUNE             0.0560025      0.15351      0.257452      0.150547      0.0513291       20     
simulate:rns:RandomNumberSaver                2.5098e-05    3.91525e-05   0.00023422    2.89235e-05   4.4783e-05       20     
[art]:TriggerResults:TriggerResultInserter    1.1867e-05    1.62328e-05   4.0951e-05     1.336e-05    8.12333e-06      20     
end_path:out1:RootOutput                       4.376e-06    5.51375e-06   1.8597e-05     4.832e-06    3.01231e-06      20     
end_path:out1:RootOutput(write)                0.0511073     0.106605      0.188332      0.0946005     0.0425868       20     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3114.09 MB
  Peak resident set size usage (VmHWM): 1423.84 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r--. 1 dune009 dune 59M Jan 14 04:50 prod_monte-carlo-011876-000002_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-011876-000002_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-06 04:40:48 UTC       justIN version: 01.06.00