justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 277571.1@dunegpschedd01.fnal.gov

Jobsub ID277571.1@dunegpschedd01.fnal.gov
Workflow ID11877
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit28800 (8 hours)
Submitted time2026-01-14 05:20:11
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-14 05:33:11
From worker nodeHostnametd808.pic.es
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 05:21:21
Input filesusertests:prod_monte-carlo-011877-000003_gen_g4.root
JobscriptExit code0
Real time10m (638s)
CPU time10m (609s = 95%)
Max RSS bytes1670541312 (1593 MiB)
Outputting started2026-01-14 05:32:00
Output files
Finished2026-01-14 05:33:11
Saved logsjustin-logs:277571.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

2800 traces tagged "gauss"
FrameSaver: q=899689 n=133802 tag=gauss
wclsFrameSaver: saving 12800 traces tagged "wiener"
FrameSaver: q=941230 n=117476 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 20000014 subRun: 0 event: 10 at 14-Jan-2026 06:27:24 CET
SimDepoSetSource got 7409 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 7409 depos spanning: [0.207299, 0.213782]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=172706 n=30277 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=180188 n=29771 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 11th record. run: 20000014 subRun: 0 event: 11 at 14-Jan-2026 06:27:48 CET
SimDepoSetSource got 792 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 792 depos spanning: [0.379224, 0.380765]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=14915.5 n=2691 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=16382 n=2452 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 12th record. run: 20000014 subRun: 0 event: 12 at 14-Jan-2026 06:27:59 CET
SimDepoSetSource got 19426 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 19426 depos spanning: [0.608674, 0.629492]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=592817 n=79015 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=640959 n=65505 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 13th record. run: 20000014 subRun: 0 event: 13 at 14-Jan-2026 06:28:42 CET
SimDepoSetSource got 239 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 239 depos spanning: [0.472114, 0.472417]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=7092.45 n=1916 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=8189.51 n=1894 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 14th record. run: 20000014 subRun: 0 event: 14 at 14-Jan-2026 06:28:57 CET
SimDepoSetSource got 12149 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 12149 depos spanning: [0.0158692, 0.0290374]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=335421 n=42258 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=343077 n=40701 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 15th record. run: 20000014 subRun: 0 event: 15 at 14-Jan-2026 06:29:17 CET
SimDepoSetSource got 14163 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14163 depos spanning: [0.780944, 0.796402]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=416294 n=67289 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=426910 n=63184 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 16th record. run: 20000014 subRun: 0 event: 16 at 14-Jan-2026 06:29:49 CET
SimDepoSetSource got 9487 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 9487 depos spanning: [0.916779, 0.929932]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=220947 n=27051 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=246533 n=23295 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 17th record. run: 20000014 subRun: 0 event: 17 at 14-Jan-2026 06:30:15 CET
SimDepoSetSource got 18945 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 18945 depos spanning: [0.648478, 0.669109]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=540026 n=56854 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=557941 n=52426 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 18th record. run: 20000014 subRun: 0 event: 18 at 14-Jan-2026 06:30:51 CET
SimDepoSetSource got 2698 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2698 depos spanning: [0.232488, 0.236764]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=86572.5 n=11927 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=93015.6 n=10377 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 19th record. run: 20000014 subRun: 0 event: 19 at 14-Jan-2026 06:31:17 CET
SimDepoSetSource got 3564 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3564 depos spanning: [0.738318, 0.743193]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=91122.2 n=17568 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=94356.7 n=17332 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 20th record. run: 20000014 subRun: 0 event: 20 at 14-Jan-2026 06:31:30 CET
SimDepoSetSource got 12489 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 12489 depos spanning: [0.373745, 0.51935]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=390343 n=58206 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=400491 n=53108 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 06:31:59 CET  Closed output file "prod_monte-carlo-011877-000003_gen_g4_detsim.root"
14-Jan-2026 06:31:59 CET  Closed input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/48/b0/prod_monte-carlo-011877-000003_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      11.0603       28.735        68.4675       24.281        16.988         20     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000362671   0.00154044    0.00345362    0.00116783    0.00106061       20     
simulate:tpcrawdecoder:WireCellToolkit          10.9491       28.4192       67.9109       23.9864       16.8673        20     
simulate:opdigi:OpDetDigitizerDUNE             0.033712      0.169813      0.373318       0.18825      0.0932031       20     
simulate:rns:RandomNumberSaver                2.1111e-05    3.81426e-05   0.000230191   2.6655e-05    4.46151e-05      20     
[art]:TriggerResults:TriggerResultInserter     1.056e-05    1.49865e-05    4.034e-05     1.438e-05    6.1604e-06       20     
end_path:out1:RootOutput                       2.96e-06     4.4665e-06     1.846e-05     3.435e-06    3.3283e-06       20     
end_path:out1:RootOutput(write)                0.0626492     0.143954      0.263323      0.140265      0.0622882       20     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3424.04 MB
  Peak resident set size usage (VmHWM): 1670.54 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 77M Jan 14 06:31 prod_monte-carlo-011877-000003_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-011877-000003_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-06 04:40:07 UTC       justIN version: 01.06.00