justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272504.0@dunegpschedd02.fnal.gov

Jobsub ID272504.0@dunegpschedd02.fnal.gov
Workflow ID11924
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit28800 (8 hours)
Submitted time2026-01-14 21:21:00
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2026-01-14 21:31:59
From worker nodeHostnameb9p04p6505.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 21:21:58
Input filesusertests:prod_monte-carlo-011924-000005_gen_g4.root
JobscriptExit code0
Real time8m (530s)
CPU time8m (512s = 96%)
Max RSS bytes1835466752 (1750 MiB)
Outputting started2026-01-14 21:30:49
Output files
Finished2026-01-14 21:31:57
Saved logsjustin-logs:272504.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

"gauss"
FrameSaver: q=685255 n=75676 tag=gauss
wclsFrameSaver: saving 12800 traces tagged "wiener"
FrameSaver: q=730474 n=64531 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 200005 subRun: 0 event: 10 at 14-Jan-2026 22:26:31 CET
SimDepoSetSource got 5235 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5235 depos spanning: [0.0954531, 2.12943]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=128416 n=23646 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=132870 n=23440 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 11th record. run: 200005 subRun: 0 event: 11 at 14-Jan-2026 22:26:41 CET
SimDepoSetSource got 14090 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14090 depos spanning: [0.374878, 0.390882]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=410134 n=56212 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=450622 n=44293 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 12th record. run: 200005 subRun: 0 event: 12 at 14-Jan-2026 22:27:02 CET
SimDepoSetSource got 5987 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5987 depos spanning: [0.0750983, 0.0816373]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=159629 n=28471 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=166346 n=27764 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 13th record. run: 200005 subRun: 0 event: 13 at 14-Jan-2026 22:27:17 CET
SimDepoSetSource got 5890 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5890 depos spanning: [0.167989, 0.174491]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=170068 n=23383 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=186956 n=18256 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 14th record. run: 200005 subRun: 0 event: 14 at 14-Jan-2026 22:27:25 CET
SimDepoSetSource got 2741 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2741 depos spanning: [0.695027, 2.47561]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=93279.9 n=12600 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=98175.6 n=11675 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 15th record. run: 200005 subRun: 0 event: 15 at 14-Jan-2026 22:27:41 CET
SimDepoSetSource got 14331 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14331 depos spanning: [0.805235, 0.822372]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=416402 n=67590 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=436449 n=56692 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 16th record. run: 200005 subRun: 0 event: 16 at 14-Jan-2026 22:28:04 CET
SimDepoSetSource got 15892 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 15892 depos spanning: [0.177248, 0.194762]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=463385 n=72344 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=487076 n=61443 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 17th record. run: 200005 subRun: 0 event: 17 at 14-Jan-2026 22:28:34 CET
SimDepoSetSource got 13080 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 13080 depos spanning: [0.0150322, 0.0298109]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=366420 n=57876 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=382108 n=50441 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 18th record. run: 200005 subRun: 0 event: 18 at 14-Jan-2026 22:28:55 CET
SimDepoSetSource got 21401 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 21401 depos spanning: [0.636838, 0.660351]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=444006 n=71068 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=463401 n=69343 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 19th record. run: 200005 subRun: 0 event: 19 at 14-Jan-2026 22:29:31 CET
SimDepoSetSource got 35927 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 35927 depos spanning: [0.841931, 0.878845]us
Retagger: tagging trace set: wiener with 12800 traces, 12800 summary
Retagger: tagging trace set: gauss with 12800 traces, 0 summary
wclsFrameSaver: saving 12800 traces tagged "gauss"
FrameSaver: q=806107 n=109155 tag=gauss
wclsFrameSaver: saving 12800 traces tagged "wiener"
FrameSaver: q=866287 n=93452 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 20th record. run: 200005 subRun: 0 event: 20 at 14-Jan-2026 22:30:26 CET
SimDepoSetSource got 1409 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 1409 depos spanning: [0.28165, 1.20779]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=49437.1 n=9471 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=52670.6 n=8995 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 22:30:48 CET  Closed output file "prod_monte-carlo-011924-000005_gen_g4_detsim.root"
14-Jan-2026 22:30:48 CET  Closed input file "root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/usertests/15/57/prod_monte-carlo-011924-000005_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      8.47342       24.0486       61.253        20.4381       14.5321        20     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000270622   0.00131382    0.00658837    0.000850822   0.00136538       20     
simulate:tpcrawdecoder:WireCellToolkit          8.33369       23.7999       60.5739       20.286        14.4297        20     
simulate:opdigi:OpDetDigitizerDUNE             0.0619137     0.138193      0.378021       0.11062      0.0709251       20     
simulate:rns:RandomNumberSaver                 1.709e-05    4.81948e-05   0.000339603   3.21655e-05   6.76914e-05      20     
[art]:TriggerResults:TriggerResultInserter     7.81e-06     1.80127e-05   5.6031e-05     1.374e-05    1.27051e-05      20     
end_path:out1:RootOutput                       2.85e-06     5.7797e-06     2.327e-05    4.5555e-06    4.58446e-06      20     
end_path:out1:RootOutput(write)                0.0390225     0.108737      0.299853      0.0921867     0.0633588       20     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3616.35 MB
  Peak resident set size usage (VmHWM): 1835.47 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r--. 1 duneprd np-comp 97M Jan 14 22:30 prod_monte-carlo-011924-000005_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-011924-000005_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-06 02:07:26 UTC       justIN version: 01.06.00