justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 277552.1@dunegpschedd01.fnal.gov

Jobsub ID277552.1@dunegpschedd01.fnal.gov
Workflow ID11876
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit28800 (8 hours)
Submitted time2026-01-14 03:36:06
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce509
Last heartbeat2026-01-14 03:49:11
From worker nodeHostnameb9p11p4236.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 03:36:48
Input filesusertests:prod_monte-carlo-011876-000003_gen_g4.root
JobscriptExit code0
Real time10m (659s)
CPU time10m (614s = 93%)
Max RSS bytes1671802880 (1594 MiB)
Outputting started2026-01-14 03:47:48
Output filesusertests:prod_monte-carlo-011876-000003_gen_g4_detsim.root
Finished2026-01-14 03:49:11
Saved logsjustin-logs:277552.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

n=13885 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=95619.2 n=13152 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 20000014 subRun: 0 event: 10 at 14-Jan-2026 04:40:52 CET
SimDepoSetSource got 7133 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 7133 depos spanning: [0.691251, 1.28095]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=211481 n=30802 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=230081 n=24245 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 11th record. run: 20000014 subRun: 0 event: 11 at 14-Jan-2026 04:41:23 CET
SimDepoSetSource got 12319 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 12319 depos spanning: [0.535094, 0.548847]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=363361 n=59621 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=372790 n=55245 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 12th record. run: 20000014 subRun: 0 event: 12 at 14-Jan-2026 04:41:57 CET
SimDepoSetSource got 2139 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2139 depos spanning: [0.176896, 0.918464]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=69605.2 n=8349 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=74577.8 n=7269 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 13th record. run: 20000014 subRun: 0 event: 13 at 14-Jan-2026 04:42:13 CET
SimDepoSetSource got 27346 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 27346 depos spanning: [0.0408599, 0.0703948]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=792032 n=73742 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=928968 n=60338 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 14th record. run: 20000014 subRun: 0 event: 14 at 14-Jan-2026 04:43:21 CET
SimDepoSetSource got 16993 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 16993 depos spanning: [0.806692, 0.825946]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=509759 n=78605 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=523303 n=74497 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 15th record. run: 20000014 subRun: 0 event: 15 at 14-Jan-2026 04:44:00 CET
SimDepoSetSource got 945 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 945 depos spanning: [0.360643, 0.361796]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=22510.2 n=3055 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=24085.4 n=2685 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 16th record. run: 20000014 subRun: 0 event: 16 at 14-Jan-2026 04:44:14 CET
SimDepoSetSource got 8024 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 8024 depos spanning: [0.181922, 0.190595]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=221108 n=39489 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=228768 n=38330 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 17th record. run: 20000014 subRun: 0 event: 17 at 14-Jan-2026 04:44:51 CET
SimDepoSetSource got 21268 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 21268 depos spanning: [0.848633, 0.872838]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=604735 n=56255 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=647357 n=47019 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 18th record. run: 20000014 subRun: 0 event: 18 at 14-Jan-2026 04:45:40 CET
SimDepoSetSource got 16265 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 16265 depos spanning: [0.321993, 0.341675]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=480861 n=73277 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=495447 n=68055 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 19th record. run: 20000014 subRun: 0 event: 19 at 14-Jan-2026 04:46:14 CET
SimDepoSetSource got 9851 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 9851 depos spanning: [0.079754, 0.484023]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=268536 n=36683 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=287979 n=29961 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 20th record. run: 20000014 subRun: 0 event: 20 at 14-Jan-2026 04:46:59 CET
SimDepoSetSource got 35663 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 35663 depos spanning: [0.263214, 0.299619]us
Retagger: tagging trace set: wiener with 12800 traces, 12800 summary
Retagger: tagging trace set: gauss with 12800 traces, 0 summary
wclsFrameSaver: saving 12800 traces tagged "gauss"
FrameSaver: q=1.07092e+06 n=140807 tag=gauss
wclsFrameSaver: saving 12800 traces tagged "wiener"
FrameSaver: q=1.16905e+06 n=114441 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 04:47:47 CET  Closed output file "prod_monte-carlo-011876-000003_gen_g4_detsim.root"
14-Jan-2026 04:47:47 CET  Closed input file "root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/usertests/65/20/prod_monte-carlo-011876-000003_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      8.18915       27.975        68.0603       25.9078       16.1752        20     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000328474   0.00444221     0.0380287    0.00106256     0.0091984       20     
simulate:tpcrawdecoder:WireCellToolkit          8.10002       27.693        67.5419       25.6445       16.0648        20     
simulate:opdigi:OpDetDigitizerDUNE             0.0456387     0.166048      0.318786      0.171162      0.0801529       20     
simulate:rns:RandomNumberSaver                 1.843e-05    0.000131802   0.00167442    4.2886e-05    0.000354771      20     
[art]:TriggerResults:TriggerResultInserter     8.02e-06     3.41646e-05   0.000317444    1.571e-05    6.5536e-05       20     
end_path:out1:RootOutput                       2.63e-06     8.62505e-06   6.3061e-05     5.665e-06    1.26642e-05      20     
end_path:out1:RootOutput(write)                0.0336787     0.110717      0.195962      0.0996605     0.0513043       20     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3424.73 MB
  Peak resident set size usage (VmHWM): 1671.8 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r--. 1 duneprd np-comp 82M Jan 14 04:47 prod_monte-carlo-011876-000003_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-011876-000003_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-06 04:40:32 UTC       justIN version: 01.06.00