justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303486.1@dunegpschedd01.fnal.gov

Jobsub ID303486.1@dunegpschedd01.fnal.gov
Workflow ID12686
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 16:00:05
SiteUS_UChicago
EntryEngage_US_MWT2_iut2_condce_mcore
Last heartbeat2026-02-03 16:27:01
From worker nodeHostnameiut2-c389.iu.edu
cpuinfoAMD EPYC 7443 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 16:03:21
Input filesusertests:prod_monte-carlo-012686-000004_gen_g4.root
JobscriptExit code0
Real time23m (1382s)
CPU time22m (1331s = 96%)
Max RSS bytes1823125504 (1738 MiB)
Outputting started2026-02-03 16:26:24
Output files
Finished2026-02-03 16:27:01
Saved logsjustin-logs:303486.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

auss"
FrameSaver: q=311469 n=35725 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=333331 n=30109 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 40th record. run: 210004 subRun: 0 event: 40 at 03-Feb-2026 11:21:41 EST
SimDepoSetSource got 4890 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 4890 depos spanning: [0.983633, 0.989749]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=114121 n=18199 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=118812 n=17116 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 41st record. run: 210004 subRun: 0 event: 41 at 03-Feb-2026 11:21:51 EST
SimDepoSetSource got 42551 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 42551 depos spanning: [0.716048, 0.763844]us
Retagger: tagging trace set: wiener with 15360 traces, 15360 summary
Retagger: tagging trace set: gauss with 15360 traces, 0 summary
wclsFrameSaver: saving 15360 traces tagged "gauss"
FrameSaver: q=1.24318e+06 n=145901 tag=gauss
wclsFrameSaver: saving 15360 traces tagged "wiener"
FrameSaver: q=1.33066e+06 n=121865 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 42nd record. run: 210004 subRun: 0 event: 42 at 03-Feb-2026 11:22:53 EST
SimDepoSetSource got 172 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 172 depos spanning: [0.783168, 0.783275]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=7086.54 n=1483 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=8258.88 n=1454 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 43rd record. run: 210004 subRun: 0 event: 43 at 03-Feb-2026 11:23:02 EST
SimDepoSetSource got 35571 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 35571 depos spanning: [0.504222, 0.540345]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=1.01519e+06 n=99175 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=1.06388e+06 n=86573 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 44th record. run: 210004 subRun: 0 event: 44 at 03-Feb-2026 11:23:51 EST
SimDepoSetSource got 7000 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 7000 depos spanning: [0.682668, 0.690676]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=169641 n=25079 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=179407 n=21801 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 45th record. run: 210004 subRun: 0 event: 45 at 03-Feb-2026 11:24:21 EST
SimDepoSetSource got 4626 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 4626 depos spanning: [0.723918, 0.731231]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=126940 n=21341 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=130128 n=21014 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 46th record. run: 210004 subRun: 0 event: 46 at 03-Feb-2026 11:24:33 EST
SimDepoSetSource got 19731 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 19731 depos spanning: [0.64597, 0.66805]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=580152 n=60090 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=600757 n=53320 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 47th record. run: 210004 subRun: 0 event: 47 at 03-Feb-2026 11:24:54 EST
SimDepoSetSource got 10033 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 10033 depos spanning: [0.225006, 0.233604]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=267193 n=42483 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=279601 n=38217 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 48th record. run: 210004 subRun: 0 event: 48 at 03-Feb-2026 11:25:07 EST
SimDepoSetSource got 15066 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 15066 depos spanning: [0.574079, 0.591354]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=423663 n=61886 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=440810 n=55230 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 49th record. run: 210004 subRun: 0 event: 49 at 03-Feb-2026 11:25:40 EST
SimDepoSetSource got 19097 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 19097 depos spanning: [0.320308, 0.338869]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=414896 n=60636 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=430190 n=58666 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 50th record. run: 210004 subRun: 0 event: 50 at 03-Feb-2026 11:26:03 EST
SimDepoSetSource got 1832 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 1832 depos spanning: [0.180331, 0.182535]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=50614.1 n=8897 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=54414.8 n=7762 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 11:26:23 EST  Closed output file "prod_monte-carlo-012686-000004_gen_g4_detsim.root"
03-Feb-2026 11:26:23 EST  Closed input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/0a/a0/prod_monte-carlo-012686-000004_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      8.33813       26.4208       62.0313       22.7228       14.8523        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000291226   0.00389226     0.0292778    0.00112055    0.00801781       50     
simulate:tpcrawdecoder:WireCellToolkit          8.24526       26.1692       61.6172       22.3906       14.7807        50     
simulate:opdigi:OpDetDigitizerDUNE             0.0191466     0.144351      0.291685      0.142797      0.0649914       50     
simulate:rns:RandomNumberSaver                 1.607e-05    2.64149e-05   0.000257503   1.89355e-05   3.35121e-05      50     
[art]:TriggerResults:TriggerResultInserter     7.243e-06    1.1224e-05    4.1418e-05    8.8815e-06    6.40667e-06      50     
end_path:out1:RootOutput                       1.984e-06    4.07234e-06   3.4134e-05     2.695e-06    5.22528e-06      50     
end_path:out1:RootOutput(write)                0.027411       0.10298      0.222734      0.0988308     0.0484846       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3587.99 MB
  Peak resident set size usage (VmHWM): 1823.13 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r--. 1 dune osgvo 259M Feb  3 11:26 prod_monte-carlo-012686-000004_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-012686-000004_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 04:28:23 UTC       justIN version: 01.06.00