justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 277536.0@dunegpschedd01.fnal.gov

Jobsub ID277536.0@dunegpschedd01.fnal.gov
Workflow ID11873
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit28800 (8 hours)
Submitted time2026-01-14 01:58:01
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2026-01-14 02:08:50
From worker nodeHostnamegpu08.pic.es
cpuinfoAMD EPYC 9124 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 02:04:12
Input filesusertests:prod_monte-carlo-011873-000003_gen_g4.root
JobscriptExit code0
Real time3m (232s)
CPU time3m (226s = 97%)
Max RSS bytes1680023552 (1602 MiB)
Outputting started2026-01-14 02:08:05
Output filesusertests:prod_monte-carlo-011873-000003_gen_g4_detsim.root
Finished2026-01-14 02:08:50
Saved logsjustin-logs:277536.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

clsDepoFluxWriter:postdrift"
Outputer: "wclsFrameSaver:spsignals"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
wclsFrameSaver: promising to produce recob::Wires named "dnnsp"
14-Jan-2026 03:04:39 CET  Initiating request to open input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/91/4d/prod_monte-carlo-011873-000003_gen_g4.root"
14-Jan-2026 03:04:40 CET  Opened input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/91/4d/prod_monte-carlo-011873-000003_gen_g4.root"
Begin processing the 1st record. run: 20000014 subRun: 0 event: 1 at 14-Jan-2026 03:04:40 CET
SimDepoSetSource got 14504 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14504 depos spanning: [0.108519, 0.121734]us
[03:04:43.877] W [  gen   ] <IncoherentAddNoise:addnoiseapa2> model EmpiricalNoiseModel:empericalnoisemeganodes produced 6174 samples instead of expected 6000, future warnings muted 
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=381436 n=57570 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=410682 n=46673 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 03:05:02 CET  Opened output file with pattern "prod_monte-carlo-011873-000003_gen_g4_detsim.root"
Begin processing the 2nd record. run: 20000014 subRun: 0 event: 2 at 14-Jan-2026 03:05:03 CET
SimDepoSetSource got 26351 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 26351 depos spanning: [0.224651, 0.247597]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=732718 n=62406 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=805086 n=52585 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 3rd record. run: 20000014 subRun: 0 event: 3 at 14-Jan-2026 03:05:25 CET
SimDepoSetSource got 2936 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2936 depos spanning: [0.633601, 0.81441]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=109560 n=14219 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=115936 n=12216 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 4th record. run: 20000014 subRun: 0 event: 4 at 14-Jan-2026 03:05:31 CET
SimDepoSetSource got 10828 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 10828 depos spanning: [0.139098, 0.150735]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=342235 n=42240 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=390643 n=33489 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 5th record. run: 20000014 subRun: 0 event: 5 at 14-Jan-2026 03:05:43 CET
SimDepoSetSource got 36520 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 36520 depos spanning: [0.470353, 1.00172]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=916353 n=119650 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=962816 n=107461 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 6th record. run: 20000014 subRun: 0 event: 6 at 14-Jan-2026 03:06:13 CET
SimDepoSetSource got 5601 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5601 depos spanning: [0.672531, 2.15155]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=189318 n=26781 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=195167 n=24119 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 7th record. run: 20000014 subRun: 0 event: 7 at 14-Jan-2026 03:06:25 CET
SimDepoSetSource got 3769 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3769 depos spanning: [0.888988, 3.84896]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=125159 n=16289 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=141116 n=13546 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 8th record. run: 20000014 subRun: 0 event: 8 at 14-Jan-2026 03:06:37 CET
SimDepoSetSource got 13555 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 13555 depos spanning: [0.138304, 0.154709]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=353614 n=36211 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=376894 n=28904 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 9th record. run: 20000014 subRun: 0 event: 9 at 14-Jan-2026 03:06:57 CET
SimDepoSetSource got 33262 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 33262 depos spanning: [0.45829, 0.500592]us
Retagger: tagging trace set: wiener with 12800 traces, 12800 summary
Retagger: tagging trace set: gauss with 12800 traces, 0 summary
wclsFrameSaver: saving 12800 traces tagged "gauss"
FrameSaver: q=971731 n=141224 tag=gauss
wclsFrameSaver: saving 12800 traces tagged "wiener"
FrameSaver: q=1.04931e+06 n=113714 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 20000014 subRun: 0 event: 10 at 14-Jan-2026 03:07:32 CET
SimDepoSetSource got 24537 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 24537 depos spanning: [0.393, 0.850883]us
Retagger: tagging trace set: wiener with 12800 traces, 12800 summary
Retagger: tagging trace set: gauss with 12800 traces, 0 summary
wclsFrameSaver: saving 12800 traces tagged "gauss"
FrameSaver: q=746547 n=102161 tag=gauss
wclsFrameSaver: saving 12800 traces tagged "wiener"
FrameSaver: q=784938 n=87092 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 03:08:04 CET  Closed output file "prod_monte-carlo-011873-000003_gen_g4_detsim.root"
14-Jan-2026 03:08:04 CET  Closed input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/91/4d/prod_monte-carlo-011873-000003_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      6.01901       20.2614       35.1257       21.1513       9.29952        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000448986   0.00120924     0.0028053    0.000784936   0.000763219      10     
simulate:tpcrawdecoder:WireCellToolkit          5.86777       20.0575       34.8237       20.9835       9.24008        10     
simulate:opdigi:OpDetDigitizerDUNE             0.0923354     0.121176      0.184608      0.110703      0.0311257       10     
simulate:rns:RandomNumberSaver                 1.674e-05    4.23494e-05   0.000191652   2.67555e-05   5.02153e-05      10     
[art]:TriggerResults:TriggerResultInserter     8.51e-06     1.34952e-05    3.554e-05    9.2305e-06    8.93976e-06      10     
end_path:out1:RootOutput                       2.68e-06      5.389e-06     1.879e-05     3.74e-06     4.61489e-06      10     
end_path:out1:RootOutput(write)                0.038721      0.0810594     0.144766      0.0737941     0.038145        10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3466.9 MB
  Peak resident set size usage (VmHWM): 1680.02 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 56M Jan 14 03:08 prod_monte-carlo-011873-000003_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-011873-000003_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-06 04:51:17 UTC       justIN version: 01.06.00