justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297623.6@dunegpschedd02.fnal.gov

Jobsub ID297623.6@dunegpschedd02.fnal.gov
Workflow ID12667
Stage ID3
User namescalvez@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 10:02:42
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-02-03 10:12:47
From worker nodeHostnameheplnc158.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 10:04:43
Input filesusertests:prod_monte-carlo-012667-000003_gen_g4.root
JobscriptExit code0
Real time6m (391s)
CPU time5m (352s = 90%)
Max RSS bytes1495658496 (1426 MiB)
Outputting started2026-02-03 10:11:15
Output files
Finished2026-02-03 10:12:47
Saved logsjustin-logs:297623.6-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Inputer: "wclsSimDepoSetSource:"
Outputer: "wclsDepoFluxWriter:postdrift"
Outputer: "wclsFrameSaver:spsignals"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
wclsFrameSaver: promising to produce recob::Wires named "dnnsp"
03-Feb-2026 10:05:46 GMT  Initiating request to open input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/usertests/86/25/prod_monte-carlo-012667-000003_gen_g4.root"
03-Feb-2026 10:05:48 GMT  Opened input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/usertests/86/25/prod_monte-carlo-012667-000003_gen_g4.root"
Begin processing the 1st record. run: 210003 subRun: 0 event: 1 at 03-Feb-2026 10:05:49 GMT
SimDepoSetSource got 101 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 101 depos spanning: [0.366176, 0.366294]us
[10:05:50.347] W [  gen   ] <IncoherentAddNoise:addnoiseapa10> model EmpiricalNoiseModel:empericalnoisemeganodes produced 6174 samples instead of expected 6000, future warnings muted 
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=5117.31 n=1684 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=6520.31 n=1647 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 10:06:03 GMT  Opened output file with pattern "prod_monte-carlo-012667-000003_gen_g4_detsim.root"
Begin processing the 2nd record. run: 210003 subRun: 0 event: 2 at 03-Feb-2026 10:06:05 GMT
SimDepoSetSource got 27056 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 27056 depos spanning: [0.652469, 0.682818]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=790057 n=105303 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=831715 n=89824 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 3rd record. run: 210003 subRun: 0 event: 3 at 03-Feb-2026 10:07:06 GMT
SimDepoSetSource got 13739 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 13739 depos spanning: [0.522822, 0.538369]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=376513 n=45774 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=389044 n=40601 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 4th record. run: 210003 subRun: 0 event: 4 at 03-Feb-2026 10:07:39 GMT
SimDepoSetSource got 24296 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 24296 depos spanning: [0.108486, 0.13582]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=738308 n=61912 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=789161 n=53175 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 5th record. run: 210003 subRun: 0 event: 5 at 03-Feb-2026 10:08:20 GMT
SimDepoSetSource got 5448 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5448 depos spanning: [0.873953, 0.880673]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=162362 n=17102 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=181725 n=13685 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 6th record. run: 210003 subRun: 0 event: 6 at 03-Feb-2026 10:08:34 GMT
SimDepoSetSource got 31019 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 31019 depos spanning: [0.392282, 0.426016]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=904678 n=114188 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=932712 n=109578 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 7th record. run: 210003 subRun: 0 event: 7 at 03-Feb-2026 10:09:33 GMT
SimDepoSetSource got 9196 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 9196 depos spanning: [0.198387, 0.208943]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=263111 n=36326 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=273724 n=32041 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 8th record. run: 210003 subRun: 0 event: 8 at 03-Feb-2026 10:09:50 GMT
SimDepoSetSource got 3693 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3693 depos spanning: [0.46195, 0.784652]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=79654.7 n=16084 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=84172.5 n=15493 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 9th record. run: 210003 subRun: 0 event: 9 at 03-Feb-2026 10:10:29 GMT
SimDepoSetSource got 4647 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 4647 depos spanning: [0.473328, 0.501147]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=128237 n=24014 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=132958 n=23533 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 210003 subRun: 0 event: 10 at 03-Feb-2026 10:10:58 GMT
SimDepoSetSource got 5376 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5376 depos spanning: [0.674739, 0.679895]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=138624 n=22864 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=145394 n=22565 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 10:11:14 GMT  Closed output file "prod_monte-carlo-012667-000003_gen_g4_detsim.root"
03-Feb-2026 10:11:14 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/usertests/86/25/prod_monte-carlo-012667-000003_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      13.2082       31.0561       61.0395       28.554        17.3302        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000680771   0.00201263    0.00707652    0.00131633     0.0018193       10     
simulate:tpcrawdecoder:WireCellToolkit          12.9477       30.6776       60.3853       28.2047       17.1701        10     
simulate:opdigi:OpDetDigitizerDUNE             0.0198478     0.213926      0.380389      0.173001      0.115983        10     
simulate:rns:RandomNumberSaver                4.3822e-05    0.000101161   0.000476937   5.9612e-05    0.00012586       10     
[art]:TriggerResults:TriggerResultInserter    1.4928e-05    2.89948e-05   7.3258e-05    2.2758e-05    1.89151e-05      10     
end_path:out1:RootOutput                       6.151e-06    1.00489e-05   3.0698e-05    7.2485e-06    7.00491e-06      10     
end_path:out1:RootOutput(write)                0.0786583     0.161892      0.314012      0.128688      0.0753428       10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3270.76 MB
  Peak resident set size usage (VmHWM): 1495.66 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 pltdune04 pltdune 42M Feb  3 10:11 prod_monte-carlo-012667-000003_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-012667-000003_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 04:31:59 UTC       justIN version: 01.06.00