justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297623.9@dunegpschedd02.fnal.gov

Jobsub ID297623.9@dunegpschedd02.fnal.gov
Workflow ID12667
Stage ID3
User namescalvez@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 10:02:42
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2026-02-03 10:13:43
From worker nodeHostnamet2wn001.physics.ox.ac.uk
cpuinfoAMD EPYC 7713P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 10:04:22
Input filesusertests:prod_monte-carlo-012667-000006_gen_g4.root
JobscriptExit code0
Real time6m (404s)
CPU time5m (340s = 84%)
Max RSS bytes1528991744 (1458 MiB)
Outputting started2026-02-03 10:11:06
Output files
Finished2026-02-03 10:13:43
Saved logsjustin-logs:297623.9-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lsFrameSaver:spsignals"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
wclsFrameSaver: promising to produce recob::Wires named "dnnsp"
03-Feb-2026 10:05:46 GMT  Initiating request to open input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/48/79/prod_monte-carlo-012667-000006_gen_g4.root"
03-Feb-2026 10:05:48 GMT  Opened input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/48/79/prod_monte-carlo-012667-000006_gen_g4.root"
Begin processing the 1st record. run: 210006 subRun: 0 event: 1 at 03-Feb-2026 10:05:49 GMT
SimDepoSetSource got 3821 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3821 depos spanning: [0.420418, 0.425134]us
[10:05:51.023] W [  gen   ] <IncoherentAddNoise:addnoiseapa4> model EmpiricalNoiseModel:empericalnoisemeganodes produced 6174 samples instead of expected 6000, future warnings muted 
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=101500 n=14575 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=104721 n=14007 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 10:06:03 GMT  Opened output file with pattern "prod_monte-carlo-012667-000006_gen_g4_detsim.root"
Begin processing the 2nd record. run: 210006 subRun: 0 event: 2 at 03-Feb-2026 10:06:06 GMT
SimDepoSetSource got 758 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 758 depos spanning: [0.730391, 0.731219]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=16724.1 n=3624 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=18077.1 n=3485 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 3rd record. run: 210006 subRun: 0 event: 3 at 03-Feb-2026 10:06:19 GMT
SimDepoSetSource got 5790 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5790 depos spanning: [0.187469, 3.48787]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=180321 n=23263 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=194359 n=19903 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 4th record. run: 210006 subRun: 0 event: 4 at 03-Feb-2026 10:06:45 GMT
SimDepoSetSource got 8307 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 8307 depos spanning: [0.105017, 0.258441]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=238943 n=34714 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=246289 n=34303 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 5th record. run: 210006 subRun: 0 event: 5 at 03-Feb-2026 10:07:01 GMT
SimDepoSetSource got 4817 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 4817 depos spanning: [0.139261, 0.144596]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=117315 n=20927 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=122564 n=20634 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 6th record. run: 210006 subRun: 0 event: 6 at 03-Feb-2026 10:07:17 GMT
SimDepoSetSource got 21464 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 21464 depos spanning: [0.881723, 4.13986]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=464492 n=67369 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=484404 n=65260 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 7th record. run: 210006 subRun: 0 event: 7 at 03-Feb-2026 10:07:52 GMT
SimDepoSetSource got 39661 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 39661 depos spanning: [0.085618, 0.123132]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=1.10135e+06 n=100746 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=1.18274e+06 n=83632 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 8th record. run: 210006 subRun: 0 event: 8 at 03-Feb-2026 10:09:03 GMT
SimDepoSetSource got 7834 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 7834 depos spanning: [0.784652, 0.793708]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=223595 n=38835 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=229273 n=35336 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 9th record. run: 210006 subRun: 0 event: 9 at 03-Feb-2026 10:09:22 GMT
SimDepoSetSource got 14410 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14410 depos spanning: [0.337671, 0.498108]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=364223 n=52530 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=381250 n=46924 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 210006 subRun: 0 event: 10 at 03-Feb-2026 10:09:55 GMT
SimDepoSetSource got 14326 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14326 depos spanning: [0.106247, 0.697298]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=417888 n=56350 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=445563 n=45610 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
03-Feb-2026 10:11:06 GMT  Closed output file "prod_monte-carlo-012667-000006_gen_g4_detsim.root"
03-Feb-2026 10:11:06 GMT  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/48/79/prod_monte-carlo-012667-000006_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      12.9132       28.9342        70.8         22.0593       17.5234        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000283747   0.00204518     0.0101535    0.000996086   0.00282082       10     
simulate:tpcrawdecoder:WireCellToolkit          12.6978       28.5384       70.0826       21.7567       17.3856        10     
simulate:opdigi:OpDetDigitizerDUNE              0.1365        0.22746      0.393303      0.206957      0.080166        10     
simulate:rns:RandomNumberSaver                2.3355e-05    5.22388e-05   0.000270451    2.825e-05    7.27867e-05      10     
[art]:TriggerResults:TriggerResultInserter     1.025e-05    1.93825e-05   5.6328e-05    1.5946e-05    1.25579e-05      10     
end_path:out1:RootOutput                       3.096e-06    5.8823e-06    2.5369e-05     3.757e-06    6.50687e-06      10     
end_path:out1:RootOutput(write)                0.0778476     0.165826      0.322467      0.139562      0.0727585       10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3287.58 MB
  Peak resident set size usage (VmHWM): 1528.99 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 dune020 dune 41M Feb  3 10:11 prod_monte-carlo-012667-000006_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-012667-000006_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 10:44:58 UTC       justIN version: 01.06.00