justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297391.52@dunegpschedd02.fnal.gov

Jobsub ID297391.52@dunegpschedd02.fnal.gov
Workflow ID12654
Stage ID3
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit28800 (8 hours)
Submitted time2026-02-02 20:06:06
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2026-02-02 21:00:07
From worker nodeHostnamewn-s04-05-07-03.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-02 20:13:23
Input filesusertests:prodgenie_monte-carlo-012654-000114_gen_g4.root
JobscriptExit code0
Real time37m (2229s)
CPU time32m (1976s = 88%)
Max RSS bytes2333712384 (2225 MiB)
Outputting started2026-02-02 20:50:32
Output files
Finished2026-02-02 21:00:07
Saved logsjustin-logs:297391.52-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

: q=23056.8 n=5434 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 40th record. run: 100114 subRun: 0 event: 40 at 02-Feb-2026 21:44:38 CET
SimDepoSetSource got 153271 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 153271 depos spanning: [6.67337e-07, 2.27444]us
Retagger: tagging trace set: wiener with 15360 traces, 15360 summary
Retagger: tagging trace set: gauss with 15360 traces, 0 summary
wclsFrameSaver: saving 15360 traces tagged "gauss"
FrameSaver: q=4.20229e+06 n=392399 tag=gauss
wclsFrameSaver: saving 15360 traces tagged "wiener"
FrameSaver: q=4.79687e+06 n=339794 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 41st record. run: 100114 subRun: 0 event: 41 at 02-Feb-2026 21:45:52 CET
SimDepoSetSource got 24408 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 24408 depos spanning: [6.67646e-07, 0.21789]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=724944 n=89234 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=814322 n=73219 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 42nd record. run: 100114 subRun: 0 event: 42 at 02-Feb-2026 21:46:16 CET
SimDepoSetSource got 24547 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 24547 depos spanning: [6.67226e-07, 1.41101]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=711029 n=86636 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=789474 n=70905 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 43rd record. run: 100114 subRun: 0 event: 43 at 02-Feb-2026 21:46:43 CET
SimDepoSetSource got 3915 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 3915 depos spanning: [8.43604e-07, 0.229398]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=128487 n=16551 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=139400 n=14527 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 44th record. run: 100114 subRun: 0 event: 44 at 02-Feb-2026 21:47:09 CET
SimDepoSetSource got 21130 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 21130 depos spanning: [6.70019e-07, 1.0049]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=573621 n=61212 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=643814 n=53383 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 45th record. run: 100114 subRun: 0 event: 45 at 02-Feb-2026 21:47:38 CET
SimDepoSetSource got 4294 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 4294 depos spanning: [1.05345e-06, 0.00494801]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=150182 n=15053 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=160177 n=13249 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 46th record. run: 100114 subRun: 0 event: 46 at 02-Feb-2026 21:47:46 CET
SimDepoSetSource got 26908 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 26908 depos spanning: [6.67143e-07, 2.56138]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=742509 n=100139 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=794279 n=83880 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 47th record. run: 100114 subRun: 0 event: 47 at 02-Feb-2026 21:48:16 CET
SimDepoSetSource got 46433 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 46433 depos spanning: [6.67603e-07, 2.37641]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=1.57057e+06 n=169255 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=1.73071e+06 n=147000 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 48th record. run: 100114 subRun: 0 event: 48 at 02-Feb-2026 21:48:47 CET
SimDepoSetSource got 50047 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 50047 depos spanning: [6.67443e-07, 1.71437]us
Retagger: tagging trace set: wiener with 15360 traces, 15360 summary
Retagger: tagging trace set: gauss with 15360 traces, 0 summary
wclsFrameSaver: saving 15360 traces tagged "gauss"
FrameSaver: q=1.67955e+06 n=190979 tag=gauss
wclsFrameSaver: saving 15360 traces tagged "wiener"
FrameSaver: q=1.93928e+06 n=156242 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 49th record. run: 100114 subRun: 0 event: 49 at 02-Feb-2026 21:49:37 CET
SimDepoSetSource got 14829 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 14829 depos spanning: [6.96437e-07, 2.70043]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=499865 n=46441 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=565281 n=40761 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 50th record. run: 100114 subRun: 0 event: 50 at 02-Feb-2026 21:49:54 CET
SimDepoSetSource got 28856 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 28856 depos spanning: [6.67425e-07, 7.98233]us
Retagger: tagging trace set: wiener with 10240 traces, 10240 summary
Retagger: tagging trace set: gauss with 10240 traces, 0 summary
wclsFrameSaver: saving 10240 traces tagged "gauss"
FrameSaver: q=967076 n=105999 tag=gauss
wclsFrameSaver: saving 10240 traces tagged "wiener"
FrameSaver: q=1.05595e+06 n=91637 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
02-Feb-2026 21:50:32 CET  Closed output file "prodgenie_monte-carlo-012654-000114_gen_g4_detsim.root"
02-Feb-2026 21:50:32 CET  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/1f/48/prodgenie_monte-carlo-012654-000114_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      7.41669       39.2627       118.668       29.6456       24.8582        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00033534     0.0162625     0.235267     0.00166193     0.0452548       50     
simulate:tpcrawdecoder:WireCellToolkit          7.31577       38.8749       117.143       29.391        24.582         50     
simulate:opdigi:OpDetDigitizerDUNE             0.0119657     0.178776      0.594871      0.141916      0.130734        50     
simulate:rns:RandomNumberSaver                1.5679e-05    2.99288e-05   0.000209493   2.56985e-05   2.67866e-05      50     
[art]:TriggerResults:TriggerResultInserter     6.983e-06    1.06371e-05   3.0257e-05    9.2575e-06    4.17017e-06      50     
end_path:out1:RootOutput                       2.104e-06    3.87088e-06   1.4637e-05    3.2765e-06    1.89105e-06      50     
end_path:out1:RootOutput(write)                0.0284248     0.192298      0.929578      0.142053      0.190687        50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4047.61 MB
  Peak resident set size usage (VmHWM): 2333.71 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 neutrino002 neutrino 661M Feb  2 21:50 prodgenie_monte-carlo-012654-000114_gen_g4_detsim.root
Wrote processed DID: usertests:prodgenie_monte-carlo-012654-000114_gen_g4.root
=== DETSIM job completed successfully ===
justIN time: 2026-02-04 04:48:35 UTC       justIN version: 01.06.00