Jobsub ID 271793.0@dunegpschedd02.fnal.gov

Jobsub ID	271793.0@dunegpschedd02.fnal.gov
Workflow ID	11873
Stage ID	3
User name	usher@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	8388608000 (8000 MiB)
	Wall seconds limit	28800 (8 hours)
Submitted time	2026-01-14 02:02:01
Site	ES_PIC
Entry	DUNE_T1_ES_PIC_ce15-multicore
Last heartbeat	2026-01-14 02:09:50
From worker node	Hostname	td814.pic.es
	cpuinfo	AMD EPYC 7452 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	10485760000 (10000 MiB)
	Wall seconds limit	216000 (60 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-14 02:02:38
Input files	usertests:prod_monte-carlo-011873-000001_gen_g4.root
Jobscript	Exit code	0
	Real time	5m (350s)
	CPU time	5m (329s = 93%)
	Max RSS bytes	1383698432 (1319 MiB)
Outputting started	2026-01-14 02:08:30
Output files	usertests:prod_monte-carlo-011873-000001_gen_g4_detsim.root
Finished	2026-01-14 02:09:50
Saved logs	justin-logs:271793.0-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

imDepoSetSource:"
Outputer: "wclsDepoFluxWriter:postdrift"
Outputer: "wclsFrameSaver:spsignals"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
wclsFrameSaver: promising to produce recob::Wires named "dnnsp"
14-Jan-2026 03:03:46 CET  Initiating request to open input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/62/44/prod_monte-carlo-011873-000001_gen_g4.root"
14-Jan-2026 03:03:50 CET  Opened input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/62/44/prod_monte-carlo-011873-000001_gen_g4.root"
Begin processing the 1st record. run: 20000014 subRun: 0 event: 1 at 14-Jan-2026 03:03:50 CET
SimDepoSetSource got 17041 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 17041 depos spanning: [0.162906, 0.179482]us
[03:03:54.249] W [  gen   ] <IncoherentAddNoise:addnoiseapa9> model EmpiricalNoiseModel:empericalnoisemeganodes produced 6174 samples instead of expected 6000, future warnings muted 
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=427900 n=67728 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=449228 n=66122 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 03:04:27 CET  Opened output file with pattern "prod_monte-carlo-011873-000001_gen_g4_detsim.root"
Begin processing the 2nd record. run: 20000014 subRun: 0 event: 2 at 14-Jan-2026 03:04:29 CET
SimDepoSetSource got 18411 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 18411 depos spanning: [0.589481, 0.609192]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=504008 n=75604 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=523741 n=73477 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 3rd record. run: 20000014 subRun: 0 event: 3 at 14-Jan-2026 03:05:07 CET
SimDepoSetSource got 1731 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 1731 depos spanning: [0.818741, 0.820743]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=46969.9 n=6129 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=49454.9 n=5359 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 4th record. run: 20000014 subRun: 0 event: 4 at 14-Jan-2026 03:05:24 CET
SimDepoSetSource got 11219 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 11219 depos spanning: [0.323056, 0.568424]us
Retagger: tagging trace set: wiener with 7680 traces, 7680 summary
Retagger: tagging trace set: gauss with 7680 traces, 0 summary
wclsFrameSaver: saving 7680 traces tagged "gauss"
FrameSaver: q=358556 n=46280 tag=gauss
wclsFrameSaver: saving 7680 traces tagged "wiener"
FrameSaver: q=407244 n=37646 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 5th record. run: 20000014 subRun: 0 event: 5 at 14-Jan-2026 03:06:08 CET
SimDepoSetSource got 517 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 517 depos spanning: [0.858313, 0.8617]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=15389.4 n=3268 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=16602.4 n=3051 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 6th record. run: 20000014 subRun: 0 event: 6 at 14-Jan-2026 03:06:20 CET
SimDepoSetSource got 10293 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 10293 depos spanning: [0.286489, 0.299863]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=325908 n=45015 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=353557 n=37065 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 7th record. run: 20000014 subRun: 0 event: 7 at 14-Jan-2026 03:06:48 CET
SimDepoSetSource got 2810 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 2810 depos spanning: [0.228638, 0.232568]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=75195.7 n=7070 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=82899 n=5872 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 8th record. run: 20000014 subRun: 0 event: 8 at 14-Jan-2026 03:07:03 CET
SimDepoSetSource got 5325 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5325 depos spanning: [0.130245, 0.442354]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=150517 n=25881 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=154970 n=25201 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 9th record. run: 20000014 subRun: 0 event: 9 at 14-Jan-2026 03:07:35 CET
SimDepoSetSource got 7812 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 7812 depos spanning: [0.782006, 0.79097]us
Retagger: tagging trace set: wiener with 5120 traces, 5120 summary
Retagger: tagging trace set: gauss with 5120 traces, 0 summary
wclsFrameSaver: saving 5120 traces tagged "gauss"
FrameSaver: q=190430 n=32458 tag=gauss
wclsFrameSaver: saving 5120 traces tagged "wiener"
FrameSaver: q=198745 n=31972 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
Begin processing the 10th record. run: 20000014 subRun: 0 event: 10 at 14-Jan-2026 03:08:08 CET
SimDepoSetSource got 5274 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSetSource: ready with 5274 depos spanning: [0.951731, 0.957603]us
Retagger: tagging trace set: wiener with 2560 traces, 2560 summary
Retagger: tagging trace set: gauss with 2560 traces, 0 summary
wclsFrameSaver: saving 2560 traces tagged "gauss"
FrameSaver: q=137416 n=13074 tag=gauss
wclsFrameSaver: saving 2560 traces tagged "wiener"
FrameSaver: q=144514 n=11072 tag=wiener
wclsFrameSaver: no traces tagged "dnnsp"
FrameSaver: q=0 n=0 tag=dnnsp
14-Jan-2026 03:08:28 CET  Closed output file "prod_monte-carlo-011873-000001_gen_g4_detsim.root"
14-Jan-2026 03:08:28 CET  Closed input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/usertests/62/44/prod_monte-carlo-011873-000001_gen_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      12.224        27.4682       44.2047       30.0641       10.6497        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000349173   0.000825824   0.00129191    0.000828557   0.000348107      10     
simulate:tpcrawdecoder:WireCellToolkit          12.0883       27.1314       43.848        29.6991       10.5488        10     
simulate:opdigi:OpDetDigitizerDUNE             0.0588298     0.193555      0.315375       0.1993       0.070669        10     
simulate:rns:RandomNumberSaver                2.1561e-05    6.09335e-05   0.000351343   2.6475e-05    9.70882e-05      10     
[art]:TriggerResults:TriggerResultInserter     1.044e-05    1.88264e-05    6.14e-05     1.41255e-05   1.4285e-05       10     
end_path:out1:RootOutput                        2.7e-06      5.254e-06     2.373e-05     3.11e-06     6.17016e-06      10     
end_path:out1:RootOutput(write)                0.0754664     0.141981      0.245248      0.125617      0.0568739       10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3129.12 MB
  Peak resident set size usage (VmHWM): 1383.7 MB
====================================================================================================
Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 30M Jan 14 03:08 prod_monte-carlo-011873-000001_gen_g4_detsim.root
Wrote processed DID: usertests:prod_monte-carlo-011873-000001_gen_g4.root
=== DETSIM job completed successfully ===