Jobsub ID 222958.9@dunegpschedd02.fnal.gov

Jobsub ID	222958.9@dunegpschedd02.fnal.gov
Workflow ID	8443
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-10-03 03:31:48
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-10-03 03:56:53
From worker node	Hostname	dunegli-7105006-0-fnpc23116.fnal.gov
	cpuinfo	AMD EPYC 9354 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-10-03 03:32:35
Input files	monte-carlo-008443-001850
Jobscript	Exit code	0
	Real time	14m (870s)
	CPU time	6m (411s = 47%)
	Max RSS bytes	3050733568 (2909 MiB)
Outputting started	2025-10-03 03:47:05
Output files	fardet-hd:prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim.root
Finished	2025-10-03 03:56:53
Saved logs	justin-logs:222958.9-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

mChannelSink : 0.16 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.13 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.13 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.12 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.12 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.12 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.12 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.11 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.1 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.09 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.08 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.05 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.05 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.05 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.04 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Drifter : 0.04 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.04 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.04 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.04 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.03 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoSetFanout : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.02 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DepoBagger : 0.01 sec
[03:46:30.065] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0.01 sec
[03:46:30.065] I [ timer  ] Timer: wcls::FrameSaver : 0.01 sec
[03:46:30.065] I [ timer  ] Timer: wcls::SimDepoSource : 0 sec
[03:46:30.065] I [ timer  ] Timer: Total node execution : 9.239999998360872 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
Begin processing the 10th record. run: 8443 subRun: 0 event: 18500 at 03-Oct-2025 03:46:41 UTC
SimDepoSource got 4745 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSource: ready with 4745 depos spanning: [1, 3.43966]us
Retagger: tagging trace set: daq with 30720 traces, 0 summary
[03:46:50.269] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.45 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.26 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.25 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.24 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.24 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.24 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.24 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.24 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.23 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.22 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.21 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.15 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.15 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.14 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.14 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.14 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.14 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.13 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.12 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.12 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.11 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 0.11 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.06 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.05 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.05 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.05 sec
[03:46:50.270] I [ timer  ] Timer: wcls::SimChannelSink : 0.04 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.04 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.04 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.04 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.03 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.03 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.03 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.03 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.02 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Drifter : 0.02 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.02 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0.02 sec
[03:46:50.270] I [ timer  ] Timer: wcls::FrameSaver : 0.02 sec
[03:46:50.270] I [ timer  ] Timer: wcls::SimDepoSource : 0.01 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoSetFanout : 0.01 sec
[03:46:50.270] I [ timer  ] Timer: WireCell::Gen::DepoBagger : 0.01 sec
[03:46:50.270] I [ timer  ] Timer: Total node execution : 8.069999968633056 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
03-Oct-2025 03:47:02 UTC  Closed output file "prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim.root"
03-Oct-2025 03:47:02 UTC  Closed input file "prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      18.897        19.9202       21.6101       19.5265      0.924359        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00049309    0.000786256   0.00106628    0.000832587   0.000163171      10     
simulate:tpcrawdecoder:WireCellToolkit          9.79348       10.7239       13.2476       10.293         1.061         10     
simulate:opdigi:OpDetDigitizerDUNE             0.0203668     0.076263      0.150135      0.0747674     0.0418078       10     
simulate:rns:RandomNumberSaver                2.0451e-05    6.28186e-05   0.000362151   2.9365e-05    9.99937e-05      10     
[art]:TriggerResults:TriggerResultInserter     8.583e-06    1.45881e-05   4.4858e-05    1.14725e-05   1.01672e-05      10     
end_path:out1:RootOutput                       3.215e-06    6.2734e-06    2.9745e-05    3.6205e-06    7.83173e-06      10     
end_path:out1:RootOutput(write)                 8.20879       9.11878       10.2336       9.21609       0.73527        10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4686.2 MB
  Peak resident set size usage (VmHWM): 3050.73 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850.root 99203
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4.root 3394254
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim.root 1800554206
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim.root.ext.json 1472
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim.root.json 1772
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim.root.temp.json 1772
written output file: prod_muminus_1-500MeV_dune10kt_1x2x6_20251003T033941Z_gen_001850_g4_detsim_hist.root 519