Jobsub ID 243184.2@dunegpschedd02.fnal.gov

Jobsub ID	243184.2@dunegpschedd02.fnal.gov
Workflow ID	9970
Stage ID	1
User name	schellma@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-13 15:49:36
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-13 17:09:44
From worker node	Hostname	dunegli-7585892-0-fnpc23011.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-13 17:06:20
Input files	monte-carlo-009970-000026
Jobscript	Exit code	0
	Real time	2m (175s)
	CPU time	0m (32s = 18%)
	Max RSS bytes	608522240 (580 MiB)
Outputting started	2025-11-13 17:09:16
Output files	usertests:prodmarley_nue_flat_es_dune10kt_1x2x2.fcl_trg_mc_2025a_detsim_dune10kt_1x2x2_notpcsigproc.fcl_detsim_v10_06_00d01_triggerana_dune10kt_1x2x2.fcl_s2500_l3500_202511111537_hists_merged_20251112T013820_f25.root
Finished	2025-11-13 17:09:44
Saved logs	justin-logs:243184.2-dunegpschedd02.fnal.gov.logs.tgz
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

4822nd record. run: 8145 subRun: 0 event: 22722 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4823rd record. run: 8145 subRun: 0 event: 22723 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4824th record. run: 8145 subRun: 0 event: 22724 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4825th record. run: 8145 subRun: 0 event: 22725 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4826th record. run: 8145 subRun: 0 event: 22726 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4827th record. run: 8145 subRun: 0 event: 22727 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4828th record. run: 8145 subRun: 0 event: 22728 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4829th record. run: 8145 subRun: 0 event: 22729 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4830th record. run: 8145 subRun: 0 event: 22730 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4831st record. run: 8145 subRun: 0 event: 22731 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4832nd record. run: 8145 subRun: 0 event: 22732 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4833rd record. run: 8145 subRun: 0 event: 22733 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4834th record. run: 8145 subRun: 0 event: 22734 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4835th record. run: 8145 subRun: 0 event: 22735 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4836th record. run: 8145 subRun: 0 event: 22736 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4837th record. run: 8145 subRun: 0 event: 22737 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4838th record. run: 8145 subRun: 0 event: 22738 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4839th record. run: 8145 subRun: 0 event: 22739 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4840th record. run: 8145 subRun: 0 event: 22740 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4841st record. run: 8145 subRun: 0 event: 22741 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4842nd record. run: 8145 subRun: 0 event: 22742 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4843rd record. run: 8145 subRun: 0 event: 22743 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4844th record. run: 8145 subRun: 0 event: 22744 at 13-Nov-2025 17:09:13 UTC
Begin processing the 4845th record. run: 8145 subRun: 0 event: 22745 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4846th record. run: 8145 subRun: 0 event: 22746 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4847th record. run: 8145 subRun: 0 event: 22747 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4848th record. run: 8145 subRun: 0 event: 22748 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4849th record. run: 8145 subRun: 0 event: 22749 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4850th record. run: 8145 subRun: 0 event: 22750 at 13-Nov-2025 17:09:14 UTC
13-Nov-2025 17:09:14 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/b0/2a/prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T064605Z_gen_000455_supernova_g4_detsim.root"
13-Nov-2025 17:09:14 UTC  Initiating request to open input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5d/99/prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T064628Z_gen_000459_supernova_g4_detsim.root"
13-Nov-2025 17:09:14 UTC  Opened input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5d/99/prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T064628Z_gen_000459_supernova_g4_detsim.root"
Begin processing the 4851st record. run: 8145 subRun: 0 event: 22901 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4852nd record. run: 8145 subRun: 0 event: 22902 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4853rd record. run: 8145 subRun: 0 event: 22903 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4854th record. run: 8145 subRun: 0 event: 22904 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4855th record. run: 8145 subRun: 0 event: 22905 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4856th record. run: 8145 subRun: 0 event: 22906 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4857th record. run: 8145 subRun: 0 event: 22907 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4858th record. run: 8145 subRun: 0 event: 22908 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4859th record. run: 8145 subRun: 0 event: 22909 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4860th record. run: 8145 subRun: 0 event: 22910 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4861st record. run: 8145 subRun: 0 event: 22911 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4862nd record. run: 8145 subRun: 0 event: 22912 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4863rd record. run: 8145 subRun: 0 event: 22913 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4864th record. run: 8145 subRun: 0 event: 22914 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4865th record. run: 8145 subRun: 0 event: 22915 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4866th record. run: 8145 subRun: 0 event: 22916 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4867th record. run: 8145 subRun: 0 event: 22917 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4868th record. run: 8145 subRun: 0 event: 22918 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4869th record. run: 8145 subRun: 0 event: 22919 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4870th record. run: 8145 subRun: 0 event: 22920 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4871st record. run: 8145 subRun: 0 event: 22921 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4872nd record. run: 8145 subRun: 0 event: 22922 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4873rd record. run: 8145 subRun: 0 event: 22923 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4874th record. run: 8145 subRun: 0 event: 22924 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4875th record. run: 8145 subRun: 0 event: 22925 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4876th record. run: 8145 subRun: 0 event: 22926 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4877th record. run: 8145 subRun: 0 event: 22927 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4878th record. run: 8145 subRun: 0 event: 22928 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4879th record. run: 8145 subRun: 0 event: 22929 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4880th record. run: 8145 subRun: 0 event: 22930 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4881st record. run: 8145 subRun: 0 event: 22931 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4882nd record. run: 8145 subRun: 0 event: 22932 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4883rd record. run: 8145 subRun: 0 event: 22933 at 13-Nov-2025 17:09:14 UTC
Begin processing the 4884th record. run: 8145 subRun: 0 event: 22934 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4885th record. run: 8145 subRun: 0 event: 22935 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4886th record. run: 8145 subRun: 0 event: 22936 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4887th record. run: 8145 subRun: 0 event: 22937 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4888th record. run: 8145 subRun: 0 event: 22938 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4889th record. run: 8145 subRun: 0 event: 22939 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4890th record. run: 8145 subRun: 0 event: 22940 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4891st record. run: 8145 subRun: 0 event: 22941 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4892nd record. run: 8145 subRun: 0 event: 22942 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4893rd record. run: 8145 subRun: 0 event: 22943 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4894th record. run: 8145 subRun: 0 event: 22944 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4895th record. run: 8145 subRun: 0 event: 22945 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4896th record. run: 8145 subRun: 0 event: 22946 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4897th record. run: 8145 subRun: 0 event: 22947 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4898th record. run: 8145 subRun: 0 event: 22948 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4899th record. run: 8145 subRun: 0 event: 22949 at 13-Nov-2025 17:09:15 UTC
Begin processing the 4900th record. run: 8145 subRun: 0 event: 22950 at 13-Nov-2025 17:09:15 UTC
13-Nov-2025 17:09:15 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5d/99/prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T064628Z_gen_000459_supernova_g4_detsim.root"

========================================================================================================================
TimeTracker printout (sec)                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================
Full event                            0.000697025   0.00342261     0.275354     0.00141343     0.0112088      4900    
------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                5.6498e-05    0.00136422     0.225545     7.0033e-05    0.00665896      4900    
end_path:triggerAna:TriggerAnaTree    0.000577648    0.0018004     0.274626     0.00113991    0.00789317      4900    
========================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1300.74 MB
  Peak resident set size usage (VmHWM): 608.522 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.

merged pass1_US_FNAL-FermiGrid_000026.json