Jobsub ID 237887.0@dunegpschedd02.fnal.gov

Jobsub ID	237887.0@dunegpschedd02.fnal.gov
Workflow ID	9627
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-05 16:19:35
Site	US_UChicago
Entry	Engage_US_MWT2_uiuc_gk02_condce_mcore
Last heartbeat	2025-11-05 17:10:45
From worker node	Hostname	iut2-c307.iu.edu
	cpuinfo	Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-05 16:20:44
Input files	fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T210204Z_gen_002537_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T191130Z_gen_001950_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T192246Z_gen_002471_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185024Z_gen_000780_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim.root
Jobscript	Exit code	0
	Real time	33m (2022s)
	CPU time	31m (1874s = 92%)
	Max RSS bytes	804421632 (767 MiB)
Outputting started	2025-11-05 16:54:27
Output files	fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T191130Z_gen_001950_supernova_g4_detsim_20251105T162738Z_reco.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185024Z_gen_000780_supernova_g4_detsim_20251105T164101Z_reco.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T210204Z_gen_002537_supernova_g4_detsim_20251105T162050Z_reco.root fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T192246Z_gen_002471_supernova_g4_detsim_20251105T163422Z_reco.root
Finished	2025-11-05 17:10:23
Saved logs	justin-logs:237887.0-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

"
Begin processing the 2nd record. run: 8111 subRun: 0 event: 149752 at 05-Nov-2025 11:48:35 EST
Begin processing the 3rd record. run: 8111 subRun: 0 event: 149753 at 05-Nov-2025 11:48:42 EST
Begin processing the 4th record. run: 8111 subRun: 0 event: 149754 at 05-Nov-2025 11:48:49 EST
Begin processing the 5th record. run: 8111 subRun: 0 event: 149755 at 05-Nov-2025 11:48:56 EST
Begin processing the 6th record. run: 8111 subRun: 0 event: 149756 at 05-Nov-2025 11:49:04 EST
Begin processing the 7th record. run: 8111 subRun: 0 event: 149757 at 05-Nov-2025 11:49:11 EST
Begin processing the 8th record. run: 8111 subRun: 0 event: 149758 at 05-Nov-2025 11:49:19 EST
Begin processing the 9th record. run: 8111 subRun: 0 event: 149759 at 05-Nov-2025 11:49:26 EST
Begin processing the 10th record. run: 8111 subRun: 0 event: 149760 at 05-Nov-2025 11:49:33 EST
Begin processing the 11th record. run: 8111 subRun: 0 event: 149761 at 05-Nov-2025 11:49:40 EST
Begin processing the 12th record. run: 8111 subRun: 0 event: 149762 at 05-Nov-2025 11:49:47 EST
Begin processing the 13th record. run: 8111 subRun: 0 event: 149763 at 05-Nov-2025 11:49:54 EST
Begin processing the 14th record. run: 8111 subRun: 0 event: 149764 at 05-Nov-2025 11:50:00 EST
Begin processing the 15th record. run: 8111 subRun: 0 event: 149765 at 05-Nov-2025 11:50:07 EST
Begin processing the 16th record. run: 8111 subRun: 0 event: 149766 at 05-Nov-2025 11:50:15 EST
Begin processing the 17th record. run: 8111 subRun: 0 event: 149767 at 05-Nov-2025 11:50:22 EST
Begin processing the 18th record. run: 8111 subRun: 0 event: 149768 at 05-Nov-2025 11:50:29 EST
Begin processing the 19th record. run: 8111 subRun: 0 event: 149769 at 05-Nov-2025 11:50:36 EST
Begin processing the 20th record. run: 8111 subRun: 0 event: 149770 at 05-Nov-2025 11:50:43 EST
Begin processing the 21st record. run: 8111 subRun: 0 event: 149771 at 05-Nov-2025 11:50:50 EST
Begin processing the 22nd record. run: 8111 subRun: 0 event: 149772 at 05-Nov-2025 11:50:57 EST
Begin processing the 23rd record. run: 8111 subRun: 0 event: 149773 at 05-Nov-2025 11:51:04 EST
Begin processing the 24th record. run: 8111 subRun: 0 event: 149774 at 05-Nov-2025 11:51:11 EST
Begin processing the 25th record. run: 8111 subRun: 0 event: 149775 at 05-Nov-2025 11:51:18 EST
Begin processing the 26th record. run: 8111 subRun: 0 event: 149776 at 05-Nov-2025 11:51:25 EST
Begin processing the 27th record. run: 8111 subRun: 0 event: 149777 at 05-Nov-2025 11:51:32 EST
Begin processing the 28th record. run: 8111 subRun: 0 event: 149778 at 05-Nov-2025 11:51:39 EST
Begin processing the 29th record. run: 8111 subRun: 0 event: 149779 at 05-Nov-2025 11:51:46 EST
Begin processing the 30th record. run: 8111 subRun: 0 event: 149780 at 05-Nov-2025 11:51:53 EST
Begin processing the 31st record. run: 8111 subRun: 0 event: 149781 at 05-Nov-2025 11:52:00 EST
Begin processing the 32nd record. run: 8111 subRun: 0 event: 149782 at 05-Nov-2025 11:52:08 EST
Begin processing the 33rd record. run: 8111 subRun: 0 event: 149783 at 05-Nov-2025 11:52:15 EST
Begin processing the 34th record. run: 8111 subRun: 0 event: 149784 at 05-Nov-2025 11:52:22 EST
Begin processing the 35th record. run: 8111 subRun: 0 event: 149785 at 05-Nov-2025 11:52:29 EST
Begin processing the 36th record. run: 8111 subRun: 0 event: 149786 at 05-Nov-2025 11:52:36 EST
Begin processing the 37th record. run: 8111 subRun: 0 event: 149787 at 05-Nov-2025 11:52:43 EST
Begin processing the 38th record. run: 8111 subRun: 0 event: 149788 at 05-Nov-2025 11:52:50 EST
Begin processing the 39th record. run: 8111 subRun: 0 event: 149789 at 05-Nov-2025 11:52:57 EST
Begin processing the 40th record. run: 8111 subRun: 0 event: 149790 at 05-Nov-2025 11:53:04 EST
Begin processing the 41st record. run: 8111 subRun: 0 event: 149791 at 05-Nov-2025 11:53:11 EST
Begin processing the 42nd record. run: 8111 subRun: 0 event: 149792 at 05-Nov-2025 11:53:18 EST
Begin processing the 43rd record. run: 8111 subRun: 0 event: 149793 at 05-Nov-2025 11:53:25 EST
Begin processing the 44th record. run: 8111 subRun: 0 event: 149794 at 05-Nov-2025 11:53:31 EST
Begin processing the 45th record. run: 8111 subRun: 0 event: 149795 at 05-Nov-2025 11:53:39 EST
Begin processing the 46th record. run: 8111 subRun: 0 event: 149796 at 05-Nov-2025 11:53:46 EST
Begin processing the 47th record. run: 8111 subRun: 0 event: 149797 at 05-Nov-2025 11:53:54 EST
Begin processing the 48th record. run: 8111 subRun: 0 event: 149798 at 05-Nov-2025 11:54:01 EST
Begin processing the 49th record. run: 8111 subRun: 0 event: 149799 at 05-Nov-2025 11:54:08 EST
Begin processing the 50th record. run: 8111 subRun: 0 event: 149800 at 05-Nov-2025 11:54:16 EST
05-Nov-2025 11:54:24 EST  Closed output file "prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root"
05-Nov-2025 11:54:24 EST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/7b/c9/prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim.root"

=========================================================================================================================================
TimeTracker printout (sec)                                 Min           Avg           Max         Median          RMS         nEvts   
=========================================================================================================================================
Full event                                               6.50377       7.01172       7.9399        6.93945      0.253367        50     
-----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.000257186    0.0015654     0.0332838    0.00036682     0.0048079       50     
makers:tpmakerTPCsimpleThr:TriggerPrimitiveMakerTPC      3.38405       3.71594       4.21255       3.68101      0.186167        50     
makers:tpmakerTPCabsRS:TriggerPrimitiveMakerTPC          3.11567       3.29114       3.72398       3.28979      0.102076        50     
[art]:TriggerResults:TriggerResultInserter             2.2091e-05    3.55029e-05   0.00013865    3.29575e-05   1.64842e-05      50     
end_path:out1:RootOutput                                3.126e-06    7.2834e-06    4.3013e-05    6.4115e-06    5.3133e-06       50     
end_path:out1:RootOutput(write)                         0.0015594    0.00245772    0.00473043    0.00219285    0.000757978      50     
=========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1484.1 MB
  Peak resident set size usage (VmHWM): 753.484 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
did.list
file.list
jobscript.log
mem.db
old_md.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185024Z_gen_000780_supernova_g4_detsim_20251105T164101Z_reco.root
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185024Z_gen_000780_supernova_g4_detsim_20251105T164101Z_reco.root.ext.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185024Z_gen_000780_supernova_g4_detsim_20251105T164101Z_reco.root.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185024Z_gen_000780_supernova_g4_detsim_20251105T164101Z_reco.root.temp.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T191130Z_gen_001950_supernova_g4_detsim_20251105T162738Z_reco.root
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T191130Z_gen_001950_supernova_g4_detsim_20251105T162738Z_reco.root.ext.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T191130Z_gen_001950_supernova_g4_detsim_20251105T162738Z_reco.root.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T191130Z_gen_001950_supernova_g4_detsim_20251105T162738Z_reco.root.temp.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T192246Z_gen_002471_supernova_g4_detsim_20251105T163422Z_reco.root
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T192246Z_gen_002471_supernova_g4_detsim_20251105T163422Z_reco.root.ext.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T192246Z_gen_002471_supernova_g4_detsim_20251105T163422Z_reco.root.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T192246Z_gen_002471_supernova_g4_detsim_20251105T163422Z_reco.root.temp.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root.ext.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T193223Z_gen_002996_supernova_g4_detsim_20251105T164737Z_reco.root.temp.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T210204Z_gen_002537_supernova_g4_detsim_20251105T162050Z_reco.root
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T210204Z_gen_002537_supernova_g4_detsim_20251105T162050Z_reco.root.ext.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T210204Z_gen_002537_supernova_g4_detsim_20251105T162050Z_reco.root.json
prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T210204Z_gen_002537_supernova_g4_detsim_20251105T162050Z_reco.root.temp.json
temp_pfns.txt
time.db
triggersim_hist.root