Jobsub ID 243569.2@dunegpschedd01.fnal.gov

Jobsub ID	243569.2@dunegpschedd01.fnal.gov
Workflow ID	9615
Stage ID	1
User name	epennacc@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-05 15:01:31
Site	US_UChicago
Entry	Engage_US_MWT2_uct2_gk02_condce_mcore
Last heartbeat	2025-11-05 16:24:49
From worker node	Hostname	mwt2-c210.campuscluster.illinois.edu
	cpuinfo	AMD EPYC 7402 24-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-05 15:40:59
Input files	fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170804Z_gen_002086_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T165451Z_gen_001259_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164537Z_gen_000860_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim.root
Jobscript	Exit code	0
	Real time	23m (1405s)
	CPU time	21m (1283s = 91%)
	Max RSS bytes	791781376 (755 MiB)
Outputting started	2025-11-05 16:04:25
Output files	fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170804Z_gen_002086_supernova_g4_detsim_20251105T154106Z_reco.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T165451Z_gen_001259_supernova_g4_detsim_20251105T154725Z_reco.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164537Z_gen_000860_supernova_g4_detsim_20251105T155311Z_reco.root
Finished	2025-11-05 16:24:49
Saved logs	justin-logs:243569.2-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

:04 CST  Opened input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/cd/3c/prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim.root"
Begin processing the 1st record. run: 8106 subRun: 0 event: 177151 at 05-Nov-2025 09:59:05 CST
05-Nov-2025 09:59:11 CST  Opened output file with pattern "prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root"
Begin processing the 2nd record. run: 8106 subRun: 0 event: 177152 at 05-Nov-2025 09:59:34 CST
Begin processing the 3rd record. run: 8106 subRun: 0 event: 177153 at 05-Nov-2025 09:59:40 CST
Begin processing the 4th record. run: 8106 subRun: 0 event: 177154 at 05-Nov-2025 09:59:46 CST
Begin processing the 5th record. run: 8106 subRun: 0 event: 177155 at 05-Nov-2025 09:59:52 CST
Begin processing the 6th record. run: 8106 subRun: 0 event: 177156 at 05-Nov-2025 09:59:58 CST
Begin processing the 7th record. run: 8106 subRun: 0 event: 177157 at 05-Nov-2025 10:00:04 CST
Begin processing the 8th record. run: 8106 subRun: 0 event: 177158 at 05-Nov-2025 10:00:10 CST
Begin processing the 9th record. run: 8106 subRun: 0 event: 177159 at 05-Nov-2025 10:00:16 CST
Begin processing the 10th record. run: 8106 subRun: 0 event: 177160 at 05-Nov-2025 10:00:21 CST
Begin processing the 11th record. run: 8106 subRun: 0 event: 177161 at 05-Nov-2025 10:00:27 CST
Begin processing the 12th record. run: 8106 subRun: 0 event: 177162 at 05-Nov-2025 10:00:33 CST
Begin processing the 13th record. run: 8106 subRun: 0 event: 177163 at 05-Nov-2025 10:00:39 CST
Begin processing the 14th record. run: 8106 subRun: 0 event: 177164 at 05-Nov-2025 10:00:44 CST
Begin processing the 15th record. run: 8106 subRun: 0 event: 177165 at 05-Nov-2025 10:00:50 CST
Begin processing the 16th record. run: 8106 subRun: 0 event: 177166 at 05-Nov-2025 10:00:56 CST
Begin processing the 17th record. run: 8106 subRun: 0 event: 177167 at 05-Nov-2025 10:01:02 CST
Begin processing the 18th record. run: 8106 subRun: 0 event: 177168 at 05-Nov-2025 10:01:08 CST
Begin processing the 19th record. run: 8106 subRun: 0 event: 177169 at 05-Nov-2025 10:01:14 CST
Begin processing the 20th record. run: 8106 subRun: 0 event: 177170 at 05-Nov-2025 10:01:19 CST
Begin processing the 21st record. run: 8106 subRun: 0 event: 177171 at 05-Nov-2025 10:01:25 CST
Begin processing the 22nd record. run: 8106 subRun: 0 event: 177172 at 05-Nov-2025 10:01:31 CST
Begin processing the 23rd record. run: 8106 subRun: 0 event: 177173 at 05-Nov-2025 10:01:37 CST
Begin processing the 24th record. run: 8106 subRun: 0 event: 177174 at 05-Nov-2025 10:01:43 CST
Begin processing the 25th record. run: 8106 subRun: 0 event: 177175 at 05-Nov-2025 10:01:49 CST
Begin processing the 26th record. run: 8106 subRun: 0 event: 177176 at 05-Nov-2025 10:01:55 CST
Begin processing the 27th record. run: 8106 subRun: 0 event: 177177 at 05-Nov-2025 10:02:00 CST
Begin processing the 28th record. run: 8106 subRun: 0 event: 177178 at 05-Nov-2025 10:02:06 CST
Begin processing the 29th record. run: 8106 subRun: 0 event: 177179 at 05-Nov-2025 10:02:12 CST
Begin processing the 30th record. run: 8106 subRun: 0 event: 177180 at 05-Nov-2025 10:02:18 CST
Begin processing the 31st record. run: 8106 subRun: 0 event: 177181 at 05-Nov-2025 10:02:23 CST
Begin processing the 32nd record. run: 8106 subRun: 0 event: 177182 at 05-Nov-2025 10:02:29 CST
Begin processing the 33rd record. run: 8106 subRun: 0 event: 177183 at 05-Nov-2025 10:02:35 CST
Begin processing the 34th record. run: 8106 subRun: 0 event: 177184 at 05-Nov-2025 10:02:41 CST
Begin processing the 35th record. run: 8106 subRun: 0 event: 177185 at 05-Nov-2025 10:02:47 CST
Begin processing the 36th record. run: 8106 subRun: 0 event: 177186 at 05-Nov-2025 10:02:52 CST
Begin processing the 37th record. run: 8106 subRun: 0 event: 177187 at 05-Nov-2025 10:02:58 CST
Begin processing the 38th record. run: 8106 subRun: 0 event: 177188 at 05-Nov-2025 10:03:04 CST
Begin processing the 39th record. run: 8106 subRun: 0 event: 177189 at 05-Nov-2025 10:03:10 CST
Begin processing the 40th record. run: 8106 subRun: 0 event: 177190 at 05-Nov-2025 10:03:15 CST
Begin processing the 41st record. run: 8106 subRun: 0 event: 177191 at 05-Nov-2025 10:03:21 CST
Begin processing the 42nd record. run: 8106 subRun: 0 event: 177192 at 05-Nov-2025 10:03:27 CST
Begin processing the 43rd record. run: 8106 subRun: 0 event: 177193 at 05-Nov-2025 10:03:33 CST
Begin processing the 44th record. run: 8106 subRun: 0 event: 177194 at 05-Nov-2025 10:03:39 CST
Begin processing the 45th record. run: 8106 subRun: 0 event: 177195 at 05-Nov-2025 10:03:46 CST
Begin processing the 46th record. run: 8106 subRun: 0 event: 177196 at 05-Nov-2025 10:03:52 CST
Begin processing the 47th record. run: 8106 subRun: 0 event: 177197 at 05-Nov-2025 10:03:57 CST
Begin processing the 48th record. run: 8106 subRun: 0 event: 177198 at 05-Nov-2025 10:04:03 CST
Begin processing the 49th record. run: 8106 subRun: 0 event: 177199 at 05-Nov-2025 10:04:09 CST
Begin processing the 50th record. run: 8106 subRun: 0 event: 177200 at 05-Nov-2025 10:04:15 CST
05-Nov-2025 10:04:22 CST  Closed output file "prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root"
05-Nov-2025 10:04:22 CST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/cd/3c/prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim.root"

=========================================================================================================================================
TimeTracker printout (sec)                                 Min           Avg           Max         Median          RMS         nEvts   
=========================================================================================================================================
Full event                                               5.53096       5.78497       6.58644       5.74136      0.198491        50     
-----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.00026528    0.00131577     0.0154927    0.000384635   0.00248562       50     
makers:tpmakerTPCsimpleThr:TriggerPrimitiveMakerTPC      2.84171       3.04639       3.65377       3.00134      0.151525        50     
makers:tpmakerTPCabsRS:TriggerPrimitiveMakerTPC          2.60979       2.73412       3.0619        2.72063      0.076896        50     
[art]:TriggerResults:TriggerResultInserter             1.3496e-05    2.36641e-05   0.000131838   2.09895e-05   1.61166e-05      50     
end_path:out1:RootOutput                                3.416e-06    7.81852e-06   3.0748e-05     6.442e-06    4.8427e-06       50     
end_path:out1:RootOutput(write)                        0.00133344    0.00254834    0.00520932    0.00183096    0.00114776       50     
=========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1534.74 MB
  Peak resident set size usage (VmHWM): 791.781 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
did.list
file.list
jobscript.log
mem.db
old_md.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164537Z_gen_000860_supernova_g4_detsim_20251105T155311Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164537Z_gen_000860_supernova_g4_detsim_20251105T155311Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164537Z_gen_000860_supernova_g4_detsim_20251105T155311Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164537Z_gen_000860_supernova_g4_detsim_20251105T155311Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T165451Z_gen_001259_supernova_g4_detsim_20251105T154725Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T165451Z_gen_001259_supernova_g4_detsim_20251105T154725Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T165451Z_gen_001259_supernova_g4_detsim_20251105T154725Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T165451Z_gen_001259_supernova_g4_detsim_20251105T154725Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170804Z_gen_002086_supernova_g4_detsim_20251105T154106Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170804Z_gen_002086_supernova_g4_detsim_20251105T154106Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170804Z_gen_002086_supernova_g4_detsim_20251105T154106Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170804Z_gen_002086_supernova_g4_detsim_20251105T154106Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T182912Z_gen_003544_supernova_g4_detsim_20251105T155850Z_reco.root.temp.json
temp_pfns.txt
time.db
triggersim_hist.root