Jobsub ID 242938.6@dunegpschedd02.fnal.gov

Jobsub ID	242938.6@dunegpschedd02.fnal.gov
Workflow ID	10083
Stage ID	1
User name	epennacc@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-13 09:29:15
Site	US_UCSD
Entry	CMSHTPC_T2_US_UCSD_gw6
Last heartbeat	2025-11-13 17:29:26
From worker node	Hostname	mh-7763-14.t2.ucsd.edu
	cpuinfo	AMD EPYC 7763 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-13 09:29:54
Input files	fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164953Z_gen_001062_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164634Z_gen_000931_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T162221Z_gen_000200_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim.root
Jobscript	Exit code	0
	Real time	7h (27898s)
	CPU time	29m (1742s = 6%)
	Max RSS bytes	769040384 (733 MiB)
Outputting started	2025-11-13 17:14:53
Output files	fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164953Z_gen_001062_supernova_g4_detsim_20251113T093003Z_reco.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164634Z_gen_000931_supernova_g4_detsim_20251113T102906Z_reco.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T162221Z_gen_000200_supernova_g4_detsim_20251113T112712Z_reco.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root
Finished	2025-11-13 17:29:26
Saved logs	justin-logs:242938.6-dunegpschedd02.fnal.gov.logs.tgz
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

:02 PST  Opened input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/16/bb/prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim.root"
Begin processing the 1st record. run: 8106 subRun: 0 event: 112301 at 13-Nov-2025 04:16:05 PST
13-Nov-2025 04:19:00 PST  Opened output file with pattern "prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root"
Begin processing the 2nd record. run: 8106 subRun: 0 event: 112302 at 13-Nov-2025 06:42:22 PST
Begin processing the 3rd record. run: 8106 subRun: 0 event: 112303 at 13-Nov-2025 06:45:11 PST
Begin processing the 4th record. run: 8106 subRun: 0 event: 112304 at 13-Nov-2025 06:48:13 PST
Begin processing the 5th record. run: 8106 subRun: 0 event: 112305 at 13-Nov-2025 06:51:09 PST
Begin processing the 6th record. run: 8106 subRun: 0 event: 112306 at 13-Nov-2025 06:54:02 PST
Begin processing the 7th record. run: 8106 subRun: 0 event: 112307 at 13-Nov-2025 06:56:57 PST
Begin processing the 8th record. run: 8106 subRun: 0 event: 112308 at 13-Nov-2025 06:59:51 PST
Begin processing the 9th record. run: 8106 subRun: 0 event: 112309 at 13-Nov-2025 07:02:52 PST
Begin processing the 10th record. run: 8106 subRun: 0 event: 112310 at 13-Nov-2025 07:05:54 PST
Begin processing the 11th record. run: 8106 subRun: 0 event: 112311 at 13-Nov-2025 07:08:52 PST
Begin processing the 12th record. run: 8106 subRun: 0 event: 112312 at 13-Nov-2025 07:11:59 PST
Begin processing the 13th record. run: 8106 subRun: 0 event: 112313 at 13-Nov-2025 07:15:10 PST
Begin processing the 14th record. run: 8106 subRun: 0 event: 112314 at 13-Nov-2025 07:18:23 PST
Begin processing the 15th record. run: 8106 subRun: 0 event: 112315 at 13-Nov-2025 07:21:16 PST
Begin processing the 16th record. run: 8106 subRun: 0 event: 112316 at 13-Nov-2025 07:24:16 PST
Begin processing the 17th record. run: 8106 subRun: 0 event: 112317 at 13-Nov-2025 07:27:16 PST
Begin processing the 18th record. run: 8106 subRun: 0 event: 112318 at 13-Nov-2025 07:30:12 PST
Begin processing the 19th record. run: 8106 subRun: 0 event: 112319 at 13-Nov-2025 07:32:58 PST
Begin processing the 20th record. run: 8106 subRun: 0 event: 112320 at 13-Nov-2025 07:35:54 PST
Begin processing the 21st record. run: 8106 subRun: 0 event: 112321 at 13-Nov-2025 07:39:02 PST
Begin processing the 22nd record. run: 8106 subRun: 0 event: 112322 at 13-Nov-2025 07:42:20 PST
Begin processing the 23rd record. run: 8106 subRun: 0 event: 112323 at 13-Nov-2025 07:45:27 PST
Begin processing the 24th record. run: 8106 subRun: 0 event: 112324 at 13-Nov-2025 07:48:34 PST
Begin processing the 25th record. run: 8106 subRun: 0 event: 112325 at 13-Nov-2025 07:51:49 PST
Begin processing the 26th record. run: 8106 subRun: 0 event: 112326 at 13-Nov-2025 07:54:52 PST
Begin processing the 27th record. run: 8106 subRun: 0 event: 112327 at 13-Nov-2025 07:57:57 PST
Begin processing the 28th record. run: 8106 subRun: 0 event: 112328 at 13-Nov-2025 08:01:01 PST
Begin processing the 29th record. run: 8106 subRun: 0 event: 112329 at 13-Nov-2025 08:03:57 PST
Begin processing the 30th record. run: 8106 subRun: 0 event: 112330 at 13-Nov-2025 08:06:59 PST
Begin processing the 31st record. run: 8106 subRun: 0 event: 112331 at 13-Nov-2025 08:10:14 PST
Begin processing the 32nd record. run: 8106 subRun: 0 event: 112332 at 13-Nov-2025 08:13:19 PST
Begin processing the 33rd record. run: 8106 subRun: 0 event: 112333 at 13-Nov-2025 08:16:27 PST
Begin processing the 34th record. run: 8106 subRun: 0 event: 112334 at 13-Nov-2025 08:19:21 PST
Begin processing the 35th record. run: 8106 subRun: 0 event: 112335 at 13-Nov-2025 08:22:28 PST
Begin processing the 36th record. run: 8106 subRun: 0 event: 112336 at 13-Nov-2025 08:25:37 PST
Begin processing the 37th record. run: 8106 subRun: 0 event: 112337 at 13-Nov-2025 08:28:46 PST
Begin processing the 38th record. run: 8106 subRun: 0 event: 112338 at 13-Nov-2025 08:31:54 PST
Begin processing the 39th record. run: 8106 subRun: 0 event: 112339 at 13-Nov-2025 08:34:59 PST
Begin processing the 40th record. run: 8106 subRun: 0 event: 112340 at 13-Nov-2025 08:38:15 PST
Begin processing the 41st record. run: 8106 subRun: 0 event: 112341 at 13-Nov-2025 08:41:27 PST
Begin processing the 42nd record. run: 8106 subRun: 0 event: 112342 at 13-Nov-2025 08:44:49 PST
Begin processing the 43rd record. run: 8106 subRun: 0 event: 112343 at 13-Nov-2025 08:48:03 PST
Begin processing the 44th record. run: 8106 subRun: 0 event: 112344 at 13-Nov-2025 08:51:10 PST
Begin processing the 45th record. run: 8106 subRun: 0 event: 112345 at 13-Nov-2025 08:54:33 PST
Begin processing the 46th record. run: 8106 subRun: 0 event: 112346 at 13-Nov-2025 08:58:01 PST
Begin processing the 47th record. run: 8106 subRun: 0 event: 112347 at 13-Nov-2025 09:01:09 PST
Begin processing the 48th record. run: 8106 subRun: 0 event: 112348 at 13-Nov-2025 09:04:37 PST
Begin processing the 49th record. run: 8106 subRun: 0 event: 112349 at 13-Nov-2025 09:08:00 PST
Begin processing the 50th record. run: 8106 subRun: 0 event: 112350 at 13-Nov-2025 09:11:19 PST
13-Nov-2025 09:14:48 PST  Closed output file "prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root"
13-Nov-2025 09:14:49 PST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/16/bb/prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim.root"

=========================================================================================================================================
TimeTracker printout (sec)                                 Min           Avg           Max         Median          RMS         nEvts   
=========================================================================================================================================
Full event                                               166.179       185.67        207.337       186.114       10.2366        50     
-----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.000230728   0.00634859     0.0721657    0.000501704    0.0173354       50     
makers:tpmakerTPCsimpleThr:TriggerPrimitiveMakerTPC      162.978       182.262       204.103       182.762       10.2529        50     
makers:tpmakerTPCabsRS:TriggerPrimitiveMakerTPC          2.78479       3.39907       3.77298       3.53785      0.294153        50     
[art]:TriggerResults:TriggerResultInserter              1.581e-05    2.72769e-05   0.000100739   2.2715e-05    1.71146e-05      50     
end_path:out1:RootOutput                                3.57e-06     6.05996e-06   4.7479e-05     4.49e-06     6.98143e-06      50     
end_path:out1:RootOutput(write)                        0.00141519    0.00195215    0.00396976    0.00194283    0.000386744      50     
=========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1534.85 MB
  Peak resident set size usage (VmHWM): 763.769 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
did.list
file.list
jobscript.log
mem.db
old_md.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T162221Z_gen_000200_supernova_g4_detsim_20251113T112712Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T162221Z_gen_000200_supernova_g4_detsim_20251113T112712Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T162221Z_gen_000200_supernova_g4_detsim_20251113T112712Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T162221Z_gen_000200_supernova_g4_detsim_20251113T112712Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164634Z_gen_000931_supernova_g4_detsim_20251113T102906Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164634Z_gen_000931_supernova_g4_detsim_20251113T102906Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164634Z_gen_000931_supernova_g4_detsim_20251113T102906Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164634Z_gen_000931_supernova_g4_detsim_20251113T102906Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164953Z_gen_001062_supernova_g4_detsim_20251113T093003Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164953Z_gen_001062_supernova_g4_detsim_20251113T093003Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164953Z_gen_001062_supernova_g4_detsim_20251113T093003Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164953Z_gen_001062_supernova_g4_detsim_20251113T093003Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T170954Z_gen_002247_supernova_g4_detsim_20251113T121541Z_reco.root.temp.json
temp_pfns.txt
time.db
triggersim_hist.root