Jobsub ID 242409.2@dunegpschedd01.fnal.gov

Jobsub ID	242409.2@dunegpschedd01.fnal.gov
Workflow ID	9554
Stage ID	1
User name	epennacc@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-03 16:05:02
Site	US_UChicago
Entry	Engage_US_MWT2_iut2_condce_mcore
Last heartbeat	2025-11-03 17:09:24
From worker node	Hostname	iut2-c282.iu.edu
	cpuinfo	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-03 16:11:41
Input files	fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164636Z_gen_000933_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T163750Z_gen_000585_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T173701Z_gen_003770_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T063210Z_gen_000253_supernova_g4_detsim.root fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim.root
Jobscript	Exit code	0
	Real time	44m (2692s)
	CPU time	39m (2376s = 88%)
	Max RSS bytes	802689024 (765 MiB)
Outputting started	2025-11-03 16:56:33
Output files	usertests:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T163750Z_gen_000585_supernova_g4_detsim_20251103T162215Z_reco.root usertests:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root usertests:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164636Z_gen_000933_supernova_g4_detsim_20251103T161201Z_reco.root usertests:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T173701Z_gen_003770_supernova_g4_detsim_20251103T163100Z_reco.root usertests:prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T063210Z_gen_000253_supernova_g4_detsim_20251103T163931Z_reco.root
Finished	2025-11-03 17:09:24
Saved logs	justin-logs:242409.2-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

03-Nov-2025 11:49:07 EST
Begin processing the 3rd record. run: 8106 subRun: 0 event: 153203 at 03-Nov-2025 11:49:16 EST
Begin processing the 4th record. run: 8106 subRun: 0 event: 153204 at 03-Nov-2025 11:49:24 EST
Begin processing the 5th record. run: 8106 subRun: 0 event: 153205 at 03-Nov-2025 11:49:33 EST
Begin processing the 6th record. run: 8106 subRun: 0 event: 153206 at 03-Nov-2025 11:49:42 EST
Begin processing the 7th record. run: 8106 subRun: 0 event: 153207 at 03-Nov-2025 11:49:51 EST
Begin processing the 8th record. run: 8106 subRun: 0 event: 153208 at 03-Nov-2025 11:50:00 EST
Begin processing the 9th record. run: 8106 subRun: 0 event: 153209 at 03-Nov-2025 11:50:09 EST
Begin processing the 10th record. run: 8106 subRun: 0 event: 153210 at 03-Nov-2025 11:50:18 EST
Begin processing the 11th record. run: 8106 subRun: 0 event: 153211 at 03-Nov-2025 11:50:27 EST
Begin processing the 12th record. run: 8106 subRun: 0 event: 153212 at 03-Nov-2025 11:50:36 EST
Begin processing the 13th record. run: 8106 subRun: 0 event: 153213 at 03-Nov-2025 11:50:45 EST
Begin processing the 14th record. run: 8106 subRun: 0 event: 153214 at 03-Nov-2025 11:50:54 EST
Begin processing the 15th record. run: 8106 subRun: 0 event: 153215 at 03-Nov-2025 11:51:03 EST
Begin processing the 16th record. run: 8106 subRun: 0 event: 153216 at 03-Nov-2025 11:51:12 EST
Begin processing the 17th record. run: 8106 subRun: 0 event: 153217 at 03-Nov-2025 11:51:21 EST
Begin processing the 18th record. run: 8106 subRun: 0 event: 153218 at 03-Nov-2025 11:51:29 EST
Begin processing the 19th record. run: 8106 subRun: 0 event: 153219 at 03-Nov-2025 11:51:38 EST
Begin processing the 20th record. run: 8106 subRun: 0 event: 153220 at 03-Nov-2025 11:51:48 EST
Begin processing the 21st record. run: 8106 subRun: 0 event: 153221 at 03-Nov-2025 11:51:58 EST
Begin processing the 22nd record. run: 8106 subRun: 0 event: 153222 at 03-Nov-2025 11:52:07 EST
Begin processing the 23rd record. run: 8106 subRun: 0 event: 153223 at 03-Nov-2025 11:52:16 EST
Begin processing the 24th record. run: 8106 subRun: 0 event: 153224 at 03-Nov-2025 11:52:25 EST
Begin processing the 25th record. run: 8106 subRun: 0 event: 153225 at 03-Nov-2025 11:52:35 EST
Begin processing the 26th record. run: 8106 subRun: 0 event: 153226 at 03-Nov-2025 11:52:44 EST
Begin processing the 27th record. run: 8106 subRun: 0 event: 153227 at 03-Nov-2025 11:52:53 EST
Begin processing the 28th record. run: 8106 subRun: 0 event: 153228 at 03-Nov-2025 11:53:02 EST
Begin processing the 29th record. run: 8106 subRun: 0 event: 153229 at 03-Nov-2025 11:53:11 EST
Begin processing the 30th record. run: 8106 subRun: 0 event: 153230 at 03-Nov-2025 11:53:20 EST
Begin processing the 31st record. run: 8106 subRun: 0 event: 153231 at 03-Nov-2025 11:53:29 EST
Begin processing the 32nd record. run: 8106 subRun: 0 event: 153232 at 03-Nov-2025 11:53:38 EST
Begin processing the 33rd record. run: 8106 subRun: 0 event: 153233 at 03-Nov-2025 11:53:47 EST
Begin processing the 34th record. run: 8106 subRun: 0 event: 153234 at 03-Nov-2025 11:53:56 EST
Begin processing the 35th record. run: 8106 subRun: 0 event: 153235 at 03-Nov-2025 11:54:05 EST
Begin processing the 36th record. run: 8106 subRun: 0 event: 153236 at 03-Nov-2025 11:54:14 EST
Begin processing the 37th record. run: 8106 subRun: 0 event: 153237 at 03-Nov-2025 11:54:23 EST
Begin processing the 38th record. run: 8106 subRun: 0 event: 153238 at 03-Nov-2025 11:54:32 EST
Begin processing the 39th record. run: 8106 subRun: 0 event: 153239 at 03-Nov-2025 11:54:40 EST
Begin processing the 40th record. run: 8106 subRun: 0 event: 153240 at 03-Nov-2025 11:54:49 EST
Begin processing the 41st record. run: 8106 subRun: 0 event: 153241 at 03-Nov-2025 11:54:58 EST
Begin processing the 42nd record. run: 8106 subRun: 0 event: 153242 at 03-Nov-2025 11:55:07 EST
Begin processing the 43rd record. run: 8106 subRun: 0 event: 153243 at 03-Nov-2025 11:55:15 EST
Begin processing the 44th record. run: 8106 subRun: 0 event: 153244 at 03-Nov-2025 11:55:24 EST
Begin processing the 45th record. run: 8106 subRun: 0 event: 153245 at 03-Nov-2025 11:55:32 EST
Begin processing the 46th record. run: 8106 subRun: 0 event: 153246 at 03-Nov-2025 11:55:41 EST
Begin processing the 47th record. run: 8106 subRun: 0 event: 153247 at 03-Nov-2025 11:55:50 EST
Begin processing the 48th record. run: 8106 subRun: 0 event: 153248 at 03-Nov-2025 11:56:00 EST
Begin processing the 49th record. run: 8106 subRun: 0 event: 153249 at 03-Nov-2025 11:56:09 EST
Begin processing the 50th record. run: 8106 subRun: 0 event: 153250 at 03-Nov-2025 11:56:17 EST
03-Nov-2025 11:56:28 EST  Closed output file "prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root"
03-Nov-2025 11:56:28 EST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/f7/27/prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim.root"

=========================================================================================================================================
TimeTracker printout (sec)                                 Min           Avg           Max         Median          RMS         nEvts   
=========================================================================================================================================
Full event                                               8.36413       8.87898       9.95372       8.85949      0.345128        50     
-----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.000330579   0.00120082    0.00709604    0.000500529   0.00185508       50     
makers:tpmakerTPCsimpleThr:TriggerPrimitiveMakerTPC      4.4195        4.71398       5.15846       4.67371      0.165587        50     
makers:tpmakerTPCabsRS:TriggerPrimitiveMakerTPC          3.83223       4.15962       4.93057       4.10165      0.249725        50     
[art]:TriggerResults:TriggerResultInserter             2.9329e-05    5.26694e-05   0.000180736   5.43465e-05   2.24826e-05      50     
end_path:out1:RootOutput                                4.106e-06    9.15278e-06   3.4579e-05    8.7895e-06    5.04861e-06      50     
end_path:out1:RootOutput(write)                        0.00187533    0.00334002    0.00526745    0.00300905    0.000911058      50     
=========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1534.75 MB
  Peak resident set size usage (VmHWM): 799.322 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
mv: cannot stat 'triggerAnaTree_hist.root': No such file or directory
============OUTPUT FILE:  prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
did.list
file.list
jobscript.log
mem.db
old_md.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T163750Z_gen_000585_supernova_g4_detsim_20251103T162215Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T163750Z_gen_000585_supernova_g4_detsim_20251103T162215Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T163750Z_gen_000585_supernova_g4_detsim_20251103T162215Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T163750Z_gen_000585_supernova_g4_detsim_20251103T162215Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164636Z_gen_000933_supernova_g4_detsim_20251103T161201Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164636Z_gen_000933_supernova_g4_detsim_20251103T161201Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164636Z_gen_000933_supernova_g4_detsim_20251103T161201Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T164636Z_gen_000933_supernova_g4_detsim_20251103T161201Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T172443Z_gen_003065_supernova_g4_detsim_20251103T164802Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T173701Z_gen_003770_supernova_g4_detsim_20251103T163100Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T173701Z_gen_003770_supernova_g4_detsim_20251103T163100Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T173701Z_gen_003770_supernova_g4_detsim_20251103T163100Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250926T173701Z_gen_003770_supernova_g4_detsim_20251103T163100Z_reco.root.temp.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T063210Z_gen_000253_supernova_g4_detsim_20251103T163931Z_reco.root
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T063210Z_gen_000253_supernova_g4_detsim_20251103T163931Z_reco.root.ext.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T063210Z_gen_000253_supernova_g4_detsim_20251103T163931Z_reco.root.json
prodmarley_nue_flat_es_dune10kt_1x2x2_20250927T063210Z_gen_000253_supernova_g4_detsim_20251103T163931Z_reco.root.temp.json
temp_pfns.txt
time.db
triggersim_hist.root