Jobsub ID 40343.97@dunegpschedd02.fnal.gov

Jobsub ID	40343.97@dunegpschedd02.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-16 06:34:40
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce514
Last heartbeat	2025-09-16 08:24:04
From worker node	Hostname	b9p20p3034.cern.ch
	cpuinfo	Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-16 06:35:24
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_175_20231203T072235Z_gen_g4_detsim_hitreco__20240508T054216Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_810_20231203T113442Z_gen_g4_detsim_hitreco__20240509T191222Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_473_20231207T003130Z_gen_g4_detsim_hitreco__20240510T061252Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65840069_403_20231122T052540Z_gen_g4_detsim_hitreco__20240507T171134Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50610358_143_20231207T081746Z_gen_g4_detsim_hitreco__20240510T031343Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_270_20231208T092202Z_gen_g4_detsim_hitreco__20240510T063521Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552530_18_20231207T114110Z_gen_g4_detsim_hitreco__20240510T054446Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413290_68_20231202T195941Z_gen_g4_detsim_hitreco__20240508T062519Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_442_20231208T070239Z_gen_g4_detsim_hitreco__20240510T054102Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_609_20231202T160913Z_gen_g4_detsim_hitreco__20240508T043430Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (6499s)
	CPU time	1h (5520s = 84%)
	Max RSS bytes	2327961600 (2220 MiB)
Outputting started	2025-09-16 08:23:44
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250916T063534Z.root
Finished	2025-09-16 08:24:04
Saved logs	justin-logs:40343.97-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.648398      0.102488      0.0703285      1100    
reco:energyrecnumurange:EnergyReco                    0.000249075    0.0561721      5.26508     0.00348341     0.285112       1100    
reco:energyrecnumumcs:EnergyReco                      0.000562443    0.0564461      5.29469     0.00410884     0.284799       1100    
reco:energyrecnumumcsllhd:EnergyReco                  0.000567917    0.0552197      4.8988      0.00413434     0.271905       1100    
reco:opdec:Deconvolution                               0.0303587      0.29515      0.837555      0.289717       0.12093       1100    
reco:ophitspe:OpHitFinderDeco                          0.0345526     0.124221      0.384448      0.150024      0.0491709      1100    
reco:opflash:OpFlashFinder                            0.000168369    0.0017414    0.00784962    0.00162552    0.000892213     1100    
reco:opslicer:OpSlicer                                3.7823e-05     0.327695       1.72851      0.247555      0.292016       1100    
reco:rns:RandomNumberSaver                            1.7645e-05    3.5356e-05    0.000586669   3.2298e-05    1.98975e-05     1100    
reco:anglereconue:NuAngularReco                       0.000146404    0.0025272     0.170455     0.000434117    0.0105994      1100    
reco:anglereconumu:NuAngularReco                      7.6179e-05    0.00232465     0.161975     0.000268496    0.0103091      1100    
reco:anglereconuepfps:NuAngularReco                   0.000120312   0.00241843     0.169169     0.000377663    0.0102979      1100    
reco:anglereconumupfps:NuAngularReco                  0.000104448   0.00236617     0.166261     0.000367754   0.00994898      1100    
reco:anglerecohits:NuAngularReco                      8.5678e-05    0.00308796     0.171296     0.000863322    0.0106244      1100    
[art]:TriggerResults:TriggerResultInserter             9.654e-06    1.56083e-05   8.6811e-05    1.48725e-05   5.05354e-06     1100    
end_path:cafmaker:CAFMaker                            0.00122985     0.0304999      2.14239     0.00586862     0.120991       1100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4013.63 MB
  Peak resident set size usage (VmHWM): 2327.96 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.989063, 0.00738148, 0.00355537, 
Output 1: 0.11031, 0.254775, 0.244943, 0.389972, 
Output 2: 0.0218892, 0.0677751, 0.304496, 0.60584, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2987!
Begin processing the 1094th record. run: 66004350 subRun: 1 event: 60994 at 16-Sep-2025 10:23:09 CEST
Boundary wire vector sizes: 2516, 3031, 1747
minwire 0: 61
minwire 1: 2251
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0532743, 0.887755, 0.0589706, 
Output 1: 0.591251, 0.387964, 0.0182548, 0.00253044, 
Output 2: 0.640662, 0.339417, 0.0187672, 0.00115437, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1905!
Begin processing the 1095th record. run: 66004350 subRun: 1 event: 60995 at 16-Sep-2025 10:23:18 CEST
Boundary wire vector sizes: 150, 151, 123
minwire 0: 462
minwire 1: 1718
minwire 2: 277
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 214, 513
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0839085, 0.0240131, 0.892079, 
Output 1: 0.0179634, 0.051313, 0.0984742, 0.832249, 
Output 2: 0.518854, 0.422927, 0.0508254, 0.00739332, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1645!
Begin processing the 1096th record. run: 66004350 subRun: 1 event: 60996 at 16-Sep-2025 10:23:22 CEST
Boundary wire vector sizes: 104, 83, 63
minwire 0: 2144
minwire 1: 214
minwire 2: 2211
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.571255, 0.00270464, 0.42604, 
Output 1: 0.0244692, 0.352401, 0.58938, 0.0337498, 
Output 2: 0.76385, 0.229001, 0.00704015, 0.000109216, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1302!
Begin processing the 1097th record. run: 66004350 subRun: 1 event: 60997 at 16-Sep-2025 10:23:25 CEST
Boundary wire vector sizes: 2397, 2119, 1165
minwire 0: 1280
minwire 1: 572
minwire 2: 1338
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0163523, 0.8142, 0.169448, 
Output 1: 0.481045, 0.500016, 0.0175349, 0.00140408, 
Output 2: 0.0868378, 0.536928, 0.317605, 0.0586292, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3024!
Begin processing the 1098th record. run: 66004350 subRun: 1 event: 60998 at 16-Sep-2025 10:23:32 CEST
Boundary wire vector sizes: 353, 406, 298
minwire 0: 907
minwire 1: 1333
minwire 2: 878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000373118, 0.997028, 0.00259893, 
Output 1: 0.0115827, 0.98424, 0.00402128, 0.000156376, 
Output 2: 0.993721, 0.00609102, 0.000182152, 6.23079e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1683!
Begin processing the 1099th record. run: 66004350 subRun: 1 event: 60999 at 16-Sep-2025 10:23:36 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 557!
Begin processing the 1100th record. run: 66004350 subRun: 1 event: 61000 at 16-Sep-2025 10:23:39 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1115!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9c/ad/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_175_20231203T072235Z_gen_g4_detsim_hitreco__20240508T054216Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c2/25/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_810_20231203T113442Z_gen_g4_detsim_hitreco__20240509T191222Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/4f/42/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_473_20231207T003130Z_gen_g4_detsim_hitreco__20240510T061252Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3a/04/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65840069_403_20231122T052540Z_gen_g4_detsim_hitreco__20240507T171134Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/84/b2/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50610358_143_20231207T081746Z_gen_g4_detsim_hitreco__20240510T031343Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/4e/39/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_270_20231208T092202Z_gen_g4_detsim_hitreco__20240510T063521Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/97/98/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552530_18_20231207T114110Z_gen_g4_detsim_hitreco__20240510T054446Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/be/89/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413290_68_20231202T195941Z_gen_g4_detsim_hitreco__20240508T062519Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/79/0b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_442_20231208T070239Z_gen_g4_detsim_hitreco__20240510T054102Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/f8/1a/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_609_20231202T160913Z_gen_g4_detsim_hitreco__20240508T043430Z_reco2.root
.:
total 48084
-rw-r--r--. 1 duneprd np-comp 18072920 Sep 16 10:23 flatcaf.root
-rw-r--r--. 1 duneprd np-comp 15107063 Sep 16 10:23 caf.root
-rw-r--r--. 1 duneprd np-comp 15107063 Sep 16 10:23 caf_fd_hd_atmo_2501_20250916T063534Z.root
-rw-r--r--. 1 duneprd np-comp   642372 Sep 16 10:23 caf_20250916T063534Z.log
-rw-r--r--. 1 duneprd np-comp   250766 Sep 16 10:23 jobscript.log
-rw-r--r--. 1 duneprd np-comp     1956 Sep 16 10:23 caf_20250916T063534Z.file
-rw-r--r--. 1 duneprd np-comp     1956 Sep 16 10:23 caf_20250916T063534Z.pfns
-rw-r--r--. 1 duneprd np-comp     1956 Sep 16 08:35 file.list
-rw-r--r--. 1 duneprd np-comp     1956 Sep 16 10:23 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp     1374 Sep 16 08:35 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp     1374 Sep 16 10:23 caf_20250916T063534Z.did
-rw-r--r--. 1 duneprd np-comp     1374 Sep 16 08:35 did.list
-rw-r--r--. 1 duneprd np-comp        0 Sep 16 08:35 debugprod.log