Jobsub ID 39983.115@dunegpschedd02.fnal.gov

Jobsub ID	39983.115@dunegpschedd02.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-15 12:13:40
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2025-09-15 14:07:40
From worker node	Hostname	comp20-14
	cpuinfo	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-15 12:14:40
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542963_648_20231202T061452Z_gen_g4_detsim_hitreco__20240507T205658Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6411871_833_20231203T122855Z_gen_g4_detsim_hitreco__20240509T200803Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_947_20231202T161024Z_gen_g4_detsim_hitreco__20240508T044650Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_400_20231201T090007Z_gen_g4_detsim_hitreco__20240507T203136Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_776_20231203T132933Z_gen_g4_detsim_hitreco__20240508T052314Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_447_20231203T053153Z_gen_g4_detsim_hitreco__20240508T075040Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_844_20231205T075757Z_gen_g4_detsim_hitreco__20240510T023046Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74521450_361_20231205T024706Z_gen_g4_detsim_hitreco__20240509T221509Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_32_20231203T091003Z_gen_g4_detsim_hitreco__20240509T190646Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_265_20231027T175135Z_gen_g4_detsim_hitreco__20240503T040433Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (5470s)
	CPU time	1h (4034s = 73%)
	Max RSS bytes	2158346240 (2058 MiB)
Outputting started	2025-09-15 13:45:51
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T121447Z.root
Finished	2025-09-15 14:07:40
Saved logs	justin-logs:39983.115-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

343      0.177685      0.102424       1100    
reco:energyrecnumurange:EnergyReco                    0.000234356    0.0373408      3.7104      0.00204539      0.19436       1100    
reco:energyrecnumumcs:EnergyReco                      0.000518674    0.0380622      3.75041     0.00235835     0.194554       1100    
reco:energyrecnumumcsllhd:EnergyReco                  0.000433612    0.0383177      3.73973     0.00241612     0.195916       1100    
reco:opdec:Deconvolution                               0.0586317     0.269463      0.724758      0.267937      0.086367       1100    
reco:ophitspe:OpHitFinderDeco                          0.141134      0.245846      0.316099       0.26618      0.0469819      1100    
reco:opflash:OpFlashFinder                            0.00018901    0.00113727    0.00290004    0.00108684    0.000483158     1100    
reco:opslicer:OpSlicer                                3.9967e-05     0.197096      0.874847      0.146815      0.170958       1100    
reco:rns:RandomNumberSaver                            1.7366e-05    2.65253e-05   0.000306975   2.45515e-05   1.06205e-05     1100    
reco:anglereconue:NuAngularReco                       0.000134684   0.00153093     0.0734898    0.000313276   0.00495639      1100    
reco:anglereconumu:NuAngularReco                       6.443e-05    0.00136323     0.0718933    0.000165268   0.00485458      1100    
reco:anglereconuepfps:NuAngularReco                   0.000100106    0.0014426     0.0707185    0.000240772   0.00491018      1100    
reco:anglereconumupfps:NuAngularReco                  8.9246e-05    0.00142337     0.0692967    0.000216611   0.00486119      1100    
reco:anglerecohits:NuAngularReco                      7.3911e-05     0.0019621     0.0857659    0.000529967   0.00591188      1100    
[art]:TriggerResults:TriggerResultInserter             8.532e-06    1.12596e-05   6.0225e-05    1.06855e-05   2.46441e-06     1100    
end_path:cafmaker:CAFMaker                             0.001277      0.0204889      1.0692      0.00389808     0.0708855      1100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3839.12 MB
  Peak resident set size usage (VmHWM): 2158.35 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Found hits: 2047!
Begin processing the 1092nd record. run: 1793 subRun: 1 event: 53192 at 15-Sep-2025 14:45:24 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 794!
Begin processing the 1093rd record. run: 1793 subRun: 1 event: 53193 at 15-Sep-2025 14:45:26 BST
Boundary wire vector sizes: 292, 368, 313
minwire 0: 595
minwire 1: 2633
minwire 2: 71
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000267129, 0.998759, 0.000973536, 
Output 1: 0.0047863, 0.990533, 0.00452293, 0.00015769, 
Output 2: 0.997662, 0.00227708, 5.87814e-05, 1.76381e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1589!
Begin processing the 1094th record. run: 1793 subRun: 1 event: 53194 at 15-Sep-2025 14:45:29 BST
Boundary wire vector sizes: 136, 176, 150
minwire 0: 1771
minwire 1: 504
minwire 2: 2050
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000579243, 0.995718, 0.00370251, 
Output 1: 0.970323, 0.0294739, 0.000132107, 7.0618e-05, 
Output 2: 0.999286, 0.000687141, 2.02346e-05, 6.60638e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1049!
Begin processing the 1095th record. run: 1793 subRun: 1 event: 53195 at 15-Sep-2025 14:45:32 BST
Boundary wire vector sizes: 181, 438, 291
minwire 0: 1496
minwire 1: 1056
minwire 2: 1410
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.934636, 0.00348007, 0.0618843, 
Output 1: 0.515739, 0.476744, 0.00720029, 0.000316609, 
Output 2: 0.388885, 0.565826, 0.0441488, 0.00114082, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2097!
Begin processing the 1096th record. run: 1793 subRun: 1 event: 53196 at 15-Sep-2025 14:45:36 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 520!
Begin processing the 1097th record. run: 1793 subRun: 1 event: 53197 at 15-Sep-2025 14:45:38 BST
Boundary wire vector sizes: 60, 47, 26
minwire 0: 2136
minwire 1: 1279
minwire 2: 1955
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000365236, 0.998541, 0.00109393, 
Output 1: 0.174385, 0.821681, 0.00374176, 0.000192275, 
Output 2: 0.99931, 0.000670116, 1.90261e-05, 7.71788e-07, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 934!
Begin processing the 1098th record. run: 1793 subRun: 1 event: 53198 at 15-Sep-2025 14:45:41 BST
Boundary wire vector sizes: 83, 69, 47
minwire 0: 1322
minwire 1: 1913
minwire 2: 990
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0380974, 0.0301071, 0.931795, 
Output 1: 0.657069, 0.335071, 0.00731996, 0.000539794, 
Output 2: 0.102719, 0.887241, 0.00994403, 9.50976e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1580!
Begin processing the 1099th record. run: 1793 subRun: 1 event: 53199 at 15-Sep-2025 14:45:44 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 372!
Begin processing the 1100th record. run: 1793 subRun: 1 event: 53200 at 15-Sep-2025 14:45:46 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1123!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/ec/13/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542963_648_20231202T061452Z_gen_g4_detsim_hitreco__20240507T205658Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/e5/9d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6411871_833_20231203T122855Z_gen_g4_detsim_hitreco__20240509T200803Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a9/19/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_947_20231202T161024Z_gen_g4_detsim_hitreco__20240508T044650Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3a/bf/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_400_20231201T090007Z_gen_g4_detsim_hitreco__20240507T203136Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/82/54/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_776_20231203T132933Z_gen_g4_detsim_hitreco__20240508T052314Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/99/97/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_447_20231203T053153Z_gen_g4_detsim_hitreco__20240508T075040Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/cb/80/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_844_20231205T075757Z_gen_g4_detsim_hitreco__20240510T023046Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/2e/a0/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74521450_361_20231205T024706Z_gen_g4_detsim_hitreco__20240509T221509Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9f/b1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_32_20231203T091003Z_gen_g4_detsim_hitreco__20240509T190646Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/35/53/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_265_20231027T175135Z_gen_g4_detsim_hitreco__20240503T040433Z_reco2.root
.:
total 50248
-rw-r--r-- 1 pltdune002 pltdune 18886446 Sep 15 14:45 flatcaf.root
-rw-r--r-- 1 pltdune002 pltdune 15783002 Sep 15 14:45 caf.root
-rw-r--r-- 1 pltdune002 pltdune 15783002 Sep 15 14:45 caf_fd_hd_atmo_2501_20250915T121447Z.root
-rw-r--r-- 1 pltdune002 pltdune   631403 Sep 15 14:45 caf_20250915T121447Z.log
-rw-r--r-- 1 pltdune002 pltdune   251701 Sep 15 14:45 jobscript.log
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 14:45 caf_20250915T121447Z.file
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 14:45 caf_20250915T121447Z.pfns
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 13:14 file.list
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 14:45 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune002 pltdune     1372 Sep 15 13:14 all-input-dids.txt
-rw-r--r-- 1 pltdune002 pltdune     1372 Sep 15 14:45 caf_20250915T121447Z.did
-rw-r--r-- 1 pltdune002 pltdune     1372 Sep 15 13:14 did.list
-rw-r--r-- 1 pltdune002 pltdune        0 Sep 15 13:15 debugprod.log