Jobsub ID 46601.150@dunegpschedd01.fnal.gov

Jobsub ID	46601.150@dunegpschedd01.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-15 12:17:41
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2025-09-15 13:55:26
From worker node	Hostname	comp22-01
	cpuinfo	Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-15 12:19:41
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6464269_697_20231207T034108Z_gen_g4_detsim_hitreco__20240510T033325Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_970_20231204T083529Z_gen_g4_detsim_hitreco__20240509T200409Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577091_874_20231204T004154Z_gen_g4_detsim_hitreco__20240509T211603Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_574_20231204T082329Z_gen_g4_detsim_hitreco__20240509T220513Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_269_20231204T081316Z_gen_g4_detsim_hitreco__20240509T214503Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400858_504_20231202T081049Z_gen_g4_detsim_hitreco__20240507T210139Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6396715_4_20231202T011603Z_gen_g4_detsim_hitreco__20240507T202033Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_743_20231202T190944Z_gen_g4_detsim_hitreco__20240508T063744Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50579427_506_20231204T044419Z_gen_g4_detsim_hitreco__20240509T205516Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74550731_265_20231207T083151Z_gen_g4_detsim_hitreco__20240510T045845Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (4424s)
	CPU time	53m (3224s = 72%)
	Max RSS bytes	2257956864 (2153 MiB)
Outputting started	2025-09-15 13:33:26
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T121950Z.root
Finished	2025-09-15 13:55:26
Saved logs	justin-logs:46601.150-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.560851      0.163425      0.0984627      1000    
reco:energyrecnumurange:EnergyReco                    0.000259925    0.0335772      2.24659     0.00230133     0.151285       1000    
reco:energyrecnumumcs:EnergyReco                      0.000454348    0.0340338      2.19484      0.0028584     0.150105       1000    
reco:energyrecnumumcsllhd:EnergyReco                  0.000361103    0.0339851      2.24933     0.00266825     0.151051       1000    
reco:opdec:Deconvolution                               0.0571051     0.265867      0.780402      0.267071      0.079675       1000    
reco:ophitspe:OpHitFinderDeco                          0.143385      0.252665      0.490327      0.274802      0.0487325      1000    
reco:opflash:OpFlashFinder                            0.000279178   0.00174137    0.00488462    0.00172273    0.000548669     1000    
reco:opslicer:OpSlicer                                9.0115e-05     0.199158      0.888785      0.162876      0.160468       1000    
reco:rns:RandomNumberSaver                            1.7174e-05    2.38433e-05   0.000300902   2.22005e-05   9.9384e-06      1000    
reco:anglereconue:NuAngularReco                       0.000139657   0.00144663     0.0685814    0.00034435    0.00469871      1000    
reco:anglereconumu:NuAngularReco                      6.2932e-05    0.00128718     0.0683685    0.000206649   0.00464208      1000    
reco:anglereconuepfps:NuAngularReco                    9.974e-05    0.00135809     0.0674833    0.000281237   0.00463992      1000    
reco:anglereconumupfps:NuAngularReco                  8.4771e-05    0.00134717     0.066866     0.000265198   0.00463306      1000    
reco:anglerecohits:NuAngularReco                      7.7535e-05    0.00180372     0.0704043    0.000561977   0.00506981      1000    
[art]:TriggerResults:TriggerResultInserter             8.553e-06    1.10344e-05   4.2509e-05    1.0333e-05    2.72202e-06     1000    
end_path:cafmaker:CAFMaker                            0.00127296     0.0179627     0.948892     0.00393986     0.0609722      1000    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3923.96 MB
  Peak resident set size usage (VmHWM): 2251.11 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Classifier summary: 
Output 0: 0.995719, 0.000442768, 0.00383823, 
Output 1: 0.603261, 0.316285, 0.05827, 0.0221839, 
Output 2: 0.920616, 0.0748212, 0.00416933, 0.000393638, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1795!
Begin processing the 994th record. run: 74550731 subRun: 1 event: 26594 at 15-Sep-2025 14:33:05 BST
Boundary wire vector sizes: 34, 37, 37
minwire 0: 582
minwire 1: 1807
minwire 2: 527
Used alternate method to get min and max wires due to vertex determination failure: 11, 310
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 63, 362
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 1, 300
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0162277, 0.00237639, 0.981396, 
Output 1: 0.555443, 0.437608, 0.00677667, 0.000172781, 
Output 2: 0.997469, 0.00243561, 9.09749e-05, 4.29304e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 741!
Begin processing the 995th record. run: 74550731 subRun: 1 event: 26595 at 15-Sep-2025 14:33:07 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 205!
Begin processing the 996th record. run: 74550731 subRun: 1 event: 26596 at 15-Sep-2025 14:33:10 BST
Boundary wire vector sizes: 135, 168, 105
minwire 0: 2195
minwire 1: 1203
minwire 2: 1812
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.1964, 0.215931, 0.587669, 
Output 1: 0.0126222, 0.144245, 0.684729, 0.158404, 
Output 2: 0.828959, 0.162021, 0.00877463, 0.000245337, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2086!
Begin processing the 997th record. run: 74550731 subRun: 1 event: 26597 at 15-Sep-2025 14:33:13 BST
Boundary wire vector sizes: 187, 249, 115
minwire 0: 410
minwire 1: 2703
minwire 2: 32
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.992615, 0.000929672, 0.00645479, 
Output 1: 0.00331042, 0.0588992, 0.913091, 0.0246991, 
Output 2: 0.965298, 0.0335373, 0.00115139, 1.35798e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1003!
Begin processing the 998th record. run: 74550731 subRun: 1 event: 26598 at 15-Sep-2025 14:33:16 BST
Boundary wire vector sizes: 247, 248, 219
minwire 0: 1452
minwire 1: 791
minwire 2: 1668
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.985231, 0.000771006, 0.0139984, 
Output 1: 0.0368549, 0.953011, 0.00986245, 0.000271941, 
Output 2: 0.977106, 0.0222606, 0.000618706, 1.50397e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1485!
Begin processing the 999th record. run: 74550731 subRun: 1 event: 26599 at 15-Sep-2025 14:33:19 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 247!
Begin processing the 1000th record. run: 74550731 subRun: 1 event: 26600 at 15-Sep-2025 14:33:21 BST
Boundary wire vector sizes: 124, 157, 127
minwire 0: 648
minwire 1: 2373
minwire 2: 107
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0991415, 0.0249356, 0.875923, 
Output 1: 0.451488, 0.471897, 0.0660893, 0.0105261, 
Output 2: 0.199973, 0.577447, 0.199827, 0.0227529, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1595!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9d/bf/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6464269_697_20231207T034108Z_gen_g4_detsim_hitreco__20240510T033325Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/59/97/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_970_20231204T083529Z_gen_g4_detsim_hitreco__20240509T200409Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c1/bd/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577091_874_20231204T004154Z_gen_g4_detsim_hitreco__20240509T211603Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/27/46/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_574_20231204T082329Z_gen_g4_detsim_hitreco__20240509T220513Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/2f/0d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_269_20231204T081316Z_gen_g4_detsim_hitreco__20240509T214503Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/d2/43/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400858_504_20231202T081049Z_gen_g4_detsim_hitreco__20240507T210139Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a6/27/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6396715_4_20231202T011603Z_gen_g4_detsim_hitreco__20240507T202033Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/46/7f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_743_20231202T190944Z_gen_g4_detsim_hitreco__20240508T063744Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/48/fb/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50579427_506_20231204T044419Z_gen_g4_detsim_hitreco__20240509T205516Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/ce/48/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74550731_265_20231207T083151Z_gen_g4_detsim_hitreco__20240510T045845Z_reco2.root
.:
total 45596
-rw-r--r-- 1 pltdune002 pltdune 17123618 Sep 15 14:33 flatcaf.root
-rw-r--r-- 1 pltdune002 pltdune 14296223 Sep 15 14:33 caf.root
-rw-r--r-- 1 pltdune002 pltdune 14296223 Sep 15 14:33 caf_fd_hd_atmo_2501_20250915T121950Z.root
-rw-r--r-- 1 pltdune002 pltdune   605881 Sep 15 14:33 caf_20250915T121950Z.log
-rw-r--r-- 1 pltdune002 pltdune   221755 Sep 15 14:33 jobscript.log
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 14:33 caf_20250915T121950Z.file
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 14:33 caf_20250915T121950Z.pfns
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 13:19 file.list
-rw-r--r-- 1 pltdune002 pltdune     1922 Sep 15 14:33 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune002 pltdune     1372 Sep 15 13:19 all-input-dids.txt
-rw-r--r-- 1 pltdune002 pltdune     1372 Sep 15 14:33 caf_20250915T121950Z.did
-rw-r--r-- 1 pltdune002 pltdune     1372 Sep 15 13:19 did.list
-rw-r--r-- 1 pltdune002 pltdune        0 Sep 15 13:20 debugprod.log