Jobsub ID 46601.151@dunegpschedd01.fnal.gov

Jobsub ID	46601.151@dunegpschedd01.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-15 12:17:41
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2025-09-15 14:27:39
From worker node	Hostname	comp20-14
	cpuinfo	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-15 12:19:32
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_654_20231203T043452Z_gen_g4_detsim_hitreco__20240508T064746Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_121_20231201T050553Z_gen_g4_detsim_hitreco__20240507T202456Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6456061_744_20231207T133853Z_gen_g4_detsim_hitreco__20240510T060229Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_206_20231201T082404Z_gen_g4_detsim_hitreco__20240507T202526Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74254738_144_20231121T151216Z_gen_g4_detsim_hitreco__20240503T070813Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_787_20231202T072156Z_gen_g4_detsim_hitreco__20240508T034449Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_1045_20231204T083731Z_gen_g4_detsim_hitreco__20240509T212030Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_367_20231208T085858Z_gen_g4_detsim_hitreco__20240510T061642Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_781_20231207T160420Z_gen_g4_detsim_hitreco__20240510T044932Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209163_198_20231123T003925Z_gen_g4_detsim_hitreco__20240507T175002Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (6066s)
	CPU time	1h (4595s = 75%)
	Max RSS bytes	2187735040 (2086 MiB)
Outputting started	2025-09-15 14:00:39
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T121939Z.root
Finished	2025-09-15 14:27:39
Saved logs	justin-logs:46601.151-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

o:energyrecnc:EnergyReco                           0.00179869     0.128089      0.709649      0.150675      0.0942619      1200    
reco:energyrecnumurange:EnergyReco                    0.000248894    0.0289217      2.21073     0.00207427     0.123806       1200    
reco:energyrecnumumcs:EnergyReco                      0.000474506    0.0295204      2.26392     0.00239772     0.124126       1200    
reco:energyrecnumumcsllhd:EnergyReco                  0.000349647    0.0294694      2.27052     0.00246063     0.124289       1200    
reco:opdec:Deconvolution                               0.0507876     0.264427      0.668763      0.262878      0.0840208      1200    
reco:ophitspe:OpHitFinderDeco                          0.140183       0.22785      0.497374      0.264576      0.0567565      1200    
reco:opflash:OpFlashFinder                            0.000137469   0.00114866    0.00326436    0.00111069    0.000471774     1200    
reco:opslicer:OpSlicer                                3.6837e-05     0.199494      0.847195       0.15421      0.168519       1200    
reco:rns:RandomNumberSaver                            1.7458e-05    2.59879e-05   0.000318543   2.43245e-05   1.01001e-05     1200    
reco:anglereconue:NuAngularReco                       0.000132192   0.00139352      0.12259     0.000327393    0.005533       1200    
reco:anglereconumu:NuAngularReco                      6.6454e-05    0.00123106     0.122326     0.000175741    0.0054974      1200    
reco:anglereconuepfps:NuAngularReco                   0.000101487   0.00130659     0.123752     0.000248389   0.00552648      1200    
reco:anglereconumupfps:NuAngularReco                  8.8672e-05    0.00129142     0.123016     0.000228159    0.0055174      1200    
reco:anglerecohits:NuAngularReco                      7.2965e-05    0.00174264     0.129661     0.000552787   0.00592402      1200    
[art]:TriggerResults:TriggerResultInserter             8.973e-06    1.11109e-05    5.438e-05    1.07035e-05   1.95081e-06     1200    
end_path:cafmaker:CAFMaker                            0.00123689     0.0173075      1.46949     0.00396697     0.0669828      1200    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3866.26 MB
  Peak resident set size usage (VmHWM): 2187.74 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 0: 0.99739, 0.000450929, 0.00215891, 
Output 1: 0.357788, 0.50395, 0.0929764, 0.0452856, 
Output 2: 0.87946, 0.108434, 0.0108666, 0.00123873, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2626!
Begin processing the 1193rd record. run: 6209163 subRun: 1 event: 39793 at 15-Sep-2025 15:00:01 BST
Boundary wire vector sizes: 99, 351, 248
minwire 0: 672
minwire 1: 1397
minwire 2: 760
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.975835, 0.000551583, 0.0236134, 
Output 1: 0.987692, 0.0113311, 0.000697806, 0.000279421, 
Output 2: 0.981992, 0.0173904, 0.000516239, 0.000101137, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1356!
Begin processing the 1194th record. run: 6209163 subRun: 1 event: 39794 at 15-Sep-2025 15:00:04 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 621!
Begin processing the 1195th record. run: 6209163 subRun: 1 event: 39795 at 15-Sep-2025 15:00:07 BST
Boundary wire vector sizes: 2631, 3246, 3077
minwire 0: 161
minwire 1: 518
minwire 2: 3
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.992171, 0.00334331, 0.00448568, 
Output 1: 0.378754, 0.461151, 0.112543, 0.0475519, 
Output 2: 0.0472809, 0.210767, 0.443614, 0.298339, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3014!
Begin processing the 1196th record. run: 6209163 subRun: 1 event: 39796 at 15-Sep-2025 15:00:14 BST
Boundary wire vector sizes: 2374, 2829, 2680
minwire 0: 861
minwire 1: 571
minwire 2: 391
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.990477, 0.00418013, 0.0053429, 
Output 1: 0.0529901, 0.139078, 0.174121, 0.633811, 
Output 2: 0.0488781, 0.132986, 0.32672, 0.491416, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3107!
Begin processing the 1197th record. run: 6209163 subRun: 1 event: 39797 at 15-Sep-2025 15:00:21 BST
Boundary wire vector sizes: 347, 512, 262
minwire 0: 868
minwire 1: 1913
minwire 2: 460
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00112688, 0.995303, 0.00357011, 
Output 1: 0.0123545, 0.163877, 0.635705, 0.188064, 
Output 2: 0.785287, 0.197784, 0.0163514, 0.0005774, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2136!
Begin processing the 1198th record. run: 6209163 subRun: 1 event: 39798 at 15-Sep-2025 15:00:25 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 790!
Begin processing the 1199th record. run: 6209163 subRun: 1 event: 39799 at 15-Sep-2025 15:00:28 BST
Boundary wire vector sizes: 1105, 988, 1211
minwire 0: 98
minwire 1: 1240
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.993538, 0.000928666, 0.00553291, 
Output 1: 0.423215, 0.574273, 0.00238343, 0.00012793, 
Output 2: 0.988787, 0.0108877, 0.000310874, 1.42377e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1862!
Begin processing the 1200th record. run: 6209163 subRun: 1 event: 39800 at 15-Sep-2025 15:00:34 BST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 634!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/7e/a1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_654_20231203T043452Z_gen_g4_detsim_hitreco__20240508T064746Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/93/ac/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_121_20231201T050553Z_gen_g4_detsim_hitreco__20240507T202456Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/44/0b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6456061_744_20231207T133853Z_gen_g4_detsim_hitreco__20240510T060229Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/45/97/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_206_20231201T082404Z_gen_g4_detsim_hitreco__20240507T202526Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/bb/88/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74254738_144_20231121T151216Z_gen_g4_detsim_hitreco__20240503T070813Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/92/47/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_787_20231202T072156Z_gen_g4_detsim_hitreco__20240508T034449Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/bf/f1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_1045_20231204T083731Z_gen_g4_detsim_hitreco__20240509T212030Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/04/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_367_20231208T085858Z_gen_g4_detsim_hitreco__20240510T061642Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/12/35/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_781_20231207T160420Z_gen_g4_detsim_hitreco__20240510T044932Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a5/f2/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209163_198_20231123T003925Z_gen_g4_detsim_hitreco__20240507T175002Z_reco2.root
.:
total 50476
-rw-r--r-- 1 pltdune002 pltdune 18896547 Sep 15 15:00 flatcaf.root
-rw-r--r-- 1 pltdune002 pltdune 15790956 Sep 15 15:00 caf.root
-rw-r--r-- 1 pltdune002 pltdune 15790956 Sep 15 15:00 caf_fd_hd_atmo_2501_20250915T121939Z.root
-rw-r--r-- 1 pltdune002 pltdune   706655 Sep 15 15:00 caf_20250915T121939Z.log
-rw-r--r-- 1 pltdune002 pltdune   261918 Sep 15 15:00 jobscript.log
-rw-r--r-- 1 pltdune002 pltdune     1986 Sep 15 15:00 caf_20250915T121939Z.file
-rw-r--r-- 1 pltdune002 pltdune     1986 Sep 15 15:00 caf_20250915T121939Z.pfns
-rw-r--r-- 1 pltdune002 pltdune     1986 Sep 15 13:19 file.list
-rw-r--r-- 1 pltdune002 pltdune     1986 Sep 15 15:00 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune002 pltdune     1374 Sep 15 13:19 all-input-dids.txt
-rw-r--r-- 1 pltdune002 pltdune     1374 Sep 15 15:00 caf_20250915T121939Z.did
-rw-r--r-- 1 pltdune002 pltdune     1374 Sep 15 13:19 did.list
-rw-r--r-- 1 pltdune002 pltdune        0 Sep 15 13:19 debugprod.log