Jobsub ID 40179.6@dunegpschedd02.fnal.gov

Jobsub ID	40179.6@dunegpschedd02.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-15 17:04:04
Site	FR_CCIN2P3
Entry	DUNE_FR_CCIN2P3_cccondorce04
Last heartbeat	2025-09-15 18:26:32
From worker node	Hostname	ccwcondor0385
	cpuinfo	AMD EPYC 9334 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	106200 (29 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-15 17:04:30
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_166_20231201T220552Z_gen_g4_detsim_hitreco__20240507T203937Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6406500_474_20231202T125358Z_gen_g4_detsim_hitreco__20240507T222008Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472938_463_20231208T095952Z_gen_g4_detsim_hitreco__20240510T064941Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_687_20231207T162615Z_gen_g4_detsim_hitreco__20240510T031834Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571580_1058_20231203T030530Z_gen_g4_detsim_hitreco__20240508T070634Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419113_622_20231203T151628Z_gen_g4_detsim_hitreco__20240508T054530Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258005_118_20231121T153309Z_gen_g4_detsim_hitreco__20240503T054053Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (4888s)
	CPU time	1h (4188s = 85%)
	Max RSS bytes	2219339776 (2116 MiB)
Outputting started	2025-09-15 18:25:59
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T170438Z.root
Finished	2025-09-15 18:26:32
Saved logs	justin-logs:40179.6-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Evaluator                              1.1538e-05     0.185177       1.20462      0.198687      0.134845        800    
reco:energyrecnumu:EnergyReco                         0.000169648    0.0583189      1.48317      0.0166182     0.0970136       800    
reco:energyrecnue:EnergyReco                          6.8033e-05    0.00157419     0.0507121    0.000472326   0.00406915       800    
reco:energyrecnc:EnergyReco                           0.000930411    0.0828724     0.589857      0.0958914     0.0661198       800    
reco:energyrecnumurange:EnergyReco                    0.000132832    0.0208552      1.32606     0.00211328     0.0836857       800    
reco:energyrecnumumcs:EnergyReco                      0.000445045    0.020759       1.32524     0.00254949     0.0809588       800    
reco:energyrecnumumcsllhd:EnergyReco                  0.000301287    0.0206548      1.32287     0.00249039     0.080488        800    
reco:opdec:Deconvolution                               0.0547508     0.188491      0.448165      0.181463      0.0655818       800    
reco:ophitspe:OpHitFinderDeco                          0.161017       0.16871      0.187596      0.167495     0.00487141       800    
reco:opflash:OpFlashFinder                            0.000119241   0.00110535     0.0044771    0.000987352   0.00058863       800    
reco:opslicer:OpSlicer                                4.2604e-05     0.187364       1.00734      0.145164      0.168264        800    
reco:rns:RandomNumberSaver                            1.3781e-05    5.07991e-05   0.00912816    3.7116e-05    0.000321656      800    
reco:anglereconue:NuAngularReco                       0.000110367   0.000996323    0.0342897    0.000364413   0.00253691       800    
reco:anglereconumu:NuAngularReco                      3.7958e-05    0.000780507    0.036239     0.000156888   0.00254304       800    
reco:anglereconuepfps:NuAngularReco                   6.7252e-05    0.000872937    0.0363788    0.000263309   0.00254929       800    
reco:anglereconumupfps:NuAngularReco                  4.9636e-05    0.000812265    0.0363922    0.000198857   0.00252863       800    
reco:anglerecohits:NuAngularReco                      6.1483e-05    0.00132037     0.0411346    0.000584111   0.00297623       800    
[art]:TriggerResults:TriggerResultInserter             7.13e-06     1.23223e-05   0.000255237   1.1878e-05    1.02228e-05      800    
end_path:cafmaker:CAFMaker                            0.000871492    0.0132182     0.484367     0.00475343     0.0328847       800    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3902.95 MB
  Peak resident set size usage (VmHWM): 2219.34 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0040063, 0.991837, 0.00415685, 
Output 1: 0.0413055, 0.200019, 0.344743, 0.413932, 
Output 2: 0.127349, 0.501477, 0.307349, 0.0638252, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2797!
Begin processing the 795th record. run: 50258005 subRun: 1 event: 23795 at 15-Sep-2025 20:25:31 CEST
Boundary wire vector sizes: 94, 85, 91
minwire 0: 603
minwire 1: 1430
minwire 2: 290
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 810, 1109
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0184732, 0.00229834, 0.979228, 
Output 1: 0.0221763, 0.145843, 0.731825, 0.100156, 
Output 2: 0.967106, 0.0312785, 0.00156078, 5.44741e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1738!
Begin processing the 796th record. run: 50258005 subRun: 1 event: 23796 at 15-Sep-2025 20:25:34 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 683!
Begin processing the 797th record. run: 50258005 subRun: 1 event: 23797 at 15-Sep-2025 20:25:36 CEST
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
Boundary wire vector sizes: 250, 149, 182
minwire 0: 680
minwire 1: 2044
minwire 2: 555
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00297396, 0.989276, 0.00775046, 
Output 1: 0.0155406, 0.0797165, 0.465611, 0.439132, 
Output 2: 0.958076, 0.039868, 0.00192546, 0.000130432, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1815!
Begin processing the 798th record. run: 50258005 subRun: 1 event: 23798 at 15-Sep-2025 20:25:39 CEST
Boundary wire vector sizes: 761, 409, 478
minwire 0: 1294
minwire 1: 300
minwire 2: 1554
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.996028, 0.00044022, 0.00353138, 
Output 1: 0.594374, 0.401685, 0.00313397, 0.000807419, 
Output 2: 0.996759, 0.00311327, 0.00011302, 1.51897e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2134!
Begin processing the 799th record. run: 50258005 subRun: 1 event: 23799 at 15-Sep-2025 20:25:42 CEST
Boundary wire vector sizes: 709, 715, 639
minwire 0: 1207
minwire 1: 1903
minwire 2: 891
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00182593, 0.984512, 0.0136622, 
Output 1: 0.0504326, 0.341666, 0.307995, 0.299906, 
Output 2: 0.61354, 0.294317, 0.0759544, 0.0161883, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2848!
Begin processing the 800th record. run: 50258005 subRun: 1 event: 23800 at 15-Sep-2025 20:25:46 CEST
Boundary wire vector sizes: 4805, 2524, 2530
minwire 0: 645
minwire 1: 944
minwire 2: 272
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0263777, 0.911186, 0.0624359, 
Output 1: 0.298177, 0.410274, 0.148815, 0.142735, 
Output 2: 0.0637457, 0.231036, 0.330758, 0.37446, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3190!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/04/02/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_166_20231201T220552Z_gen_g4_detsim_hitreco__20240507T203937Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/1a/b9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6406500_474_20231202T125358Z_gen_g4_detsim_hitreco__20240507T222008Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/02/46/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472938_463_20231208T095952Z_gen_g4_detsim_hitreco__20240510T064941Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/28/07/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_687_20231207T162615Z_gen_g4_detsim_hitreco__20240510T031834Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/da/39/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571580_1058_20231203T030530Z_gen_g4_detsim_hitreco__20240508T070634Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/bd/d3/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419113_622_20231203T151628Z_gen_g4_detsim_hitreco__20240508T054530Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/04/41/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258005_118_20231121T153309Z_gen_g4_detsim_hitreco__20240503T054053Z_reco2.root
.:
total 34376
-rw-r--r-- 1 dune001 lbno 12937664 Sep 15 20:25 flatcaf.root
-rw-r--r-- 1 dune001 lbno 10771466 Sep 15 20:25 caf.root
-rw-r--r-- 1 dune001 lbno 10771466 Sep 15 20:25 caf_fd_hd_atmo_2501_20250915T170438Z.root
-rw-r--r-- 1 dune001 lbno   473004 Sep 15 20:25 caf_20250915T170438Z.log
-rw-r--r-- 1 dune001 lbno   179129 Sep 15 20:25 jobscript.log
-rw-r--r-- 1 dune001 lbno     1337 Sep 15 20:25 caf_20250915T170438Z.file
-rw-r--r-- 1 dune001 lbno     1337 Sep 15 20:25 caf_20250915T170438Z.pfns
-rw-r--r-- 1 dune001 lbno     1337 Sep 15 19:04 file.list
-rw-r--r-- 1 dune001 lbno     1337 Sep 15 20:25 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno      964 Sep 15 19:04 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno      964 Sep 15 20:25 caf_20250915T170438Z.did
-rw-r--r-- 1 dune001 lbno      964 Sep 15 19:04 did.list
-rw-r--r-- 1 dune001 lbno        0 Sep 15 19:04 debugprod.log