Jobsub ID 47253.0@dunegpschedd01.fnal.gov

Jobsub ID	47253.0@dunegpschedd01.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-17 01:20:33
Site	ES_PIC
Entry	DUNE_T1_ES_PIC_ce16-multicore
Last heartbeat	2025-09-17 02:38:35
From worker node	Hostname	hnode56.pic.es
	cpuinfo	AMD EPYC 7402P 24-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	216000 (60 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-17 01:21:10
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74576447_257_20231208T034031Z_gen_g4_detsim_hitreco__20240510T045626Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6468412_429_20231207T075639Z_gen_g4_detsim_hitreco__20240510T061148Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_482_20231202T180752Z_gen_g4_detsim_hitreco__20240508T063800Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420351_1149_20231204T045425Z_gen_g4_detsim_hitreco__20240509T192658Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472706_184_20231208T075625Z_gen_g4_detsim_hitreco__20240510T052540Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481093_103_20231201T072643Z_gen_g4_detsim_hitreco__20240507T192325Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_640_20231202T084105Z_gen_g4_detsim_hitreco__20240507T205237Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (4620s)
	CPU time	55m (3312s = 71%)
	Max RSS bytes	2147790848 (2048 MiB)
Outputting started	2025-09-17 02:38:11
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T012119Z.root
Finished	2025-09-17 02:38:35
Saved logs	justin-logs:47253.0-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

6   0.00149208    0.000113238   0.000124628      700    
reco:cvnmap:CVNMapper                                  1.462e-05    0.00784829     0.0407848     0.0100453    0.00598145       700    
reco:cvneva:CVNEvaluator                               1.423e-05     0.302499       2.12431      0.420455      0.222499        700    
reco:energyrecnumu:EnergyReco                         0.000210165    0.266896       121.459      0.0147175      4.58869        700    
reco:energyrecnue:EnergyReco                          0.000120623    0.0025984     0.181841     0.000647012   0.00955753       700    
reco:energyrecnc:EnergyReco                            0.0018312     0.113471       0.6588       0.0331184     0.115164        700    
reco:energyrecnumurange:EnergyReco                    0.000181254    0.0407525      2.87917     0.00285052     0.186856        700    
reco:energyrecnumumcs:EnergyReco                      0.000605176    0.041021       2.82665     0.00321915     0.184131        700    
reco:energyrecnumumcsllhd:EnergyReco                   0.0004336     0.0407568      2.80437     0.00325607     0.182866        700    
reco:opdec:Deconvolution                              0.00363844     0.297601      0.804338      0.308005      0.124498        700    
reco:ophitspe:OpHitFinderDeco                         0.00144123     0.303803      0.537675      0.375584      0.190101        700    
reco:opflash:OpFlashFinder                            0.000103033   0.00126705    0.00739609    0.00119449    0.000651725      700    
reco:opslicer:OpSlicer                                3.2961e-05     0.263034       1.2671       0.197885       0.22859        700    
reco:rns:RandomNumberSaver                             1.532e-05    3.01443e-05   0.000354369   2.9116e-05    1.42934e-05      700    
reco:anglereconue:NuAngularReco                       9.2923e-05    0.00175831     0.152467     0.00037112     0.0078054       700    
reco:anglereconumu:NuAngularReco                      6.0032e-05    0.00159726     0.153417     0.000219965   0.00781045       700    
reco:anglereconuepfps:NuAngularReco                   9.7113e-05    0.00169648     0.152287     0.000318693   0.00775095       700    
reco:anglereconumupfps:NuAngularReco                  8.4572e-05    0.00167111     0.151365     0.000282077   0.00772622       700    
reco:anglerecohits:NuAngularReco                      6.8792e-05    0.00232189     0.155456     0.000718329   0.00827991       700    
[art]:TriggerResults:TriggerResultInserter             8.64e-06     1.24492e-05   4.4771e-05     1.194e-05    3.13076e-06      700    
end_path:cafmaker:CAFMaker                            0.00115386     0.0226764      1.83803     0.00434541     0.0986139       700    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3815.27 MB
  Peak resident set size usage (VmHWM): 2147.79 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 902, 724, 784
minwire 0: 2171
minwire 1: 288
minwire 2: 2390
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000643556, 0.997679, 0.0016779, 
Output 1: 0.00906514, 0.834604, 0.150264, 0.00606709, 
Output 2: 0.996307, 0.00352238, 0.000165631, 4.97812e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1890!
Begin processing the 695th record. run: 6400837 subRun: 1 event: 64095 at 17-Sep-2025 04:37:45 CEST
Boundary wire vector sizes: 323, 388, 300
minwire 0: 1500
minwire 1: 971
minwire 2: 1616
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000552617, 0.998112, 0.00133511, 
Output 1: 0.00282174, 0.0595804, 0.905204, 0.0323939, 
Output 2: 0.983019, 0.0159374, 0.00102405, 1.96154e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1708!
Begin processing the 696th record. run: 6400837 subRun: 1 event: 64096 at 17-Sep-2025 04:37:49 CEST
Boundary wire vector sizes: 186, 339, 314
minwire 0: 2750
minwire 1: 730
minwire 2: 2463
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.996963, 0.00093949, 0.00209773, 
Output 1: 0.00481929, 0.989326, 0.00572398, 0.000130496, 
Output 2: 0.997616, 0.00232011, 6.24147e-05, 1.36671e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1588!
Begin processing the 697th record. run: 6400837 subRun: 1 event: 64097 at 17-Sep-2025 04:37:53 CEST
Boundary wire vector sizes: 713, 670, 220
minwire 0: 1943
minwire 1: 308
minwire 2: 2332
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.985477, 0.00175157, 0.0127713, 
Output 1: 0.350002, 0.613655, 0.0348948, 0.00144879, 
Output 2: 0.982857, 0.0167303, 0.000400143, 1.21544e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1713!
Begin processing the 698th record. run: 6400837 subRun: 1 event: 64098 at 17-Sep-2025 04:37:57 CEST
Boundary wire vector sizes: 527, 481, 534
minwire 0: 803
minwire 1: 1938
minwire 2: 609
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00249548, 0.00139916, 0.996105, 
Output 1: 0.963492, 0.0357917, 0.00040067, 0.000315728, 
Output 2: 0.998768, 0.00119101, 3.16448e-05, 9.00128e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2666!
Begin processing the 699th record. run: 6400837 subRun: 1 event: 64099 at 17-Sep-2025 04:38:02 CEST
Boundary wire vector sizes: 469, 442, 512
minwire 0: 633
minwire 1: 2015
minwire 2: 136
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.99236, 0.00053176, 0.00710823, 
Output 1: 0.1401, 0.432105, 0.378951, 0.0488438, 
Output 2: 0.972074, 0.0263868, 0.00146365, 7.53688e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1981!
Begin processing the 700th record. run: 6400837 subRun: 1 event: 64100 at 17-Sep-2025 04:38:07 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 508!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/1e/2c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74576447_257_20231208T034031Z_gen_g4_detsim_hitreco__20240510T045626Z_reco2.root
root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/25/d5/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6468412_429_20231207T075639Z_gen_g4_detsim_hitreco__20240510T061148Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/2f/fe/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_482_20231202T180752Z_gen_g4_detsim_hitreco__20240508T063800Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/85/bf/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420351_1149_20231204T045425Z_gen_g4_detsim_hitreco__20240509T192658Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/83/cc/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472706_184_20231208T075625Z_gen_g4_detsim_hitreco__20240510T052540Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/ac/30/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481093_103_20231201T072643Z_gen_g4_detsim_hitreco__20240507T192325Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/33/9b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_640_20231202T084105Z_gen_g4_detsim_hitreco__20240507T205237Z_reco2.root
.:
total 32868
-rw-r--r-- 1 dune001 dune 12412668 Sep 17 04:38 flatcaf.root
-rw-r--r-- 1 dune001 dune 10316957 Sep 17 04:38 caf.root
-rw-r--r-- 1 dune001 dune 10316957 Sep 17 04:38 caf_fd_hd_atmo_2501_20250917T012119Z.root
-rw-r--r-- 1 dune001 dune   401918 Sep 17 04:38 caf_20250917T012119Z.log
-rw-r--r-- 1 dune001 dune   158273 Sep 17 04:38 jobscript.log
-rw-r--r-- 1 dune001 dune     1384 Sep 17 04:38 caf_20250917T012119Z.file
-rw-r--r-- 1 dune001 dune     1384 Sep 17 04:38 caf_20250917T012119Z.pfns
-rw-r--r-- 1 dune001 dune     1384 Sep 17 03:21 file.list
-rw-r--r-- 1 dune001 dune     1384 Sep 17 04:38 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune      963 Sep 17 03:21 all-input-dids.txt
-rw-r--r-- 1 dune001 dune      963 Sep 17 04:38 caf_20250917T012119Z.did
-rw-r--r-- 1 dune001 dune      963 Sep 17 03:21 did.list
-rw-r--r-- 1 dune001 dune        0 Sep 17 03:21 debugprod.log