Jobsub ID 46802.9@dunegpschedd01.fnal.gov

Jobsub ID	46802.9@dunegpschedd01.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-15 20:18:04
Site	NL_NIKHEF
Entry	VIRGO_NL_NIKHEF_brug
Last heartbeat	2025-09-15 22:01:16
From worker node	Hostname	wn-sate-050.farm.nikhef.nl
	cpuinfo	AMD EPYC 7551P 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-15 20:18:39
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50610358_518_20231207T084929Z_gen_g4_detsim_hitreco__20240510T033956Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_464_20231201T090556Z_gen_g4_detsim_hitreco__20240507T203436Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_475_20231202T160843Z_gen_g4_detsim_hitreco__20240508T041736Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517882_966_20231204T140427Z_gen_g4_detsim_hitreco__20240509T204113Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6416859_1014_20231203T084721Z_gen_g4_detsim_hitreco__20240509T201930Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74508011_969_20231203T065249Z_gen_g4_detsim_hitreco__20240508T074633Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_599_20231203T125830Z_gen_g4_detsim_hitreco__20240508T051632Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_157_20231120T133629Z_gen_g4_detsim_hitreco__20240503T054857Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (6135s)
	CPU time	1h (5289s = 86%)
	Max RSS bytes	2143543296 (2044 MiB)
Outputting started	2025-09-15 22:00:55
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T201848Z.root
Finished	2025-09-15 22:01:16
Saved logs	justin-logs:46802.9-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

584      0.607522      0.286919       1000    
reco:energyrecnumu:EnergyReco                         0.000259949    0.0760215      5.97775      0.0203986     0.244036       1000    
reco:energyrecnue:EnergyReco                          0.000184528   0.00265217     0.111422     0.000587009   0.00789338      1000    
reco:energyrecnc:EnergyReco                            0.0026505     0.0851557     0.372194      0.0972772     0.061816       1000    
reco:energyrecnumurange:EnergyReco                    0.000242456    0.0401361      5.20983      0.0029776     0.214281       1000    
reco:energyrecnumumcs:EnergyReco                      0.000516181    0.0409431      5.18343     0.00344489      0.21334       1000    
reco:energyrecnumumcsllhd:EnergyReco                  0.000408919    0.0409486      5.17991     0.00350426     0.213099       1000    
reco:opdec:Deconvolution                               0.0449412     0.235162      0.490618      0.231748      0.0811277      1000    
reco:ophitspe:OpHitFinderDeco                          0.127215      0.139618      0.189404      0.135466      0.0133844      1000    
reco:opflash:OpFlashFinder                            0.000142909    0.0012174    0.00297684    0.00116633    0.000563539     1000    
reco:opslicer:OpSlicer                                4.2601e-05     0.232912       1.06732      0.181934      0.196298       1000    
reco:rns:RandomNumberSaver                            1.7674e-05    2.58078e-05   0.000222409   2.5088e-05    7.14066e-06     1000    
reco:anglereconue:NuAngularReco                       0.00014881    0.00170141     0.0771484    0.00033781    0.00573006      1000    
reco:anglereconumu:NuAngularReco                      9.4418e-05    0.00159787     0.0755593    0.000227198   0.00569431      1000    
reco:anglereconuepfps:NuAngularReco                   0.000136547   0.00168222     0.0759957    0.000312288   0.00571885      1000    
reco:anglereconumupfps:NuAngularReco                  0.00013293    0.00168486     0.0777928    0.000298101   0.00574961      1000    
reco:anglerecohits:NuAngularReco                      0.000104316   0.00222919     0.0830324    0.000647317    0.0063082      1000    
[art]:TriggerResults:TriggerResultInserter            1.0379e-05    1.21911e-05   4.3542e-05    1.1792e-05    1.68276e-06     1000    
end_path:cafmaker:CAFMaker                            0.000965476    0.0214708     0.977881     0.00376424     0.0722053      1000    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3805.97 MB
  Peak resident set size usage (VmHWM): 2143.54 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1620!
Begin processing the 992nd record. run: 50253389 subRun: 1 event: 31592 at 15-Sep-2025 23:59:18 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 702!
Begin processing the 993rd record. run: 50253389 subRun: 1 event: 31593 at 15-Sep-2025 23:59:20 CEST
Boundary wire vector sizes: 298, 279, 221
minwire 0: 2101
minwire 1: 687
minwire 2: 1877
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00106798, 0.994371, 0.00456047, 
Output 1: 0.0564691, 0.466759, 0.450434, 0.0263377, 
Output 2: 0.916657, 0.0796614, 0.00359614, 8.54746e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2363!
Begin processing the 994th record. run: 50253389 subRun: 1 event: 31594 at 15-Sep-2025 23:59:24 CEST
Boundary wire vector sizes: 411, 560, 199
minwire 0: 1015
minwire 1: 1953
minwire 2: 619
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000657019, 0.996472, 0.00287059, 
Output 1: 0.983878, 0.0157442, 0.000187007, 0.000191259, 
Output 2: 0.999078, 0.000874584, 3.15663e-05, 1.55052e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2205!
Begin processing the 995th record. run: 50253389 subRun: 1 event: 31595 at 15-Sep-2025 23:59:28 CEST
Boundary wire vector sizes: 9406, 9288, 6564
minwire 0: 15
minwire 1: 607
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.64349, 0.257699, 0.0988104, 
Output 1: 0.243054, 0.428785, 0.201775, 0.126386, 
Output 2: 0.00827275, 0.0293562, 0.141906, 0.820465, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3426!
Begin processing the 996th record. run: 50253389 subRun: 1 event: 31596 at 16-Sep-2025 00:00:39 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 822!
Begin processing the 997th record. run: 50253389 subRun: 1 event: 31597 at 16-Sep-2025 00:00:42 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 394!
Begin processing the 998th record. run: 50253389 subRun: 1 event: 31598 at 16-Sep-2025 00:00:44 CEST
Boundary wire vector sizes: 637, 334, 349
minwire 0: 38
minwire 1: 2266
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.597737, 0.00847731, 0.393786, 
Output 1: 0.926181, 0.0722663, 0.00149256, 6.01869e-05, 
Output 2: 0.867133, 0.130538, 0.00227152, 5.67539e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1078!
Begin processing the 999th record. run: 50253389 subRun: 1 event: 31599 at 16-Sep-2025 00:00:48 CEST
Boundary wire vector sizes: 44, 46, 52
minwire 0: 163
minwire 1: 2484
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00463704, 0.00193826, 0.993425, 
Output 1: 0.251463, 0.73799, 0.0102958, 0.000250945, 
Output 2: 0.997266, 0.0026181, 0.000111498, 4.71557e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 856!
Begin processing the 1000th record. run: 50253389 subRun: 1 event: 31600 at 16-Sep-2025 00:00:51 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 192!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/35/d4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50610358_518_20231207T084929Z_gen_g4_detsim_hitreco__20240510T033956Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/da/ba/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/fc/00/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6374820_464_20231201T090556Z_gen_g4_detsim_hitreco__20240507T203436Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/c7/59/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_475_20231202T160843Z_gen_g4_detsim_hitreco__20240508T041736Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/77/9b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517882_966_20231204T140427Z_gen_g4_detsim_hitreco__20240509T204113Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/89/02/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6416859_1014_20231203T084721Z_gen_g4_detsim_hitreco__20240509T201930Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/e9/1f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74508011_969_20231203T065249Z_gen_g4_detsim_hitreco__20240508T074633Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/7a/38/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_599_20231203T125830Z_gen_g4_detsim_hitreco__20240508T051632Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/41/24/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_157_20231120T133629Z_gen_g4_detsim_hitreco__20240503T054857Z_reco2.root
.:
total 46692
-rw-r--r--. 1 dune012 dune 17550733 Sep 16 00:00 flatcaf.root
-rw-r--r--. 1 dune012 dune 14688977 Sep 16 00:00 caf.root
-rw-r--r--. 1 dune012 dune 14688977 Sep 16 00:00 caf_fd_hd_atmo_2501_20250915T201848Z.root
-rw-r--r--. 1 dune012 dune   604923 Sep 16 00:00 caf_20250915T201848Z.log
-rw-r--r--. 1 dune012 dune   223932 Sep 16 00:00 jobscript.log
-rw-r--r--. 1 dune012 dune     1701 Sep 16 00:00 caf_20250915T201848Z.file
-rw-r--r--. 1 dune012 dune     1701 Sep 16 00:00 caf_20250915T201848Z.pfns
-rw-r--r--. 1 dune012 dune     1701 Sep 15 22:18 file.list
-rw-r--r--. 1 dune012 dune     1701 Sep 16 00:00 justin-processed-pfns.txt
-rw-r--r--. 1 dune012 dune     1240 Sep 15 22:18 all-input-dids.txt
-rw-r--r--. 1 dune012 dune     1240 Sep 16 00:00 caf_20250915T201848Z.did
-rw-r--r--. 1 dune012 dune     1240 Sep 15 22:18 did.list
-rw-r--r--. 1 dune012 dune        0 Sep 15 22:19 debugprod.log