Jobsub ID 46718.5@dunegpschedd01.fnal.gov

Jobsub ID	46718.5@dunegpschedd01.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-15 16:22:51
Site	NL_NIKHEF
Entry	VIRGO_NL_NIKHEF_klomp
Last heartbeat	2025-09-15 18:05:17
From worker node	Hostname	wn-snel-029.farm.nikhef.nl
	cpuinfo	AMD EPYC 7H12 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-15 16:24:11
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481093_46_20231201T071023Z_gen_g4_detsim_hitreco__20240507T192615Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517882_354_20231204T140337Z_gen_g4_detsim_hitreco__20240509T220831Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6455553_508_20231207T114744Z_gen_g4_detsim_hitreco__20240510T053629Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_102_20231206T033237Z_gen_g4_detsim_hitreco__20240510T055900Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_487_20231204T082019Z_gen_g4_detsim_hitreco__20240509T215837Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_916_20231204T075630Z_gen_g4_detsim_hitreco__20240509T220453Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_43_20231203T091040Z_gen_g4_detsim_hitreco__20240508T075708Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_123_20231208T063539Z_gen_g4_detsim_hitreco__20240510T052252Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_476_20231122T130240Z_gen_g4_detsim_hitreco__20240503T064322Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_374_20231204T212603Z_gen_g4_detsim_hitreco__20240510T025437Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (6044s)
	CPU time	1h (5144s = 85%)
	Max RSS bytes	2145005568 (2045 MiB)
Outputting started	2025-09-15 18:04:56
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T162420Z.root
Finished	2025-09-15 18:05:17
Saved logs	justin-logs:46718.5-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.00566804      1100    
reco:energyrecnc:EnergyReco                            0.0016977     0.0916791     0.807009      0.0949002     0.0781392      1100    
reco:energyrecnumurange:EnergyReco                    0.000190207    0.0277378      2.16698     0.00240843     0.140012       1100    
reco:energyrecnumumcs:EnergyReco                      0.000393256    0.0282903      2.14601     0.00285521     0.138968       1100    
reco:energyrecnumumcsllhd:EnergyReco                  0.00026164     0.0283449      2.15437     0.00279997     0.140774       1100    
reco:opdec:Deconvolution                               0.0450645     0.241319      0.691316      0.235242      0.0924418      1100    
reco:ophitspe:OpHitFinderDeco                           0.12462      0.149048      0.483719      0.134792      0.032227       1100    
reco:opflash:OpFlashFinder                            0.000166451   0.00120395    0.00530202    0.00115687    0.000551496     1100    
reco:opslicer:OpSlicer                                4.5695e-05     0.237059       1.04369      0.198226      0.186199       1100    
reco:rns:RandomNumberSaver                            1.6931e-05    2.52875e-05   0.000295172   2.47315e-05   8.89232e-06     1100    
reco:anglereconue:NuAngularReco                       0.000140282   0.00127973     0.0887912    0.000341635   0.00450579      1100    
reco:anglereconumu:NuAngularReco                      7.4319e-05    0.00113214     0.0898342    0.000198556   0.00450426      1100    
reco:anglereconuepfps:NuAngularReco                   0.000108313   0.00122252     0.0915256    0.000295138    0.0045261      1100    
reco:anglereconumupfps:NuAngularReco                  9.5018e-05    0.00121698     0.0886972    0.000264469    0.0046456      1100    
reco:anglerecohits:NuAngularReco                      0.000114454   0.00170149     0.093687     0.000641851   0.00484572      1100    
[art]:TriggerResults:TriggerResultInserter             8.366e-06    1.11311e-05   5.1647e-05     1.076e-05    2.09401e-06     1100    
end_path:cafmaker:CAFMaker                            0.00119662     0.0154336     0.995559     0.00381386     0.0529662      1100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3807.25 MB
  Peak resident set size usage (VmHWM): 2145.01 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
minwire 0: 352
minwire 1: 1739
minwire 2: 409
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00489006, 0.911523, 0.0835867, 
Output 1: 0.109264, 0.865122, 0.0214805, 0.00413447, 
Output 2: 0.380509, 0.463938, 0.129308, 0.0262448, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2455!
Begin processing the 1095th record. run: 74518431 subRun: 1 event: 37495 at 15-Sep-2025 20:04:19 CEST
Boundary wire vector sizes: 48, 29, 33
minwire 0: 2156
minwire 1: 1096
minwire 2: 2040
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.361947, 0.622231, 0.0158221, 
Output 1: 0.229281, 0.719569, 0.0496155, 0.00153467, 
Output 2: 0.987553, 0.0121072, 0.000327026, 1.237e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 764!
Begin processing the 1096th record. run: 74518431 subRun: 1 event: 37496 at 15-Sep-2025 20:04:22 CEST
Boundary wire vector sizes: 1076, 921, 1025
minwire 0: 1583
minwire 1: 955
minwire 2: 1534
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0022001, 0.982184, 0.015616, 
Output 1: 0.115847, 0.585299, 0.280817, 0.0180372, 
Output 2: 0.969765, 0.0281552, 0.0019751, 0.000104363, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2640!
Begin processing the 1097th record. run: 74518431 subRun: 1 event: 37497 at 15-Sep-2025 20:04:26 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 854!
Begin processing the 1098th record. run: 74518431 subRun: 1 event: 37498 at 15-Sep-2025 20:04:28 CEST
Boundary wire vector sizes: 248, 302, 194
minwire 0: 1697
minwire 1: 1207
minwire 2: 1352
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000281242, 0.99852, 0.00119837, 
Output 1: 0.00373423, 0.991474, 0.00463936, 0.000152861, 
Output 2: 0.998233, 0.00171618, 4.88092e-05, 1.57216e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2499!
Begin processing the 1099th record. run: 74518431 subRun: 1 event: 37499 at 15-Sep-2025 20:04:32 CEST
Boundary wire vector sizes: 380, 309, 173
minwire 0: 763
minwire 1: 662
minwire 2: 284
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.869123, 0.000766364, 0.13011, 
Output 1: 0.141399, 0.834997, 0.0229708, 0.000633508, 
Output 2: 0.784764, 0.210241, 0.0048949, 0.000100442, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2785!
Begin processing the 1100th record. run: 74518431 subRun: 1 event: 37500 at 15-Sep-2025 20:04:35 CEST
Boundary wire vector sizes: 7822, 6734, 5545
minwire 0: 156
minwire 1: 396
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.660901, 0.337365, 0.00173408, 
Output 1: 0.196368, 0.542204, 0.212531, 0.0488971, 
Output 2: 0.106628, 0.36358, 0.354575, 0.175217, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3183!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/b6/65/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481093_46_20231201T071023Z_gen_g4_detsim_hitreco__20240507T192615Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9d/eb/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517882_354_20231204T140337Z_gen_g4_detsim_hitreco__20240509T220831Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/ec/52/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6455553_508_20231207T114744Z_gen_g4_detsim_hitreco__20240510T053629Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/95/b9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_102_20231206T033237Z_gen_g4_detsim_hitreco__20240510T055900Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/02/34/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_487_20231204T082019Z_gen_g4_detsim_hitreco__20240509T215837Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/e1/d9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_916_20231204T075630Z_gen_g4_detsim_hitreco__20240509T220453Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1a/85/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_43_20231203T091040Z_gen_g4_detsim_hitreco__20240508T075708Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/05/de/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_123_20231208T063539Z_gen_g4_detsim_hitreco__20240510T052252Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1b/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_476_20231122T130240Z_gen_g4_detsim_hitreco__20240503T064322Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9d/30/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_374_20231204T212603Z_gen_g4_detsim_hitreco__20240510T025437Z_reco2.root
.:
total 45824
-rw-r--r--. 1 dune012 dune 17206048 Sep 15 20:04 flatcaf.root
-rw-r--r--. 1 dune012 dune 14382731 Sep 15 20:04 caf.root
-rw-r--r--. 1 dune012 dune 14382731 Sep 15 20:04 caf_fd_hd_atmo_2501_20250915T162420Z.root
-rw-r--r--. 1 dune012 dune   650336 Sep 15 20:04 caf_20250915T162420Z.log
-rw-r--r--. 1 dune012 dune   248696 Sep 15 20:04 jobscript.log
-rw-r--r--. 1 dune012 dune     1916 Sep 15 20:04 caf_20250915T162420Z.file
-rw-r--r--. 1 dune012 dune     1916 Sep 15 20:04 caf_20250915T162420Z.pfns
-rw-r--r--. 1 dune012 dune     1916 Sep 15 18:24 file.list
-rw-r--r--. 1 dune012 dune     1916 Sep 15 20:04 justin-processed-pfns.txt
-rw-r--r--. 1 dune012 dune     1374 Sep 15 18:24 all-input-dids.txt
-rw-r--r--. 1 dune012 dune     1374 Sep 15 20:04 caf_20250915T162420Z.did
-rw-r--r--. 1 dune012 dune     1374 Sep 15 18:24 did.list
-rw-r--r--. 1 dune012 dune        0 Sep 15 18:24 debugprod.log