Jobsub ID 47830.7@dunegpschedd01.fnal.gov

Jobsub ID	47830.7@dunegpschedd01.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-17 22:53:44
Site	NL_SURFsara
Entry	DUNE_SurfSARA_arc03
Last heartbeat	2025-09-17 23:54:20
From worker node	Hostname	wn-da-20.gina.surf.nl
	cpuinfo	AMD EPYC 7702P 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-17 22:55:12
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571580_557_20231203T022320Z_gen_g4_detsim_hitreco__20240508T071515Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_223_20231203T105301Z_gen_g4_detsim_hitreco__20240508T042836Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_802_20231201T170355Z_gen_g4_detsim_hitreco__20240507T201424Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6403947_48_20231202T114541Z_gen_g4_detsim_hitreco__20240507T212220Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1499_3_20231028T021142Z_gen_g4_detsim_hitreco__20240503T045312Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481105_383_20231201T105701Z_gen_g4_detsim_hitreco__20240507T193536Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50573344_1074_20231203T085628Z_gen_g4_detsim_hitreco__20240508T080809Z_reco2.root
Jobscript	Exit code	0
	Real time	58m (3506s)
	CPU time	49m (2970s = 84%)
	Max RSS bytes	2142355456 (2043 MiB)
Outputting started	2025-09-17 23:53:38
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T225522Z.root
Finished	2025-09-17 23:54:20
Saved logs	justin-logs:47830.7-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.000122497   0.00147944    9.7586e-05    0.000101994      800    
reco:cvnmap:CVNMapper                                 1.7503e-05    0.00702148     0.0309875    0.00884824    0.00489852       800    
reco:cvneva:CVNEvaluator                              1.4979e-05     0.256527        2.331       0.347657      0.177864        800    
reco:energyrecnumu:EnergyReco                         0.000234477    0.0522175      2.83332      0.0147523     0.157482        800    
reco:energyrecnue:EnergyReco                          0.000130739   0.00205607     0.0836178    0.000532135   0.00572839       800    
reco:energyrecnc:EnergyReco                           0.00180564     0.0617955     0.291888      0.069894      0.0467166       800    
reco:energyrecnumurange:EnergyReco                    0.000217856    0.0265216      2.63669     0.00233374     0.145614        800    
reco:energyrecnumumcs:EnergyReco                      0.000480067    0.0271465      2.6607       0.002778      0.145323        800    
reco:energyrecnumumcsllhd:EnergyReco                  0.000373182    0.0271809      2.65291     0.00274635      0.14603        800    
reco:opdec:Deconvolution                               0.0492942     0.221065       0.49025      0.222914      0.0743187       800    
reco:ophitspe:OpHitFinderDeco                          0.131347      0.142012      0.231327      0.140691      0.0081464       800    
reco:opflash:OpFlashFinder                            0.000134957   0.00108526    0.00355039    0.00104115    0.000491016      800    
reco:opslicer:OpSlicer                                3.3243e-05     0.213975      0.882015      0.167594      0.176381        800    
reco:rns:RandomNumberSaver                            1.8294e-05    2.90939e-05   0.000304601   2.8534e-05    1.27806e-05      800    
reco:anglereconue:NuAngularReco                       0.000133927   0.00129409     0.0618463    0.000340946   0.00422498       800    
reco:anglereconumu:NuAngularReco                      6.6326e-05    0.00116184     0.0638216    0.000188038   0.00427969       800    
reco:anglereconuepfps:NuAngularReco                   0.000107716   0.00126069     0.0641662    0.000280626   0.00431897       800    
reco:anglereconumupfps:NuAngularReco                  9.5021e-05    0.00121324     0.0623505    0.000247086   0.00427215       800    
reco:anglerecohits:NuAngularReco                      7.8561e-05    0.00171273     0.0676068    0.000581609    0.0047207       800    
[art]:TriggerResults:TriggerResultInserter             8.707e-06    1.09228e-05   0.000135188   1.0255e-05    5.18567e-06      800    
end_path:cafmaker:CAFMaker                            0.00113603     0.0155743     0.714355     0.00354401     0.0502993       800    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3807.5 MB
  Peak resident set size usage (VmHWM): 2142.36 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.972144, 0.00184555, 0.0260102, 
Output 1: 0.186017, 0.7521, 0.0591904, 0.00269238, 
Output 2: 0.964622, 0.0337139, 0.00159548, 6.86207e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1775!
Begin processing the 794th record. run: 50573344 subRun: 1 event: 107494 at 18-Sep-2025 01:53:16 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1590!
Begin processing the 795th record. run: 50573344 subRun: 1 event: 107495 at 18-Sep-2025 01:53:19 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1426!
Begin processing the 796th record. run: 50573344 subRun: 1 event: 107496 at 18-Sep-2025 01:53:21 CEST
Boundary wire vector sizes: 676, 693, 677
minwire 0: 1331
minwire 1: 2113
minwire 2: 773
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00627124, 0.603808, 0.38992, 
Output 1: 0.227419, 0.757234, 0.0138576, 0.00149007, 
Output 2: 0.93173, 0.0650154, 0.00301653, 0.000237737, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2538!
Begin processing the 797th record. run: 50573344 subRun: 1 event: 107497 at 18-Sep-2025 01:53:24 CEST
Boundary wire vector sizes: 551, 577, 403
minwire 0: 196
minwire 1: 595
minwire 2: 201
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.17735, 0.351784, 0.470866, 
Output 1: 0.110588, 0.284347, 0.302377, 0.302688, 
Output 2: 0.234557, 0.549156, 0.183689, 0.0325983, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2408!
Begin processing the 798th record. run: 50573344 subRun: 1 event: 107498 at 18-Sep-2025 01:53:28 CEST
Boundary wire vector sizes: 814, 810, 271
minwire 0: 869
minwire 1: 1598
minwire 2: 671
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.996897, 0.000377469, 0.0027252, 
Output 1: 0.0477394, 0.41586, 0.510901, 0.0254996, 
Output 2: 0.855245, 0.135674, 0.00887871, 0.000202646, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2691!
Begin processing the 799th record. run: 50573344 subRun: 1 event: 107499 at 18-Sep-2025 01:53:31 CEST
Boundary wire vector sizes: 87, 95, 81
minwire 0: 1896
minwire 1: 1263
minwire 2: 1541
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.711808, 0.00112994, 0.287062, 
Output 1: 0.718201, 0.271617, 0.00978262, 0.000399915, 
Output 2: 0.722603, 0.269883, 0.00729345, 0.000220859, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2001!
Begin processing the 800th record. run: 50573344 subRun: 1 event: 107500 at 18-Sep-2025 01:53:34 CEST
Boundary wire vector sizes: 111, 119, 85
minwire 0: 1819
minwire 1: 791
minwire 2: 1913
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000764752, 0.985529, 0.0137058, 
Output 1: 0.861442, 0.136691, 0.00168713, 0.000180399, 
Output 2: 0.995334, 0.00450611, 0.000143333, 1.69147e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1518!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c9/85/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571580_557_20231203T022320Z_gen_g4_detsim_hitreco__20240508T071515Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/03/14/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_223_20231203T105301Z_gen_g4_detsim_hitreco__20240508T042836Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/2a/4c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_802_20231201T170355Z_gen_g4_detsim_hitreco__20240507T201424Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/6d/3f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6403947_48_20231202T114541Z_gen_g4_detsim_hitreco__20240507T212220Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/57/c0/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1499_3_20231028T021142Z_gen_g4_detsim_hitreco__20240503T045312Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/90/81/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481105_383_20231201T105701Z_gen_g4_detsim_hitreco__20240507T193536Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/01/1d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50573344_1074_20231203T085628Z_gen_g4_detsim_hitreco__20240508T080809Z_reco2.root
.:
total 33352
-rw-r--r--. 1 dune003 dune 12559984 Sep 18 01:53 flatcaf.root
-rw-r--r--. 1 dune003 dune 10437761 Sep 18 01:53 caf.root
-rw-r--r--. 1 dune003 dune 10437761 Sep 18 01:53 caf_fd_hd_atmo_2501_20250917T225522Z.root
-rw-r--r--. 1 dune003 dune   475323 Sep 18 01:53 caf_20250917T225522Z.log
-rw-r--r--. 1 dune003 dune   183944 Sep 18 01:53 jobscript.log
-rw-r--r--. 1 dune003 dune     1301 Sep 18 01:53 caf_20250917T225522Z.file
-rw-r--r--. 1 dune003 dune     1301 Sep 18 01:53 caf_20250917T225522Z.pfns
-rw-r--r--. 1 dune003 dune     1301 Sep 18 00:55 file.list
-rw-r--r--. 1 dune003 dune     1301 Sep 18 01:53 justin-processed-pfns.txt
-rw-r--r--. 1 dune003 dune      958 Sep 18 00:55 all-input-dids.txt
-rw-r--r--. 1 dune003 dune      958 Sep 18 01:53 caf_20250917T225522Z.did
-rw-r--r--. 1 dune003 dune      958 Sep 18 00:55 did.list
-rw-r--r--. 1 dune003 dune        0 Sep 18 00:55 debugprod.log