Jobsub ID 40343.38@dunegpschedd02.fnal.gov

Jobsub ID	40343.38@dunegpschedd02.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-16 06:34:40
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce515
Last heartbeat	2025-09-16 08:14:50
From worker node	Hostname	b9p20p1449.cern.ch
	cpuinfo	Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-16 06:35:25
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_842_20231204T083201Z_gen_g4_detsim_hitreco__20240509T194754Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74505925_1069_20231202T164130Z_gen_g4_detsim_hitreco__20240508T030221Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_978_20231202T152210Z_gen_g4_detsim_hitreco__20240508T031941Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420853_475_20231204T035000Z_gen_g4_detsim_hitreco__20240509T205215Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_175_20231208T053137Z_gen_g4_detsim_hitreco__20240510T043335Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_741_20231207T001640Z_gen_g4_detsim_hitreco__20240510T063709Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467222_555_20231207T052530Z_gen_g4_detsim_hitreco__20240510T044836Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_436_20231204T081921Z_gen_g4_detsim_hitreco__20240509T215605Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420351_1065_20231204T045220Z_gen_g4_detsim_hitreco__20240509T191032Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_465_20231201T125710Z_gen_g4_detsim_hitreco__20240507T190545Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (5943s)
	CPU time	1h (5134s = 86%)
	Max RSS bytes	2307657728 (2200 MiB)
Outputting started	2025-09-16 08:14:29
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250916T063535Z.root
Finished	2025-09-16 08:14:50
Saved logs	justin-logs:40343.38-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eco:energyrecnc:EnergyReco                           0.00218596     0.0916196     0.975501      0.109773      0.0773115      1000    
reco:energyrecnumurange:EnergyReco                    0.000297609    0.0874425      15.9853      0.003724       0.5939        1000    
reco:energyrecnumumcs:EnergyReco                      0.000672485    0.0822697      14.6378     0.00458102     0.533339       1000    
reco:energyrecnumumcsllhd:EnergyReco                  0.000511488    0.0802117      14.2956     0.00461213     0.523748       1000    
reco:opdec:Deconvolution                               0.0448154     0.315078       0.98634      0.306833      0.121236       1000    
reco:ophitspe:OpHitFinderDeco                          0.0267262     0.141397      0.207779       0.15171      0.0357969      1000    
reco:opflash:OpFlashFinder                            0.000202461   0.00185972    0.00736444    0.00171593    0.000925148     1000    
reco:opslicer:OpSlicer                                5.8089e-05     0.349824       1.78704      0.270789      0.303094       1000    
reco:rns:RandomNumberSaver                             2.054e-05    3.51982e-05   0.000522407   3.3393e-05    1.79453e-05     1000    
reco:anglereconue:NuAngularReco                       0.000156682   0.00256015     0.195635     0.000493598    0.010365       1000    
reco:anglereconumu:NuAngularReco                       7.918e-05    0.00232105     0.193935     0.000318009   0.00983126      1000    
reco:anglereconuepfps:NuAngularReco                   0.000123273   0.00243083     0.194279     0.000431593   0.00985398      1000    
reco:anglereconumupfps:NuAngularReco                  0.000110915   0.00241687      0.19311     0.000407569   0.00980784      1000    
reco:anglerecohits:NuAngularReco                      0.000124157   0.00320293     0.190649     0.000942015    0.0105077      1000    
[art]:TriggerResults:TriggerResultInserter            1.0543e-05    1.58045e-05   6.2822e-05     1.546e-05    3.90153e-06     1000    
end_path:cafmaker:CAFMaker                            0.00125955     0.0293573      1.84684     0.00564709     0.109619       1000    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3995.93 MB
  Peak resident set size usage (VmHWM): 2307.66 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000944132, 0.993079, 0.00597636, 
Output 1: 0.605431, 0.389473, 0.00433186, 0.000764662, 
Output 2: 0.51181, 0.446654, 0.0385256, 0.00300982, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2734!
Begin processing the 994th record. run: 50515285 subRun: 1 event: 46594 at 16-Sep-2025 10:14:04 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 381!
Begin processing the 995th record. run: 50515285 subRun: 1 event: 46595 at 16-Sep-2025 10:14:06 CEST
Boundary wire vector sizes: 982, 1108, 911
minwire 0: 1333
minwire 1: 1151
minwire 2: 922
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.028699, 0.289983, 0.681318, 
Output 1: 0.0755321, 0.753887, 0.165458, 0.00512293, 
Output 2: 0.0509358, 0.534928, 0.380343, 0.0337934, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3220!
Begin processing the 996th record. run: 50515285 subRun: 1 event: 46596 at 16-Sep-2025 10:14:11 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 134!
Begin processing the 997th record. run: 50515285 subRun: 1 event: 46597 at 16-Sep-2025 10:14:13 CEST
Boundary wire vector sizes: 193, 262, 218
minwire 0: 1838
minwire 1: 1122
minwire 2: 1331
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00659918, 0.170872, 0.822529, 
Output 1: 0.137663, 0.856331, 0.0056804, 0.000325421, 
Output 2: 0.947035, 0.0517667, 0.0011573, 4.12585e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2065!
Begin processing the 998th record. run: 50515285 subRun: 1 event: 46598 at 16-Sep-2025 10:14:16 CEST
Boundary wire vector sizes: 499, 502, 389
minwire 0: 848
minwire 1: 2164
minwire 2: 184
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00885913, 0.870382, 0.120758, 
Output 1: 0.136876, 0.258858, 0.254354, 0.349912, 
Output 2: 0.175785, 0.425532, 0.272836, 0.125847, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2030!
Begin processing the 999th record. run: 50515285 subRun: 1 event: 46599 at 16-Sep-2025 10:14:20 CEST
Boundary wire vector sizes: 498, 605, 502
minwire 0: 1179
minwire 1: 1081
minwire 2: 845
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.206071, 0.156573, 0.637356, 
Output 1: 0.242724, 0.391035, 0.241553, 0.124687, 
Output 2: 0.109558, 0.202647, 0.351098, 0.336697, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2287!
Begin processing the 1000th record. run: 50515285 subRun: 1 event: 46600 at 16-Sep-2025 10:14:24 CEST
Boundary wire vector sizes: 318, 530, 252
minwire 0: 1249
minwire 1: 775
minwire 2: 1258
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.993003, 0.000994347, 0.00600225, 
Output 1: 0.141676, 0.840817, 0.017006, 0.000500433, 
Output 2: 0.982423, 0.0171735, 0.000393372, 1.03509e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1934!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/64/a1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_842_20231204T083201Z_gen_g4_detsim_hitreco__20240509T194754Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/eb/e1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74505925_1069_20231202T164130Z_gen_g4_detsim_hitreco__20240508T030221Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3e/42/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_978_20231202T152210Z_gen_g4_detsim_hitreco__20240508T031941Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/81/31/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420853_475_20231204T035000Z_gen_g4_detsim_hitreco__20240509T205215Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/07/d6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_175_20231208T053137Z_gen_g4_detsim_hitreco__20240510T043335Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/59/de/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_741_20231207T001640Z_gen_g4_detsim_hitreco__20240510T063709Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/73/f1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467222_555_20231207T052530Z_gen_g4_detsim_hitreco__20240510T044836Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/49/ff/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_436_20231204T081921Z_gen_g4_detsim_hitreco__20240509T215605Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3e/69/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420351_1065_20231204T045220Z_gen_g4_detsim_hitreco__20240509T191032Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/55/2f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_465_20231201T125710Z_gen_g4_detsim_hitreco__20240507T190545Z_reco2.root
.:
total 45300
-rw-r--r--. 1 duneprd np-comp 17053993 Sep 16 10:14 flatcaf.root
-rw-r--r--. 1 duneprd np-comp 14217358 Sep 16 10:14 caf.root
-rw-r--r--. 1 duneprd np-comp 14217358 Sep 16 10:14 caf_fd_hd_atmo_2501_20250916T063535Z.root
-rw-r--r--. 1 duneprd np-comp   593688 Sep 16 10:14 caf_20250916T063535Z.log
-rw-r--r--. 1 duneprd np-comp   224657 Sep 16 10:14 jobscript.log
-rw-r--r--. 1 duneprd np-comp     1885 Sep 16 10:14 caf_20250916T063535Z.file
-rw-r--r--. 1 duneprd np-comp     1885 Sep 16 10:14 caf_20250916T063535Z.pfns
-rw-r--r--. 1 duneprd np-comp     1885 Sep 16 08:35 file.list
-rw-r--r--. 1 duneprd np-comp     1885 Sep 16 10:14 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp     1375 Sep 16 08:35 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp     1375 Sep 16 10:14 caf_20250916T063535Z.did
-rw-r--r--. 1 duneprd np-comp     1375 Sep 16 08:35 did.list
-rw-r--r--. 1 duneprd np-comp        0 Sep 16 08:35 debugprod.log