Jobsub ID 40343.45@dunegpschedd02.fnal.gov

Jobsub ID	40343.45@dunegpschedd02.fnal.gov
Workflow ID	2501
Stage ID	1
User name	pgranger@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-16 06:34:40
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce511
Last heartbeat	2025-09-16 07:40:13
From worker node	Hostname	b9p09p4833.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-16 06:35:31
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6406796_203_20231202T132558Z_gen_g4_detsim_hitreco__20240507T223145Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6464269_751_20231207T034132Z_gen_g4_detsim_hitreco__20240510T042354Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65974923_271_20231201T075335Z_gen_g4_detsim_hitreco__20240507T191728Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577113_524_20231204T014815Z_gen_g4_detsim_hitreco__20240509T215642Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_83_20231208T055810Z_gen_g4_detsim_hitreco__20240510T052115Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_624_20231206T235017Z_gen_g4_detsim_hitreco__20240510T041153Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_881_20231204T075543Z_gen_g4_detsim_hitreco__20240509T220304Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_490_20231203T104001Z_gen_g4_detsim_hitreco__20240508T075607Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6370890_153_20231201T073735Z_gen_g4_detsim_hitreco__20240507T201539Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419766_1082_20231204T010313Z_gen_g4_detsim_hitreco__20240509T214204Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (3857s)
	CPU time	50m (3003s = 77%)
	Max RSS bytes	2091372544 (1994 MiB)
Outputting started	2025-09-16 07:39:50
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250916T063544Z.root
Finished	2025-09-16 07:40:13
Saved logs	justin-logs:40343.45-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0108521   0.00212976     0.076052     0.000600242   0.00583587      1000    
reco:energyrecnc:EnergyReco                           0.00145327     0.0810311     0.326437      0.104795      0.0609512      1000    
reco:energyrecnumurange:EnergyReco                    0.000177882    0.0380036      3.81018     0.00284948     0.190902       1000    
reco:energyrecnumumcs:EnergyReco                      0.000517616    0.0374938      3.75584     0.00317455     0.185885       1000    
reco:energyrecnumumcsllhd:EnergyReco                  0.000337874    0.0364804      3.66809     0.00315348     0.179528       1000    
reco:opdec:Deconvolution                               0.0164812      0.20572       0.83433      0.204245      0.0736541      1000    
reco:ophitspe:OpHitFinderDeco                          0.0325952     0.138004      0.178612      0.148312      0.0330032      1000    
reco:opflash:OpFlashFinder                            0.000130271   0.00128669    0.00474977    0.00122477    0.000590283     1000    
reco:opslicer:OpSlicer                                 3.947e-05     0.246511       1.19076      0.190795      0.209938       1000    
reco:rns:RandomNumberSaver                             1.573e-05    3.97575e-05   0.000507765   3.6596e-05    1.84659e-05     1000    
reco:anglereconue:NuAngularReco                       8.1511e-05    0.00138189     0.0697009    0.00039182    0.00439888      1000    
reco:anglereconumu:NuAngularReco                      6.0741e-05    0.00120913     0.0689133    0.000225037   0.00441483      1000    
reco:anglereconuepfps:NuAngularReco                   8.4421e-05    0.00131098     0.0682637    0.000329994    0.0044507      1000    
reco:anglereconumupfps:NuAngularReco                  7.8201e-05    0.00126822     0.0681104    0.000279438   0.00443568      1000    
reco:anglerecohits:NuAngularReco                      9.3881e-05    0.00187524     0.0745916    0.000689357   0.00503519      1000    
[art]:TriggerResults:TriggerResultInserter             8.43e-06     1.42766e-05   8.1421e-05     1.361e-05    4.6391e-06      1000    
end_path:cafmaker:CAFMaker                            0.00096873     0.0168667      0.89243     0.00429942     0.054131       1000    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3810.89 MB
  Peak resident set size usage (VmHWM): 2091.37 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1539!
Begin processing the 992nd record. run: 6419766 subRun: 1 event: 108292 at 16-Sep-2025 09:39:26 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 426!
Begin processing the 993rd record. run: 6419766 subRun: 1 event: 108293 at 16-Sep-2025 09:39:28 CEST
Boundary wire vector sizes: 237, 221, 169
minwire 0: 716
minwire 1: 1753
minwire 2: 485
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00548596, 0.0594914, 0.935023, 
Output 1: 0.968368, 0.030559, 0.000876808, 0.000195788, 
Output 2: 0.993482, 0.00637622, 0.000122153, 1.99488e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1648!
Begin processing the 994th record. run: 6419766 subRun: 1 event: 108294 at 16-Sep-2025 09:39:31 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 837!
Begin processing the 995th record. run: 6419766 subRun: 1 event: 108295 at 16-Sep-2025 09:39:33 CEST
Boundary wire vector sizes: 223, 127, 104
minwire 0: 1621
minwire 1: 940
minwire 2: 1773
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.993719, 0.00097803, 0.00530315, 
Output 1: 0.100162, 0.889642, 0.00988798, 0.000307977, 
Output 2: 0.992705, 0.00714945, 0.000142268, 3.29038e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1429!
Begin processing the 996th record. run: 6419766 subRun: 1 event: 108296 at 16-Sep-2025 09:39:36 CEST
Boundary wire vector sizes: 212, 271, 250
minwire 0: 460
minwire 1: 2070
minwire 2: 212
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000651933, 0.986071, 0.0132775, 
Output 1: 0.0590609, 0.930065, 0.0106165, 0.000257275, 
Output 2: 0.99532, 0.00456927, 0.000106835, 3.90483e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1265!
Begin processing the 997th record. run: 6419766 subRun: 1 event: 108297 at 16-Sep-2025 09:39:38 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 263!
Begin processing the 998th record. run: 6419766 subRun: 1 event: 108298 at 16-Sep-2025 09:39:40 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 560!
Begin processing the 999th record. run: 6419766 subRun: 1 event: 108299 at 16-Sep-2025 09:39:42 CEST
Boundary wire vector sizes: 463, 348, 321
minwire 0: 935
minwire 1: 1187
minwire 2: 811
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.40579, 0.0225241, 0.571686, 
Output 1: 0.983109, 0.0162692, 0.000480194, 0.000141271, 
Output 2: 0.34691, 0.630335, 0.0215956, 0.00115947, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2230!
Begin processing the 1000th record. run: 6419766 subRun: 1 event: 108300 at 16-Sep-2025 09:39:45 CEST
Boundary wire vector sizes: 221, 255, 246
minwire 0: 626
minwire 1: 1624
minwire 2: 673
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00678943, 0.819178, 0.174033, 
Output 1: 0.00385333, 0.9855, 0.0102685, 0.000378757, 
Output 2: 0.977321, 0.0221184, 0.000547548, 1.32339e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1343!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9c/70/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6406796_203_20231202T132558Z_gen_g4_detsim_hitreco__20240507T223145Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/2f/2e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6464269_751_20231207T034132Z_gen_g4_detsim_hitreco__20240510T042354Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/39/ce/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65974923_271_20231201T075335Z_gen_g4_detsim_hitreco__20240507T191728Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1f/fb/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577113_524_20231204T014815Z_gen_g4_detsim_hitreco__20240509T215642Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/8f/e4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_83_20231208T055810Z_gen_g4_detsim_hitreco__20240510T052115Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/d9/c9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_624_20231206T235017Z_gen_g4_detsim_hitreco__20240510T041153Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/53/4f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_881_20231204T075543Z_gen_g4_detsim_hitreco__20240509T220304Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/04/67/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_490_20231203T104001Z_gen_g4_detsim_hitreco__20240508T075607Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/b1/9a/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6370890_153_20231201T073735Z_gen_g4_detsim_hitreco__20240507T201539Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/df/60/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419766_1082_20231204T010313Z_gen_g4_detsim_hitreco__20240509T214204Z_reco2.root
.:
total 38496
-rw-r--r--. 1 duneprd np-comp 14467438 Sep 16 09:39 flatcaf.root
-rw-r--r--. 1 duneprd np-comp 12025509 Sep 16 09:39 caf.root
-rw-r--r--. 1 duneprd np-comp 12025509 Sep 16 09:39 caf_fd_hd_atmo_2501_20250916T063544Z.root
-rw-r--r--. 1 duneprd np-comp   593953 Sep 16 09:39 caf_20250916T063544Z.log
-rw-r--r--. 1 duneprd np-comp   216689 Sep 16 09:39 jobscript.log
-rw-r--r--. 1 duneprd np-comp     1884 Sep 16 09:39 caf_20250916T063544Z.file
-rw-r--r--. 1 duneprd np-comp     1884 Sep 16 09:39 caf_20250916T063544Z.pfns
-rw-r--r--. 1 duneprd np-comp     1884 Sep 16 08:35 file.list
-rw-r--r--. 1 duneprd np-comp     1884 Sep 16 09:39 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp     1374 Sep 16 08:35 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp     1374 Sep 16 09:39 caf_20250916T063544Z.did
-rw-r--r--. 1 duneprd np-comp     1374 Sep 16 08:35 did.list
-rw-r--r--. 1 duneprd np-comp        0 Sep 16 08:35 debugprod.log