justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 40131.0@dunegpschedd02.fnal.gov

Jobsub ID40131.0@dunegpschedd02.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-15 16:55:53
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2025-09-15 17:39:26
From worker nodeHostnameccwcondor0386
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-15 16:57:49
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66029763_419_20231207T032137Z_gen_g4_detsim_hitreco__20240510T024042Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50649927_101_20231208T043454Z_gen_g4_detsim_hitreco__20240510T065425Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_1118_20231203T073258Z_gen_g4_detsim_hitreco__20240508T073027Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515286_395_20231201T144829Z_gen_g4_detsim_hitreco__20240507T190648Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419624_1136_20231204T031823Z_gen_g4_detsim_hitreco__20240509T212349Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_568_20231202T114117Z_gen_g4_detsim_hitreco__20240507T215908Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66029763_375_20231207T031848Z_gen_g4_detsim_hitreco__20240510T023726Z_reco2.root
JobscriptExit code0
Real time41m (2464s)
CPU time30m (1802s = 73%)
Max RSS bytes2297851904 (2191 MiB)
Outputting started2025-09-15 17:38:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T165757Z.root
Finished2025-09-15 17:39:26
Saved logsjustin-logs:40131.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.000638682    0.0795358       700    
reco:pandorapid:Chi2ParticleID                        2.1853e-05    9.96039e-05   0.000852401   7.4208e-05    8.61901e-05      700    
reco:cvnmap:CVNMapper                                 1.1328e-05    0.00523239     0.0313408    0.00473673    0.00494118       700    
reco:cvneva:CVNEvaluator                              1.0936e-05     0.160899       1.28028      0.183914       0.11469        700    
reco:energyrecnumu:EnergyReco                         0.000152561    0.0586244      5.79084      0.0112545     0.238271        700    
reco:energyrecnue:EnergyReco                          6.4277e-05    0.00144959     0.0970328    0.000352168   0.00540299       700    
reco:energyrecnc:EnergyReco                           0.000932622    0.0807881     0.405942      0.092653      0.0623504       700    
reco:energyrecnumurange:EnergyReco                    0.000124448    0.0262418      5.24743     0.00124871      0.21358        700    
reco:energyrecnumumcs:EnergyReco                      0.000318583    0.0266466      5.25725     0.00199936     0.214013        700    
reco:energyrecnumumcsllhd:EnergyReco                  0.00017875     0.0264953      5.28697     0.00193236     0.214852        700    
reco:opdec:Deconvolution                               0.0579856     0.169002      0.440354      0.164704      0.0534565       700    
reco:ophitspe:OpHitFinderDeco                          0.161283      0.168936      0.194047      0.169245     0.00333574       700    
reco:opflash:OpFlashFinder                            8.0241e-05    0.000872209   0.00273913    0.000784172   0.000444617      700    
reco:opslicer:OpSlicer                                4.3896e-05     0.154123       1.03213      0.115257      0.139425        700    
reco:rns:RandomNumberSaver                            1.5003e-05    3.07242e-05   0.000262437   2.7016e-05    1.53822e-05      700    
reco:anglereconue:NuAngularReco                       0.000111148   0.000976562    0.0734937    0.000287475   0.00375258       700    
reco:anglereconumu:NuAngularReco                      3.6255e-05    0.000805364    0.078979     0.000124703   0.00389439       700    
reco:anglereconuepfps:NuAngularReco                   6.4708e-05    0.000877799    0.0762469    0.000201715   0.00384202       700    
reco:anglereconumupfps:NuAngularReco                  4.8273e-05    0.000834881    0.0740594    0.000160313   0.00378145       700    
reco:anglerecohits:NuAngularReco                      4.3837e-05    0.00127943     0.0812061    0.000476943   0.00425052       700    
[art]:TriggerResults:TriggerResultInserter             6.841e-06    1.00168e-05   3.0306e-05    8.7735e-06    2.97152e-06      700    
end_path:cafmaker:CAFMaker                            0.000814343    0.0123934     0.994232     0.00320162     0.0486163       700    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3987.6 MB
  Peak resident set size usage (VmHWM): 2297.85 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 0: 0.980305, 0.000911799, 0.0187834, 
Output 1: 0.973836, 0.0253325, 0.000707381, 0.000123936, 
Output 2: 0.981777, 0.0178889, 0.000307274, 2.73502e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 486!
Begin processing the 693rd record. run: 66029763 subRun: 1 event: 37593 at 15-Sep-2025 19:38:30 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 794!
Begin processing the 694th record. run: 66029763 subRun: 1 event: 37594 at 15-Sep-2025 19:38:32 CEST
Boundary wire vector sizes: 2013, 1851, 1133
minwire 0: 397
minwire 1: 2302
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0035256, 0.992431, 0.00404318, 
Output 1: 0.00730516, 0.0295722, 0.0890526, 0.87407, 
Output 2: 0.833559, 0.145551, 0.0170736, 0.00381567, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2526!
Begin processing the 695th record. run: 66029763 subRun: 1 event: 37595 at 15-Sep-2025 19:38:37 CEST
Boundary wire vector sizes: 45, 66, 30
minwire 0: 1010
minwire 1: 1514
minwire 2: 1091
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.995482, 0.00126356, 0.00325418, 
Output 1: 0.0333881, 0.956101, 0.0101476, 0.000363135, 
Output 2: 0.997058, 0.00285743, 8.22594e-05, 2.57152e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 755!
Begin processing the 696th record. run: 66029763 subRun: 1 event: 37596 at 15-Sep-2025 19:38:39 CEST
Boundary wire vector sizes: 150, 172, 90
minwire 0: 2105
minwire 1: 1393
minwire 2: 1796
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00501062, 0.00657082, 0.988419, 
Output 1: 0.872741, 0.120231, 0.00601551, 0.00101321, 
Output 2: 0.996891, 0.00303212, 7.05738e-05, 6.5839e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2053!
Begin processing the 697th record. run: 66029763 subRun: 1 event: 37597 at 15-Sep-2025 19:38:42 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 888!
Begin processing the 698th record. run: 66029763 subRun: 1 event: 37598 at 15-Sep-2025 19:38:44 CEST
Boundary wire vector sizes: 1015, 1289, 1140
minwire 0: 2015
minwire 1: 903
minwire 2: 1820
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000300207, 0.998718, 0.000982193, 
Output 1: 0.0034546, 0.992596, 0.00381001, 0.000139527, 
Output 2: 0.997302, 0.00262228, 7.33048e-05, 2.2944e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2574!
Begin processing the 699th record. run: 66029763 subRun: 1 event: 37599 at 15-Sep-2025 19:38:48 CEST
Boundary wire vector sizes: 125, 186, 135
minwire 0: 539
minwire 1: 2720
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.85628, 0.0555358, 0.0881841, 
Output 1: 0.0800424, 0.7936, 0.119205, 0.00715261, 
Output 2: 0.108605, 0.86566, 0.0255344, 0.000200577, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1285!
Begin processing the 700th record. run: 66029763 subRun: 1 event: 37600 at 15-Sep-2025 19:38:50 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 178!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3b/90/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66029763_419_20231207T032137Z_gen_g4_detsim_hitreco__20240510T024042Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/88/e6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50649927_101_20231208T043454Z_gen_g4_detsim_hitreco__20240510T065425Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/f0/f2/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_1118_20231203T073258Z_gen_g4_detsim_hitreco__20240508T073027Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9e/e9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515286_395_20231201T144829Z_gen_g4_detsim_hitreco__20240507T190648Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c7/05/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419624_1136_20231204T031823Z_gen_g4_detsim_hitreco__20240509T212349Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/7f/2d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_568_20231202T114117Z_gen_g4_detsim_hitreco__20240507T215908Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/34/1c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66029763_375_20231207T031848Z_gen_g4_detsim_hitreco__20240510T023726Z_reco2.root
.:
total 32340
-rw-r--r-- 1 dune001 lbno 12183926 Sep 15 19:38 flatcaf.root
-rw-r--r-- 1 dune001 lbno 10129682 Sep 15 19:38 caf.root
-rw-r--r-- 1 dune001 lbno 10129682 Sep 15 19:38 caf_fd_hd_atmo_2501_20250915T165757Z.root
-rw-r--r-- 1 dune001 lbno   418421 Sep 15 19:38 caf_20250915T165757Z.log
-rw-r--r-- 1 dune001 lbno   170426 Sep 15 19:38 jobscript.log
-rw-r--r-- 1 dune001 lbno     1309 Sep 15 19:38 caf_20250915T165757Z.file
-rw-r--r-- 1 dune001 lbno     1309 Sep 15 19:38 caf_20250915T165757Z.pfns
-rw-r--r-- 1 dune001 lbno     1309 Sep 15 18:57 file.list
-rw-r--r-- 1 dune001 lbno     1309 Sep 15 19:38 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno      966 Sep 15 18:57 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno      966 Sep 15 19:38 caf_20250915T165757Z.did
-rw-r--r-- 1 dune001 lbno      966 Sep 15 18:57 did.list
-rw-r--r-- 1 dune001 lbno        0 Sep 15 18:58 debugprod.log
justIN time: 2025-09-19 04:30:41 UTC       justIN version: 01.05.00