justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 40082.90@dunegpschedd02.fnal.gov

Jobsub ID40082.90@dunegpschedd02.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-15 15:12:49
SiteBR_CBPF
EntryDUNE_BR_CBPF_ce03
Last heartbeat2025-09-15 16:24:32
From worker nodeHostnamewn77
cpuinfoAMD EPYC 7713P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-15 15:14:26
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422149_337_20231204T120406Z_gen_g4_detsim_hitreco__20240509T220408Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_174_20231203T043759Z_gen_g4_detsim_hitreco__20240508T073012Z_reco2.root
JobscriptExit code0
Real time55m (3353s)
CPU time25m (1525s = 45%)
Max RSS bytes1987633152 (1895 MiB)
Outputting started2025-09-15 16:10:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T151439Z.root
Finished2025-09-15 16:24:28
Saved logsjustin-logs:40082.90-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

254.313       8.25126       25.8803        200    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.206908      0.328124      0.457022      0.244797      0.106013        200    
reco:pandora:StandardPandora                            2.48413       7.12749       244.935       2.8976        23.8318        200    
reco:pandoraTrack:LArPandoraTrackCreation             0.000138196    0.0130747     0.614759     0.00232407     0.0524041       200    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000172157    0.0219923     0.913004     0.00461272     0.0765373       200    
reco:pandoracalo:Calorimetry                          0.000104764    0.0501701      2.25536      0.0016002     0.217835        200    
reco:pandorapid:Chi2ParticleID                        3.8512e-05    0.000224175   0.00169375    0.000164717   0.000221372      200    
reco:cvnmap:CVNMapper                                 2.4651e-05    0.00754748     0.0595694    0.00846521    0.00672064       200    
reco:cvneva:CVNEvaluator                              1.8701e-05     0.426485       3.11843      0.570551      0.323991        200    
reco:energyrecnumu:EnergyReco                         0.000335415    0.453421       6.18559      0.0195313     0.696058        200    
reco:energyrecnue:EnergyReco                          0.000185098   0.00454694     0.172831     0.000801821    0.0157182       200    
reco:energyrecnc:EnergyReco                           0.00330556     0.919282       2.34787       1.2461       0.641787        200    
reco:energyrecnumurange:EnergyReco                    0.000375097    0.0766976      6.34166     0.00406392     0.468591        200    
reco:energyrecnumumcs:EnergyReco                      0.00089399     0.0765726      6.21798     0.00467855     0.459121        200    
reco:energyrecnumumcsllhd:EnergyReco                  0.000728112    0.0776167      6.10408     0.00469233     0.454373        200    
reco:opdec:Deconvolution                               0.636317       1.04039       2.07403       1.03219      0.209735        200    
reco:ophitspe:OpHitFinderDeco                           1.85287       1.89229       2.02058       1.90661      0.0256161       200    
reco:opflash:OpFlashFinder                            0.000369426   0.00172935    0.00435834    0.00162117    0.000821611      200    
reco:opslicer:OpSlicer                                 0.002426      0.347651       1.74428      0.269027      0.309763        200    
reco:rns:RandomNumberSaver                            2.6121e-05    3.89894e-05   0.000383767   3.6552e-05    2.55116e-05      200    
reco:anglereconue:NuAngularReco                       0.000214689   0.00313366     0.182591     0.000487182    0.0146771       200    
reco:anglereconumu:NuAngularReco                      0.000113465   0.00297903     0.180051     0.000351525    0.0144801       200    
reco:anglereconuepfps:NuAngularReco                   0.000166748   0.00305411      0.17198     0.000454815    0.0138671       200    
reco:anglereconumupfps:NuAngularReco                  0.000161537   0.00304548     0.180466     0.00044127     0.0143685       200    
reco:anglerecohits:NuAngularReco                      0.000489242   0.00398848     0.188354      0.0010264     0.0153582       200    
[art]:TriggerResults:TriggerResultInserter             1.091e-05    1.85777e-05   0.000111065   1.72205e-05   9.42543e-06      200    
end_path:cafmaker:CAFMaker                            0.00152264     0.0396731      2.13546     0.00531764     0.173869        200    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3776.44 MB
  Peak resident set size usage (VmHWM): 1987.63 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 849!
Begin processing the 195th record. run: 50572032 subRun: 1 event: 17495 at 15-Sep-2025 13:05:25 -03
Boundary wire vector sizes: 78, 62, 25
minwire 0: 1197
minwire 1: 1673
minwire 2: 1166
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.994544, 0.000571933, 0.00488418, 
Output 1: 0.024071, 0.820224, 0.15075, 0.00495469, 
Output 2: 0.994224, 0.00559177, 0.000180358, 4.06417e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1339!
Begin processing the 196th record. run: 50572032 subRun: 1 event: 17496 at 15-Sep-2025 13:05:36 -03
Boundary wire vector sizes: 457, 316, 300
minwire 0: 1992
minwire 1: 758
minwire 2: 2096
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00315808, 0.992047, 0.00479494, 
Output 1: 0.0140178, 0.0476793, 0.164236, 0.774067, 
Output 2: 0.939169, 0.0555439, 0.00451958, 0.000767388, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1957!
Begin processing the 197th record. run: 50572032 subRun: 1 event: 17497 at 15-Sep-2025 13:05:48 -03
Boundary wire vector sizes: 16607, 12169, 10622
minwire 0: 840
minwire 1: 511
minwire 2: 310
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0195108, 0.951407, 0.0290827, 
Output 1: 0.156653, 0.232014, 0.235283, 0.37605, 
Output 2: 0.0256409, 0.0767253, 0.28757, 0.610064, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3796!
Begin processing the 198th record. run: 50572032 subRun: 1 event: 17498 at 15-Sep-2025 13:09:42 -03
Boundary wire vector sizes: 225, 236, 249
minwire 0: 1713
minwire 1: 664
minwire 2: 2016
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00153749, 0.97993, 0.0185324, 
Output 1: 0.990651, 0.00914419, 0.00011996, 8.5314e-05, 
Output 2: 0.998408, 0.00153331, 4.12904e-05, 1.69386e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1107!
Begin processing the 199th record. run: 50572032 subRun: 1 event: 17499 at 15-Sep-2025 13:09:54 -03
Boundary wire vector sizes: 532, 456, 379
minwire 0: 1174
minwire 1: 540
minwire 2: 983
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00901043, 0.807494, 0.183495, 
Output 1: 0.192232, 0.666447, 0.13055, 0.0107713, 
Output 2: 0.338629, 0.520253, 0.128434, 0.0126852, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2741!
Begin processing the 200th record. run: 50572032 subRun: 1 event: 17500 at 15-Sep-2025 13:10:06 -03
Boundary wire vector sizes: 514, 378, 319
minwire 0: 367
minwire 1: 2169
minwire 2: 144
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000540545, 0.996168, 0.00329134, 
Output 1: 0.99603, 0.00376366, 9.86847e-05, 0.000108097, 
Output 2: 0.998072, 0.00184815, 5.34822e-05, 2.58485e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1997!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/09/9f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422149_337_20231204T120406Z_gen_g4_detsim_hitreco__20240509T220408Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/d2/14/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_174_20231203T043759Z_gen_g4_detsim_hitreco__20240508T073012Z_reco2.root
.:
total 10824
-rw-r--r-- 1 nobody nobody 4148313 Sep 15 13:10 flatcaf.root
-rw-r--r-- 1 nobody nobody 3356953 Sep 15 13:10 caf.root
-rw-r--r-- 1 nobody nobody 3356953 Sep 15 13:10 caf_fd_hd_atmo_2501_20250915T151439Z.root
-rw-r--r-- 1 nobody nobody  129277 Sep 15 13:10 caf_20250915T151439Z.log
-rw-r--r-- 1 nobody nobody   52576 Sep 15 13:10 jobscript.log
-rw-r--r-- 1 nobody nobody     373 Sep 15 13:10 caf_20250915T151439Z.file
-rw-r--r-- 1 nobody nobody     373 Sep 15 13:10 caf_20250915T151439Z.pfns
-rw-r--r-- 1 nobody nobody     373 Sep 15 12:14 file.list
-rw-r--r-- 1 nobody nobody     373 Sep 15 13:10 justin-processed-pfns.txt
-rw-r--r-- 1 nobody nobody     275 Sep 15 12:14 all-input-dids.txt
-rw-r--r-- 1 nobody nobody     275 Sep 15 13:10 caf_20250915T151439Z.did
-rw-r--r-- 1 nobody nobody     275 Sep 15 12:14 did.list
-rw-r--r-- 1 nobody nobody       0 Sep 15 12:15 debugprod.log
justIN time: 2025-09-18 23:56:36 UTC       justIN version: 01.05.00