justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 48503.4@dunegpschedd01.fnal.gov

Jobsub ID48503.4@dunegpschedd01.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 12:50:26
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-09-18 14:02:09
From worker nodeHostnamehnode59.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 12:50:54
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_231_20231207T052258Z_gen_g4_detsim_hitreco__20240510T063059Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_139_20231202T170748Z_gen_g4_detsim_hitreco__20240507T202945Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_45_20231205T105626Z_gen_g4_detsim_hitreco__20240510T062921Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_224_20231202T075625Z_gen_g4_detsim_hitreco__20240507T205015Z_reco2.root
JobscriptExit code0
Real time1h (4233s)
CPU time57m (3446s = 81%)
Max RSS bytes2142945280 (2043 MiB)
Outputting started2025-09-18 14:01:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250918T125110Z.root
Finished2025-09-18 14:02:09
Saved logsjustin-logs:48503.4-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

.000129564    0.0162435      1.02977     0.00256384     0.0710116       400    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000179186    0.0151079     0.461226      0.0038274     0.037065        400    
reco:pandoracalo:Calorimetry                          0.000107404    0.045305       2.32982     0.00135004     0.217771        400    
reco:pandorapid:Chi2ParticleID                        4.0821e-05    0.000265141   0.00287325    0.000176681   0.000283017      400    
reco:cvnmap:CVNMapper                                 2.1791e-05     0.0229159     0.0897323     0.0248268     0.0170375       400    
reco:cvneva:CVNEvaluator                               1.71e-05      0.424383       4.04159      0.504585      0.331205        400    
reco:energyrecnumu:EnergyReco                         0.00029045     0.192845       18.2382      0.0266947      1.01036        400    
reco:energyrecnue:EnergyReco                          0.000124405   0.00776433     0.886164     0.000882985    0.0490716       400    
reco:energyrecnc:EnergyReco                           0.00200401     0.156239       1.25673      0.177722      0.122935        400    
reco:energyrecnumurange:EnergyReco                    0.000358392    0.0848385      8.18103      0.0034648     0.527623        400    
reco:energyrecnumumcs:EnergyReco                       0.0006125     0.0737854      7.12986     0.00423119     0.425483        400    
reco:energyrecnumumcsllhd:EnergyReco                  0.000423975    0.0917996      11.5574     0.00435907     0.650788        400    
reco:opdec:Deconvolution                               0.126534      0.357019      0.666929      0.359037      0.106002        400    
reco:ophitspe:OpHitFinderDeco                          0.363782      0.394626       0.46589        0.378       0.0326056       400    
reco:opflash:OpFlashFinder                            0.000263249   0.00232914    0.00850302    0.00204789    0.00127251       400    
reco:opslicer:OpSlicer                                5.8702e-05     0.303339       1.63284      0.220207      0.278587        400    
reco:rns:RandomNumberSaver                            2.1081e-05    6.51232e-05   0.000819257   5.0551e-05    5.07091e-05      400    
reco:anglereconue:NuAngularReco                       0.000198166   0.00357624     0.355413     0.000669712    0.0197919       400    
reco:anglereconumu:NuAngularReco                      8.0103e-05    0.00321939     0.321095     0.000262964    0.0184636       400    
reco:anglereconuepfps:NuAngularReco                   0.000131185   0.00348673     0.382409      0.0004535     0.0210567       400    
reco:anglereconumupfps:NuAngularReco                  0.000110334    0.0033184     0.344774     0.000334212    0.0193248       400    
reco:anglerecohits:NuAngularReco                      0.000115964    0.0044745     0.391908     0.00102421     0.0219232       400    
[art]:TriggerResults:TriggerResultInserter             1.038e-05    2.84045e-05   0.000192637   1.9615e-05    2.25746e-05      400    
end_path:cafmaker:CAFMaker                            0.00230001     0.0407302      2.20683     0.00834884     0.156084        400    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3804.98 MB
  Peak resident set size usage (VmHWM): 2142.95 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 392nd record. run: 6400837 subRun: 1 event: 22492 at 18-Sep-2025 15:59:58 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 957!
Begin processing the 393rd record. run: 6400837 subRun: 1 event: 22493 at 18-Sep-2025 16:00:02 CEST
Boundary wire vector sizes: 2327, 2617, 1520
minwire 0: 1427
minwire 1: 877
minwire 2: 1513
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0109368, 0.977877, 0.011186, 
Output 1: 0.279344, 0.677737, 0.0394634, 0.00345518, 
Output 2: 0.219138, 0.716626, 0.0619673, 0.00226934, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3128!
Begin processing the 394th record. run: 6400837 subRun: 1 event: 22494 at 18-Sep-2025 16:00:12 CEST
Boundary wire vector sizes: 369, 404, 263
minwire 0: 297
minwire 1: 2424
minwire 2: 69
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0633652, 0.148919, 0.787716, 
Output 1: 0.0301962, 0.0790121, 0.140722, 0.75007, 
Output 2: 0.687175, 0.268143, 0.037572, 0.00710918, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1558!
Begin processing the 395th record. run: 6400837 subRun: 1 event: 22495 at 18-Sep-2025 16:00:19 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 982!
Begin processing the 396th record. run: 6400837 subRun: 1 event: 22496 at 18-Sep-2025 16:00:24 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 774!
Begin processing the 397th record. run: 6400837 subRun: 1 event: 22497 at 18-Sep-2025 16:00:28 CEST
Boundary wire vector sizes: 3238, 3231, 1751
minwire 0: 1232
minwire 1: 637
minwire 2: 1265
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0147875, 0.92413, 0.0610828, 
Output 1: 0.534925, 0.418428, 0.0353311, 0.0113164, 
Output 2: 0.0341202, 0.169518, 0.390081, 0.406281, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3242!
Begin processing the 398th record. run: 6400837 subRun: 1 event: 22498 at 18-Sep-2025 16:00:40 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 862!
Begin processing the 399th record. run: 6400837 subRun: 1 event: 22499 at 18-Sep-2025 16:00:45 CEST
Boundary wire vector sizes: 6092, 6006, 3822
minwire 0: 464
minwire 1: 1773
minwire 2: 260
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0124996, 0.977689, 0.0098118, 
Output 1: 0.355862, 0.442018, 0.104385, 0.0977359, 
Output 2: 0.0410467, 0.142857, 0.361393, 0.454703, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3422!
Begin processing the 400th record. run: 6400837 subRun: 1 event: 22500 at 18-Sep-2025 16:01:21 CEST
Boundary wire vector sizes: 33, 40, 31
minwire 0: 1320
minwire 1: 1255
minwire 2: 1366
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00538463, 0.00199413, 0.992621, 
Output 1: 0.307103, 0.677762, 0.0146636, 0.000472203, 
Output 2: 0.998479, 0.00144573, 7.18164e-05, 3.5085e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1108!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/88/d8/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_231_20231207T052258Z_gen_g4_detsim_hitreco__20240510T063059Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/e9/67/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_139_20231202T170748Z_gen_g4_detsim_hitreco__20240507T202945Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a8/0e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_45_20231205T105626Z_gen_g4_detsim_hitreco__20240510T062921Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/c3/b1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_224_20231202T075625Z_gen_g4_detsim_hitreco__20240507T205015Z_reco2.root
.:
total 19912
-rw-r--r-- 1 dune001 dune 7538387 Sep 18 16:01 flatcaf.root
-rw-r--r-- 1 dune001 dune 6226468 Sep 18 16:01 caf.root
-rw-r--r-- 1 dune001 dune 6226468 Sep 18 16:01 caf_fd_hd_atmo_2501_20250918T125110Z.root
-rw-r--r-- 1 dune001 dune  252158 Sep 18 16:01 caf_20250918T125110Z.log
-rw-r--r-- 1 dune001 dune   93768 Sep 18 16:01 jobscript.log
-rw-r--r-- 1 dune001 dune     836 Sep 18 16:01 caf_20250918T125110Z.file
-rw-r--r-- 1 dune001 dune     836 Sep 18 16:01 caf_20250918T125110Z.pfns
-rw-r--r-- 1 dune001 dune     836 Sep 18 14:51 file.list
-rw-r--r-- 1 dune001 dune     836 Sep 18 16:01 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune     550 Sep 18 14:51 all-input-dids.txt
-rw-r--r-- 1 dune001 dune     550 Sep 18 16:01 caf_20250918T125110Z.did
-rw-r--r-- 1 dune001 dune     550 Sep 18 14:51 did.list
-rw-r--r-- 1 dune001 dune       0 Sep 18 14:51 debugprod.log
justIN time: 2025-09-18 21:56:42 UTC       justIN version: 01.05.00