justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 40234.9@dunegpschedd02.fnal.gov

Jobsub ID40234.9@dunegpschedd02.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-15 22:26:11
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-09-15 23:15:25
From worker nodeHostnamehnode75.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-15 22:26:57
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50538503_742_20231201T175007Z_gen_g4_detsim_hitreco__20240507T191524Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65974923_74_20231201T042205Z_gen_g4_detsim_hitreco__20240507T192540Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_0_20231208T091224Z_gen_g4_detsim_hitreco__20240510T062935Z_reco2.root
JobscriptExit code0
Real time47m (2877s)
CPU time37m (2245s = 78%)
Max RSS bytes2156630016 (2056 MiB)
Outputting started2025-09-15 23:14:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/001/caf_fd_hd_atmo_2501_20250915T222708Z.root
Finished2025-09-15 23:15:25
Saved logsjustin-logs:40234.9-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

300    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0394846     0.0873773     0.144724      0.0804656     0.0265444       300    
reco:pandora:StandardPandora                           0.247583       6.27197       811.302       1.67523       48.0139        300    
reco:pandoraTrack:LArPandoraTrackCreation             0.000112313   0.00974182     0.601155     0.00178045     0.0388522       300    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000142243    0.0200825      0.64889      0.0037384     0.0625101       300    
reco:pandoracalo:Calorimetry                          8.8642e-05     0.0415674      2.90968     0.00124312     0.212512        300    
reco:pandorapid:Chi2ParticleID                         3.266e-05    0.000185364   0.00270474    0.000140478   0.000197393      300    
reco:cvnmap:CVNMapper                                 2.0331e-05    0.00999127     0.0635813     0.0110419    0.00729914       300    
reco:cvneva:CVNEvaluator                               1.499e-05     0.345451       2.17277      0.433051      0.219164        300    
reco:energyrecnumu:EnergyReco                         0.000257846    0.150368       8.0146       0.0148325     0.587853        300    
reco:energyrecnue:EnergyReco                          0.000116672   0.00373304     0.221836     0.000715457    0.0150725       300    
reco:energyrecnc:EnergyReco                           0.00156562     0.195943      0.691923      0.225663      0.127646        300    
reco:energyrecnumurange:EnergyReco                    0.000350068    0.0798241      8.89799     0.00294056     0.570587        300    
reco:energyrecnumumcs:EnergyReco                      0.000658554    0.0722555      7.66244     0.00348716     0.485773        300    
reco:energyrecnumumcsllhd:EnergyReco                  0.000573292    0.0708815      6.25042     0.00354834     0.434152        300    
reco:opdec:Deconvolution                               0.120221      0.338887      0.635346       0.34262      0.0825989       300    
reco:ophitspe:OpHitFinderDeco                          0.347783      0.423848      0.486587      0.449026      0.0489608       300    
reco:opflash:OpFlashFinder                            0.000220235   0.00155024    0.00590814    0.00146133    0.000711294      300    
reco:opslicer:OpSlicer                                0.000124893     0.27782       1.30534      0.219536      0.224495        300    
reco:rns:RandomNumberSaver                            2.3171e-05    4.02596e-05   0.000462401   3.5581e-05    2.72818e-05      300    
reco:anglereconue:NuAngularReco                       0.000174454   0.00198613     0.0840332    0.000469525   0.00644956       300    
reco:anglereconumu:NuAngularReco                      9.0202e-05    0.00180959     0.0843139    0.000279746   0.00652889       300    
reco:anglereconuepfps:NuAngularReco                   0.000133103   0.00194445     0.0946662    0.000370849   0.00691798       300    
reco:anglereconumupfps:NuAngularReco                  0.000114303    0.0019175     0.0988709    0.000335122   0.00712068       300    
reco:anglerecohits:NuAngularReco                      0.000139603    0.0026142     0.103139     0.000767347   0.00774023       300    
[art]:TriggerResults:TriggerResultInserter             8.47e-06     1.57203e-05   6.1322e-05    1.46905e-05   5.23378e-06      300    
end_path:cafmaker:CAFMaker                            0.00149343     0.0266253      1.21932     0.00541526     0.0924844       300    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3819.4 MB
  Peak resident set size usage (VmHWM): 2156.63 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 579, 415, 383
minwire 0: 1436
minwire 1: 62
minwire 2: 1511
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.83806, 0.00245618, 0.159483, 
Output 1: 0.26129, 0.401114, 0.253084, 0.0845114, 
Output 2: 0.055982, 0.287974, 0.452827, 0.203218, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1627!
Begin processing the 292nd record. run: 6472782 subRun: 1 event: 92 at 16-Sep-2025 01:13:35 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1115!
Begin processing the 293rd record. run: 6472782 subRun: 1 event: 93 at 16-Sep-2025 01:13:39 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 412!
Begin processing the 294th record. run: 6472782 subRun: 1 event: 94 at 16-Sep-2025 01:13:42 CEST
Boundary wire vector sizes: 7407, 10244, 6293
minwire 0: 733
minwire 1: 340
minwire 2: 485
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.334091, 0.662574, 0.00333462, 
Output 1: 0.283862, 0.615482, 0.0936186, 0.00703702, 
Output 2: 0.105672, 0.471245, 0.3448, 0.0782821, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3380!
Begin processing the 295th record. run: 6472782 subRun: 1 event: 95 at 16-Sep-2025 01:14:29 CEST
Boundary wire vector sizes: 776, 767, 693
minwire 0: 1280
minwire 1: 734
minwire 2: 1152
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0255208, 0.965229, 0.00925056, 
Output 1: 0.0963543, 0.88358, 0.0192453, 0.000820464, 
Output 2: 0.331706, 0.624409, 0.0428501, 0.00103541, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2469!
Begin processing the 296th record. run: 6472782 subRun: 1 event: 96 at 16-Sep-2025 01:14:34 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1073!
Begin processing the 297th record. run: 6472782 subRun: 1 event: 97 at 16-Sep-2025 01:14:38 CEST
Boundary wire vector sizes: 82, 57, 32
minwire 0: 1288
minwire 1: 1127
minwire 2: 1547
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000496607, 0.998203, 0.0013009, 
Output 1: 0.00907793, 0.986791, 0.0039963, 0.000134196, 
Output 2: 0.998702, 0.00125873, 3.81482e-05, 1.21861e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1001!
Begin processing the 298th record. run: 6472782 subRun: 1 event: 98 at 16-Sep-2025 01:14:42 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 775!
Begin processing the 299th record. run: 6472782 subRun: 1 event: 99 at 16-Sep-2025 01:14:45 CEST
Boundary wire vector sizes: 230, 156, 206
minwire 0: 1474
minwire 1: 1988
minwire 2: 1102
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00640763, 0.00449947, 0.989093, 
Output 1: 0.931572, 0.0668162, 0.00139462, 0.000216861, 
Output 2: 0.993982, 0.00588432, 0.000122572, 1.1607e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1577!
Begin processing the 300th record. run: 6472782 subRun: 1 event: 100 at 16-Sep-2025 01:14:50 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 794!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/48/35/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50538503_742_20231201T175007Z_gen_g4_detsim_hitreco__20240507T191524Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c0/41/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65974923_74_20231201T042205Z_gen_g4_detsim_hitreco__20240507T192540Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/50/be/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_0_20231208T091224Z_gen_g4_detsim_hitreco__20240510T062935Z_reco2.root
.:
total 16776
-rw-r--r-- 1 dune001 dune 6372688 Sep 16 01:14 flatcaf.root
-rw-r--r-- 1 dune001 dune 5240777 Sep 16 01:14 caf.root
-rw-r--r-- 1 dune001 dune 5240777 Sep 16 01:14 caf_fd_hd_atmo_2501_20250915T222708Z.root
-rw-r--r-- 1 dune001 dune  195256 Sep 16 01:14 caf_20250915T222708Z.log
-rw-r--r-- 1 dune001 dune   80064 Sep 16 01:14 jobscript.log
-rw-r--r-- 1 dune001 dune     577 Sep 16 01:14 caf_20250915T222708Z.file
-rw-r--r-- 1 dune001 dune     577 Sep 16 01:14 caf_20250915T222708Z.pfns
-rw-r--r-- 1 dune001 dune     577 Sep 16 00:27 file.list
-rw-r--r-- 1 dune001 dune     577 Sep 16 01:14 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune     410 Sep 16 00:27 all-input-dids.txt
-rw-r--r-- 1 dune001 dune     410 Sep 16 01:14 caf_20250915T222708Z.did
-rw-r--r-- 1 dune001 dune     410 Sep 16 00:27 did.list
-rw-r--r-- 1 dune001 dune       0 Sep 16 00:27 debugprod.log
justIN time: 2025-09-18 20:36:09 UTC       justIN version: 01.05.00