justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 47886.0@dunegpschedd01.fnal.gov

Jobsub ID47886.0@dunegpschedd01.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-17 23:19:45
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-09-17 23:40:55
From worker nodeHostnamewn-la-22.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-17 23:20:45
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_517_20231202T114026Z_gen_g4_detsim_hitreco__20240507T215951Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207207_268_20231120T133335Z_gen_g4_detsim_hitreco__20240503T061930Z_reco2.root
JobscriptExit code0
Real time19m (1155s)
CPU time15m (942s = 81%)
Max RSS bytes2149109760 (2049 MiB)
Outputting started2025-09-17 23:40:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T232058Z.root
Finished2025-09-17 23:40:55
Saved logsjustin-logs:47886.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

3.30537       43.7646       2.37681       5.09261        300    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0159785     0.0398817     0.175153      0.0391823     0.0151578       300    
reco:pandora:StandardPandora                           0.0998873      2.24553       38.5652       1.39585       4.27318        300    
reco:pandoraTrack:LArPandoraTrackCreation             0.00011341    0.00549597     0.173951     0.00115737      0.01763        300    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000152288    0.0101495     0.267784     0.00217842     0.028036        300    
reco:pandoracalo:Calorimetry                           9.363e-05     0.0174926     0.570693     0.00072281     0.0738551       300    
reco:pandorapid:Chi2ParticleID                        3.4112e-05    0.000132192   0.00111833    0.000104802   0.000117573      300    
reco:cvnmap:CVNMapper                                 1.7787e-05    0.00607508     0.0179226    0.00725075    0.00425314       300    
reco:cvneva:CVNEvaluator                              1.7216e-05     0.227211       1.88212       0.30847      0.171277        300    
reco:energyrecnumu:EnergyReco                         0.000279528    0.0612561      4.01718      0.0125353     0.254503        300    
reco:energyrecnue:EnergyReco                          0.000109644   0.00183093     0.0622835    0.000439157   0.00556668       300    
reco:energyrecnc:EnergyReco                           0.00153812     0.0581926     0.277329      0.0705209     0.0470171       300    
reco:energyrecnumurange:EnergyReco                    0.000210696    0.0240145      2.57777      0.001776      0.164727        300    
reco:energyrecnumumcs:EnergyReco                      0.000546716    0.0240416      2.5679      0.00200003     0.161744        300    
reco:energyrecnumumcsllhd:EnergyReco                  0.00038108     0.0252848      2.5575      0.00202844     0.168658        300    
reco:opdec:Deconvolution                               0.0502439      0.18899      0.445644      0.190845      0.0634243       300    
reco:ophitspe:OpHitFinderDeco                          0.135988       0.15002      0.258808      0.148759      0.0103458       300    
reco:opflash:OpFlashFinder                            0.000159118   0.00109066    0.00311646    0.00105723    0.000497251      300    
reco:opslicer:OpSlicer                                3.6724e-05     0.197958      0.844423      0.163336      0.174167        300    
reco:rns:RandomNumberSaver                            2.2273e-05    3.63843e-05   0.000299275   3.49425e-05   1.63606e-05      300    
reco:anglereconue:NuAngularReco                       0.000159078   0.00109992     0.0370931    0.00034496     0.0033108       300    
reco:anglereconumu:NuAngularReco                      5.8426e-05    0.000896209    0.036089     0.000139689   0.00323522       300    
reco:anglereconuepfps:NuAngularReco                   0.000101661    0.0010023     0.0361802    0.000241922   0.00326408       300    
reco:anglereconumupfps:NuAngularReco                  7.7687e-05    0.000952176    0.036273     0.000183371   0.00327655       300    
reco:anglerecohits:NuAngularReco                      7.1256e-05    0.00144293     0.0394134    0.000532922    0.0037177       300    
[art]:TriggerResults:TriggerResultInserter             9.244e-06    1.30172e-05   0.000116134   1.16025e-05   7.38446e-06      300    
end_path:cafmaker:CAFMaker                            0.00125892     0.0140678     0.425216     0.00342579     0.0413343       300    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3814.99 MB
  Peak resident set size usage (VmHWM): 2149.11 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1894!
Begin processing the 293rd record. run: 6207207 subRun: 1 event: 53793 at 18-Sep-2025 01:39:34 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 430!
Begin processing the 294th record. run: 6207207 subRun: 1 event: 53794 at 18-Sep-2025 01:39:36 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 686!
Begin processing the 295th record. run: 6207207 subRun: 1 event: 53795 at 18-Sep-2025 01:39:38 CEST
Boundary wire vector sizes: 122, 100, 88
minwire 0: 2094
minwire 1: 460
minwire 2: 2418
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000868517, 0.992537, 0.00659436, 
Output 1: 0.994268, 0.00547521, 0.00013241, 0.000124486, 
Output 2: 0.998371, 0.00156486, 4.40032e-05, 1.99793e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 884!
Begin processing the 296th record. run: 6207207 subRun: 1 event: 53796 at 18-Sep-2025 01:39:40 CEST
Boundary wire vector sizes: 36, 37, 34
minwire 0: 1227
minwire 1: 1153
minwire 2: 1198
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 200, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0166866, 0.0035984, 0.979715, 
Output 1: 0.00559614, 0.0717194, 0.880278, 0.042406, 
Output 2: 0.968057, 0.030568, 0.00134875, 2.6302e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1574!
Begin processing the 297th record. run: 6207207 subRun: 1 event: 53797 at 18-Sep-2025 01:39:43 CEST
Boundary wire vector sizes: 152, 77, 89
minwire 0: 1143
minwire 1: 1209
minwire 2: 1155
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0086796, 0.00352954, 0.987791, 
Output 1: 0.0793407, 0.735943, 0.170508, 0.0142083, 
Output 2: 0.986811, 0.0125526, 0.000611317, 2.51782e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1721!
Begin processing the 298th record. run: 6207207 subRun: 1 event: 53798 at 18-Sep-2025 01:39:45 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 329!
Begin processing the 299th record. run: 6207207 subRun: 1 event: 53799 at 18-Sep-2025 01:39:47 CEST
Boundary wire vector sizes: 3976, 3170, 2997
minwire 0: 717
minwire 1: 808
minwire 2: 765
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0115923, 0.884264, 0.104144, 
Output 1: 0.075594, 0.214382, 0.274298, 0.435726, 
Output 2: 0.109047, 0.280015, 0.334653, 0.276285, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2810!
Begin processing the 300th record. run: 6207207 subRun: 1 event: 53800 at 18-Sep-2025 01:39:57 CEST
Boundary wire vector sizes: 161, 225, 128
minwire 0: 592
minwire 1: 2397
minwire 2: 254
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00993033, 0.0612916, 0.928778, 
Output 1: 0.793423, 0.198145, 0.00804686, 0.000384959, 
Output 2: 0.986887, 0.0128366, 0.000263218, 1.31686e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1307!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/0e/28/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_517_20231202T114026Z_gen_g4_detsim_hitreco__20240507T215951Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/93/27/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207207_268_20231120T133335Z_gen_g4_detsim_hitreco__20240503T061930Z_reco2.root
.:
total 13652
-rw-r--r--. 1 dune003 dune 5182055 Sep 18 01:39 flatcaf.root
-rw-r--r--. 1 dune003 dune 4222622 Sep 18 01:40 caf.root
-rw-r--r--. 1 dune003 dune 4222622 Sep 18 01:40 caf_fd_hd_atmo_2501_20250917T232058Z.root
-rw-r--r--. 1 dune003 dune  188193 Sep 18 01:40 caf_20250917T232058Z.log
-rw-r--r--. 1 dune003 dune   71281 Sep 18 01:40 jobscript.log
-rw-r--r--. 1 dune003 dune     372 Sep 18 01:40 caf_20250917T232058Z.file
-rw-r--r--. 1 dune003 dune     372 Sep 18 01:40 caf_20250917T232058Z.pfns
-rw-r--r--. 1 dune003 dune     372 Sep 18 01:20 file.list
-rw-r--r--. 1 dune003 dune     372 Sep 18 01:40 justin-processed-pfns.txt
-rw-r--r--. 1 dune003 dune     274 Sep 18 01:20 all-input-dids.txt
-rw-r--r--. 1 dune003 dune     274 Sep 18 01:40 caf_20250917T232058Z.did
-rw-r--r--. 1 dune003 dune     274 Sep 18 01:20 did.list
-rw-r--r--. 1 dune003 dune       0 Sep 18 01:21 debugprod.log
justIN time: 2025-09-18 19:36:59 UTC       justIN version: 01.05.00