justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 47395.11@dunegpschedd01.fnal.gov

Jobsub ID47395.11@dunegpschedd01.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-17 07:12:53
SiteUS_UChicago
EntryEngage_US_MWT2_uct2_condce_mcore
Last heartbeat2025-09-17 07:30:24
From worker nodeHostnameiut2-c322.iu.edu
cpuinfoAMD EPYC 7402 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-17 07:13:19
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6491894_630_20231208T014623Z_gen_g4_detsim_hitreco__20240510T031631Z_reco2.root
JobscriptExit code0
Real time16m (1006s)
CPU time9m (596s = 59%)
Max RSS bytes2144894976 (2045 MiB)
Outputting started2025-09-17 07:30:06
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T071327Z.root
Finished2025-09-17 07:30:24
Saved logsjustin-logs:47395.11-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

RMS         nEvts   
========================================================================================================================================
Full event                                              3.44108       7.59353       136.785       4.66422       14.6406        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0921907     0.161957      0.254647      0.169222      0.0491718       100    
reco:pandora:StandardPandora                            1.6644        4.64877       130.875       2.09452       13.7267        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000137329   0.00907594     0.182214     0.00177313      0.0247         100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000232016    0.0142964     0.192446     0.00397128     0.0297908       100    
reco:pandoracalo:Calorimetry                          0.000107343    0.0324193     0.668108     0.00151817     0.112039        100    
reco:pandorapid:Chi2ParticleID                        3.3794e-05    0.000214531   0.00266971    0.000145154   0.000292413      100    
reco:cvnmap:CVNMapper                                 2.0909e-05     0.0159235     0.0519988     0.0135865     0.0132995       100    
reco:cvneva:CVNEvaluator                              1.6411e-05     0.364264       2.75896      0.437062      0.314022        100    
reco:energyrecnumu:EnergyReco                         0.00032961     0.178417       3.19583      0.0189873     0.360361        100    
reco:energyrecnue:EnergyReco                          0.000132269   0.00322822     0.0609313    0.000573642   0.00853626       100    
reco:energyrecnc:EnergyReco                           0.00237558     0.319441      0.880504      0.389504      0.227212        100    
reco:energyrecnumurange:EnergyReco                    0.000308301    0.0514167      2.76087     0.00288718     0.287499        100    
reco:energyrecnumumcs:EnergyReco                      0.000612503    0.0530211      2.73416     0.00351143     0.287681        100    
reco:energyrecnumumcsllhd:EnergyReco                  0.000475485    0.0554545      2.97707     0.00367671     0.309838        100    
reco:opdec:Deconvolution                               0.310327      0.515352      0.864018      0.526443      0.100267        100    
reco:ophitspe:OpHitFinderDeco                          0.821645      0.827895      0.922373      0.826393     0.00991633       100    
reco:opflash:OpFlashFinder                            0.000361451    0.0015382    0.00541155    0.00142163    0.000731059      100    
reco:opslicer:OpSlicer                                0.00201599     0.283471       1.07168      0.209387      0.247776        100    
reco:rns:RandomNumberSaver                             3.172e-05    5.67142e-05   0.000540758   5.2169e-05    4.97005e-05      100    
reco:anglereconue:NuAngularReco                       0.000203242   0.00225582     0.0474017    0.000457367   0.00613558       100    
reco:anglereconumu:NuAngularReco                      8.5471e-05    0.00193747     0.0456217    0.000248384   0.00604868       100    
reco:anglereconuepfps:NuAngularReco                   0.000116078   0.00214512     0.0453946    0.000344218   0.00613343       100    
reco:anglereconumupfps:NuAngularReco                  0.000102442   0.00201438     0.0456884    0.000305647   0.00606982       100    
reco:anglerecohits:NuAngularReco                      0.000410284   0.00293953     0.0512571    0.000809336   0.00698932       100    
[art]:TriggerResults:TriggerResultInserter             9.548e-06    1.68321e-05   0.000101171   1.4888e-05    9.39716e-06      100    
end_path:cafmaker:CAFMaker                            0.00217064     0.0442589      1.49317     0.00603424     0.167811        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3807.97 MB
  Peak resident set size usage (VmHWM): 2144.89 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.614425, 0.363697, 0.0218779, 
Output 1: 0.232707, 0.465072, 0.189518, 0.112703, 
Output 2: 0.0433045, 0.133123, 0.314213, 0.509359, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3463!
Begin processing the 94th record. run: 6491894 subRun: 1 event: 63094 at 17-Sep-2025 03:29:11 EDT

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 524!
Begin processing the 95th record. run: 6491894 subRun: 1 event: 63095 at 17-Sep-2025 03:29:16 EDT
Boundary wire vector sizes: 4034, 4124, 2642
minwire 0: 1101
minwire 1: 983
minwire 2: 781
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.424099, 0.547992, 0.0279083, 
Output 1: 0.17707, 0.408684, 0.234697, 0.179548, 
Output 2: 0.0219185, 0.0735125, 0.254367, 0.650202, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3234!
Begin processing the 96th record. run: 6491894 subRun: 1 event: 63096 at 17-Sep-2025 03:29:33 EDT
Boundary wire vector sizes: 227, 237, 175
minwire 0: 2083
minwire 1: 642
minwire 2: 2108
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00211531, 0.989382, 0.00850279, 
Output 1: 0.0234769, 0.186675, 0.439502, 0.350347, 
Output 2: 0.809987, 0.169457, 0.0189278, 0.00162864, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1652!
Begin processing the 97th record. run: 6491894 subRun: 1 event: 63097 at 17-Sep-2025 03:29:39 EDT
Boundary wire vector sizes: 553, 488, 385
minwire 0: 1681
minwire 1: 1133
minwire 2: 1541
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.709266, 0.0123447, 0.278389, 
Output 1: 0.335489, 0.436742, 0.183448, 0.0443202, 
Output 2: 0.157095, 0.453054, 0.308548, 0.0813023, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2690!
Begin processing the 98th record. run: 6491894 subRun: 1 event: 63098 at 17-Sep-2025 03:29:46 EDT
Boundary wire vector sizes: 65, 47, 46
minwire 0: 1369
minwire 1: 1083
minwire 2: 1486
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.10982, 0.0135743, 0.876606, 
Output 1: 0.0757352, 0.304517, 0.4542, 0.165548, 
Output 2: 0.584043, 0.404241, 0.0114762, 0.000240335, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1352!
Begin processing the 99th record. run: 6491894 subRun: 1 event: 63099 at 17-Sep-2025 03:29:52 EDT
Boundary wire vector sizes: 1867, 1160, 947
minwire 0: 1216
minwire 1: 1022
minwire 2: 1033
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.938694, 0.0510704, 0.0102353, 
Output 1: 0.303495, 0.54004, 0.114412, 0.0420532, 
Output 2: 0.0301531, 0.111228, 0.32992, 0.528699, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3134!
Begin processing the 100th record. run: 6491894 subRun: 1 event: 63100 at 17-Sep-2025 03:30:00 EDT

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 461!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/e8/de/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6491894_630_20231208T014623Z_gen_g4_detsim_hitreco__20240510T031631Z_reco2.root
.:
total 5248
-rw-r--r--. 1 dune osgvo 2062701 Sep 17 03:30 flatcaf.root
-rw-r--r--. 1 dune osgvo 1576021 Sep 17 03:30 caf.root
-rw-r--r--. 1 dune osgvo 1576021 Sep 17 03:30 caf_fd_hd_atmo_2501_20250917T071327Z.root
-rw-r--r--. 1 dune osgvo   73418 Sep 17 03:30 caf_20250917T071327Z.log
-rw-r--r--. 1 dune osgvo   40355 Sep 17 03:30 jobscript.log
-rw-r--r--. 1 dune osgvo     218 Sep 17 03:30 caf_20250917T071327Z.file
-rw-r--r--. 1 dune osgvo     218 Sep 17 03:30 caf_20250917T071327Z.pfns
-rw-r--r--. 1 dune osgvo     218 Sep 17 03:13 file.list
-rw-r--r--. 1 dune osgvo     218 Sep 17 03:30 justin-processed-pfns.txt
-rw-r--r--. 1 dune osgvo     137 Sep 17 03:13 all-input-dids.txt
-rw-r--r--. 1 dune osgvo     137 Sep 17 03:30 caf_20250917T071327Z.did
-rw-r--r--. 1 dune osgvo     137 Sep 17 03:13 did.list
-rw-r--r--. 1 dune osgvo       0 Sep 17 03:13 debugprod.log
justIN time: 2025-09-18 16:07:08 UTC       justIN version: 01.05.00