justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 47837.20@dunegpschedd01.fnal.gov

Jobsub ID47837.20@dunegpschedd01.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-17 22:59:44
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-09-17 23:11:29
From worker nodeHostnamehnode61.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-17 23:00:29
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400858_404_20231202T080315Z_gen_g4_detsim_hitreco__20240507T210131Z_reco2.root
JobscriptExit code0
Real time10m (633s)
CPU time6m (419s = 66%)
Max RSS bytes2136588288 (2037 MiB)
Outputting started2025-09-17 23:11:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T230038Z.root
Finished2025-09-17 23:11:29
Saved logsjustin-logs:47837.20-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

===============================================================================================================================
Full event                                              1.21529       4.99121       116.508       3.17935       11.8844        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0516959     0.091904      0.174756      0.0858799     0.0290358       100    
reco:pandora:StandardPandora                           0.316125       3.1518        106.716       1.59822       10.8582        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000113904   0.00482997     0.0982817    0.000937084    0.0157072       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000155765   0.00646292     0.0867432    0.00195245     0.0127647       100    
reco:pandoracalo:Calorimetry                          8.2422e-05     0.0173599     0.644323     0.000681891    0.0820386       100    
reco:pandorapid:Chi2ParticleID                        3.1361e-05    0.000132349   0.000767304   9.31375e-05   0.000122822      100    
reco:cvnmap:CVNMapper                                 1.8711e-05    0.00835469     0.0410305    0.00978196    0.00727743       100    
reco:cvneva:CVNEvaluator                               1.487e-05     0.267332       1.84286      0.307359      0.246341        100    
reco:energyrecnumu:EnergyReco                         0.000209507    0.109698       1.14889     0.00959274     0.181384        100    
reco:energyrecnue:EnergyReco                          0.000113474   0.00196467     0.0422961    0.000429428   0.00617307       100    
reco:energyrecnc:EnergyReco                           0.00184337     0.166346      0.484752      0.214643      0.119005        100    
reco:energyrecnumurange:EnergyReco                    0.000208536    0.0489509      1.7135      0.00249279     0.224819        100    
reco:energyrecnumumcs:EnergyReco                      0.000546927    0.0464968      1.42142      0.0028644      0.20394        100    
reco:energyrecnumumcsllhd:EnergyReco                  0.000429254    0.0440116      1.33795      0.0028741     0.188532        100    
reco:opdec:Deconvolution                                0.15389      0.338145      0.633857      0.346612      0.0907553       100    
reco:ophitspe:OpHitFinderDeco                          0.458198      0.464539      0.474201      0.464215     0.00282947       100    
reco:opflash:OpFlashFinder                            0.000141564   0.00107969    0.00283485    0.00104846    0.000535146      100    
reco:opslicer:OpSlicer                                3.9851e-05     0.194218      0.858139      0.147956      0.176112        100    
reco:rns:RandomNumberSaver                            1.9151e-05    3.12447e-05   0.000301359   2.7906e-05    2.76899e-05      100    
reco:anglereconue:NuAngularReco                       0.000143904   0.00147046     0.0368634    0.000321495   0.00518855       100    
reco:anglereconumu:NuAngularReco                      6.2622e-05    0.00133018     0.0368669    0.000153229   0.00528139       100    
reco:anglereconuepfps:NuAngularReco                   0.000100523   0.00145758     0.0370075    0.000248108    0.0053538       100    
reco:anglereconumupfps:NuAngularReco                  8.6723e-05    0.00139581     0.0367408    0.000208472   0.00533037       100    
reco:anglerecohits:NuAngularReco                      0.000101344   0.00198631     0.0409421    0.000567952    0.0061773       100    
[art]:TriggerResults:TriggerResultInserter             8.65e-06     1.19926e-05   5.0122e-05     1.139e-05    4.55494e-06      100    
end_path:cafmaker:CAFMaker                            0.00158345     0.0187164      0.40146     0.00375915     0.0637749       100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3802.29 MB
  Peak resident set size usage (VmHWM): 2136.59 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Classifier summary: 
Output 0: 0.00756715, 0.00359077, 0.988842, 
Output 1: 0.00900754, 0.972989, 0.017538, 0.000465569, 
Output 2: 0.995236, 0.00463749, 0.000123171, 2.98892e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1149!
Begin processing the 94th record. run: 6400858 subRun: 1 event: 40494 at 18-Sep-2025 01:08:33 CEST
Boundary wire vector sizes: 1388, 741, 1191
minwire 0: 984
minwire 1: 816
minwire 2: 978
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.997409, 0.000450252, 0.00214096, 
Output 1: 0.44787, 0.44841, 0.0780529, 0.0256667, 
Output 2: 0.882944, 0.107685, 0.00857294, 0.000798576, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2240!
Begin processing the 95th record. run: 6400858 subRun: 1 event: 40495 at 18-Sep-2025 01:08:41 CEST
Boundary wire vector sizes: 8831, 6948, 6059
minwire 0: 1211
minwire 1: 426
minwire 2: 1241
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0218483, 0.728417, 0.249735, 
Output 1: 0.265151, 0.347263, 0.184632, 0.202954, 
Output 2: 0.0439096, 0.118182, 0.272641, 0.565267, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3327!
Begin processing the 96th record. run: 6400858 subRun: 1 event: 40496 at 18-Sep-2025 01:10:40 CEST
Boundary wire vector sizes: 175, 260, 191
minwire 0: 636
minwire 1: 2386
minwire 2: 52
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0196187, 0.354881, 0.6255, 
Output 1: 0.318329, 0.616823, 0.0615424, 0.00330544, 
Output 2: 0.914892, 0.0811478, 0.00376967, 0.000191033, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1556!
Begin processing the 97th record. run: 6400858 subRun: 1 event: 40497 at 18-Sep-2025 01:10:45 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 463!
Begin processing the 98th record. run: 6400858 subRun: 1 event: 40498 at 18-Sep-2025 01:10:49 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 250!
Begin processing the 99th record. run: 6400858 subRun: 1 event: 40499 at 18-Sep-2025 01:10:52 CEST
Boundary wire vector sizes: 83, 178, 153
minwire 0: 1109
minwire 1: 1873
minwire 2: 852
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.98864, 0.000869905, 0.0104903, 
Output 1: 0.013808, 0.980578, 0.00547373, 0.000140602, 
Output 2: 0.9898, 0.00999634, 0.000199907, 4.05839e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2022!
Begin processing the 100th record. run: 6400858 subRun: 1 event: 40500 at 18-Sep-2025 01:10:57 CEST
Boundary wire vector sizes: 43, 38, 34
minwire 0: 1451
minwire 1: 1120
minwire 2: 1251
Used alternate method to get min and max wires due to vertex determination failure: 110, 409
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 461, 760
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 102, 401
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00312188, 0.00161178, 0.995266, 
Output 1: 0.819808, 0.17874, 0.00135196, 9.91699e-05, 
Output 2: 0.999069, 0.000898904, 3.0028e-05, 2.57418e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1629!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/af/b4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400858_404_20231202T080315Z_gen_g4_detsim_hitreco__20240507T210131Z_reco2.root
.:
total 4696
-rw-r--r-- 1 dune001 dune 1851322 Sep 18 01:11 flatcaf.root
-rw-r--r-- 1 dune001 dune 1402680 Sep 18 01:11 caf.root
-rw-r--r-- 1 dune001 dune 1402680 Sep 18 01:11 caf_fd_hd_atmo_2501_20250917T230038Z.root
-rw-r--r-- 1 dune001 dune   70948 Sep 18 01:11 caf_20250917T230038Z.log
-rw-r--r-- 1 dune001 dune   30516 Sep 18 01:11 jobscript.log
-rw-r--r-- 1 dune001 dune     186 Sep 18 01:11 caf_20250917T230038Z.file
-rw-r--r-- 1 dune001 dune     186 Sep 18 01:11 caf_20250917T230038Z.pfns
-rw-r--r-- 1 dune001 dune     186 Sep 18 01:00 file.list
-rw-r--r-- 1 dune001 dune     186 Sep 18 01:11 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune     137 Sep 18 01:00 all-input-dids.txt
-rw-r--r-- 1 dune001 dune     137 Sep 18 01:11 caf_20250917T230038Z.did
-rw-r--r-- 1 dune001 dune     137 Sep 18 01:00 did.list
-rw-r--r-- 1 dune001 dune       0 Sep 18 01:00 debugprod.log
justIN time: 2025-09-18 20:16:23 UTC       justIN version: 01.05.00