justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 47113.8@dunegpschedd01.fnal.gov

Jobsub ID47113.8@dunegpschedd01.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-16 17:36:09
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_dissel
Last heartbeat2025-09-16 18:13:38
From worker nodeHostnamewn-sate-049.farm.nikhef.nl
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-16 17:36:56
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_344_20231207T040715Z_gen_g4_detsim_hitreco__20240510T054155Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_338_20231206T032949Z_gen_g4_detsim_hitreco__20240510T034610Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577091_1159_20231204T005716Z_gen_g4_detsim_hitreco__20240509T203306Z_reco2.root
JobscriptExit code0
Real time36m (2177s)
CPU time32m (1942s = 89%)
Max RSS bytes2041626624 (1947 MiB)
Outputting started2025-09-16 18:13:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250916T173706Z.root
Finished2025-09-16 18:13:38
Saved logsjustin-logs:47113.8-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

---------------------------------------------------------------
source:RootInput(read)                                0.00991797     0.0375323     0.179403      0.0290911     0.0305135       300    
reco:pandora:StandardPandora                           0.0681963      4.93525       373.171       2.01045       22.865         300    
reco:pandoraTrack:LArPandoraTrackCreation             0.00014366     0.0119786     0.537986     0.00181876     0.0449533       300    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000190778    0.0160142     0.264532     0.00364415     0.0387861       300    
reco:pandoracalo:Calorimetry                          0.000114494    0.0359259      2.00928     0.00131411     0.173577        300    
reco:pandorapid:Chi2ParticleID                        4.0075e-05    0.000191078   0.00289042    0.00014127    0.000218706      300    
reco:cvnmap:CVNMapper                                 2.5278e-05     0.0133007     0.0502487     0.016351      0.0100048       300    
reco:cvneva:CVNEvaluator                              1.9486e-05     0.458053       3.31191      0.638159      0.341609        300    
reco:energyrecnumu:EnergyReco                         0.000340709     0.11758       8.13971      0.0230167     0.596922        300    
reco:energyrecnue:EnergyReco                          0.000185879   0.00440172      0.17382     0.000756629    0.016028        300    
reco:energyrecnc:EnergyReco                           0.00288041     0.0731858     0.457033      0.0664693     0.0656253       300    
reco:energyrecnumurange:EnergyReco                    0.000334777    0.0759647      5.7825      0.00333278     0.446844        300    
reco:energyrecnumumcs:EnergyReco                      0.000650289    0.0775317      6.14934     0.00394529     0.463743        300    
reco:energyrecnumumcsllhd:EnergyReco                  0.000418344    0.0767149      6.15637     0.00404385     0.458793        300    
reco:opdec:Deconvolution                               0.0308766     0.225889      0.544434      0.220311      0.0877927       300    
reco:ophitspe:OpHitFinderDeco                          0.0811867     0.105026      0.160919      0.0900574     0.0241764       300    
reco:opflash:OpFlashFinder                            0.000149069   0.00145669    0.00517978    0.00139383    0.000658066      300    
reco:opslicer:OpSlicer                                4.2961e-05     0.245419      0.830965      0.191463      0.204527        300    
reco:rns:RandomNumberSaver                            2.0077e-05    3.95047e-05   0.000568336   3.6539e-05    3.14882e-05      300    
reco:anglereconue:NuAngularReco                       0.000172674   0.00254753     0.118097     0.00043999    0.00987148       300    
reco:anglereconumu:NuAngularReco                      9.2884e-05    0.00230401     0.106568     0.000238372   0.00929729       300    
reco:anglereconuepfps:NuAngularReco                   0.000136155   0.00246479      0.11903     0.000369307   0.00970352       300    
reco:anglereconumupfps:NuAngularReco                  0.00013346    0.00243583      0.10749     0.000322544   0.00948568       300    
reco:anglerecohits:NuAngularReco                      0.000102913   0.00316874     0.123379     0.000749871    0.0106328       300    
[art]:TriggerResults:TriggerResultInserter            1.1772e-05    1.72557e-05   8.8726e-05    1.67865e-05   5.45179e-06      300    
end_path:cafmaker:CAFMaker                            0.00136625      0.03339       1.2762      0.00513863     0.121098        300    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3700.91 MB
  Peak resident set size usage (VmHWM): 2041.63 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 0: 0.988527, 0.00272981, 0.00874274, 
Output 1: 0.0402101, 0.937314, 0.0218294, 0.000646329, 
Output 2: 0.943495, 0.0544426, 0.00201345, 4.91737e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2940!
Begin processing the 293rd record. run: 50577091 subRun: 1 event: 115993 at 16-Sep-2025 20:12:46 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 487!
Begin processing the 294th record. run: 50577091 subRun: 1 event: 115994 at 16-Sep-2025 20:12:49 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 636!
Begin processing the 295th record. run: 50577091 subRun: 1 event: 115995 at 16-Sep-2025 20:12:51 CEST
Boundary wire vector sizes: 518, 539, 213
minwire 0: 686
minwire 1: 2170
minwire 2: 231
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00036711, 0.998446, 0.00118658, 
Output 1: 0.0348928, 0.953479, 0.0109725, 0.00065606, 
Output 2: 0.997389, 0.00250319, 0.00010266, 5.10417e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2039!
Begin processing the 296th record. run: 50577091 subRun: 1 event: 115996 at 16-Sep-2025 20:12:56 CEST
Boundary wire vector sizes: 248, 162, 188
minwire 0: 2251
minwire 1: 1085
minwire 2: 2007
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000315222, 0.998367, 0.00131818, 
Output 1: 0.0046131, 0.99096, 0.00425972, 0.000167385, 
Output 2: 0.998332, 0.00161975, 4.67043e-05, 1.61303e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2062!
Begin processing the 297th record. run: 50577091 subRun: 1 event: 115997 at 16-Sep-2025 20:12:59 CEST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 572!
Begin processing the 298th record. run: 50577091 subRun: 1 event: 115998 at 16-Sep-2025 20:13:02 CEST
Boundary wire vector sizes: 268, 414, 243
minwire 0: 2751
minwire 1: 391
minwire 2: 2717
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00055931, 0.997781, 0.00165953, 
Output 1: 0.00530498, 0.978747, 0.0154453, 0.000502779, 
Output 2: 0.99796, 0.00196592, 7.21228e-05, 2.41306e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1398!
Begin processing the 299th record. run: 50577091 subRun: 1 event: 115999 at 16-Sep-2025 20:13:05 CEST
Boundary wire vector sizes: 124, 126, 129
minwire 0: 1414
minwire 1: 1410
minwire 2: 1350
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000380579, 0.998133, 0.00148612, 
Output 1: 0.0109286, 0.984831, 0.00412748, 0.000112565, 
Output 2: 0.998767, 0.00118999, 4.12347e-05, 1.3348e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1694!
Begin processing the 300th record. run: 50577091 subRun: 1 event: 116000 at 16-Sep-2025 20:13:09 CEST
Boundary wire vector sizes: 244, 271, 179
minwire 0: 593
minwire 1: 2121
minwire 2: 416
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0014873, 0.980424, 0.0180883, 
Output 1: 0.00622549, 0.146445, 0.820461, 0.0268681, 
Output 2: 0.985924, 0.0133488, 0.000706767, 2.05241e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1533!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/d1/71/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_344_20231207T040715Z_gen_g4_detsim_hitreco__20240510T054155Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/4d/36/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_338_20231206T032949Z_gen_g4_detsim_hitreco__20240510T034610Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/80/7c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577091_1159_20231204T005716Z_gen_g4_detsim_hitreco__20240509T203306Z_reco2.root
.:
total 15948
-rw-r--r--. 1 dune012 dune 6057720 Sep 16 20:13 flatcaf.root
-rw-r--r--. 1 dune012 dune 4979812 Sep 16 20:13 caf.root
-rw-r--r--. 1 dune012 dune 4979812 Sep 16 20:13 caf_fd_hd_atmo_2501_20250916T173706Z.root
-rw-r--r--. 1 dune012 dune  183759 Sep 16 20:13 caf_20250916T173706Z.log
-rw-r--r--. 1 dune012 dune   78618 Sep 16 20:13 jobscript.log
-rw-r--r--. 1 dune012 dune     589 Sep 16 20:13 caf_20250916T173706Z.file
-rw-r--r--. 1 dune012 dune     589 Sep 16 20:13 caf_20250916T173706Z.pfns
-rw-r--r--. 1 dune012 dune     589 Sep 16 19:37 file.list
-rw-r--r--. 1 dune012 dune     589 Sep 16 20:13 justin-processed-pfns.txt
-rw-r--r--. 1 dune012 dune     414 Sep 16 19:37 all-input-dids.txt
-rw-r--r--. 1 dune012 dune     414 Sep 16 20:13 caf_20250916T173706Z.did
-rw-r--r--. 1 dune012 dune     414 Sep 16 19:37 did.list
-rw-r--r--. 1 dune012 dune       0 Sep 16 19:37 debugprod.log
justIN time: 2025-09-19 04:26:31 UTC       justIN version: 01.05.00