Jobsub ID 44203.2@dunegpschedd02.fnal.gov
| Jobsub ID | 44203.2@dunegpschedd02.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-20 11:40:57 | |
| Site | NL_NIKHEF | |
| Entry | VIRGO_NL_NIKHEF_juk | |
| Last heartbeat | 2025-09-20 12:05:21 | |
| From worker node | Hostname | wn-lot-026.farm.nikhef.nl |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Started | 2025-09-20 11:42:52 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517726_159_20231204T100452Z_gen_g4_detsim_hitreco__20240509T213840Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_866_20231204T075513Z_gen_g4_detsim_hitreco__20240509T220142Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_750_20231204T223958Z_gen_g4_detsim_hitreco__20240510T032742Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_628_20231203T130738Z_gen_g4_detsim_hitreco__20240508T052211Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 22m (1332s) | |
| CPU time | 19m (1175s = 88%) | |
| Max RSS bytes | 2053775360 (1958 MiB) | |
| Outputting started | 2025-09-20 12:05:05 | |
| Output files | ||
| Finished | 2025-09-20 12:05:21 | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
65.2051 1.27026 4.46538 400 reco:pandoraTrack:LArPandoraTrackCreation 0.000118793 0.00556827 0.169929 0.00127861 0.01483 400 reco:pandoraShower:LArPandoraModularShowerCreation 0.00016505 0.0101721 0.317958 0.00296777 0.0250135 400 reco:pandoracalo:Calorimetry 9.3626e-05 0.0143473 1.12377 0.000922936 0.0694335 400 reco:pandorapid:Chi2ParticleID 3.3894e-05 0.000128621 0.00192235 0.000103539 0.000129425 400 reco:cvnmap:CVNMapper 1.7503e-05 0.00726427 0.037463 0.00872264 0.00478763 400 reco:cvneva:CVNEvaluator 1.5649e-05 0.285632 1.874 0.372613 0.181354 400 reco:energyrecnumu:EnergyReco 0.000231424 0.046605 1.23676 0.0106032 0.114246 400 reco:energyrecnue:EnergyReco 0.000143609 0.00204939 0.0643921 0.000486417 0.00567116 400 reco:energyrecnc:EnergyReco 0.00225183 0.0464374 0.279665 0.0138259 0.0519117 400 reco:energyrecnumurange:EnergyReco 0.000196108 0.0266299 1.0485 0.00241673 0.099996 400 reco:energyrecnumumcs:EnergyReco 0.000641153 0.0273867 1.04605 0.00277768 0.0998197 400 reco:energyrecnumumcsllhd:EnergyReco 0.000369563 0.0273833 1.05943 0.00282553 0.100003 400 reco:opdec:Deconvolution 0.00790238 0.185007 0.400239 0.18379 0.072207 400 reco:ophitspe:OpHitFinderDeco 0.00235663 0.0705831 0.151295 0.073178 0.0635449 400 reco:opflash:OpFlashFinder 0.000143138 0.00109251 0.00493977 0.00105602 0.000514263 400 reco:opslicer:OpSlicer 3.6629e-05 0.215903 0.870779 0.184083 0.174288 400 reco:rns:RandomNumberSaver 1.633e-05 2.55494e-05 0.000316002 2.4546e-05 1.60905e-05 400 reco:anglereconue:NuAngularReco 0.000121838 0.00136824 0.0674129 0.000324143 0.00448393 400 reco:anglereconumu:NuAngularReco 7.3838e-05 0.00131763 0.0895074 0.000197236 0.00536529 400 reco:anglereconuepfps:NuAngularReco 0.000114826 0.00139334 0.091371 0.000283698 0.00544787 400 reco:anglereconumupfps:NuAngularReco 0.000105869 0.00132333 0.0731746 0.000257889 0.00472582 400 reco:anglerecohits:NuAngularReco 8.6282e-05 0.00176715 0.0470087 0.000582101 0.0041441 400 [art]:TriggerResults:TriggerResultInserter 9.427e-06 1.09989e-05 5.5254e-05 1.066e-05 2.5021e-06 400 end_path:cafmaker:CAFMaker 0.00117127 0.0161723 0.47721 0.00361402 0.0451709 400 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3719.36 MB Peak resident set size usage (VmHWM): 2053.78 MB ==================================================================================================== lar exit code 0 === Start last 100 lines of lar log file === minwire 0: 505 minwire 1: 2025 minwire 2: 295 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00271204, 0.00154724, 0.995741, Output 1: 0.111474, 0.883326, 0.00494254, 0.000257611, Output 2: 0.997962, 0.00192771, 0.000102825, 7.03701e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 946! Begin processing the 395th record. run: 50574832 subRun: 1 event: 62895 at 20-Sep-2025 14:04:37 CEST Boundary wire vector sizes: 175, 162, 136 minwire 0: 2233 minwire 1: 149 minwire 2: 2624 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00716898, 0.011866, 0.980965, Output 1: 0.047417, 0.939487, 0.0126423, 0.000453381, Output 2: 0.991418, 0.00827385, 0.00029709, 1.12605e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 884! Begin processing the 396th record. run: 50574832 subRun: 1 event: 62896 at 20-Sep-2025 14:04:40 CEST Boundary wire vector sizes: 251, 323, 237 minwire 0: 1469 minwire 1: 1597 minwire 2: 1171 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00267186, 0.00356573, 0.993762, Output 1: 0.973774, 0.024859, 0.00100935, 0.000357517, Output 2: 0.993832, 0.00601157, 0.000135027, 2.09613e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2366! Begin processing the 397th record. run: 50574832 subRun: 1 event: 62897 at 20-Sep-2025 14:04:43 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 855! Begin processing the 398th record. run: 50574832 subRun: 1 event: 62898 at 20-Sep-2025 14:04:45 CEST Boundary wire vector sizes: 197, 208, 193 minwire 0: 2104 minwire 1: 1321 minwire 2: 1784 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00586151, 0.582735, 0.411404, Output 1: 0.550514, 0.426566, 0.0221729, 0.000747408, Output 2: 0.980585, 0.0191134, 0.000291103, 1.10153e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1802! Begin processing the 399th record. run: 50574832 subRun: 1 event: 62899 at 20-Sep-2025 14:04:48 CEST Boundary wire vector sizes: 4596, 5448, 1804 minwire 0: 183 minwire 1: 1811 minwire 2: 19 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.000967536, 0.993384, 0.00564818, Output 1: 0.419944, 0.43002, 0.103414, 0.0466212, Output 2: 0.617664, 0.267723, 0.0892038, 0.0254098, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3055! Begin processing the 400th record. run: 50574832 subRun: 1 event: 62900 at 20-Sep-2025 14:05:01 CEST Boundary wire vector sizes: 78, 52, 47 minwire 0: 252 minwire 1: 2362 minwire 2: 67 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.423238, 0.152613, 0.424149, Output 1: 0.0244258, 0.259735, 0.498478, 0.217361, Output 2: 0.758392, 0.228473, 0.0126942, 0.000440607, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1014! Art has completed and will exit with status 0. === End last 100 lines of lar log file === processed files root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/d8/da/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517726_159_20231204T100452Z_gen_g4_detsim_hitreco__20240509T213840Z_reco2.root root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/32/c7/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_866_20231204T075513Z_gen_g4_detsim_hitreco__20240509T220142Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/06/25/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_750_20231204T223958Z_gen_g4_detsim_hitreco__20240510T032742Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/8b/cf/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_628_20231203T130738Z_gen_g4_detsim_hitreco__20240508T052211Z_reco2.root .: total 17940 -rw-r--r--. 1 dune012 dune 6795221 Sep 20 14:05 flatcaf.root -rw-r--r--. 1 dune012 dune 5586893 Sep 20 14:05 caf.root -rw-r--r--. 1 dune012 dune 5586893 Sep 20 14:05 caf_fd_hd_atmo_2501_20250920T114300Z.root -rw-r--r--. 1 dune012 dune 257294 Sep 20 14:05 caf_20250920T114300Z.log -rw-r--r--. 1 dune012 dune 98210 Sep 20 14:05 jobscript.log -rw-r--r--. 1 dune012 dune 808 Sep 20 14:05 caf_20250920T114300Z.file -rw-r--r--. 1 dune012 dune 808 Sep 20 14:05 caf_20250920T114300Z.pfns -rw-r--r--. 1 dune012 dune 808 Sep 20 13:42 file.list -rw-r--r--. 1 dune012 dune 808 Sep 20 14:05 justin-processed-pfns.txt -rw-r--r--. 1 dune012 dune 552 Sep 20 13:42 all-input-dids.txt -rw-r--r--. 1 dune012 dune 552 Sep 20 14:05 caf_20250920T114300Z.did -rw-r--r--. 1 dune012 dune 552 Sep 20 13:42 did.list -rw-r--r--. 1 dune012 dune 0 Sep 20 13:43 debugprod.log