Jobsub ID 48328.7@dunegpschedd01.fnal.gov
| Jobsub ID | 48328.7@dunegpschedd01.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-18 10:34:20 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce01 | |
| Last heartbeat | 2025-09-18 11:53:57 | |
| From worker node | Hostname | wn1207081.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Started | 2025-09-18 10:35:01 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_231_20231207T052258Z_gen_g4_detsim_hitreco__20240510T063059Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_139_20231202T170748Z_gen_g4_detsim_hitreco__20240507T202945Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_45_20231205T105626Z_gen_g4_detsim_hitreco__20240510T062921Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_224_20231202T075625Z_gen_g4_detsim_hitreco__20240507T205015Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 1h (4703s) | |
| CPU time | 1h (4174s = 88%) | |
| Max RSS bytes | 2122117120 (2023 MiB) | |
| Outputting started | 2025-09-18 11:53:25 | |
| Output files | ||
| Finished | 2025-09-18 11:53:57 | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
153108 0.0170991 1.18732 0.00253065 0.0747586 400 reco:pandoraShower:LArPandoraModularShowerCreation 0.000208332 0.0222779 1.11445 0.0050322 0.0700289 400 reco:pandoracalo:Calorimetry 0.000125175 0.0546671 2.77494 0.00168254 0.250364 400 reco:pandorapid:Chi2ParticleID 4.746e-05 0.000260205 0.0029313 0.000193574 0.000273922 400 reco:cvnmap:CVNMapper 2.7692e-05 0.0124588 0.104835 0.0144449 0.0101775 400 reco:cvneva:CVNEvaluator 2.0629e-05 0.502186 3.42299 0.667977 0.341641 400 reco:energyrecnumu:EnergyReco 0.000427854 0.132885 5.08189 0.0316561 0.462956 400 reco:energyrecnue:EnergyReco 0.000193784 0.00570262 0.346973 0.000986443 0.0232582 400 reco:energyrecnc:EnergyReco 0.00304488 0.106902 0.57816 0.104815 0.0830886 400 reco:energyrecnumurange:EnergyReco 0.000331443 0.0843552 5.27317 0.00449473 0.405011 400 reco:energyrecnumumcs:EnergyReco 0.000914058 0.0814683 5.03268 0.00530717 0.380142 400 reco:energyrecnumumcsllhd:EnergyReco 0.00073393 0.0799176 5.09725 0.00527741 0.372849 400 reco:opdec:Deconvolution 0.047484 0.326312 0.68464 0.316022 0.124838 400 reco:ophitspe:OpHitFinderDeco 0.12531 0.168174 0.280321 0.140515 0.050811 400 reco:opflash:OpFlashFinder 0.000240502 0.0021668 0.00620608 0.00203052 0.00102451 400 reco:opslicer:OpSlicer 6.0744e-05 0.41521 1.97198 0.301845 0.363307 400 reco:rns:RandomNumberSaver 2.7282e-05 4.63809e-05 0.000555814 4.2009e-05 2.84357e-05 400 reco:anglereconue:NuAngularReco 0.000237596 0.00378832 0.2392 0.000551747 0.016294 400 reco:anglereconumu:NuAngularReco 0.000108043 0.00360316 0.250203 0.000335696 0.016656 400 reco:anglereconuepfps:NuAngularReco 0.000195136 0.00375231 0.245855 0.000493102 0.0165177 400 reco:anglereconumupfps:NuAngularReco 0.000176422 0.00371741 0.237049 0.000455426 0.0162577 400 reco:anglerecohits:NuAngularReco 0.00013869 0.00481487 0.250475 0.0011359 0.0175005 400 [art]:TriggerResults:TriggerResultInserter 1.2854e-05 2.08537e-05 0.000108714 1.9015e-05 8.42765e-06 400 end_path:cafmaker:CAFMaker 0.00224298 0.0463693 2.685 0.0068444 0.189036 400 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3804.52 MB Peak resident set size usage (VmHWM): 2122.12 MB ==================================================================================================== lar exit code 0 === Start last 100 lines of lar log file === Begin processing the 392nd record. run: 6400837 subRun: 1 event: 22492 at 18-Sep-2025 12:51:55 BST Running Ophitfinder with InputDigiType = 'recob' Found hits: 957! Begin processing the 393rd record. run: 6400837 subRun: 1 event: 22493 at 18-Sep-2025 12:51:59 BST Boundary wire vector sizes: 2327, 2617, 1520 minwire 0: 1427 minwire 1: 877 minwire 2: 1513 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0109368, 0.977877, 0.011186, Output 1: 0.279344, 0.677737, 0.0394634, 0.00345518, Output 2: 0.219138, 0.716626, 0.0619673, 0.00226934, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3128! Begin processing the 394th record. run: 6400837 subRun: 1 event: 22494 at 18-Sep-2025 12:52:09 BST Boundary wire vector sizes: 369, 404, 263 minwire 0: 297 minwire 1: 2424 minwire 2: 69 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0633652, 0.148919, 0.787716, Output 1: 0.0301962, 0.0790121, 0.140722, 0.75007, Output 2: 0.687175, 0.268143, 0.037572, 0.00710918, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1558! Begin processing the 395th record. run: 6400837 subRun: 1 event: 22495 at 18-Sep-2025 12:52:13 BST Running Ophitfinder with InputDigiType = 'recob' Found hits: 982! Begin processing the 396th record. run: 6400837 subRun: 1 event: 22496 at 18-Sep-2025 12:52:16 BST Running Ophitfinder with InputDigiType = 'recob' Found hits: 774! Begin processing the 397th record. run: 6400837 subRun: 1 event: 22497 at 18-Sep-2025 12:52:19 BST Boundary wire vector sizes: 3238, 3231, 1751 minwire 0: 1232 minwire 1: 637 minwire 2: 1265 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0147875, 0.92413, 0.0610828, Output 1: 0.534925, 0.418428, 0.0353311, 0.0113164, Output 2: 0.0341202, 0.169518, 0.390081, 0.406281, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3242! Begin processing the 398th record. run: 6400837 subRun: 1 event: 22498 at 18-Sep-2025 12:52:34 BST Running Ophitfinder with InputDigiType = 'recob' Found hits: 862! Begin processing the 399th record. run: 6400837 subRun: 1 event: 22499 at 18-Sep-2025 12:52:38 BST Boundary wire vector sizes: 6092, 6006, 3822 minwire 0: 464 minwire 1: 1773 minwire 2: 260 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0124996, 0.977689, 0.0098118, Output 1: 0.355862, 0.442018, 0.104385, 0.0977359, Output 2: 0.0410467, 0.142857, 0.361393, 0.454703, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3422! Begin processing the 400th record. run: 6400837 subRun: 1 event: 22500 at 18-Sep-2025 12:53:18 BST Boundary wire vector sizes: 33, 40, 31 minwire 0: 1320 minwire 1: 1255 minwire 2: 1366 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00538463, 0.00199413, 0.992621, Output 1: 0.307103, 0.677762, 0.0146636, 0.000472203, Output 2: 0.998479, 0.00144573, 7.18164e-05, 3.5085e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1108! Art has completed and will exit with status 0. === End last 100 lines of lar log file === processed files root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/88/d8/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_231_20231207T052258Z_gen_g4_detsim_hitreco__20240510T063059Z_reco2.root root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/e9/67/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_139_20231202T170748Z_gen_g4_detsim_hitreco__20240507T202945Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a8/0e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_45_20231205T105626Z_gen_g4_detsim_hitreco__20240510T062921Z_reco2.root root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/c3/b1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_224_20231202T075625Z_gen_g4_detsim_hitreco__20240507T205015Z_reco2.root .: total 19912 -rw-r--r-- 1 dune01 dune01 7538387 Sep 18 12:53 flatcaf.root -rw-r--r-- 1 dune01 dune01 6226238 Sep 18 12:53 caf.root -rw-r--r-- 1 dune01 dune01 6226238 Sep 18 12:53 caf_fd_hd_atmo_2501_20250918T103514Z.root -rw-r--r-- 1 dune01 dune01 251759 Sep 18 12:53 caf_20250918T103514Z.log -rw-r--r-- 1 dune01 dune01 93444 Sep 18 12:53 jobscript.log -rw-r--r-- 1 dune01 dune01 836 Sep 18 12:53 caf_20250918T103514Z.file -rw-r--r-- 1 dune01 dune01 836 Sep 18 12:53 caf_20250918T103514Z.pfns -rw-r--r-- 1 dune01 dune01 836 Sep 18 11:35 file.list -rw-r--r-- 1 dune01 dune01 836 Sep 18 12:53 justin-processed-pfns.txt -rw-r--r-- 1 dune01 dune01 550 Sep 18 11:35 all-input-dids.txt -rw-r--r-- 1 dune01 dune01 550 Sep 18 12:53 caf_20250918T103514Z.did -rw-r--r-- 1 dune01 dune01 550 Sep 18 11:35 did.list -rw-r--r-- 1 dune01 dune01 0 Sep 18 11:35 debugprod.log