Jobsub ID 48448.3@dunegpschedd01.fnal.gov
| Jobsub ID | 48448.3@dunegpschedd01.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-18 11:56:24 | |
| Site | ES_PIC | |
| Entry | DUNE_T1_ES_PIC_ce15-multicore | |
| Last heartbeat | 2025-09-18 12:47:53 | |
| From worker node | Hostname | hnode64.pic.es |
| cpuinfo | AMD EPYC 7402P 24-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 216000 (60 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Started | 2025-09-18 11:57:03 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_231_20231207T052258Z_gen_g4_detsim_hitreco__20240510T063059Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_139_20231202T170748Z_gen_g4_detsim_hitreco__20240507T202945Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_45_20231205T105626Z_gen_g4_detsim_hitreco__20240510T062921Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_224_20231202T075625Z_gen_g4_detsim_hitreco__20240507T205015Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 50m (3019s) | |
| CPU time | 37m (2263s = 74%) | |
| Max RSS bytes | 2142040064 (2042 MiB) | |
| Outputting started | 2025-09-18 12:47:23 | |
| Output files | ||
| Finished | 2025-09-18 12:47:53 | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
oraTrackCreation 0.000106313 0.00909582 0.382201 0.00141565 0.0347999 400 reco:pandoraShower:LArPandoraModularShowerCreation 0.000148324 0.0121039 0.714692 0.00278505 0.042322 400 reco:pandoracalo:Calorimetry 8.5602e-05 0.0304034 1.56538 0.000920568 0.141841 400 reco:pandorapid:Chi2ParticleID 3.2271e-05 0.000150893 0.00186452 0.000105938 0.000170123 400 reco:cvnmap:CVNMapper 1.586e-05 0.0100291 0.0496132 0.0111712 0.00783017 400 reco:cvneva:CVNEvaluator 1.46e-05 0.286313 2.93095 0.337899 0.224162 400 reco:energyrecnumu:EnergyReco 0.000223646 0.10722 3.12848 0.0182497 0.245817 400 reco:energyrecnue:EnergyReco 0.000121093 0.00320501 0.140902 0.000564902 0.0117148 400 reco:energyrecnc:EnergyReco 0.00179116 0.155626 0.538259 0.175732 0.110265 400 reco:energyrecnumurange:EnergyReco 0.000183456 0.0477 2.03215 0.00288089 0.213188 400 reco:energyrecnumumcs:EnergyReco 0.000581877 0.0497595 2.63127 0.00337508 0.228184 400 reco:energyrecnumumcsllhd:EnergyReco 0.000411172 0.046971 2.6535 0.00336999 0.209579 400 reco:opdec:Deconvolution 0.124553 0.330492 0.617693 0.335488 0.0914633 400 reco:ophitspe:OpHitFinderDeco 0.36061 0.407554 0.456084 0.40747 0.0388546 400 reco:opflash:OpFlashFinder 0.000146764 0.00136878 0.00524211 0.00129654 0.000633575 400 reco:opslicer:OpSlicer 3.2881e-05 0.247098 1.15789 0.19312 0.213814 400 reco:rns:RandomNumberSaver 1.8461e-05 3.24163e-05 0.000450244 3.0416e-05 2.21521e-05 400 reco:anglereconue:NuAngularReco 0.000146824 0.00228136 0.166824 0.000368651 0.0105286 400 reco:anglereconumu:NuAngularReco 6.2402e-05 0.00206368 0.154302 0.000187006 0.00993981 400 reco:anglereconuepfps:NuAngularReco 0.000103633 0.00218641 0.158502 0.000293498 0.0101843 400 reco:anglereconumupfps:NuAngularReco 8.9883e-05 0.00216911 0.166624 0.000252438 0.0105438 400 reco:anglerecohits:NuAngularReco 7.5952e-05 0.00276727 0.158885 0.00065358 0.0105191 400 [art]:TriggerResults:TriggerResultInserter 8.72e-06 1.35392e-05 7.4992e-05 1.26405e-05 5.33935e-06 400 end_path:cafmaker:CAFMaker 0.00143624 0.0267464 1.46014 0.00445485 0.10698 400 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3804.52 MB Peak resident set size usage (VmHWM): 2142.04 MB ==================================================================================================== lar exit code 0 === Start last 100 lines of lar log file === Begin processing the 392nd record. run: 6400837 subRun: 1 event: 22492 at 18-Sep-2025 14:46:25 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 957! Begin processing the 393rd record. run: 6400837 subRun: 1 event: 22493 at 18-Sep-2025 14:46:28 CEST Boundary wire vector sizes: 2327, 2617, 1520 minwire 0: 1427 minwire 1: 877 minwire 2: 1513 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0109368, 0.977877, 0.011186, Output 1: 0.279344, 0.677737, 0.0394634, 0.00345518, Output 2: 0.219138, 0.716626, 0.0619673, 0.00226934, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3128! Begin processing the 394th record. run: 6400837 subRun: 1 event: 22494 at 18-Sep-2025 14:46:35 CEST Boundary wire vector sizes: 369, 404, 263 minwire 0: 297 minwire 1: 2424 minwire 2: 69 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0633652, 0.148919, 0.787716, Output 1: 0.0301962, 0.0790121, 0.140722, 0.75007, Output 2: 0.687175, 0.268143, 0.037572, 0.00710918, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1558! Begin processing the 395th record. run: 6400837 subRun: 1 event: 22495 at 18-Sep-2025 14:46:39 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 982! Begin processing the 396th record. run: 6400837 subRun: 1 event: 22496 at 18-Sep-2025 14:46:42 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 774! Begin processing the 397th record. run: 6400837 subRun: 1 event: 22497 at 18-Sep-2025 14:46:46 CEST Boundary wire vector sizes: 3238, 3231, 1751 minwire 0: 1232 minwire 1: 637 minwire 2: 1265 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0147875, 0.92413, 0.0610828, Output 1: 0.534925, 0.418428, 0.0353311, 0.0113164, Output 2: 0.0341202, 0.169518, 0.390081, 0.406281, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3242! Begin processing the 398th record. run: 6400837 subRun: 1 event: 22498 at 18-Sep-2025 14:46:55 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 862! Begin processing the 399th record. run: 6400837 subRun: 1 event: 22499 at 18-Sep-2025 14:46:58 CEST Boundary wire vector sizes: 6092, 6006, 3822 minwire 0: 464 minwire 1: 1773 minwire 2: 260 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0124996, 0.977689, 0.0098118, Output 1: 0.355862, 0.442018, 0.104385, 0.0977359, Output 2: 0.0410467, 0.142857, 0.361393, 0.454703, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3422! Begin processing the 400th record. run: 6400837 subRun: 1 event: 22500 at 18-Sep-2025 14:47:18 CEST Boundary wire vector sizes: 33, 40, 31 minwire 0: 1320 minwire 1: 1255 minwire 2: 1366 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00538463, 0.00199413, 0.992621, Output 1: 0.307103, 0.677762, 0.0146636, 0.000472203, Output 2: 0.998479, 0.00144573, 7.18164e-05, 3.5085e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1108! Art has completed and will exit with status 0. === End last 100 lines of lar log file === processed files root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/88/d8/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_231_20231207T052258Z_gen_g4_detsim_hitreco__20240510T063059Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/e9/67/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_139_20231202T170748Z_gen_g4_detsim_hitreco__20240507T202945Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a8/0e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_45_20231205T105626Z_gen_g4_detsim_hitreco__20240510T062921Z_reco2.root root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/c3/b1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_224_20231202T075625Z_gen_g4_detsim_hitreco__20240507T205015Z_reco2.root .: total 19908 -rw-r--r-- 1 dune001 dune 7538387 Sep 18 14:47 flatcaf.root -rw-r--r-- 1 dune001 dune 6226505 Sep 18 14:47 caf.root -rw-r--r-- 1 dune001 dune 6226505 Sep 18 14:47 caf_fd_hd_atmo_2501_20250918T115714Z.root -rw-r--r-- 1 dune001 dune 252159 Sep 18 14:47 caf_20250918T115714Z.log -rw-r--r-- 1 dune001 dune 93618 Sep 18 14:47 jobscript.log -rw-r--r-- 1 dune001 dune 806 Sep 18 14:47 caf_20250918T115714Z.file -rw-r--r-- 1 dune001 dune 806 Sep 18 14:47 caf_20250918T115714Z.pfns -rw-r--r-- 1 dune001 dune 806 Sep 18 13:57 file.list -rw-r--r-- 1 dune001 dune 806 Sep 18 14:47 justin-processed-pfns.txt -rw-r--r-- 1 dune001 dune 550 Sep 18 13:57 all-input-dids.txt -rw-r--r-- 1 dune001 dune 550 Sep 18 14:47 caf_20250918T115714Z.did -rw-r--r-- 1 dune001 dune 550 Sep 18 13:57 did.list -rw-r--r-- 1 dune001 dune 0 Sep 18 13:57 debugprod.log