justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279277.177@dunegpschedd01.fnal.gov

Jobsub ID279277.177@dunegpschedd01.fnal.gov
Workflow ID11972
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-15 19:38:14
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce515
Last heartbeat2026-01-15 20:46:44
From worker nodeHostnameb9p07p0819.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-15 19:40:05
Input filesfardet-hd:nu_dune10kt_1x2x6_1100_19_20230826T033411Z_gen_g4_detsim_hitreco__20240229T180234Z_reco2.root
JobscriptExit code0
Real time7m (442s)
CPU time4m (293s = 66%)
Max RSS bytes1227243520 (1170 MiB)
Outputting started2026-01-15 19:47:28
Output files
Finished2026-01-15 20:46:44
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

is /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/ad554f6b0d8af5cd51272760a301f448969f3ed8/localProducts_larsoft_v10_13_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/0e/e1/nu_dune10kt_1x2x6_1100_19_20230826T033411Z_gen_g4_detsim_hitreco__20240229T180234Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1100 subRun: 1 event: 1923 at 15-Jan-2026 20:43:26 CET
Begin processing the 24th record. run: 1100 subRun: 1 event: 1924 at 15-Jan-2026 20:43:29 CET
Begin processing the 25th record. run: 1100 subRun: 1 event: 1925 at 15-Jan-2026 20:43:34 CET
Begin processing the 26th record. run: 1100 subRun: 1 event: 1926 at 15-Jan-2026 20:43:37 CET
Begin processing the 27th record. run: 1100 subRun: 1 event: 1927 at 15-Jan-2026 20:43:39 CET
Begin processing the 28th record. run: 1100 subRun: 1 event: 1928 at 15-Jan-2026 20:43:41 CET
Begin processing the 29th record. run: 1100 subRun: 1 event: 1929 at 15-Jan-2026 20:43:44 CET
Begin processing the 30th record. run: 1100 subRun: 1 event: 1930 at 15-Jan-2026 20:43:46 CET
Begin processing the 31st record. run: 1100 subRun: 1 event: 1931 at 15-Jan-2026 20:43:49 CET
Begin processing the 32nd record. run: 1100 subRun: 1 event: 1932 at 15-Jan-2026 20:43:51 CET
Begin processing the 33rd record. run: 1100 subRun: 1 event: 1933 at 15-Jan-2026 20:43:54 CET
Begin processing the 34th record. run: 1100 subRun: 1 event: 1934 at 15-Jan-2026 20:43:56 CET
Begin processing the 35th record. run: 1100 subRun: 1 event: 1935 at 15-Jan-2026 20:43:59 CET
Begin processing the 36th record. run: 1100 subRun: 1 event: 1936 at 15-Jan-2026 20:44:01 CET
Begin processing the 37th record. run: 1100 subRun: 1 event: 1937 at 15-Jan-2026 20:44:04 CET
Begin processing the 38th record. run: 1100 subRun: 1 event: 1938 at 15-Jan-2026 20:44:07 CET
Begin processing the 39th record. run: 1100 subRun: 1 event: 1939 at 15-Jan-2026 20:44:09 CET
Begin processing the 40th record. run: 1100 subRun: 1 event: 1940 at 15-Jan-2026 20:44:13 CET
Begin processing the 41st record. run: 1100 subRun: 1 event: 1941 at 15-Jan-2026 20:44:16 CET
Begin processing the 42nd record. run: 1100 subRun: 1 event: 1942 at 15-Jan-2026 20:44:18 CET
Begin processing the 43rd record. run: 1100 subRun: 1 event: 1943 at 15-Jan-2026 20:44:20 CET
Begin processing the 44th record. run: 1100 subRun: 1 event: 1944 at 15-Jan-2026 20:44:23 CET
Begin processing the 45th record. run: 1100 subRun: 1 event: 1945 at 15-Jan-2026 20:44:26 CET
Begin processing the 46th record. run: 1100 subRun: 1 event: 1946 at 15-Jan-2026 20:44:28 CET
Begin processing the 47th record. run: 1100 subRun: 1 event: 1947 at 15-Jan-2026 20:44:30 CET
Begin processing the 48th record. run: 1100 subRun: 1 event: 1948 at 15-Jan-2026 20:44:34 CET
Begin processing the 49th record. run: 1100 subRun: 1 event: 1949 at 15-Jan-2026 20:44:37 CET
Begin processing the 50th record. run: 1100 subRun: 1 event: 1950 at 15-Jan-2026 20:44:39 CET
Begin processing the 51st record. run: 1100 subRun: 1 event: 1951 at 15-Jan-2026 20:44:46 CET
Begin processing the 52nd record. run: 1100 subRun: 1 event: 1952 at 15-Jan-2026 20:44:49 CET
Begin processing the 53rd record. run: 1100 subRun: 1 event: 1953 at 15-Jan-2026 20:44:52 CET
Begin processing the 54th record. run: 1100 subRun: 1 event: 1954 at 15-Jan-2026 20:44:54 CET
Begin processing the 55th record. run: 1100 subRun: 1 event: 1955 at 15-Jan-2026 20:44:57 CET
Begin processing the 56th record. run: 1100 subRun: 1 event: 1956 at 15-Jan-2026 20:44:59 CET
Begin processing the 57th record. run: 1100 subRun: 1 event: 1957 at 15-Jan-2026 20:45:02 CET
Begin processing the 58th record. run: 1100 subRun: 1 event: 1958 at 15-Jan-2026 20:45:07 CET
Begin processing the 59th record. run: 1100 subRun: 1 event: 1959 at 15-Jan-2026 20:45:09 CET
Begin processing the 60th record. run: 1100 subRun: 1 event: 1960 at 15-Jan-2026 20:45:13 CET
Begin processing the 61st record. run: 1100 subRun: 1 event: 1961 at 15-Jan-2026 20:45:15 CET
Begin processing the 62nd record. run: 1100 subRun: 1 event: 1962 at 15-Jan-2026 20:45:18 CET
Begin processing the 63rd record. run: 1100 subRun: 1 event: 1963 at 15-Jan-2026 20:45:21 CET
Begin processing the 64th record. run: 1100 subRun: 1 event: 1964 at 15-Jan-2026 20:45:23 CET
Begin processing the 65th record. run: 1100 subRun: 1 event: 1965 at 15-Jan-2026 20:45:26 CET
Begin processing the 66th record. run: 1100 subRun: 1 event: 1966 at 15-Jan-2026 20:45:28 CET
Begin processing the 67th record. run: 1100 subRun: 1 event: 1967 at 15-Jan-2026 20:45:31 CET
Begin processing the 68th record. run: 1100 subRun: 1 event: 1968 at 15-Jan-2026 20:45:33 CET
Begin processing the 69th record. run: 1100 subRun: 1 event: 1969 at 15-Jan-2026 20:45:37 CET
Begin processing the 70th record. run: 1100 subRun: 1 event: 1970 at 15-Jan-2026 20:45:39 CET
Begin processing the 71st record. run: 1100 subRun: 1 event: 1971 at 15-Jan-2026 20:45:42 CET
Begin processing the 72nd record. run: 1100 subRun: 1 event: 1972 at 15-Jan-2026 20:45:44 CET
Begin processing the 73rd record. run: 1100 subRun: 1 event: 1973 at 15-Jan-2026 20:45:47 CET
Begin processing the 74th record. run: 1100 subRun: 1 event: 1974 at 15-Jan-2026 20:45:51 CET
Begin processing the 75th record. run: 1100 subRun: 1 event: 1975 at 15-Jan-2026 20:45:54 CET
Begin processing the 76th record. run: 1100 subRun: 1 event: 1976 at 15-Jan-2026 20:45:56 CET
Begin processing the 77th record. run: 1100 subRun: 1 event: 1977 at 15-Jan-2026 20:46:01 CET
Begin processing the 78th record. run: 1100 subRun: 1 event: 1978 at 15-Jan-2026 20:46:03 CET
Begin processing the 79th record. run: 1100 subRun: 1 event: 1979 at 15-Jan-2026 20:46:06 CET
Begin processing the 80th record. run: 1100 subRun: 1 event: 1980 at 15-Jan-2026 20:46:11 CET
Begin processing the 81st record. run: 1100 subRun: 1 event: 1981 at 15-Jan-2026 20:46:13 CET
Begin processing the 82nd record. run: 1100 subRun: 1 event: 1982 at 15-Jan-2026 20:46:16 CET
Begin processing the 83rd record. run: 1100 subRun: 1 event: 1983 at 15-Jan-2026 20:46:19 CET
Begin processing the 84th record. run: 1100 subRun: 1 event: 1984 at 15-Jan-2026 20:46:22 CET
Begin processing the 85th record. run: 1100 subRun: 1 event: 1985 at 15-Jan-2026 20:46:24 CET
Begin processing the 86th record. run: 1100 subRun: 1 event: 1986 at 15-Jan-2026 20:46:27 CET
Begin processing the 87th record. run: 1100 subRun: 1 event: 1987 at 15-Jan-2026 20:46:31 CET
Begin processing the 88th record. run: 1100 subRun: 1 event: 1988 at 15-Jan-2026 20:46:34 CET
Begin processing the 89th record. run: 1100 subRun: 1 event: 1989 at 15-Jan-2026 20:46:36 CET
Begin processing the 90th record. run: 1100 subRun: 1 event: 1990 at 15-Jan-2026 20:46:38 CET
Begin processing the 91st record. run: 1100 subRun: 1 event: 1991 at 15-Jan-2026 20:46:40 CET
Begin processing the 92nd record. run: 1100 subRun: 1 event: 1992 at 15-Jan-2026 20:46:43 CET
Begin processing the 93rd record. run: 1100 subRun: 1 event: 1993 at 15-Jan-2026 20:46:46 CET
Begin processing the 94th record. run: 1100 subRun: 1 event: 1994 at 15-Jan-2026 20:47:05 CET
Begin processing the 95th record. run: 1100 subRun: 1 event: 1995 at 15-Jan-2026 20:47:07 CET
Begin processing the 96th record. run: 1100 subRun: 1 event: 1996 at 15-Jan-2026 20:47:11 CET
Begin processing the 97th record. run: 1100 subRun: 1 event: 1997 at 15-Jan-2026 20:47:14 CET
Begin processing the 98th record. run: 1100 subRun: 1 event: 1998 at 15-Jan-2026 20:47:18 CET
Begin processing the 99th record. run: 1100 subRun: 1 event: 1999 at 15-Jan-2026 20:47:20 CET
Begin processing the 100th record. run: 1100 subRun: 1 event: 2000 at 15-Jan-2026 20:47:23 CET
15-Jan-2026 20:47:27 CET  Closed output file "nu_dune10kt_1x2x6_1100_19_20230826T033411Z_gen_g4_detsim_hitreco__20240229T180234Z_reco2_reco2.root"
15-Jan-2026 20:47:27 CET  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/0e/e1/nu_dune10kt_1x2x6_1100_19_20230826T033411Z_gen_g4_detsim_hitreco__20240229T180234Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.133518       2.63981       17.6641       2.28063       1.67176        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0244875     0.0450735     0.0769204     0.0439879    0.00915594       100    
reco:pandora:StandardPandora                   0.100408       2.58498       17.5822       2.22111       1.6671         100    
[art]:TriggerResults:TriggerResultInserter     1.059e-05    1.83764e-05   6.0141e-05    1.70105e-05   7.7467e-06       100    
end_path:out1:RootOutput                       2.75e-06     3.88147e-06   1.7291e-05     3.595e-06    1.58522e-06      100    
end_path:out1:RootOutput(write)               0.00106318    0.00953407     0.0427458    0.00678197     0.0078535       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2217.84 MB
  Peak resident set size usage (VmHWM): 1227.24 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 04:50:35 UTC       justIN version: 01.06.00