justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 45517.6@dunegpschedd01.fnal.gov

Jobsub ID45517.6@dunegpschedd01.fnal.gov
Workflow ID2530
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-09-12 08:35:40
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw7
Last heartbeat2025-09-12 08:36:40
From worker nodeHostnamemh-7763-15.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-09-12 08:36:16
Input filesusertests:prod_beam_p1GeV_protodunehd_1_45515_11_1757664887_gen_largeant_ionscint_pdext.root
JobscriptExit code2
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-09-12 08:36:40
Saved logsjustin-logs:45517.6-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

/products/dune/dune_pardata/v01_84_00
DUNETRIGGER_VERSION=v10_10_02d00
DUNESIM_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunesim/v10_10_02d00
DUNEOPDET_VERSION=v10_10_02d00
SETUP_DUNE_PARDATA=dune_pardata v01_84_00 -f NULL -z /cvmfs/dune.opensciencegrid.org/products/dune
DUNERECO_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunereco/v10_10_02d00/slf7.x86_64.e26.prof
DUNEEXAMPLES_VERSION=v10_10_02d00
DUNERECO_INC=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunereco/v10_10_02d00/include
DUNEUTIL_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneutil/v10_10_02d00
DUNEPDLEGACY_DIR=/cvmfs/dune.opensciencegrid.org/products/dune/dunepdlegacy/v1_01_04
DUNECORE_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunecore/v10_10_02d00
DUNECALIB_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunecalib/v10_10_02d00/slf7.x86_64.e26.prof
DUNEANA_VERSION=v10_10_02d00
DUNETRIGGER_DIR=/cvmfs/dune.opensciencegrid.org/products/dune/dunetrigger/v10_10_02d00
DUNEPROTOTYPES_VERSION=v10_10_02d00
DUNEOPDET_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneopdet/v10_10_02d00
DUNEPROTOTYPES_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneprototypes/v10_10_02d00
SETUP_DUNEDATAPREP=dunedataprep v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
DUNETRIGGER_FQ_DIR=/cvmfs/dune.opensciencegrid.org/products/dune/dunetrigger/v10_10_02d00/slf7.x86_64.e26.prof
DUNEOPDET_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneopdet/v10_10_02d00/slf7.x86_64.e26.prof
PROTODUNEANA_VERSION=v10_10_02d00
SETUP_DUNEPROTOTYPES=duneprototypes v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
DUNEEXAMPLES_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneexamples/v10_10_02d00/slf7.x86_64.e26.prof
DUNEDATAPREP_VERSION=v10_10_02d00
DUNECORE_INC=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunecore/v10_10_02d00/include
DUNEPDLEGACY_INC=/cvmfs/dune.opensciencegrid.org/products/dune/dunepdlegacy/v1_01_04/include
SETUP_DUNEANA=duneana v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
DUNEPROTOTYPES_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneprototypes/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNEDETDATAFORMATS_VERSION=v4_4_5
DUNEUTIL_VERSION=v10_10_02d00
DUNESW_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunesw/v10_10_02d00/slf7.x86_64.e26.prof/lib
SETUP_PROTODUNEANA=protoduneana v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
SETUP_DUNESIM=dunesim v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
PROTODUNEANA_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/protoduneana/v10_10_02d00/slf7.x86_64.e26.prof
DUNESIM_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunesim/v10_10_02d00/slf7.x86_64.e26.prof/lib
SETUP_DUNEOPDET=duneopdet v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
DUNESIM_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunesim/v10_10_02d00/slf7.x86_64.e26.prof
SETUP_DUNEPDLEGACY=dunepdlegacy v1_01_04 -f Linux64bit+3.10-2.17 -z /cvmfs/dune.opensciencegrid.org/products/dune -q e26:prof
DUNECALIB_VERSION=v10_10_02d00
SETUP_DUNEDAQDATAFORMATS=dunedaqdataformats v4_4_5 -f NULL -z /cvmfs/dune.opensciencegrid.org/products/dune
DUNEEXAMPLES_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneexamples/v10_10_02d00
DUNECORE_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunecore/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNEPDLEGACY_LIB=/cvmfs/dune.opensciencegrid.org/products/dune/dunepdlegacy/v1_01_04/slf7.x86_64.e26.prof/lib
SETUP_DUNETRIGGER=dunetrigger v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/dune.opensciencegrid.org/products/dune -q e26:prof
DUNEANA_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneana/v10_10_02d00/slf7.x86_64.e26.prof/lib
SETUP_DUNECORE=dunecore v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
SETUP_DUNEANAOBJ=duneanaobj v03_10_00 -f Linux64bit+3.10-2.17 -z /cvmfs/dune.opensciencegrid.org/products/dune -q e26:prof
DUNETRIGGER_LIB=/cvmfs/dune.opensciencegrid.org/products/dune/dunetrigger/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNESIM_VERSION=v10_10_02d00
DUNECALIB_INC=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunecalib/v10_10_02d00/include
DUNEOPDET_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneopdet/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNEUTIL_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneutil/v10_10_02d00/slf7.x86_64.e26.prof
DUNEANAOBJ_LIB=/cvmfs/dune.opensciencegrid.org/products/dune/duneanaobj/v03_10_00/slf7.x86_64.e26.prof/lib
DUNEPROTOTYPES_FQ_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneprototypes/v10_10_02d00/slf7.x86_64.e26.prof
PROTODUNEANA_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/protoduneana/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNEPROTOTYPES_INC=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneprototypes/v10_10_02d00/include
DUNEEXAMPLES_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/duneexamples/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNESW_TAR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19
SETUP_DUNECALIB=dunecalib v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
DUNERECO_VERSION=v10_10_02d00
DUNEDETDATAFORMATS_INC=/cvmfs/dune.opensciencegrid.org/products/dune/dunedetdataformats/v4_4_5/include
SETUP_DUNESW=dunesw v10_10_02d00 -f Linux64bit+3.10-2.17 -z /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19 -q e26:prof
PROTODUNEANA_INC=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/protoduneana/v10_10_02d00/include
DUNEDATAPREP_LIB=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunedataprep/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNERECO_DIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/a8d662004a27199f7333fa42fe897af3511c5d19/dunereco/v10_10_02d00
Installing yaml
Collecting pyyaml
  Downloading PyYAML-6.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (737 kB)
Installing collected packages: pyyaml
Successfully installed pyyaml-6.0.2
WARNING: You are using pip version 21.2.4; however, version 25.2 is available.
You should consider upgrading via the '/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python3 -m pip install --upgrade pip' command.
Done
Will use justin-get-file
input file: prod_beam_p1GeV_protodunehd_1_45515_11_1757664887_gen_largeant_ionscint_pdext.root
jobsub_id: 45517.6@dunegpschedd01.fnal.gov
Getting run subrun from usertests:prod_beam_p1GeV_protodunehd_1_45515_11_1757664887_gen_largeant_ionscint_pdext.root
Traceback (most recent call last):
  File "/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/3e6e5d515730bbcc646e819bad8f6ee60c9dce9a/beam_job_utils.py", line 392, in <module>
    routines[args.routine](args)
  File "/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/3e6e5d515730bbcc646e819bad8f6ee60c9dce9a/beam_job_utils.py", line 47, in get_run_subrun
    print(md['metadata']['core.runs'][0],
KeyError: 'core.runs'
error in get_run_subrun

nevents: 10
input flag: -i root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/99/db/prod_beam_p1GeV_protodunehd_1_45515_11_1757664887_gen_largeant_ionscint_pdext.root --parent usertests:prod_beam_p1GeV_protodunehd_1_45515_11_1757664887_gen_largeant_ionscint_pdext.root
usage: beam_job_utils.py [-h] [-i I] [--yaml YAML] [-o O] [--json JSON]
                         [--run RUN] [--subrun SUBRUN]
                         [--exclude EXCLUDE [EXCLUDE ...]]
                         [--overrides OVERRIDES [OVERRIDES ...]]
                         [--parent PARENT] [--event EVENT] [--nevents NEVENTS]
                         [--past_fcls PAST_FCLS [PAST_FCLS ...]]
                         [--past_apps PAST_APPS [PAST_APPS ...]]
                         [--past_vers PAST_VERS [PAST_VERS ...]] [--set_esr]
                         {get_run_subrun,make_metadata,run_job,get_artroot_nevents}
beam_job_utils.py: error: argument --run: expected one argument
Error running. Exiting with 2
justIN time: 2025-09-18 23:24:54 UTC       justIN version: 01.05.00