Jobsub ID 302481.1@dunegpschedd01.fnal.gov

Jobsub ID	302481.1@dunegpschedd01.fnal.gov
Workflow ID	12588
Stage ID	1
User name	schellma@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-30 17:55:23
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2026-01-30 17:56:45
From worker node	Hostname	dunegli-8332462-0-fnpc19101.fnal.gov
	cpuinfo	AMD EPYC 7502 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	none_processed
Started	2026-01-30 17:56:30
Input files	hd-protodune-det-reco:np04hd_raw_run032672_0225_dataflow0_datawriter_0_20241105T200250_reco_stage1_reco_stage2_20241105T231250_keepup.root
Outputting started	2026-01-30 17:56:38
Output files
Finished	2026-01-30 17:56:45
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Current working directory is /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft
/home/justin-get-file: line 41: all-input-dids.txt: Read-only file system
../justin-jobscript: line 54: all-input-dids.txt: Read-only file system
Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune-det-reco/0a/0b/np04hd_raw_run032672_0225_dataflow0_datawriter_0_20241105T200250_reco_stage1_reco_stage2_20241105T231250_keepup.root
TARDIR /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb
CODE DIR test_larsoft
localProductsdir /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof
 check that there is a setup in /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof
-rw-r--r-- 1 65534 65534 4584 Jan 30 16:05 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof/setup
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
MRB_INSTALL is /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02d01
MRB_QUALS=e26:prof
MRB_TOP=test_larsoft
MRB_SOURCE=test_larsoft/srcs
MRB_BUILDDIR=test_larsoft/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof

PRODUCTS=/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof:/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof/:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof
 got here
local product directory is /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/f0fd9778f987a22e820118ca8f96cbed59213fdb/test_larsoft/localProducts_larsoft_v09_91_02d01_e26_prof
----------- this block should be empty ------------------
Error encountered when setting up product: dunesw
ERROR: Version conflict -- dependency tree requires versions conflicting with current setup of product duneana: version v09_91_02d01 vs v09_85_00d00
Error encountered when setting up product: protoduneana
ERROR: Version conflict -- dependency tree requires versions conflicting with current setup of product duneprototypes: version v09_91_02d01 vs v09_85_00d00
---------------------------------------------------------
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e26-p3913-prof/lib/libXrdPosixPreload.so
../justin-jobscript: line 96: np04hd_raw_run032672_0225_dataflow0_datawriter_0_20241105T200250_reco_stage1_reco_stage2_20241105T231250_keepup_analyze_2026-01-30T_175638Z.log: Read-only file system
=== Start last 1000 lines of lar log file ===
tail: cannot open 'np04hd_raw_run032672_0225_dataflow0_datawriter_0_20241105T200250_reco_stage1_reco_stage2_20241105T231250_keepup_ana_2026-01-30T_175638Z.log' for reading: No such file or directory
=== End last 1000 lines of lar log file ===
lar exit code 0
../justin-jobscript: line 109: justin-processed-pfns.txt: Read-only file system
tar: *.log: Cannot stat: No such file or directory
tar (child): 302481.1_dunegpschedd01.fnal.gov.logs.tgz: Cannot open: Read-only file system
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now