justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 22174.0@dunegpschedd01.fnal.gov

Jobsub ID22174.0@dunegpschedd01.fnal.gov
Workflow ID696
Stage ID1
User nametwalton@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-08 18:35:18
SiteUS_Swan
EntryHCC_US_Omaha_swan
Last heartbeat2025-08-08 18:37:53
From worker nodeHostnamec2208
cpuinfoIntel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?False
Job stateoutputting_failed
Started2025-08-08 18:37:11
Input filesneardet-2x2-minerva:TS1_00050017_0001_numip_v09_2407081843_RawData.dat
JobscriptExit code0
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started2025-08-08 18:37:27
Output files
Finished2025-08-08 18:37:53
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

The software is Mx2 v102r2p1
The script path is [/cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/setup]


================================================================
*                                                              *
*          Welcome to the MINERVA Software Framework           *
*                   for ProtoDUNE-ND                           *
*                                                              *
================================================================

==============================================================
Configuration Summary
--------------------------------------------------------------
* Release         : v102r2p1
* Install area    : /cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1
* Platform        : sl7
* cmt config      : x86_64-slc7-gcc49-opt
* ROOT version    : 5.34.36
* Python version  : 2.7.11
* Debug mode      : Off
 
 --- projects will be searched in $CMTPROJECTPATH set by the user
 /cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/minerva:/cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/lhcb:/cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/lcg/external 
==============================================================

Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
Complete the compiler setup
/cvmfs/larsoft.opensciencegrid.org/products/gcc/v4_9_3/Linux64bit+3.10-2.17/bin/gcc




Fri Aug 08 18:37:13 UTC 2025

The Minervascripts is /cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/minerva/MinervaScripts
 --------------------------------------------------------------------
 --- Minerva environment is set for version v102r2p1 
 --- packages will be searched for in CMTPROJECTPATH: 
 /cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/minerva:/cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/lhcb:/cvmfs/minerva.opensciencegrid.org/minerva2x2/products/releases/v102r2p1/lcg/external
 --------------------------------------------------------------------
------------------------------------------
Configuring environment for project Minerva_v102r2p1  (from /srv/home/cmtuser) 
CMT version v1r20p20090520.
------------------------------------------
Installing the cmt directory
Creating a new project file



Setup DUNE and Metacat
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
The node working directory /srv/home/cmtuser
		host is c2208.swan.hcc.unl.edu
		justin site is US_Swan
		the current directory is /srv/home/cmtuser
Checking if tarfiles exist in the directory [/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/30882cd3bd464083c90dda109b2f8ccd8641e379].
total 15K
drwxr-xr-x    3 990 985 4.0K Aug  7 19:46 .
drwxr-xr-x 1188 990 985 4.0K Aug  8 17:10 ..
-rw-r--r--    1 990 985    0 Aug  7 19:46 .cvmfscatalog
-rw-r--r--    1 990 985   37 Aug  7 19:46 .publisher
drwxr-xr-x    5 990 985 4.0K May 16 16:27 Minerva_v102r2p1



../justin-jobscript: line 100: cd: /home/workspace: No such file or directory
Running the cmt setup script.
#CMT---> Warning: package PlotUtils v* Ana not found (requested by RecoStudies)
#CMT---> Warning: package Flux v* Ana not found (requested by SystemTests)
The Systemtests setup is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/30882cd3bd464083c90dda109b2f8ccd8641e379/Minerva_v102r2p1/Tools/SystemTests
../justin-jobscript: line 152: cd: /home/workspace: No such file or directory


Retrieving the file.
/srv/home/justin-get-file: line 41: all-input-dids.txt: Read-only file system
	The file data identifier (DID) is [neardet-2x2-minerva:TS1_00050017_0001_numip_v09_2407081843_RawData.dat]
	The file physical file name (PFN) is [root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/neardet-2x2-minerva/8c/60/TS1_00050017_0001_numip_v09_2407081843_RawData.dat]
	The file Rucio storage element (RSE) is [DUNE_US_FNAL_DISK_STAGE]

Using rucio to download file [neardet-2x2-minerva:TS1_00050017_0001_numip_v09_2407081843_RawData.dat]
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
2025-08-08 18:37:26,653	INFO	Processing 1 item(s) for input
2025-08-08 18:37:27,184	INFO	No preferred protocol impl in rucio.cfg: No section: 'download'
2025-08-08 18:37:27,252	ERROR	[Errno 30] Read-only file system: '/home/workspace'
2025-08-08 18:37:27,252	ERROR	
Rucio exited with an unexpected/unknown error.
Please rerun the last command with the "-v" option to gather more information.
If you're sure there is a problem with Rucio itself, please follow up at: https://github.com/rucio/rucio/issues
mv: cannot stat '/home/workspace/neardet-2x2-minerva/*': No such file or directory
The input file [TS1_00050017_0001_numip_v09_2407081843_RawData.dat] does not exist.
justIN time: 2025-09-18 23:23:42 UTC       justIN version: 01.05.00