Workflow 11056, Stage 1

Priority	50
Processors	1
Wall seconds	80000
Image	/cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest
RSS bytes	2097152000 (2000 MiB)
Max distance for inputs	100.0
Enabled input RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, MONTECARLO, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled output RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled sites	CERN
Scope	usertests
Events for this stage

Output patterns

	Destination	Pattern	Lifetime	For next stage	RSE expression
1	Rucio usertests:fnal-w11056s1p1	prodgenie_nu_dune10kt_1x2x6__trg_mc_2025a_tpg__tpg_dune10kt_1x2x6__reco__v10_12_01d01__rootcheck__TEST-READ_l000020_20251206T022330*.log	864000	False

Environment variables

Name	Value
CONFIG_DIR	/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/9ded7fc9d5087562e8b67c757d4bf6e3feac7bb4
DUNE_QUALIFIER	e26:prof
DUNE_VERSION	v10_12_01d01
MERGE_CONFIG	pass1_CERN

File states

Total files	Finding	Unallocated	Allocated	Outputting	Processed	Not found	Failed
4	0	4	0	0	0	0	0

Job states

Total	Submitted	Started	Processing	Outputting	Finished	Notused	Aborted	Stalled	Jobscript error	Outputting failed	None processed
16	0	0	0	0	0	16	0	0	0	0	0

Stats of processed input files as CSV or JSON, and of uploaded output files as CSV or JSON (up to 10000 files included)

Jobscript

#!/bin/sh
: <<'EOF'

Hello World jobscript for justIN

Submit a workflow like this to run 10 jobs:

justin simple-workflow --monte-carlo 10 --jobscript hello-world.jobscript


Or like this to run jobs and put the output file into Rucio-managed storage:

justin simple-workflow \
     --monte-carlo 10 \
     --jobscript hello-world.jobscript \
     --description 'Hello World!!!' \
     --scope usertests \
     --output-pattern 'hello-world-*.txt:output-test-01'

EOF

# Try to get an unprocessed file from this stage
did_pfn_rse=`$JUSTIN_PATH/justin-get-file`
if [ "$did_pfn_rse" == "" ]; then
  echo "Nothing to process - exit jobscript"
  exit 0
fi
# extract job key from the did_pfn_rse string, which is of the form:
# monte-carlo-000001-000001 000001 MONTECARLO
jobkey=`echo $did_pfn_rse | cut -f2 -d' '`

# Get job configuration
while [ ! -d $CONFIG_DIR ]; do
  echo "Waiting for config directory $CONFIG_DIR to be created"
  sleep 5
done
config="${MERGE_CONFIG}_${jobkey}.json"
echo "Config file: $config"
cat $CONFIG_DIR/$config

# Setup DUNE environment
#DUNE_VERSION=`grep 'dune_version' $CONFIG_DIR/$config | cut -d'"' -f4`
#DUNE_QUALIFIER=`grep 'dune_qualifier' $CONFIG_DIR/$config | cut -d'"' -f4`
DUNE_VERSION=${DUNE_VERSION:-v10_12_01d00}
DUNE_QUALIFIER=${DUNE_QUALIFIER:-e26:prof}
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup dunesw "$DUNE_VERSION" -q "$DUNE_QUALIFIER"

python3 $CONFIG_DIR/do_merge.py $CONFIG_DIR/$config $OUT_DIR

exitcode=$?
if [ $exitcode -ne 0 ];
then
  echo "failed to merge $config $exitcode"
  exit $exitcode
fi

echo "$jobkey" > justin-processed-pfns.txt

echo "merged $config"
echo "merged $config" >merge-`date +%s.%N.txt`

exit 0