Workflow 10386, Stage 1

Workflow	10386
Priority	50
Processors	1
Wall seconds	80000
Image	/cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest
RSS bytes	2097152000 (2000 MiB)
Max distance for inputs	100.0
Enabled input RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, MONTECARLO, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled output RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled sites	CERN
Scope	usertests
Events for this stage

Output patterns

	Destination	Pattern	Lifetime	For next stage	RSE expression
1	Rucio usertests:fnal-w10386s1p1	prod_eminus_1-500MeV_dune10kt_1x2x6__trg_mc_2025a_tpg__tpg_dune10kt_1x2x6__reco__v10_12_01d01__triggerana_dune10kt_1x2x6__v10_12_01d01__s2000__l2000__ntuple__EMINUS_20251119T212954*.root	604800	False

Environment variables

Name	Value
CONFIG_DIR	/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/d32c88394661c467c1f5ce011ee659e88b320206
DUNE_QUALIFIER	e26:prof
DUNE_VERSION	v10_12_01d01
MERGE_CONFIG	pass1_CERN

File states

Total files	Finding	Unallocated	Allocated	Outputting	Processed	Not found	Failed
20	0	0	0	0	18	0	2

Job states

Total	Submitted	Started	Processing	Outputting	Finished	Notused	Aborted	Stalled	Jobscript error	Outputting failed	None processed
70	0	0	0	0	22	3	21	23	0	0	1

RSEs used

Name	Inputs	Outputs
MONTECARLO	30	0
DUNE_CERN_EOS	0	3

Stats of processed input files as CSV or JSON, and of uploaded output files as CSV or JSON (up to 10000 files included)

File reset events, by site

Site	Allocated	Outputting
CERN	8	2

Jobscript

#!/bin/sh
: <<'EOF'

Hello World jobscript for justIN

Submit a workflow like this to run 10 jobs:

justin simple-workflow --monte-carlo 10 --jobscript hello-world.jobscript


Or like this to run jobs and put the output file into Rucio-managed storage:

justin simple-workflow \
     --monte-carlo 10 \
     --jobscript hello-world.jobscript \
     --description 'Hello World!!!' \
     --scope usertests \
     --output-pattern 'hello-world-*.txt:output-test-01'

EOF

# Try to get an unprocessed file from this stage
did_pfn_rse=`$JUSTIN_PATH/justin-get-file`
if [ "$did_pfn_rse" == "" ]; then
  echo "Nothing to process - exit jobscript"
  exit 0
fi
# extract job key from the did_pfn_rse string, which is of the form:
# monte-carlo-000001-000001 000001 MONTECARLO
jobkey=`echo $did_pfn_rse | cut -f2 -d' '`

# Get job configuration
while [ ! -d $CONFIG_DIR ]; do
  echo "Waiting for config directory $CONFIG_DIR to be created"
  sleep 5
done
config="${MERGE_CONFIG}_${jobkey}.json"
echo "Config file: $config"
cat $CONFIG_DIR/$config

# Setup DUNE environment
#DUNE_VERSION=`grep 'dune_version' $CONFIG_DIR/$config | cut -d'"' -f4`
#DUNE_QUALIFIER=`grep 'dune_qualifier' $CONFIG_DIR/$config | cut -d'"' -f4`
DUNE_VERSION=${DUNE_VERSION:-v10_12_01d00}
DUNE_QUALIFIER=${DUNE_QUALIFIER:-e26:prof}
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup dunesw "$DUNE_VERSION" -q "$DUNE_QUALIFIER"

python3 $CONFIG_DIR/do_merge.py $CONFIG_DIR/$config $OUT_DIR

exitcode=$?
if [ $exitcode -ne 0 ];
then
  echo "failed to merge $config $exitcode"
  exit $exitcode
fi

echo "$jobkey" > justin-processed-pfns.txt

echo "merged $config"
echo "merged $config" >merge-`date +%s.%N.txt`

exit 0