Workflow 10409, Stage 1

Workflow	10409
Priority	50
Processors	1
Wall seconds	80000
Image	/cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest
RSS bytes	2097152000 (2000 MiB)
Max distance for inputs	100.0
Enabled input RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, MONTECARLO, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled output RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled sites	CERN
Scope	usertests
Events for this stage

Output patterns

	Destination	Pattern	Lifetime	For next stage	RSE expression
1	Rucio usertests:fnal-w10409s1p1	prodmarley_nue_flat_es_dune10kt_1x2x2__trg_mc_2025a_tpg__tpg_dune10kt_1x2x2__reco__v10_12_01d01__triggerana_dune10kt_1x2x2__v10_12_01d01__ntuple__FLAT-ES_l002000_20251120T160840*.root	604800	False

Environment variables

Name	Value
CONFIG_DIR	/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/923204154354f8894035af8a2a3a17b0a55881df
DUNE_QUALIFIER	e26:prof
DUNE_VERSION	v10_12_01d01
MERGE_CONFIG	pass1_CERN

File states

Total files	Finding	Unallocated	Allocated	Outputting	Processed	Not found	Failed
20	0	0	0	0	17	0	3

Job states

Total	Submitted	Started	Processing	Outputting	Finished	Notused	Aborted	Stalled	Jobscript error	Outputting failed	None processed
112	0	0	0	0	25	1	18	12	0	0	56

RSEs used

Name	Inputs	Outputs
MONTECARLO	35	0
DUNE_CERN_EOS	0	13
DUNE_IT_INFN_CNAF	0	2
PRAGUE	0	1

Stats of processed input files as CSV or JSON, and of uploaded output files as CSV or JSON (up to 10000 files included)

File reset events, by site

Site	Allocated	Outputting
CERN	12	3

Jobscript

#!/bin/sh
: <<'EOF'

Hello World jobscript for justIN

Submit a workflow like this to run 10 jobs:

justin simple-workflow --monte-carlo 10 --jobscript hello-world.jobscript


Or like this to run jobs and put the output file into Rucio-managed storage:

justin simple-workflow \
     --monte-carlo 10 \
     --jobscript hello-world.jobscript \
     --description 'Hello World!!!' \
     --scope usertests \
     --output-pattern 'hello-world-*.txt:output-test-01'

EOF

# Try to get an unprocessed file from this stage
did_pfn_rse=`$JUSTIN_PATH/justin-get-file`
if [ "$did_pfn_rse" == "" ]; then
  echo "Nothing to process - exit jobscript"
  exit 0
fi
# extract job key from the did_pfn_rse string, which is of the form:
# monte-carlo-000001-000001 000001 MONTECARLO
jobkey=`echo $did_pfn_rse | cut -f2 -d' '`

# Get job configuration
while [ ! -d $CONFIG_DIR ]; do
  echo "Waiting for config directory $CONFIG_DIR to be created"
  sleep 5
done
config="${MERGE_CONFIG}_${jobkey}.json"
echo "Config file: $config"
cat $CONFIG_DIR/$config

# Setup DUNE environment
#DUNE_VERSION=`grep 'dune_version' $CONFIG_DIR/$config | cut -d'"' -f4`
#DUNE_QUALIFIER=`grep 'dune_qualifier' $CONFIG_DIR/$config | cut -d'"' -f4`
DUNE_VERSION=${DUNE_VERSION:-v10_12_01d00}
DUNE_QUALIFIER=${DUNE_QUALIFIER:-e26:prof}
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup dunesw "$DUNE_VERSION" -q "$DUNE_QUALIFIER"

python3 $CONFIG_DIR/do_merge.py $CONFIG_DIR/$config $OUT_DIR

exitcode=$?
if [ $exitcode -ne 0 ];
then
  echo "failed to merge $config $exitcode"
  exit $exitcode
fi

echo "$jobkey" > justin-processed-pfns.txt

echo "merged $config"
echo "merged $config" >merge-`date +%s.%N.txt`

exit 0