justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303741.3@dunegpschedd01.fnal.gov

Jobsub ID303741.3@dunegpschedd01.fnal.gov
Workflow ID12722
Stage ID1
User namecalcuttj@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-02-04 01:30:31
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce515
Last heartbeat2026-02-04 01:32:01
From worker nodeHostnameb9p12p0201.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statenone_processed
Started2026-02-04 01:31:07
Input files
Outputting started2026-02-04 01:31:43
Output files
Finished2026-02-04 01:32:01
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
Pip installing snakemake
  WARNING: The script pulptest is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script tabulate is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script humanfriendly is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts docutils, rst2html, rst2html4, rst2html5, rst2latex, rst2man, rst2odt, rst2pseudoxml, rst2s5, rst2xetex and rst2xml are installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script normalizer is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script yte is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts jupyter, jupyter-migrate and jupyter-troubleshoot are installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script jsonschema is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script jupyter-trust is installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts snakemake and snakemake-bash-completion are installed in '/home/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

[notice] A new release of pip is available: 23.0 -> 26.0
[notice] To update, run: pip install --upgrade pip
7.32.4
workflow/Snakefile
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Job stats:
job      count
-----  -------
all          1
jgf          1
total        2

Select jobs to execute...
Calling justin get file -- 0
No files available from this stage
[Wed Feb  4 02:31:43 2026]
Error in rule jgf:
    jobid: 1
    output: justin_input_pfns.txt, justin_input_dids.txt
    shell:
        
        touch justin_input_pfns.txt justin_input_dids.txt
        for ((i=0; i < 1; i++)); do
            echo "Calling justin get file -- $i"
            DID_PFN_RSE=$($JUSTIN_PATH/justin-get-file)
            echo $?
            if [ "$DID_PFN_RSE" == "" ] ; then
                echo "Could not get file. Exiting loop"
                break
            fi
            echo "justin-get-file successful: $DID_PFN_RSE"
            did=$(echo $DID_PFN_RSE | cut -d' ' -f1)
            pfn=$(echo $DID_PFN_RSE | cut -d' ' -f2)
            echo $did >> justin_input_dids.txt
            echo $pfn >> justin_input_pfns.txt
        done
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job jgf since they might be corrupted:
justin_input_pfns.txt, justin_input_dids.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2026-02-04T023142.154518.snakemake.log
Done running snakemake
justIN time: 2026-02-04 07:36:50 UTC       justIN version: 01.06.00