Jobsub ID 225659.60@dunegpschedd02.fnal.gov
| Jobsub ID | 225659.60@dunegpschedd02.fnal.gov |
| Workflow ID | 8648 |
| Stage ID | 1 |
| User name | twalton@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 2 |
| GPU | No |
| RSS bytes | 2097152000 (2000 MiB) |
| Wall seconds limit | 7200 (2 hours) |
| Submitted time | 2025-10-07 00:25:35 |
| Site | UK_Bristol |
| Entry | CMSHTPC_T2_UK_SGrid_Bristol_lcgce02 |
| Last heartbeat | 2025-10-07 00:27:04 |
| From worker node | Hostname | hd76.dice.priv |
| cpuinfo | AMD EPYC 7551P 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 2 |
| RSS bytes | 2097152000 (2000 MiB) |
| Wall seconds limit | 259200 (72 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-10-07 00:26:59 |
| Input files | monte-carlo-008648-000093
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-10-07 00:27:04 |
| Saved logs | justin-logs:225659.60-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
The node working directory /home/workspace
host is hd76.dice.priv
justin site is UK_Bristol
the current directory is /home/workspace
The workspace directory is /home/workspace
Retrieving the file from the path [/home].
The file data identifier (DID) is [monte-carlo-008648-000093]
The file physical file name (PFN) is [000093]
The file Rucio storage element (RSE) is [MONTECARLO]
The input file is 000093
seed is [93], and run number is [93]
Using the index [92] to get the flux file [/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash//Flux/g4numi/20220409/QGSP_BERT/MediumEnergyNewTarget_OnAxis/antineutrino/flux/g4numiv6_minervame_me000z-200i_94_0001.dk2nu]
The output GENIE file names are [NuMI_RHC_Rock_2025_10_07_00_27_00_v3_06_00c_1e15.000093.GHEP.root, NuMI_RHC_Rock_2025_10_07_00_27_00_v3_06_00c_1e15.000093.GTRAC.root]
Setup the dune software (genie) software [setup dunesw v09_78_01d01 -q e20:prof].
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
ERROR: Error in call to fopen: No space left on device
ERROR: Unable to open file /tmp/upspEor6m.
Error encountered when setting up product: gdb
ERROR: Error in call to fopen: No space left on device
ERROR: Unable to open file /tmp/upsErqj2K.
/tmp/upsW3yQUs: line 14: /cvmfs/dune.opensciencegrid.org/products/dune/ups/v6_0_7/Linux64bit+3.10-2.17/bin/setup_fail: No space left on device
Error encountered when setting up product: ninja
ERROR: Error in call to fopen: No space left on device
ERROR: Unable to open file /tmp/upsEkk4HV.
/tmp/upsW3yQUs: line 14: /cvmfs/dune.opensciencegrid.org/products/dune/ups/v6_0_7/Linux64bit+3.10-2.17/bin/setup_fail: No space left on device
Error encountered when setting up product: dunesw
ERROR: Error in call to fopen: No space left on device
ERROR: Unable to open file /tmp/upsVsESQU.
/tmp/upsW3yQUs: line 14: /cvmfs/dune.opensciencegrid.org/products/dune/ups/v6_0_7/Linux64bit+3.10-2.17/bin/setup_fail: No space left on device
Executing the genie workflow for data stream [genie] and input file [000093]
The GDK2NUFLUXXML is [/cvmfs/dune.opensciencegrid.org/dunend/2x2//releases/v1.5.0/ndlar_prod_scripts/ND_Production/run-genie/flux/GNuMIFlux.xml]
Running the genie command [gevgen_fnal -e 1e15 -f /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash//Flux/g4numi/20220409/QGSP_BERT/MediumEnergyNewTarget_OnAxis/antineutrino/flux/g4numiv6_minervame_me000z-200i_94_0001.dk2nu,MiniRun5-Rock -g /cvmfs/dune.opensciencegrid.org/dunend/2x2//releases/v1.5.0/ndlar_prod_scripts/ND_Production/geometry/Merged2x2MINERvA_v4/Merged2x2MINERvA_v4_justRock.gdml -r 93 -L cm -D g_cm3 --cross-sections /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash//Flux/2x2EventGeneration/XSec/numi/genie/v3r4p00/AR2320i00000-k250-e1000/gxspl-NUsmall.xml --tune AR23_20i_00_000 --seed 93 -o NuMI_RHC_Rock_2025_10_07_00_27_00_v3_06_00c_1e15.000093 -m /cvmfs/dune.opensciencegrid.org/dunend/2x2//releases/v1.5.0/ndlar_prod_scripts/ND_Production/run-genie/maxpath/Merged2x2MINERvA_v4_justRock.AR23_20i_00_000.maxpath.xml]
../justin-jobscript: line 196: gevgen_fnal: command not found
FATAL::The file [NuMI_RHC_Rock_2025_10_07_00_27_00_v3_06_00c_1e15.000093.93.ghep.root] does not exist! Will not continue.
FATAL::The file [NuMI_RHC_Rock_2025_10_07_00_27_00_v3_06_00c_1e15.000093.GTRAC.root] does not exist! Will not continue.
