Jobsub ID 248931.35@dunegpschedd02.fnal.gov

Jobsub ID	248931.35@dunegpschedd02.fnal.gov
Workflow ID	10402
Stage ID	1
User name	calcuttj@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-20 17:50:15
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-20 19:23:52
From worker node	Hostname	dunegli-7674014-0-fnpc23021.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-20 18:24:24
Input files	hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242724_187_1_20251113T045950Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_103_1_20251113T042717Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242731_116_1_20251113T053708Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_104_1_20251113T042643Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242728_71_1_20251113T052148Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242719_71_1_20251113T043728Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248437_54_1_20251113T034332Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242578_96_1_20251113T010956Z_detsim_reco1.root
Jobscript	Exit code	0
	Real time	17m (1033s)
	CPU time	16m (975s = 94%)
	Max RSS bytes	3328540672 (3174 MiB)
Outputting started	2025-11-20 18:41:37
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/pdhd_neg5gev_prod_test_ana/fnal/10402/1/001/pdspana_248931_35_1_10402_20251120T182427Z.root
Finished	2025-11-20 19:23:52
Saved logs	justin-logs:248931.35-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

TURBO_INC=/cvmfs/larsoft.opensciencegrid.org/products/libjpeg_turbo/v2_1_0/Linux64bit+3.10-2.17/include
LAREVENTDISPLAY_LIB=/cvmfs/larsoft.opensciencegrid.org/products/lareventdisplay/v10_00_23/slf7.x86_64.e26.prof/lib
PPFX_VERSION=v02_20_05
NUG4_VERSION=v1_16_08
SETUP_LARDATAALG=lardataalg v10_01_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
LARCOREALG_INC=/cvmfs/larsoft.opensciencegrid.org/products/larcorealg/v10_00_03/include
G4PARTICLEHPDATA=/cvmfs/larsoft.opensciencegrid.org/products/g4tendl/v1_3_2/G4TENDL1.3.2
SETUP_LARANA=larana v10_00_24 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_LARSOFT=larsoft v10_10_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_CETLIB=cetlib v3_18_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_FHICLCPP=fhiclcpp v4_18_04 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
OPENBLAS_INC=/cvmfs/larsoft.opensciencegrid.org/products/openblas/v0_3_23/Linux64bit+3.10-2.17-e26/include
PYTHON_INCLUDE=/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/include/python3.9
DUNE_VERSION=v10_10_02d00
_=/usr/bin/env
Justin specific env vars
JUSTIN_SCOPE=usertests
JUSTIN_WALL_SECONDS=80000
JUSTIN_ALLOCATOR=https://justin-allocator-fnal.dune.hep.ac.uk/api/allocator/
JUSTIN_SITE_NAME=US_FNAL-FermiGrid
JUSTIN_SAM_WEB_URI=https://justin.dune.hep.ac.uk/api/samweb/248931.35@dunegpschedd02.fnal.gov/yL5UQ5B2t82cfLqf0hETd3TmnsKLdi8bf5wStVwD1bAgCRWA7tT0WHaAhFQW_33_FASijTr-M6OpsGVJTy-IgA
JUSTIN_PROCESSORS=1
JUSTIN_MQL=files from hd-protodune:hd-protodune__pdhd_mc_2025a__full-reconstructed__10_10_04d00__standard_reco_protodunehd_MC__pdhd_5GeV_h4input_cosmics__out1__neg5GeV__validation ordered skip 1000 limit 1000
JUSTIN_JOBSCRIPT_SECRET=yL5UQ5B2t82cfLqf0hETd3TmnsKLdi8bf5wStVwD1bAgCRWA7tT0WHaAhFQW_33_FASijTr-M6OpsGVJTy-IgA
JUSTIN_PATH=/home
JUSTIN_TIMESTAMP=1763663064
JUSTIN_JOBSUB_ID=248931.35@dunegpschedd02.fnal.gov
JUSTIN_WORKFLOW_ID=10402
JUSTIN_STAGE_ID=1
JUSTIN_RSS_MIB=3999
Will use justin-get-file
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242724_187_1_20251113T045950Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/c3/81/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242724_187_1_20251113T045950Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_103_1_20251113T042717Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/b8/f8/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_103_1_20251113T042717Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242731_116_1_20251113T053708Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/7c/a6/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242731_116_1_20251113T053708Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_104_1_20251113T042643Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/2d/42/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_104_1_20251113T042643Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242728_71_1_20251113T052148Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/4a/aa/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242728_71_1_20251113T052148Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242719_71_1_20251113T043728Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/75/0b/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242719_71_1_20251113T043728Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248437_54_1_20251113T034332Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/8b/d9/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248437_54_1_20251113T034332Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242578_96_1_20251113T010956Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/e9/d3/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242578_96_1_20251113T010956Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 1
Stdout b''
Stderr b'No files available from this stage\n'
root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/c3/81/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242724_187_1_20251113T045950Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/b8/f8/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_103_1_20251113T042717Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/7c/a6/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242731_116_1_20251113T053708Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/2d/42/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_104_1_20251113T042643Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/4a/aa/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242728_71_1_20251113T052148Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/75/0b/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242719_71_1_20251113T043728Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/8b/d9/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248437_54_1_20251113T034332Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/e9/d3/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242578_96_1_20251113T010956Z_detsim_reco1.root
Files retrieved: 8
PFN list:
root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/c3/81/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242724_187_1_20251113T045950Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/b8/f8/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_103_1_20251113T042717Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/7c/a6/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242731_116_1_20251113T053708Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/2d/42/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_104_1_20251113T042643Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/4a/aa/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242728_71_1_20251113T052148Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/75/0b/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242719_71_1_20251113T043728Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/8b/d9/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248437_54_1_20251113T034332Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/e9/d3/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242578_96_1_20251113T010956Z_detsim_reco1.root
Running pdspana
Ran successfully
Forming metadata
formed
{
  "parents": [
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242724_187_1_20251113T045950Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_103_1_20251113T042717Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242731_116_1_20251113T053708Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248443_104_1_20251113T042643Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242728_71_1_20251113T052148Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242719_71_1_20251113T043728Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248437_54_1_20251113T034332Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242578_96_1_20251113T010956Z_detsim_reco1.root"
    }
  ],
  "metadata": {
    "core.application": "protoduneana.pdspana",
    "core.application.family": "protoduneana",
    "core.application.name": "pdspana",
    "core.application.version": "v10_10_02d00",
    "core.data_tier": "root-tuple-virtual",
    "core.file_format": "root",
    "dune.fcl_version_tag": "v10_10_02d00",
    "dune.fcl_name": "pdhd_ana_MC_nosce_noreco.fcl",
    "core.runs": [
      20250627
    ],
    "core.runs_subruns": [
      2025062700001
    ],
    "core.group": "dune",
    "dune_mc.detector_type": "hd-protodune",
    "dune_mc.liquid_flow": "no",
    "dune_mc.space_charge": "no",
    "core.run_type": "hd-protodune",
    "core.file_type": "mc",
    "core.data_stream": "out1",
    "core.event_count": 80
  }
}