Jobsub ID 252492.2@dunegpschedd01.fnal.gov

Jobsub ID	252492.2@dunegpschedd01.fnal.gov
Workflow ID	10234
Stage ID	1
User name	calcuttj@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-16 02:36:31
Site	US_UChicago
Entry	Engage_US_MWT2_uct2_condce_mcore
Last heartbeat	2025-11-16 03:13:06
From worker node	Hostname	uct2-c590.mwt2.org
	cpuinfo	AMD EPYC 7302 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-16 02:46:55
Input files	hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242669_78_1_20251113T040458Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242659_87_1_20251113T033940Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_118_1_20251113T042415Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248389_116_1_20251113T030913Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242733_120_1_20251113T054134Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248431_0_1_20251113T033923Z_detsim_reco1.root hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_80_1_20251113T042400Z_detsim_reco1.root
Jobscript	Exit code	0
	Real time	25m (1557s)
	CPU time	22m (1324s = 85%)
	Max RSS bytes	3840110592 (3662 MiB)
Outputting started	2025-11-16 03:12:52
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/pdhd_neg1gev_prod_test_ana/fnal/10234/1/001/pdspana_252492_2_1_10234_20251116T024701Z.root
Finished	2025-11-16 03:13:06
Saved logs	justin-logs:252492.2-dunegpschedd01.fnal.gov.logs.tgz
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

STALL_BASE=/cvmfs/larsoft.opensciencegrid.org/products/mrb/v6_09_11/slf7.x86_64/CPAN
GIT_EXEC_PATH=/cvmfs/larsoft.opensciencegrid.org/products/git/v2_45_1/Linux64bit+3.10-2.17/libexec/git-core
PROTODUNEANA_INC=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/07f2e72e47054dc7d64054ec0998a3497e8f8ac8/protoduneana/v10_10_02d00/include
DUNEDATAPREP_LIB=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/07f2e72e47054dc7d64054ec0998a3497e8f8ac8/dunedataprep/v10_10_02d00/slf7.x86_64.e26.prof/lib
DUNERECO_DIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/07f2e72e47054dc7d64054ec0998a3497e8f8ac8/dunereco/v10_10_02d00
WIRECELL_LIB=/cvmfs/larsoft.opensciencegrid.org/products/wirecell/v0_30_6/Linux64bit+3.10-2.17-e26-prof/lib
TRITON_INC=/cvmfs/larsoft.opensciencegrid.org/products/triton/v2_25_0d/Linux64bit+3.10-2.17-e26/include
LIBJPEG_TURBO_INC=/cvmfs/larsoft.opensciencegrid.org/products/libjpeg_turbo/v2_1_0/Linux64bit+3.10-2.17/include
LAREVENTDISPLAY_LIB=/cvmfs/larsoft.opensciencegrid.org/products/lareventdisplay/v10_00_23/slf7.x86_64.e26.prof/lib
PPFX_VERSION=v02_20_05
NUG4_VERSION=v1_16_08
SETUP_LARDATAALG=lardataalg v10_01_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
LARCOREALG_INC=/cvmfs/larsoft.opensciencegrid.org/products/larcorealg/v10_00_03/include
G4PARTICLEHPDATA=/cvmfs/larsoft.opensciencegrid.org/products/g4tendl/v1_3_2/G4TENDL1.3.2
SETUP_LARANA=larana v10_00_24 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_LARSOFT=larsoft v10_10_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_CETLIB=cetlib v3_18_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_FHICLCPP=fhiclcpp v4_18_04 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
OPENBLAS_INC=/cvmfs/larsoft.opensciencegrid.org/products/openblas/v0_3_23/Linux64bit+3.10-2.17-e26/include
PYTHON_INCLUDE=/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/include/python3.9
DUNE_VERSION=v10_10_02d00
_=/usr/bin/env
Justin specific env vars
JUSTIN_SCOPE=usertests
JUSTIN_WALL_SECONDS=80000
JUSTIN_ALLOCATOR=https://justin-allocator-fnal.dune.hep.ac.uk/api/allocator/
JUSTIN_SITE_NAME=US_UChicago
JUSTIN_SAM_WEB_URI=https://justin.dune.hep.ac.uk/api/samweb/252492.2@dunegpschedd01.fnal.gov/I7r2cqITVvE4Iu9fMBZb5d9UnGdeHPP5Kbc20Ggx2bh9ZauYAueXs4pSJQ22hcmoOfGaHoUoeDXw4n0bjU9bcw
JUSTIN_PROCESSORS=1
JUSTIN_MQL=files from hd-protodune:hd-protodune__pdhd_mc_2025a__full-reconstructed__10_10_04d00__standard_reco_protodunehd_MC__pdhd_1GeV_h4input_cosmics__out1__neg1GeV__validation limit 350
JUSTIN_JOBSCRIPT_SECRET=I7r2cqITVvE4Iu9fMBZb5d9UnGdeHPP5Kbc20Ggx2bh9ZauYAueXs4pSJQ22hcmoOfGaHoUoeDXw4n0bjU9bcw
JUSTIN_PATH=/home
JUSTIN_TIMESTAMP=1763261215
JUSTIN_JOBSUB_ID=252492.2@dunegpschedd01.fnal.gov
JUSTIN_WORKFLOW_ID=10234
JUSTIN_STAGE_ID=1
JUSTIN_RSS_MIB=3999
Will use justin-get-file
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242669_78_1_20251113T040458Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/0c/b2/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242669_78_1_20251113T040458Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242659_87_1_20251113T033940Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/fd/20/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242659_87_1_20251113T033940Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_118_1_20251113T042415Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/71/b8/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_118_1_20251113T042415Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248389_116_1_20251113T030913Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/28/7e/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248389_116_1_20251113T030913Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242733_120_1_20251113T054134Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/0e/e9/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242733_120_1_20251113T054134Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248431_0_1_20251113T033923Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/04/7e/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248431_0_1_20251113T033923Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 0
Stdout b'hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_80_1_20251113T042400Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/16/00/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_80_1_20251113T042400Z_detsim_reco1.root DUNE_US_FNAL_DISK_STAGE'
Stderr b''
Return code 1
Stdout b''
Stderr b'No files available from this stage\n'
root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/0c/b2/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242669_78_1_20251113T040458Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/fd/20/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242659_87_1_20251113T033940Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/71/b8/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_118_1_20251113T042415Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/28/7e/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248389_116_1_20251113T030913Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/0e/e9/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242733_120_1_20251113T054134Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/04/7e/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248431_0_1_20251113T033923Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/16/00/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_80_1_20251113T042400Z_detsim_reco1.root
Files retrieved: 7
PFN list:
root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/0c/b2/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242669_78_1_20251113T040458Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/fd/20/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242659_87_1_20251113T033940Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/71/b8/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_118_1_20251113T042415Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/28/7e/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248389_116_1_20251113T030913Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/0e/e9/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242733_120_1_20251113T054134Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/04/7e/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248431_0_1_20251113T033923Z_detsim_reco1.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/hd-protodune/16/00/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_80_1_20251113T042400Z_detsim_reco1.root
Running pdspana
Ran successfully
Forming metadata
formed
{
  "parents": [
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242669_78_1_20251113T040458Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242659_87_1_20251113T033940Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_118_1_20251113T042415Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248389_116_1_20251113T030913Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242733_120_1_20251113T054134Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_248431_0_1_20251113T033923Z_detsim_reco1.root"
    },
    {
      "namespace": "hd-protodune",
      "name": "pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_242672_80_1_20251113T042400Z_detsim_reco1.root"
    }
  ],
  "metadata": {
    "core.application": "protoduneana.pdspana",
    "core.application.family": "protoduneana",
    "core.application.name": "pdspana",
    "core.application.version": "v10_10_02d00",
    "core.data_tier": "root-tuple-virtual",
    "core.file_format": "root",
    "dune.fcl_version_tag": "v10_10_02d00",
    "dune.fcl_name": "pdhd_ana_MC_nosce_noreco.fcl",
    "core.runs": [
      20250620,
      20250623,
      20250619
    ],
    "core.runs_subruns": [
      2025062000001,
      2025062300001,
      2025061900001
    ],
    "core.group": "dune",
    "dune_mc.detector_type": "hd-protodune",
    "dune_mc.liquid_flow": "no",
    "dune_mc.space_charge": "no",
    "core.run_type": "hd-protodune",
    "core.file_type": "mc",
    "core.data_stream": "out1",
    "core.event_count": 70
  }
}