Jobsub ID 256664.96@dunegpschedd01.fnal.gov

Jobsub ID	256664.96@dunegpschedd01.fnal.gov
Workflow ID	10652
Stage ID	1
User name	pgranger@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-11-25 00:01:44
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-25 03:09:57
From worker node	Hostname	dunegli-7724529-0-fnpc18003.fnal.gov
	cpuinfo	AMD EPYC 7551 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-11-25 00:08:35
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2.root
Jobscript	Exit code	1
	Real time	0m (0s)
	CPU time	0m (0s = 0%)
	Max RSS bytes	0 (0 MiB)
Outputting started
Output files
Finished	2025-11-25 03:09:57
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

Input PFN = root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1a/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
Using custom sources from /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v10_13_00
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract
MRB_SOURCE=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract/srcs
MRB_BUILDDIR=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract/localProducts_larsoft_v10_13_00_e26_prof

PRODUCTS=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract/localProducts_larsoft_v10_13_00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune/testproducts:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract/localProducts_larsoft_v10_13_00_e26_prof

local product directory is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/6732c92ca827e3ed7cdd1b6142039e514c333adc/pds_extract/localProducts_larsoft_v10_13_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
lar exit code 20
=== Start last 100 lines of lar log file ===
The following paths have not been enabled for execution and will be ignored:
  stream1
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_13_00d00/gdml/dune10kt_v6_refactored_1x2x6.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Error: Unsupported GDML Tag Used :gdml_simple_extension. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :extension. Please Check Geometry/Schema.
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1149
Info in <TGeoManager::CloseGeometry>: 67545 nodes/ 5439 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to  primaryGeneratorActionsMap_
registering to   eventActionsMap_
registering to   trackingActionsMap_
registering to  steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
25-Nov-2025 00:09:10 UTC  Initiating request to open input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1a/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2.root"
25-Nov-2025 00:09:10 UTC  Initiating request to open input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1a/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2.root"
%MSG-s ArtException:  AtmoAnalysis:analysistree@Construction  25-Nov-2025 00:09:11 UTC ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
  RootInputFileSequence::initFile(): Input file root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1a/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
%MSG-s ArtException:  AtmoAnalysis:analysistree@Construction  25-Nov-2025 00:09:11 UTC ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
  RootInputFileSequence::initFile(): Input file root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1a/44/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
Art has completed and will exit with status 20.
=== End last 100 lines of lar log file ===
.:
total 28
-rw-r--r-- 1 dunegli fnalgrid 5192 Nov 25 00:09 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 4159 Nov 25 00:09 debug.log
-rw-r--r-- 1 dunegli fnalgrid 3107 Nov 25 00:09 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2_reco_2025-11-25T_000855Z.log
-rw-r--r-- 1 dunegli fnalgrid  519 Nov 25 00:09 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_140_20231120T134148Z_gen_g4_detsim_hitreco__20240503T062412Z_reco2_ana_2025-11-25T_000855Z.root
-rw-r--r-- 1 dunegli fnalgrid  137 Nov 25 00:08 all-input-dids.txt