Jobsub ID 248945.66@dunegpschedd02.fnal.gov

Jobsub ID	248945.66@dunegpschedd02.fnal.gov
Workflow ID	10410
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-20 18:42:27
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2025-11-20 19:32:28
From worker node	Hostname	comp03-03
	cpuinfo	Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	jobscript_error
Started	2025-11-20 19:02:04
Input files	hd-protodune:pdhd_prod_beam__243868_56_1_20251106T031220Z_gen_g4_IonScintPDExt.root_241956_18_1_20251112T061046Z_PDInt.root
Jobscript	Exit code	20
	Real time	0m (0s)
	CPU time	0m (0s = 0%)
	Max RSS bytes	0 (0 MiB)
Outputting started
Output files
Finished	2025-11-20 19:32:28
Saved logs	justin-logs:248945.66-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
Justin processors: 1
name pdhd_prod_beam__243868_56_1_20251106T031220Z_gen_g4_IonScintPDExt.root_241956_18_1_20251112T061046Z_PDInt.root
Found input file URL root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/3f/9f/pdhd_prod_beam__243868_56_1_20251106T031220Z_gen_g4_IonScintPDExt.root_241956_18_1_20251112T061046Z_PDInt.root at DUNE_UK_MANCHESTER_CEPH
Using gen fcl: pdhd_5GeV_h4input_cosmics.fcl
RUNNING STAGE DetsimReco
%MSG-i MF_INIT_OK:  Early 20-Nov-2025 19:02:25 GMT JobSetup
Messagelogger initialization complete.
%MSG
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_10_04d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25610 nodes/ 5537 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
%MSG-i AuxDetGeometryCore:  Early 20-Nov-2025 19:02:28 GMT JobSetup
New detector geometry loaded from
	/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_10_04d00/gdml/protodunehd_v6_refactored.gdml
%MSG
%MSG-i GeometryCore:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Sorting volumes...
%MSG
%MSG-i GeometryCore:  Early 20-Nov-2025 19:02:28 GMT JobSetup
New detector geometry loaded from
	/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_10_04d00/gdml/protodunehd_v6_refactored.gdml
%MSG
%MSG-i DuneApaWireReadoutGeom:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Initializing wire readout...
%MSG
Cryostat 0:
  10240 total channels
  4 APAs
  For all identical APA:
    Number of channels per APA = 2560
    U channels per APA = 800
    V channels per APA = 800
    Z channels per APA = 960
    Pitch in U Plane = 0.4669
    Pitch in V Plane = 0.4669
    Pitch in Z Plane = 0.4792
%MSG-i SimpleChannelStatusService:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Loaded from configuration:
  - 0 bad channels
  - 0 noisy channels
  - largest channel ID: 10239, largest present: 10239
%MSG
%MSG-i setupProvider<DetectorPropertiesStandard>:  Early  20-Nov-2025 19:02:28 GMT JobSetup
Asked to ignore 1 keys: 'InheritNumberTimeSamples'
%MSG
%MSG-i LArG4DetectorService::Ctr:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Reading stepLimit(s) from the configuration file, for volume(s):
%MSG
%MSG-i LArG4DetectorService::Ctr:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Volume: volTPCActiveInner, stepLimit: 0.3
%MSG
%MSG-i LArG4DetectorService::Ctr:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Volume: volTPCActiveOuter, stepLimit: 0.3
%MSG
registering to  primaryGeneratorActionsMap_
registering to   eventActionsMap_
registering to   trackingActionsMap_
registering to  steppingActionsMap_
%MSG-i ParticleListActionService:  Early 20-Nov-2025 19:02:28 GMT JobSetup
The full tracking information will not be stored for particles resulting from the following processes: 
{ "conv" "LowEnConversion" "Pair" "compt" "Compt" "Brem" "phot" "Photo" "Ion" "annihil" }

%MSG
%MSG-i ParticleListActionService:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Trajectory sparsification enabled with SparsifyMargin : 0.015

%MSG
%MSG-i PhotonVisibilityService:  Early 20-Nov-2025 19:02:28 GMT JobSetup
PhotonVisbilityService initializing
%MSG
%MSG-i SignalShapingServiceDUNE:  Early 20-Nov-2025 19:02:28 GMT JobSetup
Getting Filter from .fcl file
%MSG
%MSG-i SignalShapingServiceDUNE:  Early 20-Nov-2025 19:02:28 GMT JobSetup
 using the field response provided from a .root file 
%MSG
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Using custom SPE response taken from SPE_NP04_FBK_2024_without_pretrigger.dat
Using custom SPE response taken from SPE_NP04_HPK_2024_without_pretrigger.dat
%MSG-i RANDOM:  OpDetDigitizerProtoDUNEHD:opdigi@Construction  20-Nov-2025 19:02:29 GMT ModuleConstruction
Instantiated HepJamesRandom engine "opdigi:0:" with seed 56276694.
%MSG
Generating waveforms of 5255.74us = 328483 Samples
	TimeBegin: -2255.74 
	fTimeEnd: 3000 
	SampleFreq: 62.5 MHz
	ReadoutWindow: 328483 ticks
	PreTrigger: 0 ticks
	SampleSize: 16 ns
Full streaming channels: 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 
SPE Templates: 
SPE_NP04_FBK_2024_without_pretrigger.dat 4 14 24 34 40 42 45 46 47 49 50 52 55 56 57 59 60 62 65 66 67 69 70 72 75 76 77 79 84 85 86 87 94 95 96 97 104 105 106 107 114 115 116 117 120 121 124 125 127 129 130 131 134 135 137 139 140 141 144 145 147 149 150 151 154 155 157 159 
SPE_NP04_HPK_2024_without_pretrigger.dat 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19 20 21 22 23 25 26 27 28 29 30 31 32 33 35 36 37 38 39 41 43 44 48 51 53 54 58 61 63 64 68 71 73 74 78 80 81 82 83 88 89 90 91 92 93 98 99 100 101 102 103 108 109 110 111 112 113 118 119 122 123 126 128 132 133 136 138 142 143 146 148 152 153 156 158 
Inputer: "wclsSimDepoSource:"
Outputer: "wclsSimChannelSink:postdrift"
Outputer: "wclsFrameSaver:simdigits"
wclsFrameSaver: promising to produce raw::RawDigits named "daq"
20-Nov-2025 19:02:47 GMT  Initiating request to open input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/3f/9f/pdhd_prod_beam__243868_56_1_20251106T031220Z_gen_g4_IonScintPDExt.root_241956_18_1_20251112T061046Z_PDInt.root"
%MSG-s ArtException:  TriggerResultInserter:TriggerResults@Construction  20-Nov-2025 19:17:33 GMT ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
  ---- FatalRootError BEGIN
    Fatal Root Error: TNetXNGFile::Open
    [FATAL] Socket timeout
    ROOT severity: 3000
  ---- FatalRootError END
  
  RootInputFileSequence::initFile(): Input file root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/3f/9f/pdhd_prod_beam__243868_56_1_20251106T031220Z_gen_g4_IonScintPDExt.root_241956_18_1_20251112T061046Z_PDInt.root was not found or could not be opened.
---- FileOpenError END
%MSG
Art has completed and will exit with status 20.
Detsim returns 20