justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271541.15@dunegpschedd02.fnal.gov

Jobsub ID271541.15@dunegpschedd02.fnal.gov
Workflow ID11845
Stage ID1
User namejlechevr@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-13 17:09:39
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2026-01-13 17:11:53
From worker nodeHostnameccwcondor0377
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2026-01-13 17:10:50
Input fileshd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_264056_78_1_20251205T231948Z_detsim_reco1.root
JobscriptExit code20
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2026-01-13 17:11:53
Saved logsjustin-logs:271541.15-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

riton/v2_25_0d/Linux64bit+3.10-2.17-e26/include
LIBJPEG_TURBO_INC=/cvmfs/larsoft.opensciencegrid.org/products/libjpeg_turbo/v2_1_0/Linux64bit+3.10-2.17/include
LAREVENTDISPLAY_LIB=/cvmfs/larsoft.opensciencegrid.org/products/lareventdisplay/v10_00_35/slf7.x86_64.e26.prof/lib
PPFX_VERSION=v02_20_07
NUG4_VERSION=v1_16_11
SETUP_LARDATAALG=lardataalg v10_02_01 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
LARCOREALG_INC=/cvmfs/larsoft.opensciencegrid.org/products/larcorealg/v10_00_03/include
G4PARTICLEHPDATA=/cvmfs/larsoft.opensciencegrid.org/products/g4tendl/v1_3_2/G4TENDL1.3.2
SETUP_LARANA=larana v10_01_08 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_LARSOFT=larsoft v10_16_00 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_CETLIB=cetlib v3_18_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_FHICLCPP=fhiclcpp v4_18_04 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
OPENBLAS_INC=/cvmfs/larsoft.opensciencegrid.org/products/openblas/v0_3_23/Linux64bit+3.10-2.17-e26/include
PYTHON_INCLUDE=/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/include/python3.9
_=/usr/bin/env
Will use justin-get-file
pfn: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/e8/11/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_264056_78_1_20251205T231948Z_detsim_reco1.root
did: hd-protodune:pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_264056_78_1_20251205T231948Z_detsim_reco1.root
Running reco stage1
%MSG-i MF_INIT_OK:  Early 13-Jan-2026 18:11:08 CET JobSetup
Messagelogger initialization complete.
%MSG
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25610 nodes/ 5537 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
%MSG-i GeometryCore:  Early 13-Jan-2026 18:11:22 CET JobSetup
Sorting volumes...
%MSG
%MSG-i GeometryCore:  Early 13-Jan-2026 18:11:22 CET JobSetup
New detector geometry loaded from
	/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunehd_v6_refactored.gdml
%MSG
%MSG-i DuneApaWireReadoutGeom:  Early 13-Jan-2026 18:11:22 CET JobSetup
Initializing wire readout...
%MSG
Cryostat 0:
  10240 total channels
  4 APAs
  For all identical APA:
    Number of channels per APA = 2560
    U channels per APA = 800
    V channels per APA = 800
    Z channels per APA = 960
    Pitch in U Plane = 0.4669
    Pitch in V Plane = 0.4669
    Pitch in Z Plane = 0.4792
DuneDeconvolutionService::ctor: DuneDeconvolutionService:
DuneDeconvolutionService::ctor:                LogLevel: 1
DuneRoiBuildingService::ctor: DuneRoiBuildingService:
DuneRoiBuildingService::ctor:     LogLevel: 1
DuneRoiBuildingService::ctor:  NSigmaStart: 3
DuneRoiBuildingService::ctor:    NSigmaEnd: 1
DuneRoiBuildingService::ctor:       PadLow: 50
DuneRoiBuildingService::ctor:      PadHigh: 50
StandardAdcWireBuildingService::ctor: StandardAdcWireBuildingService:
StandardAdcWireBuildingService::ctor:     LogLevel: 1
%MSG-i AuxDetGeometryCore:  Early 13-Jan-2026 18:11:22 CET JobSetup
New detector geometry loaded from
	/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunehd_v6_refactored.gdml
%MSG
GeoApaChannelGroupService::ctor: Group 0 (apa0) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 1 (apa1) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 2 (apa2) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 3 (apa3) has 2560 channels from 4/4 readout planes.
%MSG-i SimpleChannelStatusService:  Early 13-Jan-2026 18:11:22 CET JobSetup
Loaded from configuration:
  - 0 bad channels
  - 0 noisy channels
  - largest channel ID: 10239, largest present: 10239
%MSG
%MSG-i setupProvider<DetectorPropertiesStandard>:  Early  13-Jan-2026 18:11:22 CET JobSetup
Asked to ignore 1 keys: 'InheritNumberTimeSamples'
%MSG
PDSP Channel Map: Building RCE TPC wiremap from file protoDUNETPCChannelMap_RCE_v4.txt
PDSP Channel Map: Building FELIX TPC wiremap from file protoDUNETPCChannelMap_RCE_v4.txt
PDSP Channel Map: Building SSP channel map from file protoDUNESSPChannelMap_v1.txt
DuneToolManager::fclFilename: Taking fcl name from command line: lar -c pdhd_MichelAnalysis.fcl -n -1 root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/e8/11/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_264056_78_1_20251205T231948Z_detsim_reco1.root
AcdDigitReader::ctor:     LogLevel: 1
DuneToolManager::getPrivate: ERROR: Tool name is blank
StandardRawDigitExtractService::ctor: Retrieved digit read tool digitReader
StandardRawDigitExtractService::ctor: StandardRawDigitExtractService:
StandardRawDigitExtractService::ctor:         LogLevel: 1
StandardRawDigitExtractService::ctor:    DigitReadTool: digitReader
StandardRawDigitExtractService::ctor:   PedestalOption: 1
StandardRawDigitExtractService::ctor:     FlagStuckOff: 0
StandardRawDigitExtractService::ctor:      FlagStuckOn: 0
ToolBasedRawDigitPrepService::ctor:      Fetching digitReader
AcdDigitReader::ctor:     LogLevel: 1
ToolBasedRawDigitPrepService::ctor:     Found tool digitReader @ 0x98d7a70
ToolBasedRawDigitPrepService::ctor:      Fetching adcSampleFiller
AdcSampleFiller::ctor: Configuration parameters:
AdcSampleFiller::ctor:       LogLevel: 1
AdcSampleFiller::ctor:   AdcUnderflow: 0
AdcSampleFiller::ctor:    AdcOverflow: 4095
ToolBasedRawDigitPrepService::ctor:     Found tool adcSampleFiller @ 0x7a74940
ToolBasedRawDigitPrepService::ctor:      Fetching adcScaleAdcToKe
AdcSampleScaler::ctor: Parameters:
AdcSampleScaler::ctor:          LogLevel: 1
AdcSampleScaler::ctor:       ScaleFactor: 0.025
AdcSampleScaler::ctor:        OutputUnit: ke/tick
AdcSampleScaler::ctor:         InputUnit: ADC count
ToolBasedRawDigitPrepService::ctor:     Found tool adcScaleAdcToKe @ 0x8bc18f0
ToolBasedRawDigitPrepService::ctor:      Fetching pdspTailPedRemovalZKe
DuneToolManager::getPrivate: ERROR: No such tool name: pdspTailPedRemovalZKe
ToolBasedRawDigitPrepService::ctor: ERROR: Unable to retrieve display tool pdspTailPedRemovalZKe
ToolBasedRawDigitPrepService::ctor: Fetching wire building service.
ToolBasedRawDigitPrepService::ctor:   Wire building service: @0x29a70c0
ToolBasedRawDigitPrepService::ctor: ToolBasedRawDigitPrepService:
ToolBasedRawDigitPrepService::ctor:                     LogLevel: 3
ToolBasedRawDigitPrepService::ctor:                      DoWires: 1
ToolBasedRawDigitPrepService::ctor:      ADC channel tools:
ToolBasedRawDigitPrepService::ctor:            digitReader
ToolBasedRawDigitPrepService::ctor:            adcSampleFiller
ToolBasedRawDigitPrepService::ctor:            adcScaleAdcToKe
%MSG-i SignalShapingServiceDUNE:  Early 13-Jan-2026 18:11:27 CET JobSetup
Getting Filter from .fcl file
%MSG
%MSG-i SignalShapingServiceDUNE:  Early 13-Jan-2026 18:11:27 CET JobSetup
 using the field response provided from a .root file 
%MSG
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Detector Properties:
  Detector Geometry: protodunehdv6  (PDHD)
  Tick Window: [0, 6000]
  Top Bounds: (0.0651,0,-0.59375) -> (353.486,607.499,463.126)
  Bot Bounds: (-353.686,0,-0.59375) -> (-0.2651,607.499,463.126)
Analysis Parameters:
  Track Length Cut: 30 cm
  Fiducial Length: 20 cm
  Barycenter Radius: 10 cm
  Michel Space Radius: 20 cm
13-Jan-2026 18:11:28 CET  Initiating request to open input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/e8/11/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_264056_78_1_20251205T231948Z_detsim_reco1.root"
%MSG-i Root_Information:  MichelAnalysis:checks@Construction  TNetXNGFile::Open() 13-Jan-2026 18:11:29 CET ModuleConstruction
[ERROR] Server responded with an error: [3011] Unable to access file; it does not exist.
ROOT severity: 3000
%MSG
ToolBasedRawDigitPrepService:dtor: Event count: 0
ToolBasedRawDigitPrepService:dtor:  Call count: 0
ToolBasedRawDigitPrepService:dtor: Time report for 4 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.00 sec
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   0.00 sec
%MSG-s ArtException:  MichelAnalysis:checks@Construction  13-Jan-2026 18:11:30 CET ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
  RootInputFileSequence::initFile(): Input file root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/hd-protodune/e8/11/pdhd_prod_beam__gen_g4_IonScintPDExt_PDInt_264056_78_1_20251205T231948Z_detsim_reco1.root was not found or could not be opened.
---- FileOpenError END
%MSG
Art has completed and will exit with status 20.
Error in reco1
justIN time: 2026-02-04 05:49:07 UTC       justIN version: 01.06.00