justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269045.0@dunegpschedd01.fnal.gov

Jobsub ID269045.0@dunegpschedd01.fnal.gov
Workflow ID11246
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-18 12:30:46
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-12-18 12:33:16
From worker nodeHostnamecn116.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-12-18 12:31:30
Input filesvd-protodune-top:3430_3_c_pvdt.test
JobscriptExit code9
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-12-18 12:33:16
Saved logsjustin-logs:269045.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

=/cvmfs/larsoft.opensciencegrid.org/products/geant4reweight/v01_20_14
DUNERECO_VERSION=v10_14_01d00
LARPANDORA_LIB=/cvmfs/larsoft.opensciencegrid.org/products/larpandora/v10_01_08/slf7.x86_64.e26.prof/lib
IFDH_ART_INC=/cvmfs/larsoft.opensciencegrid.org/products/ifdh_art/v2_17_08/slf7.x86_64.e26.prof/inc
MARLEY_LIB=/cvmfs/larsoft.opensciencegrid.org/products/marley/v1_2_1d/Linux64bit+3.10-2.17-e26-prof/lib
SETUP_LAREVT=larevt v10_00_16 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
HDF5_INC=/cvmfs/larsoft.opensciencegrid.org/products/hdf5/v1_12_2a/Linux64bit+3.10-2.17-e26-prof/include
SETUP_CANVAS=canvas v3_16_04 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
DUNEDETDATAFORMATS_INC=/cvmfs/dune.opensciencegrid.org/products/dune/dunedetdataformats/v4_4_5/include
SETUP_TBB=tbb v2021_9_0 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26
PYTHIA_INC=/cvmfs/larsoft.opensciencegrid.org/products/pythia/v6_4_28x/Linux64bit+3.10-2.17-e26-prof/include
SETUP_DUNESW=dunesw v10_14_01d00 -f Linux64bit+3.10-2.17 -z /cvmfs/dune.opensciencegrid.org/products/dune -q e26:prof
PERL_MM_OPT=INSTALL_BASE=/cvmfs/larsoft.opensciencegrid.org/products/mrb/v6_09_11/slf7.x86_64/CPAN
GIT_EXEC_PATH=/cvmfs/larsoft.opensciencegrid.org/products/git/v2_45_1/Linux64bit+3.10-2.17/libexec/git-core
PROTODUNEANA_INC=/cvmfs/dune.opensciencegrid.org/products/dune/protoduneana/v10_14_01d00/include
DUNEDATAPREP_LIB=/cvmfs/dune.opensciencegrid.org/products/dune/dunedataprep/v10_14_01d00/slf7.x86_64.e26.prof/lib
DUNERECO_DIR=/cvmfs/dune.opensciencegrid.org/products/dune/dunereco/v10_14_01d00
WIRECELL_LIB=/cvmfs/larsoft.opensciencegrid.org/products/wirecell/v0_32_1/Linux64bit+3.10-2.17-e26-prof/lib
TRITON_INC=/cvmfs/larsoft.opensciencegrid.org/products/triton/v2_25_0d/Linux64bit+3.10-2.17-e26/include
LIBJPEG_TURBO_INC=/cvmfs/larsoft.opensciencegrid.org/products/libjpeg_turbo/v2_1_0/Linux64bit+3.10-2.17/include
LAREVENTDISPLAY_LIB=/cvmfs/larsoft.opensciencegrid.org/products/lareventdisplay/v10_00_32/slf7.x86_64.e26.prof/lib
PPFX_VERSION=v02_20_07
NUG4_VERSION=v1_16_11
SETUP_LARDATAALG=lardataalg v10_01_05 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
LARCOREALG_INC=/cvmfs/larsoft.opensciencegrid.org/products/larcorealg/v10_00_03/include
G4PARTICLEHPDATA=/cvmfs/larsoft.opensciencegrid.org/products/g4tendl/v1_3_2/G4TENDL1.3.2
SETUP_LARANA=larana v10_01_05 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_LARSOFT=larsoft v10_14_01 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_CETLIB=cetlib v3_18_02 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
SETUP_FHICLCPP=fhiclcpp v4_18_04 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e26:prof
OPENBLAS_INC=/cvmfs/larsoft.opensciencegrid.org/products/openblas/v0_3_23/Linux64bit+3.10-2.17-e26/include
PYTHON_INCLUDE=/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/include/python3.9
_=/usr/bin/env
Will use justin-get-file
pfn: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune-top/3b/22/3430_3_c_pvdt.test
did: vd-protodune-top:3430_3_c_pvdt.test
Running reco stage1
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  opflash
  ophit
  opslicer
  pandoraShower
  pandoraStdcalo
  pandoraWriter
  pandoracalipid
  pandorapid
  pdvddaphne
  rns
  tpcrawdecoder
  wclsdatavdfilter
StandardAdcWireBuildingService::ctor: StandardAdcWireBuildingService:
StandardAdcWireBuildingService::ctor:     LogLevel: 1
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/5497b66c5bd1d091f76ee66f56fcf1dfe1b03fda/protodunevd_v5_ggd_laser.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Error: Unsupported GDML Tag Used :gdml_simple_extension. Please Check Geometry/Schema.
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16484 nodes/ 2469 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
GeoApaChannelGroupService::ctor: Group 0 (apa0) has 3072 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 1 (apa1) has 3072 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 2 (apa2) has 3072 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 3 (apa3) has 3072 channels from 3/3 readout planes.
DAPHNE Channel Map: Building DAPHNE channel map from file PDVD_PDS_Mapping_v09162025.json
PDSP Channel Map: Building RCE TPC wiremap from file protoDUNETPCChannelMap_RCE_v4.txt
PDSP Channel Map: Building FELIX TPC wiremap from file protoDUNETPCChannelMap_RCE_v4.txt
PDSP Channel Map: Building SSP channel map from file protoDUNESSPChannelMap_v1.txt
DuneToolManager::fclFilename: Taking fcl name from command line: lar -c tde_reco_stage1_protodunevd_keepup.fcl -n -1 root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune-top/3b/22/3430_3_c_pvdt.test
ToolBasedRawDigitPrepService::ctor:      Fetching digitReader
AcdDigitReader::ctor:     LogLevel: 1
ToolBasedRawDigitPrepService::ctor:     Found tool digitReader @ 0xd407900
ToolBasedRawDigitPrepService::ctor:      Fetching vdcb_adcChannelRawRmsFiller
DuneToolManager::getPrivate: ERROR: No such tool name: vdcb_adcChannelRawRmsFiller
ToolBasedRawDigitPrepService::ctor: ERROR: Unable to retrieve display tool vdcb_adcChannelRawRmsFiller
ToolBasedRawDigitPrepService::ctor:      Fetching adcSampleFiller
AdcSampleFiller::ctor: Configuration parameters:
AdcSampleFiller::ctor:       LogLevel: 1
AdcSampleFiller::ctor:   AdcUnderflow: 0
AdcSampleFiller::ctor:    AdcOverflow: 4095
ToolBasedRawDigitPrepService::ctor:     Found tool adcSampleFiller @ 0xd44f920
ToolBasedRawDigitPrepService::ctor: Fetching wire building service.
ToolBasedRawDigitPrepService::ctor:   Wire building service: @0x90d32c0
ToolBasedRawDigitPrepService::ctor: ToolBasedRawDigitPrepService:
ToolBasedRawDigitPrepService::ctor:                     LogLevel: 3
ToolBasedRawDigitPrepService::ctor:                      DoWires: 1
ToolBasedRawDigitPrepService::ctor:      ADC channel tools:
ToolBasedRawDigitPrepService::ctor:            digitReader
ToolBasedRawDigitPrepService::ctor:            adcSampleFiller
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
TPC Channel Map: Building TPC wiremap from file PD2VDTopTPCChannelMap_v3.txt
DataPrepModule::reconfigure: Raw digits will be obtained from event data store.
DataPrepModule::reconfigure:              LogLevel: 1
DataPrepModule::reconfigure:           DecoderTool: 
DataPrepModule::reconfigure:            DigitLabel: tpcrawdecoder:daq (tpcrawdecoder, daq)
DataPrepModule::reconfigure:           StatusLabel: same-as-digit (tpcrawdecoder, daq)
DataPrepModule::reconfigure:           TriggerLabel: same-as-digit (tpcrawdecoder, daq)
DataPrepModule::reconfigure:         TimeStampName: 
DataPrepModule::reconfigure:       OutputDigitName: 
DataPrepModule::reconfigure:              WireName: dataprep
DataPrepModule::reconfigure:               DoAssns: 1
DataPrepModule::reconfigure:              DoGroups: 1
DataPrepModule::reconfigure:         ChannelRanges: []
DataPrepModule::reconfigure:        BeamEventLabel: 
DataPrepModule::reconfigure:    IntermediateStates: []
DataPrepModule::reconfigure:   OnlineChannelMapTool: 
DataPrepModule::reconfigure:       KeepChannelBegin: 0
DataPrepModule::reconfigure:         KeepChannelEnd: 0
DataPrepModule::reconfigure:           SkipChannels: []
DataPrepModule::reconfigure:              KeepFembs: []
DataPrepModule::ctor: Wires ande associations will be saved with name dataprep
MultiPandoraApiImpl::DeletePandoraInstances - unable to find daughter instances associated with primary 0
ToolBasedRawDigitPrepService:dtor: Event count: 0
ToolBasedRawDigitPrepService:dtor:  Call count: 0
ToolBasedRawDigitPrepService:dtor: Time report for 3 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.00 sec
ToolBasedRawDigitPrepService:dtor:    vdcb_adcChannelRawRmsFiller:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.00 sec
%MSG-s ArtException:  WireCellToolkit:wclsdatavd@Construction  18-Dec-2025 12:32:52 GMT ModuleConstruction
cet::exception caught in art
---- Configuration BEGIN
  make_tool: 
  ---- Configuration BEGIN
    UnknownPlugin
    ---- Configuration BEGIN
      Unable to load requested library /cvmfs/larsoft.opensciencegrid.org/products/larwirecell/v10_01_26/slf7.x86_64.e26.prof/lib/liblarwirecell_Tools_WCLS_tool.so
      libjsonnet.so.0: cannot open shared object file: No such file or directory
    ---- Configuration END
    Plugin WCLS with version Unknown was not registered.
    Perhaps your plugin type is misspelled or is not a plugin recognized by this
    system.
  ---- Configuration END
  Exception caught while processing plugin spec: WCLS
---- Configuration END
%MSG
Art has completed and will exit with status 9.
Error in reco1
justIN time: 2025-12-19 22:24:32 UTC       justIN version: 01.05.03