justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257662.3@dunegpschedd02.fnal.gov

Jobsub ID257662.3@dunegpschedd02.fnal.gov
Workflow ID11002
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-04 18:35:47
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-04 18:38:40
From worker nodeHostnamedunegli-7972588-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-04 18:36:55
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213407Z_gen_004236_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1m (80s)
CPU time1m (74s = 92%)
Max RSS bytes1127202816 (1074 MiB)
Outputting started2025-12-04 18:38:15
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11002/1/001/analysisLowE_20251204T183655Z.root
Finished2025-12-04 18:38:40
Saved logsjustin-logs:257662.3-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dout planes.
GeoApaChannelGroupService::ctor: Group 110 (apa110) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 111 (apa111) has 864 channels from 3/3 readout planes.
DuneToolManager::fclFilename: Taking fcl name from command line: lar -c /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/efb56533980f7cc34a52c63ccbd291bf352d2723/reco2_Cheating.fcl -n 10 -o matt_reco2.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a8/19/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213407Z_gen_004236_supernova_g4stage1_g4stage2_detsim_reco.root
AcdDigitReader::ctor:     LogLevel: 1
DuneToolManager::getPrivate: ERROR: Tool name is blank
StandardRawDigitExtractService::ctor: Retrieved digit read tool digitReader
StandardRawDigitExtractService::ctor: StandardRawDigitExtractService:
StandardRawDigitExtractService::ctor:         LogLevel: 1
StandardRawDigitExtractService::ctor:    DigitReadTool: digitReader
StandardRawDigitExtractService::ctor:   PedestalOption: 1
StandardRawDigitExtractService::ctor:     FlagStuckOff: 0
StandardRawDigitExtractService::ctor:      FlagStuckOn: 0
StandardRawDigitPrepService::ctor: Fetching extract service.
StandardRawDigitPrepService::ctor: Fetching channel status provider.
StandardRawDigitPrepService::ctor:   Channel status provider: @0xcffdcc0
StandardRawDigitPrepService::ctor:   Extract service: @0xd9debe0
StandardRawDigitPrepService::ctor: Fetching deconvolution service.
StandardRawDigitPrepService::ctor:   Deconvolution service: @0xcffc160
StandardRawDigitPrepService::ctor: Fetching ROI building service.
StandardRawDigitPrepService::ctor:   ROI building service: @0xcffcc50
StandardRawDigitPrepService::ctor: Fetching wire building service.
StandardRawDigitPrepService::ctor:   Wire building service: @0xcffbff0
StandardRawDigitPrepService::ctor: StandardRawDigitPrepService:
StandardRawDigitPrepService::ctor:              LogLevel: 1
StandardRawDigitPrepService::ctor:               SkipBad: 1
StandardRawDigitPrepService::ctor:             SkipNoisy: 0
StandardRawDigitPrepService::ctor:   ChannelStatusOnline: 0
StandardRawDigitPrepService::ctor:          DoMitigation: 0
StandardRawDigitPrepService::ctor:  DoEarlySignalFinding: 0
StandardRawDigitPrepService::ctor:        DoNoiseRemoval: 0
StandardRawDigitPrepService::ctor:       DoDeconvolution: 1
StandardRawDigitPrepService::ctor:  DoPedestalAdjustment: 0
StandardRawDigitPrepService::ctor:                 DoROI: 1
StandardRawDigitPrepService::ctor:               DoWires: 1
StandardRawDigitPrepService::ctor:                DoDump: 0
StandardRawDigitPrepService::ctor:  DoIntermediateStates: 0
StandardRawDigitPrepService::ctor: No display tools.
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
*************************************************************************************************************************************
Unique Ptrs that are added to the event
*************************************************************************************************************************************
* Data Product Name: InitialTrack             * Instance Name:  * Type: std::vector<recob::Track, std::allocator<recob::Track> >*   *
* Data Product Name: ShowerPCA                * Instance Name:  * Type: std::vector<recob::PCAxis, std::allocator<recob::PCAxis> >* *
* Data Product Name: shower                   * Instance Name:  * Type: std::vector<recob::Shower, std::allocator<recob::Shower> >* *
* Association Name:  PFParticlePCAxisAssn     * Instance Name:  * Type: art::Assns<recob::PFParticle, recob::PCAxis, void>*         *
* Association Name:  ShowerPCAxisAssn         * Instance Name:  * Type: art::Assns<recob::Shower, recob::PCAxis, void>*             *
* Association Name:  ShowerTrackAssn          * Instance Name:  * Type: art::Assns<recob::Shower, recob::Track, void>*              *
* Association Name:  ShowerTrackHitAssn       * Instance Name:  * Type: art::Assns<recob::Track, recob::Hit, void>*                 *
* Association Name:  clusterAssociationsbase  * Instance Name:  * Type: art::Assns<recob::Shower, recob::Cluster, void>*            *
* Association Name:  hitAssociationsbase      * Instance Name:  * Type: art::Assns<recob::Shower, recob::Hit, void>*                *
* Association Name:  pfShowerAssociationsbase * Instance Name:  * Type: art::Assns<recob::Shower, recob::PFParticle, void>*         *
* Association Name:  spShowerAssociationsbase * Instance Name:  * Type: art::Assns<recob::Shower, recob::SpacePoint, void>*         *
*************************************************************************************************************************************
04-Dec-2025 18:37:58 UTC  Initiating request to open input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a8/19/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213407Z_gen_004236_supernova_g4stage1_g4stage2_detsim_reco.root"
04-Dec-2025 18:38:00 UTC  Opened input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a8/19/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213407Z_gen_004236_supernova_g4stage1_g4stage2_detsim_reco.root"
EventReadingAlgorithm - nothing to do; neither geometry nor event file specified.
pLocalAlgorithm->ReadSettings(TiXmlHandle(pXmlElement)) throw STATUS_CODE_NOT_INITIALIZED
    in function: CreateAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Managers/AlgorithmManager.cc line#: 135
Failure in reading pandora settings, STATUS_CODE_NOT_INITIALIZED
PandoraApi::ReadSettings(*m_pPrimaryPandora, fullConfigFileName) throw STATUS_CODE_FAILURE
    in function: ConfigurePandoraInstances
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandora/larpandora/LArPandoraInterface/StandardPandora_module.cc line#: 120
04-Dec-2025 18:38:00 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a8/19/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T213407Z_gen_004236_supernova_g4stage1_g4stage2_detsim_reco.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
[ No processed events ]
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19860.7 MB
  Peak resident set size usage (VmHWM): 1127.2 MB
====================================================================================================
%MSG-s ArtException:  PostEndJob 04-Dec-2025 18:38:00 UTC ModuleEndJob
---- Unknown BEGIN
  An unknown exception was thrown while processing module StandardPandora/pandora during beginJob
---- Unknown END
%MSG
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
The following module label is either not assigned to any path,
or it has been assigned to ignored path(s):
  out1
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v09_91_03d00/gdml/dunevd10kt_3view_30deg_v5_refactored_1x8x14ref.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Error: Unsupported GDML Tag Used :gdml_simple_extension. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :extension. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 462
Info in <TGeoManager::CloseGeometry>: 98051 nodes/ 598 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
%MSG-s ArtException:  AnalyzeEvents:ana@Construction 04-Dec-2025 18:38:15 UTC  ModuleConstruction
cet::exception caught in art
---- CatalogServiceError BEGIN
  Input file not found: matt_reco2.root.
---- CatalogServiceError END
%MSG
Art has completed and will exit with status 24.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 20:39:19 UTC       justIN version: 01.06.00