justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263785.1@dunegpschedd01.fnal.gov

Jobsub ID263785.1@dunegpschedd01.fnal.gov
Workflow ID11011
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-05 11:53:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-06 03:20:30
From worker nodeHostnamedunegli-7989091-0-fnpc23016.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 02:30:16
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T001224Z_gen_001933_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time14m (894s)
CPU time14m (884s = 98%)
Max RSS bytes1968005120 (1876 MiB)
Outputting started2025-12-06 02:45:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11011/1/001/analysisLowE_20251206T023017Z.root
Finished2025-12-06 03:20:30
Saved logsjustin-logs:263785.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nnelGroupService::ctor: Group 100 (apa100) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 101 (apa101) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 102 (apa102) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 103 (apa103) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 104 (apa104) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 105 (apa105) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 106 (apa106) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 107 (apa107) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 108 (apa108) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 109 (apa109) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 110 (apa110) has 864 channels from 3/3 readout planes.
GeoApaChannelGroupService::ctor: Group 111 (apa111) has 864 channels from 3/3 readout planes.
DuneToolManager::fclFilename: Taking fcl name from command line: lar -c /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/88d4aa701d0ca8a9cb8c08bd4f8cd1850c370000/reco2_Cheating.fcl -n 10 -o matt_reco2.root root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ed/17/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T001224Z_gen_001933_supernova_g4stage1_g4stage2_detsim_reco.root
AcdDigitReader::ctor:     LogLevel: 1
DuneToolManager::getPrivate: ERROR: Tool name is blank
StandardRawDigitExtractService::ctor: Retrieved digit read tool digitReader
StandardRawDigitExtractService::ctor: StandardRawDigitExtractService:
StandardRawDigitExtractService::ctor:         LogLevel: 1
StandardRawDigitExtractService::ctor:    DigitReadTool: digitReader
StandardRawDigitExtractService::ctor:   PedestalOption: 1
StandardRawDigitExtractService::ctor:     FlagStuckOff: 0
StandardRawDigitExtractService::ctor:      FlagStuckOn: 0
StandardRawDigitPrepService::ctor: Fetching extract service.
StandardRawDigitPrepService::ctor: Fetching channel status provider.
StandardRawDigitPrepService::ctor:   Channel status provider: @0xcee44c0
StandardRawDigitPrepService::ctor:   Extract service: @0x5a75350
StandardRawDigitPrepService::ctor: Fetching deconvolution service.
StandardRawDigitPrepService::ctor:   Deconvolution service: @0xcee3e10
StandardRawDigitPrepService::ctor: Fetching ROI building service.
StandardRawDigitPrepService::ctor:   ROI building service: @0xcee1190
StandardRawDigitPrepService::ctor: Fetching wire building service.
StandardRawDigitPrepService::ctor:   Wire building service: @0xcee1160
StandardRawDigitPrepService::ctor: StandardRawDigitPrepService:
StandardRawDigitPrepService::ctor:              LogLevel: 1
StandardRawDigitPrepService::ctor:               SkipBad: 1
StandardRawDigitPrepService::ctor:             SkipNoisy: 0
StandardRawDigitPrepService::ctor:   ChannelStatusOnline: 0
StandardRawDigitPrepService::ctor:          DoMitigation: 0
StandardRawDigitPrepService::ctor:  DoEarlySignalFinding: 0
StandardRawDigitPrepService::ctor:        DoNoiseRemoval: 0
StandardRawDigitPrepService::ctor:       DoDeconvolution: 1
StandardRawDigitPrepService::ctor:  DoPedestalAdjustment: 0
StandardRawDigitPrepService::ctor:                 DoROI: 1
StandardRawDigitPrepService::ctor:               DoWires: 1
StandardRawDigitPrepService::ctor:                DoDump: 0
StandardRawDigitPrepService::ctor:  DoIntermediateStates: 0
StandardRawDigitPrepService::ctor: No display tools.
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
*************************************************************************************************************************************
Unique Ptrs that are added to the event
*************************************************************************************************************************************
* Data Product Name: InitialTrack             * Instance Name:  * Type: std::vector<recob::Track, std::allocator<recob::Track> >*   *
* Data Product Name: ShowerPCA                * Instance Name:  * Type: std::vector<recob::PCAxis, std::allocator<recob::PCAxis> >* *
* Data Product Name: shower                   * Instance Name:  * Type: std::vector<recob::Shower, std::allocator<recob::Shower> >* *
* Association Name:  PFParticlePCAxisAssn     * Instance Name:  * Type: art::Assns<recob::PFParticle, recob::PCAxis, void>*         *
* Association Name:  ShowerPCAxisAssn         * Instance Name:  * Type: art::Assns<recob::Shower, recob::PCAxis, void>*             *
* Association Name:  ShowerTrackAssn          * Instance Name:  * Type: art::Assns<recob::Shower, recob::Track, void>*              *
* Association Name:  ShowerTrackHitAssn       * Instance Name:  * Type: art::Assns<recob::Track, recob::Hit, void>*                 *
* Association Name:  clusterAssociationsbase  * Instance Name:  * Type: art::Assns<recob::Shower, recob::Cluster, void>*            *
* Association Name:  hitAssociationsbase      * Instance Name:  * Type: art::Assns<recob::Shower, recob::Hit, void>*                *
* Association Name:  pfShowerAssociationsbase * Instance Name:  * Type: art::Assns<recob::Shower, recob::PFParticle, void>*         *
* Association Name:  spShowerAssociationsbase * Instance Name:  * Type: art::Assns<recob::Shower, recob::SpacePoint, void>*         *
*************************************************************************************************************************************
06-Dec-2025 02:30:58 UTC  Initiating request to open input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ed/17/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T001224Z_gen_001933_supernova_g4stage1_g4stage2_detsim_reco.root"
06-Dec-2025 02:30:59 UTC  Opened input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ed/17/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T001224Z_gen_001933_supernova_g4stage1_g4stage2_detsim_reco.root"
Begin processing the 1st record. run: 5027 subRun: 0 event: 19321 at 06-Dec-2025 02:31:00 UTC
PandoraMonitoring, only able to use default TApplication (limited functionality).
PandoraMonitoring::InitializeEve(): DISPLAY environment not set
PandoraMonitoring::InitializeEve(): Caught TEveException: TEveManager::Create ROOT is running in batch mode.
PandoraMonitoring::InitializeEve(): Attempt to release ROOT from batch mode.
Failure in algorithm Alg0002, LArVisualMonitoring, unknown exception
PandoraContentApi::AddToPfo(*this, pNeutrinoPfo, pNeutrinoVertex) throw STATUS_CODE_NOT_ALLOWED
    in function: AddNeutrinoVertex
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoracontent/LArCheating/CheatingNeutrinoCreationAlgorithm.cc line#: 119
Failure in algorithm Alg0023, LArCheatingNeutrinoCreation, STATUS_CODE_NOT_ALLOWED
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

PandoraMonitoring::InitializeEve(): DISPLAY environment not set
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
The following module label is either not assigned to any path,
or it has been assigned to ignored path(s):
  out1
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v09_91_03d00/gdml/dunevd10kt_3view_30deg_v5_refactored_1x8x14ref.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Error: Unsupported GDML Tag Used :gdml_simple_extension. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :extension. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :color. Please Check Geometry/Schema.
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 462
Info in <TGeoManager::CloseGeometry>: 98051 nodes/ 598 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
%MSG-s ArtException:  AnalyzeEvents:ana@Construction 06-Dec-2025 02:45:11 UTC  ModuleConstruction
cet::exception caught in art
---- CatalogServiceError BEGIN
  Input file not found: matt_reco2.root.
---- CatalogServiceError END
%MSG
Art has completed and will exit with status 24.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 20:39:39 UTC       justIN version: 01.06.00