justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20848.0@dunegpschedd01.fnal.gov

Jobsub ID20848.0@dunegpschedd01.fnal.gov
Workflow ID379
Stage ID1
User nameayankele@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-04 18:26:09
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-08-04 18:32:49
From worker nodeHostnametdm005.pic.es
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-04 18:26:57
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_551_20231201T212748Z_gen_g4_detsim_hitreco__20240508T045246Z_reco2.root
JobscriptExit code0
Real time3m (208s)
CPU time0m (46s = 22%)
Max RSS bytes797196288 (760 MiB)
Outputting started2025-08-04 18:30:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ayankele/fnal/00379/1/001/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_551_20231201T212748Z_gen_g4_detsim_hitreco__20240508T045246Z_reco2_reco2_2025-08-04T_182850Z.root
Finished2025-08-04 18:32:49
Saved logsjustin-logs:20848.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

TOP=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu
MRB_SOURCE=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/srcs
MRB_BUILDDIR=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/localProducts_larsoft_v10_03_01_e26_prof

PRODUCTS=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/localProducts_larsoft_v10_03_01_e26_prof:/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/localProducts_larsoft_v10_03_01_e26_prof/:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/localProducts_larsoft_v10_03_01_e26_prof

local product directory is /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/localProducts_larsoft_v10_03_01_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
=== Start last 100 lines of lar log file ===
GeoApaChannelGroupService::ctor: Group 103 (apa103) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 104 (apa104) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 105 (apa105) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 106 (apa106) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 107 (apa107) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 108 (apa108) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 109 (apa109) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 110 (apa110) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 111 (apa111) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 112 (apa112) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 113 (apa113) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 114 (apa114) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 115 (apa115) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 116 (apa116) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 117 (apa117) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 118 (apa118) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 119 (apa119) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 120 (apa120) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 121 (apa121) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 122 (apa122) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 123 (apa123) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 124 (apa124) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 125 (apa125) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 126 (apa126) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 127 (apa127) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 128 (apa128) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 129 (apa129) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 130 (apa130) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 131 (apa131) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 132 (apa132) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 133 (apa133) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 134 (apa134) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 135 (apa135) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 136 (apa136) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 137 (apa137) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 138 (apa138) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 139 (apa139) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 140 (apa140) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 141 (apa141) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 142 (apa142) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 143 (apa143) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 144 (apa144) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 145 (apa145) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 146 (apa146) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 147 (apa147) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 148 (apa148) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 149 (apa149) has 2560 channels from 4/4 readout planes.
DuneToolManager::fclFilename: Taking fcl name from command line: lar -c /cvmfs/fifeuser4.opensciencegrid.org/sw/dune/ae8acaf9e82f18f5e2535e3ed17d6c420ab2a21b/preprocess_atmnu/recoenergys.fcl -o atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_551_20231201T212748Z_gen_g4_detsim_hitreco__20240508T045246Z_reco2_reco2_2025-08-04T_182850Z.root root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/70/ea/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_551_20231201T212748Z_gen_g4_detsim_hitreco__20240508T045246Z_reco2.root
AcdDigitReader::ctor:     LogLevel: 1
DuneToolManager::getPrivate: ERROR: Tool name is blank
StandardRawDigitExtractService::ctor: Retrieved digit read tool digitReader
StandardRawDigitExtractService::ctor: StandardRawDigitExtractService:
StandardRawDigitExtractService::ctor:         LogLevel: 1
StandardRawDigitExtractService::ctor:    DigitReadTool: digitReader
StandardRawDigitExtractService::ctor:   PedestalOption: 1
StandardRawDigitExtractService::ctor:     FlagStuckOff: 0
StandardRawDigitExtractService::ctor:      FlagStuckOn: 0
StandardRawDigitPrepService::ctor: Fetching extract service.
StandardRawDigitPrepService::ctor: Fetching channel status provider.
StandardRawDigitPrepService::ctor:   Channel status provider: @0x878b150
StandardRawDigitPrepService::ctor:   Extract service: @0xcf20a40
StandardRawDigitPrepService::ctor: Fetching deconvolution service.
StandardRawDigitPrepService::ctor:   Deconvolution service: @0x74c5c80
StandardRawDigitPrepService::ctor: Fetching ROI building service.
StandardRawDigitPrepService::ctor:   ROI building service: @0x74c7850
StandardRawDigitPrepService::ctor: Fetching wire building service.
StandardRawDigitPrepService::ctor:   Wire building service: @0x74c65c0
StandardRawDigitPrepService::ctor: StandardRawDigitPrepService:
StandardRawDigitPrepService::ctor:              LogLevel: 1
StandardRawDigitPrepService::ctor:               SkipBad: 1
StandardRawDigitPrepService::ctor:             SkipNoisy: 0
StandardRawDigitPrepService::ctor:   ChannelStatusOnline: 0
StandardRawDigitPrepService::ctor:          DoMitigation: 0
StandardRawDigitPrepService::ctor:  DoEarlySignalFinding: 0
StandardRawDigitPrepService::ctor:        DoNoiseRemoval: 0
StandardRawDigitPrepService::ctor:       DoDeconvolution: 1
StandardRawDigitPrepService::ctor:  DoPedestalAdjustment: 0
StandardRawDigitPrepService::ctor:                 DoROI: 1
StandardRawDigitPrepService::ctor:               DoWires: 1
StandardRawDigitPrepService::ctor:                DoDump: 0
StandardRawDigitPrepService::ctor:  DoIntermediateStates: 0
StandardRawDigitPrepService::ctor: No display tools.
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
recoenergy RecoEnergyS::RecoEnergyS(fhicl::ParameterSet const& pset) 
04-Aug-2025 20:30:24 CEST  Initiating request to open input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/70/ea/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_551_20231201T212748Z_gen_g4_detsim_hitreco__20240508T045246Z_reco2.root"
%MSG-s ArtException:  TriggerResultInserter:TriggerResults@Construction  04-Aug-2025 20:30:26 CEST ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
  ---- FatalRootError BEGIN
    Fatal Root Error: TNetXNGFile::Open
    [FATAL] TLS error: resource temporarily unavailable
    ROOT severity: 3000
  ---- FatalRootError END
  
  RootInputFileSequence::initFile(): Input file root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/70/ea/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_551_20231201T212748Z_gen_g4_detsim_hitreco__20240508T045246Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
Art has completed and will exit with status 20.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2025-08-05 04:34:30 UTC       justIN version: 01.04.00