justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293901.172@dunegpschedd02.fnal.gov

Jobsub ID293901.172@dunegpschedd02.fnal.gov
Workflow ID12287
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-23 00:41:42
SiteUS_BNL
EntryDUNE_US_BNL_spoolce02
Last heartbeat2026-01-23 03:34:32
From worker nodeHostnamespool1484.sdcc.bnl.gov
cpuinfoIntel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-23 02:39:47
Input filesmonte-carlo-012287-000455
JobscriptExit code0
Real time54m (3248s)
CPU time53m (3213s = 98%)
Max RSS bytes2477580288 (2362 MiB)
Outputting started2026-01-23 03:33:57
Output files
Finished2026-01-23 03:34:32
Saved logsjustin-logs:293901.172-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

o in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
22-Jan-2026 22:16:10 EST  Initiating request to open input file "prod_mpvmpr_protodunesp_1_293901_172_1769136008_gen_g4_stage1_detsim.root"
22-Jan-2026 22:16:11 EST  Opened input file "prod_mpvmpr_protodunesp_1_293901_172_1769136008_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36442 traces, 36442 summary
Retagger: tagging trace set: gauss with 34269 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34269 traces tagged "gauss"
FrameSaver: q=2.24234e+06 n=472854 tag=gauss
wclsFrameSaver: saving 36442 traces tagged "wiener"
FrameSaver: q=2.44323e+06 n=424610 tag=wiener
22-Jan-2026 22:18:31 EST  Opened output file with pattern "prod_mpvmpr_protodunesp_1_293901_172_1769136008_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42887 traces, 42887 summary
Retagger: tagging trace set: gauss with 38082 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38082 traces tagged "gauss"
FrameSaver: q=3.11503e+06 n=523261 tag=gauss
wclsFrameSaver: saving 42887 traces tagged "wiener"
FrameSaver: q=3.37188e+06 n=451755 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 53665 traces, 53665 summary
Retagger: tagging trace set: gauss with 46399 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 46399 traces tagged "gauss"
FrameSaver: q=4.44089e+06 n=734241 tag=gauss
wclsFrameSaver: saving 53665 traces tagged "wiener"
FrameSaver: q=4.76328e+06 n=652797 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 49567 traces, 49567 summary
Retagger: tagging trace set: gauss with 44824 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44824 traces tagged "gauss"
FrameSaver: q=4.19902e+06 n=693813 tag=gauss
wclsFrameSaver: saving 49567 traces tagged "wiener"
FrameSaver: q=4.58307e+06 n=619926 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 52770 traces, 52770 summary
Retagger: tagging trace set: gauss with 44764 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44764 traces tagged "gauss"
FrameSaver: q=3.4254e+06 n=654684 tag=gauss
wclsFrameSaver: saving 52770 traces tagged "wiener"
FrameSaver: q=3.67951e+06 n=566940 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36339 traces, 36339 summary
Retagger: tagging trace set: gauss with 34124 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34124 traces tagged "gauss"
FrameSaver: q=2.22891e+06 n=460991 tag=gauss
wclsFrameSaver: saving 36339 traces tagged "wiener"
FrameSaver: q=2.36971e+06 n=403035 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27887 traces, 27887 summary
Retagger: tagging trace set: gauss with 28520 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 28520 traces tagged "gauss"
FrameSaver: q=1.52858e+06 n=396383 tag=gauss
wclsFrameSaver: saving 27887 traces tagged "wiener"
FrameSaver: q=1.63255e+06 n=364740 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  hitpdune
  pandora
  pandora2Shower
  pandora2Track
  pandora2cali
  pandora2calipid
  pandora2pid
  pandoraShower
  pandoraTrack
  pandoraWriter
  pandoracalipid
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
22-Jan-2026 22:16:10 EST  Initiating request to open input file "prod_mpvmpr_protodunesp_1_293901_172_1769136008_gen_g4_stage1_detsim.root"
22-Jan-2026 22:16:11 EST  Opened input file "prod_mpvmpr_protodunesp_1_293901_172_1769136008_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36442 traces, 36442 summary
Retagger: tagging trace set: gauss with 34269 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34269 traces tagged "gauss"
FrameSaver: q=2.24234e+06 n=472854 tag=gauss
wclsFrameSaver: saving 36442 traces tagged "wiener"
FrameSaver: q=2.44323e+06 n=424610 tag=wiener
22-Jan-2026 22:18:31 EST  Opened output file with pattern "prod_mpvmpr_protodunesp_1_293901_172_1769136008_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42887 traces, 42887 summary
Retagger: tagging trace set: gauss with 38082 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38082 traces tagged "gauss"
FrameSaver: q=3.11503e+06 n=523261 tag=gauss
wclsFrameSaver: saving 42887 traces tagged "wiener"
FrameSaver: q=3.37188e+06 n=451755 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 53665 traces, 53665 summary
Retagger: tagging trace set: gauss with 46399 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 46399 traces tagged "gauss"
FrameSaver: q=4.44089e+06 n=734241 tag=gauss
wclsFrameSaver: saving 53665 traces tagged "wiener"
FrameSaver: q=4.76328e+06 n=652797 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 49567 traces, 49567 summary
Retagger: tagging trace set: gauss with 44824 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44824 traces tagged "gauss"
FrameSaver: q=4.19902e+06 n=693813 tag=gauss
wclsFrameSaver: saving 49567 traces tagged "wiener"
FrameSaver: q=4.58307e+06 n=619926 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 52770 traces, 52770 summary
Retagger: tagging trace set: gauss with 44764 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44764 traces tagged "gauss"
FrameSaver: q=3.4254e+06 n=654684 tag=gauss
wclsFrameSaver: saving 52770 traces tagged "wiener"
FrameSaver: q=3.67951e+06 n=566940 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36339 traces, 36339 summary
Retagger: tagging trace set: gauss with 34124 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34124 traces tagged "gauss"
FrameSaver: q=2.22891e+06 n=460991 tag=gauss
wclsFrameSaver: saving 36339 traces tagged "wiener"
FrameSaver: q=2.36971e+06 n=403035 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27887 traces, 27887 summary
Retagger: tagging trace set: gauss with 28520 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 28520 traces tagged "gauss"
FrameSaver: q=1.52858e+06 n=396383 tag=gauss
wclsFrameSaver: saving 27887 traces tagged "wiener"
FrameSaver: q=1.63255e+06 n=364740 tag=wiener

ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
    "dune.campaign": "sp-protodune-spine-reco",
    "core.start_time": 1769136006.0,
    "core.end_time": 1769139231.0
  }
}
justIN time: 2026-02-04 04:33:41 UTC       justIN version: 01.06.00