justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293712.10@dunegpschedd02.fnal.gov

Jobsub ID293712.10@dunegpschedd02.fnal.gov
Workflow ID12265
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-22 16:01:21
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1_multicore
Last heartbeat2026-01-22 17:25:42
From worker nodeHostnamenode2b01.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-22 16:54:47
Input filesmonte-carlo-012265-000782
JobscriptExit code0
Real time30m (1823s)
CPU time30m (1804s = 98%)
Max RSS bytes2483081216 (2368 MiB)
Outputting started2026-01-22 17:25:12
Output files
Finished2026-01-22 17:25:42
Saved logsjustin-logs:293712.10-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

n
**> Sorted 2 hits, longest distance: 0.827207
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.001
**> PCA len: 2.34676, wid: 0.000752195, height: 1.50491e-08, ratio: 2.00069e-05
**> Sorted 2 hits, longest distance: 1.13629
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.169941
**> PCA len: 0.493553, wid: 0.0838748, height: 4.39284e-09, ratio: 5.23738e-08
**> Sorted 2 hits, longest distance: 0.799996
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.124497
**> PCA len: 0.809768, wid: 0.100813, height: -nan, ratio: -nan
**> Sorted 2 hits, longest distance: 0.411431
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.00104861
**> PCA len: 2.01704, wid: 0.00211509, height: -nan, ratio: -nan
**> Sorted 2 hits, longest distance: 0.841134
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.616406
**> PCA len: 1.39567, wid: 0.8603, height: 3.22001e-09, ratio: 3.74289e-09
**> Sorted 2 hits, longest distance: 0.576713
**> Best path has 3 hits, 2 edges

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  hitpdune
  pandora
  pandora2Shower
  pandora2Track
  pandora2cali
  pandora2calipid
  pandora2pid
  pandoraShower
  pandoraTrack
  pandoraWriter
  pandoracalipid
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
22-Jan-2026 17:18:21 GMT  Initiating request to open input file "prod_mpvmpr_protodunesp_1_293712_10_1769100894_gen_g4_stage1_detsim.root"
22-Jan-2026 17:18:22 GMT  Opened input file "prod_mpvmpr_protodunesp_1_293712_10_1769100894_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 47562 traces, 47562 summary
Retagger: tagging trace set: gauss with 42789 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42789 traces tagged "gauss"
FrameSaver: q=3.63873e+06 n=632545 tag=gauss
wclsFrameSaver: saving 47562 traces tagged "wiener"
FrameSaver: q=3.95667e+06 n=569260 tag=wiener
22-Jan-2026 17:19:59 GMT  Opened output file with pattern "prod_mpvmpr_protodunesp_1_293712_10_1769100894_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 37770 traces, 37770 summary
Retagger: tagging trace set: gauss with 35885 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 35885 traces tagged "gauss"
FrameSaver: q=2.34586e+06 n=528888 tag=gauss
wclsFrameSaver: saving 37770 traces tagged "wiener"
FrameSaver: q=2.51267e+06 n=479174 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 31665 traces, 31665 summary
Retagger: tagging trace set: gauss with 30811 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 30811 traces tagged "gauss"
FrameSaver: q=1.68802e+06 n=409241 tag=gauss
wclsFrameSaver: saving 31665 traces tagged "wiener"
FrameSaver: q=1.81329e+06 n=362052 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 39414 traces, 39414 summary
Retagger: tagging trace set: gauss with 36485 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 36485 traces tagged "gauss"
FrameSaver: q=2.61889e+06 n=552690 tag=gauss
wclsFrameSaver: saving 39414 traces tagged "wiener"
FrameSaver: q=2.76295e+06 n=491782 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  hitpdune
  pandora
  pandora2Shower
  pandora2Track
  pandora2cali
  pandora2calipid
  pandora2pid
  pandoraShower
  pandoraTrack
  pandoraWriter
  pandoracalipid
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
22-Jan-2026 17:18:21 GMT  Initiating request to open input file "prod_mpvmpr_protodunesp_1_293712_10_1769100894_gen_g4_stage1_detsim.root"
22-Jan-2026 17:18:22 GMT  Opened input file "prod_mpvmpr_protodunesp_1_293712_10_1769100894_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 47562 traces, 47562 summary
Retagger: tagging trace set: gauss with 42789 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42789 traces tagged "gauss"
FrameSaver: q=3.63873e+06 n=632545 tag=gauss
wclsFrameSaver: saving 47562 traces tagged "wiener"
FrameSaver: q=3.95667e+06 n=569260 tag=wiener
22-Jan-2026 17:19:59 GMT  Opened output file with pattern "prod_mpvmpr_protodunesp_1_293712_10_1769100894_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 37770 traces, 37770 summary
Retagger: tagging trace set: gauss with 35885 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 35885 traces tagged "gauss"
FrameSaver: q=2.34586e+06 n=528888 tag=gauss
wclsFrameSaver: saving 37770 traces tagged "wiener"
FrameSaver: q=2.51267e+06 n=479174 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 31665 traces, 31665 summary
Retagger: tagging trace set: gauss with 30811 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 30811 traces tagged "gauss"
FrameSaver: q=1.68802e+06 n=409241 tag=gauss
wclsFrameSaver: saving 31665 traces tagged "wiener"
FrameSaver: q=1.81329e+06 n=362052 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 39414 traces, 39414 summary
Retagger: tagging trace set: gauss with 36485 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 36485 traces tagged "gauss"
FrameSaver: q=2.61889e+06 n=552690 tag=gauss
wclsFrameSaver: saving 39414 traces tagged "wiener"
FrameSaver: q=2.76295e+06 n=491782 tag=wiener

ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
    "dune.campaign": "sp-protodune-spine-reco",
    "core.start_time": 1769100894.0,
    "core.end_time": 1769102709.0
  }
}
justIN time: 2026-02-04 04:32:42 UTC       justIN version: 01.06.00