justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273371.0@dunegpschedd02.fnal.gov

Jobsub ID273371.0@dunegpschedd02.fnal.gov
Workflow ID11899
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-15 14:13:56
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_dissel_multicore
Last heartbeat2026-01-15 14:41:22
From worker nodeHostnamewn-pijl-029.farm.nikhef.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-15 14:15:11
Input filesmonte-carlo-011899-000482
JobscriptExit code0
Real time25m (1546s)
CPU time25m (1536s = 99%)
Max RSS bytes1916882944 (1828 MiB)
Outputting started2026-01-15 14:40:59
Output files
Finished2026-01-15 14:41:22
Saved logsjustin-logs:273371.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

st path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.163027
**> PCA len: 1.62377, wid: 0.264719, height: 6.23637e-09, ratio: 2.35584e-08
**> Sorted 2 hits, longest distance: 1.1732
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.089438
**> PCA len: 1.96439, wid: 0.175691, height: 1.23284e-08, ratio: 7.0171e-08
**> Sorted 2 hits, longest distance: 1.10613
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.264951
**> PCA len: 1.07985, wid: 0.286107, height: -nan, ratio: -nan
**> Sorted 2 hits, longest distance: 0.964027
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.254537
**> PCA len: 1.01847, wid: 0.259238, height: 3.9525e-09, ratio: 1.52466e-08
**> Sorted 2 hits, longest distance: 0.958679
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.175599
**> PCA len: 1.2025, wid: 0.211159, height: 2.44448e-09, ratio: 1.15765e-08
**> Sorted 2 hits, longest distance: 0.953621
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.143235
**> PCA len: 1.62428, wid: 0.232654, height: -nan, ratio: -nan
**> Sorted 2 hits, longest distance: 1.09758
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.670835
**> PCA len: 0.638746, wid: 0.428494, height: -nan, ratio: -nan
**> Sorted 2 hits, longest distance: 0.513671
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.013145
**> PCA len: 2.32017, wid: 0.0304986, height: 3.61722e-10, ratio: 1.18603e-08
**> Sorted 2 hits, longest distance: 0.887929
**> Best path has 3 hits, 2 edges
************* Finding best path with A* in cluster *****************
**> There are 3 hits, 2 isolated hits, the alpha parameter is 0.0137795
**> PCA len: 2.11827, wid: 0.0291887, height: 0, ratio: 0
**> Sorted 2 hits, longest distance: 1.0821
**> Best path has 3 hits, 2 edges

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunevd_v5_ggd.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16484 nodes/ 2469 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
15-Jan-2026 15:36:54 CET  Initiating request to open input file "prod_muon_400MeV_protodunevd_1_273371_0_1768486524_gen_g4_stage1_g4_stage2_detsim.root"
15-Jan-2026 15:36:54 CET  Opened input file "prod_muon_400MeV_protodunevd_1_273371_0_1768486524_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27693 traces, 27693 summary
Retagger: tagging trace set: gauss with 21550 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 21550 traces tagged "gauss"
FrameSaver: q=5.16414e+06 n=386663 tag=gauss
wclsFrameSaver: saving 27693 traces tagged "wiener"
FrameSaver: q=5.46958e+06 n=359279 tag=wiener
15-Jan-2026 15:38:11 CET  Opened output file with pattern "prod_muon_400MeV_protodunevd_1_273371_0_1768486524_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 17001 traces, 17001 summary
Retagger: tagging trace set: gauss with 12013 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 12013 traces tagged "gauss"
FrameSaver: q=2.10961e+06 n=194760 tag=gauss
wclsFrameSaver: saving 17001 traces tagged "wiener"
FrameSaver: q=2.3087e+06 n=172747 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27573 traces, 27573 summary
Retagger: tagging trace set: gauss with 20653 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 20653 traces tagged "gauss"
FrameSaver: q=4.31174e+06 n=323474 tag=gauss
wclsFrameSaver: saving 27573 traces tagged "wiener"
FrameSaver: q=4.66199e+06 n=290901 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunevd_v5_ggd.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16484 nodes/ 2469 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
15-Jan-2026 15:36:54 CET  Initiating request to open input file "prod_muon_400MeV_protodunevd_1_273371_0_1768486524_gen_g4_stage1_g4_stage2_detsim.root"
15-Jan-2026 15:36:54 CET  Opened input file "prod_muon_400MeV_protodunevd_1_273371_0_1768486524_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27693 traces, 27693 summary
Retagger: tagging trace set: gauss with 21550 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 21550 traces tagged "gauss"
FrameSaver: q=5.16414e+06 n=386663 tag=gauss
wclsFrameSaver: saving 27693 traces tagged "wiener"
FrameSaver: q=5.46958e+06 n=359279 tag=wiener
15-Jan-2026 15:38:11 CET  Opened output file with pattern "prod_muon_400MeV_protodunevd_1_273371_0_1768486524_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 17001 traces, 17001 summary
Retagger: tagging trace set: gauss with 12013 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 12013 traces tagged "gauss"
FrameSaver: q=2.10961e+06 n=194760 tag=gauss
wclsFrameSaver: saving 17001 traces tagged "wiener"
FrameSaver: q=2.3087e+06 n=172747 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27573 traces, 27573 summary
Retagger: tagging trace set: gauss with 20653 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 20653 traces tagged "gauss"
FrameSaver: q=4.31174e+06 n=323474 tag=gauss
wclsFrameSaver: saving 27573 traces tagged "wiener"
FrameSaver: q=4.66199e+06 n=290901 tag=wiener

ls: cannot access *larcvpdvd*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunevd_mpvmpr_doublets.fcl",
    "dune.campaign": "vd-protodune-spine-reco",
    "core.start_time": 1768486523.0,
    "core.end_time": 1768488055.0
  }
}
justIN time: 2026-02-04 06:23:51 UTC       justIN version: 01.06.00