justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 277668.2@dunegpschedd01.fnal.gov

Jobsub ID277668.2@dunegpschedd01.fnal.gov
Workflow ID11883
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-14 10:40:28
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_brug_multicore
Last heartbeat2026-01-14 11:14:39
From worker nodeHostnamewn-pep-010.farm.nikhef.nl
cpuinfoIntel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 10:41:42
Input filesmonte-carlo-011883-000861
JobscriptExit code0
Real time32m (1958s)
CPU time32m (1927s = 98%)
Max RSS bytes1858408448 (1772 MiB)
Outputting started2026-01-14 11:14:20
Output files
Finished2026-01-14 11:14:39
Saved logsjustin-logs:277668.2-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

uss"
FrameSaver: q=5.38164e+06 n=346456 tag=gauss
wclsFrameSaver: saving 27807 traces tagged "wiener"
FrameSaver: q=5.80616e+06 n=319459 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 21879 traces, 21879 summary
Retagger: tagging trace set: gauss with 18354 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 18354 traces tagged "gauss"
FrameSaver: q=3.21827e+06 n=290470 tag=gauss
wclsFrameSaver: saving 21879 traces tagged "wiener"
FrameSaver: q=3.38139e+06 n=273721 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunevd_v5_ggd.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16484 nodes/ 2469 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
14-Jan-2026 11:58:05 CET  Initiating request to open input file "prod_muon_400MeV_protodunevd_1_277668_2_1768387309_gen_g4_stage1_g4_stage2_detsim.root"
14-Jan-2026 11:58:06 CET  Opened input file "prod_muon_400MeV_protodunevd_1_277668_2_1768387309_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 15696 traces, 15696 summary
Retagger: tagging trace set: gauss with 13693 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 13693 traces tagged "gauss"
FrameSaver: q=2.35521e+06 n=227545 tag=gauss
wclsFrameSaver: saving 15696 traces tagged "wiener"
FrameSaver: q=2.45936e+06 n=217101 tag=wiener
14-Jan-2026 11:59:06 CET  Opened output file with pattern "prod_muon_400MeV_protodunevd_1_277668_2_1768387309_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 17491 traces, 17491 summary
Retagger: tagging trace set: gauss with 14622 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 14622 traces tagged "gauss"
FrameSaver: q=2.31215e+06 n=212563 tag=gauss
wclsFrameSaver: saving 17491 traces tagged "wiener"
FrameSaver: q=2.45629e+06 n=196105 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 18939 traces, 18939 summary
Retagger: tagging trace set: gauss with 14222 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 14222 traces tagged "gauss"
FrameSaver: q=2.66961e+06 n=231970 tag=gauss
wclsFrameSaver: saving 18939 traces tagged "wiener"
FrameSaver: q=2.9025e+06 n=212663 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 16182 traces, 16182 summary
Retagger: tagging trace set: gauss with 11627 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 11627 traces tagged "gauss"
FrameSaver: q=1.9364e+06 n=175868 tag=gauss
wclsFrameSaver: saving 16182 traces tagged "wiener"
FrameSaver: q=2.12475e+06 n=155398 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 13387 traces, 13387 summary
Retagger: tagging trace set: gauss with 11210 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 11210 traces tagged "gauss"
FrameSaver: q=1.76631e+06 n=179262 tag=gauss
wclsFrameSaver: saving 13387 traces tagged "wiener"
FrameSaver: q=1.8848e+06 n=167947 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 22830 traces, 22830 summary
Retagger: tagging trace set: gauss with 16820 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 16820 traces tagged "gauss"
FrameSaver: q=4.5223e+06 n=289169 tag=gauss
wclsFrameSaver: saving 22830 traces tagged "wiener"
FrameSaver: q=4.8896e+06 n=259744 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 22004 traces, 22004 summary
Retagger: tagging trace set: gauss with 16579 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 16579 traces tagged "gauss"
FrameSaver: q=3.0886e+06 n=277221 tag=gauss
wclsFrameSaver: saving 22004 traces tagged "wiener"
FrameSaver: q=3.32723e+06 n=253645 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 19019 traces, 19019 summary
Retagger: tagging trace set: gauss with 15532 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 15532 traces tagged "gauss"
FrameSaver: q=2.83658e+06 n=245619 tag=gauss
wclsFrameSaver: saving 19019 traces tagged "wiener"
FrameSaver: q=2.99521e+06 n=226479 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 23435 traces, 23435 summary
Retagger: tagging trace set: gauss with 19381 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 19381 traces tagged "gauss"
FrameSaver: q=4.81138e+06 n=295513 tag=gauss
wclsFrameSaver: saving 23435 traces tagged "wiener"
FrameSaver: q=5.18624e+06 n=275412 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 15092 traces, 15092 summary
Retagger: tagging trace set: gauss with 12535 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 12535 traces tagged "gauss"
FrameSaver: q=1.84193e+06 n=184355 tag=gauss
wclsFrameSaver: saving 15092 traces tagged "wiener"
FrameSaver: q=1.98796e+06 n=170953 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 29249 traces, 29249 summary
Retagger: tagging trace set: gauss with 21613 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 21613 traces tagged "gauss"
FrameSaver: q=4.35336e+06 n=345945 tag=gauss
wclsFrameSaver: saving 29249 traces tagged "wiener"
FrameSaver: q=4.72205e+06 n=309995 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 18940 traces, 18940 summary
Retagger: tagging trace set: gauss with 13457 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 13457 traces tagged "gauss"
FrameSaver: q=2.20162e+06 n=205291 tag=gauss
wclsFrameSaver: saving 18940 traces tagged "wiener"
FrameSaver: q=2.39009e+06 n=177433 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 17590 traces, 17590 summary
Retagger: tagging trace set: gauss with 14590 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 14590 traces tagged "gauss"
FrameSaver: q=2.35401e+06 n=222593 tag=gauss
wclsFrameSaver: saving 17590 traces tagged "wiener"
FrameSaver: q=2.49086e+06 n=205299 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 27807 traces, 27807 summary
Retagger: tagging trace set: gauss with 20071 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 20071 traces tagged "gauss"
FrameSaver: q=5.38164e+06 n=346456 tag=gauss
wclsFrameSaver: saving 27807 traces tagged "wiener"
FrameSaver: q=5.80616e+06 n=319459 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 21879 traces, 21879 summary
Retagger: tagging trace set: gauss with 18354 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 18354 traces tagged "gauss"
FrameSaver: q=3.21827e+06 n=290470 tag=gauss
wclsFrameSaver: saving 21879 traces tagged "wiener"
FrameSaver: q=3.38139e+06 n=273721 tag=wiener

ls: cannot access *larcvpdvd*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunevd_mpvmpr_doublets.fcl",
    "dune.campaign": "vd-protodune-spine-reco",
    "core.start_time": 1768387308.0,
    "core.end_time": 1768389258.0
  }
}
justIN time: 2026-02-04 11:23:30 UTC       justIN version: 01.06.00