justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 299467.1@dunegpschedd01.fnal.gov

Jobsub ID299467.1@dunegpschedd01.fnal.gov
Workflow ID12266
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-22 19:11:20
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2026-01-22 20:25:31
From worker nodeHostnamemahagon17.farm.particle.cz
cpuinfoAMD EPYC 7301 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-22 19:12:23
Input filesmonte-carlo-012266-000871
JobscriptExit code0
Real time1h (4278s)
CPU time1h (4190s = 97%)
Max RSS bytes3635474432 (3467 MiB)
Outputting started2026-01-22 20:23:45
Output files
Finished2026-01-22 20:25:31
Saved logsjustin-logs:299467.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

"wiener"
FrameSaver: q=3.54703e+06 n=530390 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 49019 traces, 49019 summary
Retagger: tagging trace set: gauss with 42016 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42016 traces tagged "gauss"
FrameSaver: q=3.80606e+06 n=634949 tag=gauss
wclsFrameSaver: saving 49019 traces tagged "wiener"
FrameSaver: q=4.09203e+06 n=558079 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 51662 traces, 51662 summary
Retagger: tagging trace set: gauss with 44545 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44545 traces tagged "gauss"
FrameSaver: q=3.53654e+06 n=667617 tag=gauss
wclsFrameSaver: saving 51662 traces tagged "wiener"
FrameSaver: q=3.80462e+06 n=589569 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 39999 traces, 39999 summary
Retagger: tagging trace set: gauss with 39121 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 39121 traces tagged "gauss"
FrameSaver: q=3.17554e+06 n=614849 tag=gauss
wclsFrameSaver: saving 39999 traces tagged "wiener"
FrameSaver: q=3.47941e+06 n=570042 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36956 traces, 36956 summary
Retagger: tagging trace set: gauss with 34251 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34251 traces tagged "gauss"
FrameSaver: q=2.37442e+06 n=465311 tag=gauss
wclsFrameSaver: saving 36956 traces tagged "wiener"
FrameSaver: q=2.54209e+06 n=410461 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48888 traces, 48888 summary
Retagger: tagging trace set: gauss with 41469 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 41469 traces tagged "gauss"
FrameSaver: q=3.84976e+06 n=584445 tag=gauss
wclsFrameSaver: saving 48888 traces tagged "wiener"
FrameSaver: q=4.26618e+06 n=514164 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41592 traces, 41592 summary
Retagger: tagging trace set: gauss with 38002 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38002 traces tagged "gauss"
FrameSaver: q=3.10295e+06 n=574251 tag=gauss
wclsFrameSaver: saving 41592 traces tagged "wiener"
FrameSaver: q=3.34398e+06 n=513577 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  hitpdune
  pandora
  pandora2Shower
  pandora2Track
  pandora2cali
  pandora2calipid
  pandora2pid
  pandoraShower
  pandoraTrack
  pandoraWriter
  pandoracalipid
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
22-Jan-2026 20:52:12 CET  Initiating request to open input file "prod_mpvmpr_protodunesp_1_299467_1_1769109158_gen_g4_stage1_detsim.root"
22-Jan-2026 20:52:14 CET  Opened input file "prod_mpvmpr_protodunesp_1_299467_1_1769109158_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 44510 traces, 44510 summary
Retagger: tagging trace set: gauss with 42150 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42150 traces tagged "gauss"
FrameSaver: q=3.34183e+06 n=655501 tag=gauss
wclsFrameSaver: saving 44510 traces tagged "wiener"
FrameSaver: q=3.5008e+06 n=592952 tag=wiener
22-Jan-2026 20:54:54 CET  Opened output file with pattern "prod_mpvmpr_protodunesp_1_299467_1_1769109158_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 34696 traces, 34696 summary
Retagger: tagging trace set: gauss with 32052 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 32052 traces tagged "gauss"
FrameSaver: q=1.87184e+06 n=425216 tag=gauss
wclsFrameSaver: saving 34696 traces tagged "wiener"
FrameSaver: q=2.02269e+06 n=371462 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 32498 traces, 32498 summary
Retagger: tagging trace set: gauss with 32013 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 32013 traces tagged "gauss"
FrameSaver: q=1.91838e+06 n=463778 tag=gauss
wclsFrameSaver: saving 32498 traces tagged "wiener"
FrameSaver: q=2.03229e+06 n=422593 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 50647 traces, 50647 summary
Retagger: tagging trace set: gauss with 43051 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43051 traces tagged "gauss"
FrameSaver: q=3.25902e+06 n=608750 tag=gauss
wclsFrameSaver: saving 50647 traces tagged "wiener"
FrameSaver: q=3.54703e+06 n=530390 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 49019 traces, 49019 summary
Retagger: tagging trace set: gauss with 42016 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42016 traces tagged "gauss"
FrameSaver: q=3.80606e+06 n=634949 tag=gauss
wclsFrameSaver: saving 49019 traces tagged "wiener"
FrameSaver: q=4.09203e+06 n=558079 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 51662 traces, 51662 summary
Retagger: tagging trace set: gauss with 44545 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44545 traces tagged "gauss"
FrameSaver: q=3.53654e+06 n=667617 tag=gauss
wclsFrameSaver: saving 51662 traces tagged "wiener"
FrameSaver: q=3.80462e+06 n=589569 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 39999 traces, 39999 summary
Retagger: tagging trace set: gauss with 39121 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 39121 traces tagged "gauss"
FrameSaver: q=3.17554e+06 n=614849 tag=gauss
wclsFrameSaver: saving 39999 traces tagged "wiener"
FrameSaver: q=3.47941e+06 n=570042 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36956 traces, 36956 summary
Retagger: tagging trace set: gauss with 34251 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34251 traces tagged "gauss"
FrameSaver: q=2.37442e+06 n=465311 tag=gauss
wclsFrameSaver: saving 36956 traces tagged "wiener"
FrameSaver: q=2.54209e+06 n=410461 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48888 traces, 48888 summary
Retagger: tagging trace set: gauss with 41469 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 41469 traces tagged "gauss"
FrameSaver: q=3.84976e+06 n=584445 tag=gauss
wclsFrameSaver: saving 48888 traces tagged "wiener"
FrameSaver: q=4.26618e+06 n=514164 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41592 traces, 41592 summary
Retagger: tagging trace set: gauss with 38002 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38002 traces tagged "gauss"
FrameSaver: q=3.10295e+06 n=574251 tag=gauss
wclsFrameSaver: saving 41592 traces tagged "wiener"
FrameSaver: q=3.34398e+06 n=513577 tag=wiener

ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
    "dune.campaign": "sp-protodune-spine-reco",
    "core.start_time": 1769109157.0,
    "core.end_time": 1769113417.0
  }
}
justIN time: 2026-02-04 16:16:46 UTC       justIN version: 01.06.00