justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 299904.136@dunegpschedd01.fnal.gov

Jobsub ID299904.136@dunegpschedd01.fnal.gov
Workflow ID12308
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-23 11:52:23
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_26
Last heartbeat2026-01-23 14:24:31
From worker nodeHostnamewn-a4-16
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-23 13:13:04
Input filesmonte-carlo-012308-000520
JobscriptExit code0
Real time1h (4058s)
CPU time1h (4172s = 102%)
Max RSS bytes16853426176 (16072 MiB)
Outputting started2026-01-23 14:20:47
Output files
Finished2026-01-23 14:24:31
Saved logsjustin-logs:299904.136-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ener"
FrameSaver: q=3.28942e+06 n=612289 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41308 traces, 41308 summary
Retagger: tagging trace set: gauss with 36226 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 36226 traces tagged "gauss"
FrameSaver: q=2.9626e+06 n=514345 tag=gauss
wclsFrameSaver: saving 41308 traces tagged "wiener"
FrameSaver: q=3.26818e+06 n=451990 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48619 traces, 48619 summary
Retagger: tagging trace set: gauss with 44293 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44293 traces tagged "gauss"
FrameSaver: q=3.82682e+06 n=744552 tag=gauss
wclsFrameSaver: saving 48619 traces tagged "wiener"
FrameSaver: q=4.06022e+06 n=681100 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  hitpdune
  pandora
  pandora2Shower
  pandora2Track
  pandora2cali
  pandora2calipid
  pandora2pid
  pandoraShower
  pandoraTrack
  pandoraWriter
  pandoracalipid
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
23-Jan-2026 13:47:15 GMT  Initiating request to open input file "prod_mpvmpr_protodunesp_1_299904_136_1769173998_gen_g4_stage1_detsim.root"
23-Jan-2026 13:47:15 GMT  Opened input file "prod_mpvmpr_protodunesp_1_299904_136_1769173998_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 44188 traces, 44188 summary
Retagger: tagging trace set: gauss with 39320 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 39320 traces tagged "gauss"
FrameSaver: q=2.89573e+06 n=573145 tag=gauss
wclsFrameSaver: saving 44188 traces tagged "wiener"
FrameSaver: q=3.08822e+06 n=507570 tag=wiener
23-Jan-2026 13:48:59 GMT  Opened output file with pattern "prod_mpvmpr_protodunesp_1_299904_136_1769173998_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 29741 traces, 29741 summary
Retagger: tagging trace set: gauss with 30023 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 30023 traces tagged "gauss"
FrameSaver: q=1.88866e+06 n=423004 tag=gauss
wclsFrameSaver: saving 29741 traces tagged "wiener"
FrameSaver: q=2.00222e+06 n=385615 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 43362 traces, 43362 summary
Retagger: tagging trace set: gauss with 38440 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38440 traces tagged "gauss"
FrameSaver: q=3.06074e+06 n=570288 tag=gauss
wclsFrameSaver: saving 43362 traces tagged "wiener"
FrameSaver: q=3.31619e+06 n=506547 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36860 traces, 36860 summary
Retagger: tagging trace set: gauss with 33989 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 33989 traces tagged "gauss"
FrameSaver: q=2.18506e+06 n=458803 tag=gauss
wclsFrameSaver: saving 36860 traces tagged "wiener"
FrameSaver: q=2.34839e+06 n=401487 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41431 traces, 41431 summary
Retagger: tagging trace set: gauss with 37543 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 37543 traces tagged "gauss"
FrameSaver: q=2.68276e+06 n=537697 tag=gauss
wclsFrameSaver: saving 41431 traces tagged "wiener"
FrameSaver: q=2.88849e+06 n=473420 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 47208 traces, 47208 summary
Retagger: tagging trace set: gauss with 40956 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 40956 traces tagged "gauss"
FrameSaver: q=3.50414e+06 n=611775 tag=gauss
wclsFrameSaver: saving 47208 traces tagged "wiener"
FrameSaver: q=3.78709e+06 n=539769 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 37340 traces, 37340 summary
Retagger: tagging trace set: gauss with 36007 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 36007 traces tagged "gauss"
FrameSaver: q=2.92153e+06 n=541097 tag=gauss
wclsFrameSaver: saving 37340 traces tagged "wiener"
FrameSaver: q=3.10804e+06 n=492228 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 43017 traces, 43017 summary
Retagger: tagging trace set: gauss with 38181 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38181 traces tagged "gauss"
FrameSaver: q=2.50883e+06 n=553037 tag=gauss
wclsFrameSaver: saving 43017 traces tagged "wiener"
FrameSaver: q=2.68286e+06 n=482892 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41029 traces, 41029 summary
Retagger: tagging trace set: gauss with 37799 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 37799 traces tagged "gauss"
FrameSaver: q=3.43635e+06 n=560133 tag=gauss
wclsFrameSaver: saving 41029 traces tagged "wiener"
FrameSaver: q=3.84251e+06 n=502212 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36135 traces, 36135 summary
Retagger: tagging trace set: gauss with 34295 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34295 traces tagged "gauss"
FrameSaver: q=2.62571e+06 n=514551 tag=gauss
wclsFrameSaver: saving 36135 traces tagged "wiener"
FrameSaver: q=2.83515e+06 n=467580 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 37811 traces, 37811 summary
Retagger: tagging trace set: gauss with 35238 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 35238 traces tagged "gauss"
FrameSaver: q=2.5672e+06 n=523568 tag=gauss
wclsFrameSaver: saving 37811 traces tagged "wiener"
FrameSaver: q=2.79315e+06 n=477873 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 40164 traces, 40164 summary
Retagger: tagging trace set: gauss with 39716 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 39716 traces tagged "gauss"
FrameSaver: q=3.13193e+06 n=658162 tag=gauss
wclsFrameSaver: saving 40164 traces tagged "wiener"
FrameSaver: q=3.28942e+06 n=612289 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41308 traces, 41308 summary
Retagger: tagging trace set: gauss with 36226 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 36226 traces tagged "gauss"
FrameSaver: q=2.9626e+06 n=514345 tag=gauss
wclsFrameSaver: saving 41308 traces tagged "wiener"
FrameSaver: q=3.26818e+06 n=451990 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48619 traces, 48619 summary
Retagger: tagging trace set: gauss with 44293 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 44293 traces tagged "gauss"
FrameSaver: q=3.82682e+06 n=744552 tag=gauss
wclsFrameSaver: saving 48619 traces tagged "wiener"
FrameSaver: q=4.06022e+06 n=681100 tag=wiener

ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
    "dune.campaign": "sp-protodune-spine-reco",
    "core.start_time": 1769173997.0,
    "core.end_time": 1769178038.0
  }
}
justIN time: 2026-02-04 04:28:49 UTC       justIN version: 01.06.00