justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293907.101@dunegpschedd02.fnal.gov

Jobsub ID293907.101@dunegpschedd02.fnal.gov
Workflow ID12286
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-23 00:47:41
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_26
Last heartbeat2026-01-23 05:00:30
From worker nodeHostnamewn-a4-10
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-23 04:05:02
Input filesmonte-carlo-012286-000961
JobscriptExit code0
Real time54m (3285s)
CPU time52m (3173s = 96%)
Max RSS bytes2432094208 (2319 MiB)
Outputting started2026-01-23 04:59:49
Output files
Finished2026-01-23 05:00:30
Saved logsjustin-logs:293907.101-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gged "wiener"
FrameSaver: q=2.44158e+06 n=390399 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 44154 traces, 44154 summary
Retagger: tagging trace set: gauss with 38948 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38948 traces tagged "gauss"
FrameSaver: q=2.6856e+06 n=551004 tag=gauss
wclsFrameSaver: saving 44154 traces tagged "wiener"
FrameSaver: q=2.91835e+06 n=484848 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 35796 traces, 35796 summary
Retagger: tagging trace set: gauss with 31828 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31828 traces tagged "gauss"
FrameSaver: q=1.9481e+06 n=409572 tag=gauss
wclsFrameSaver: saving 35796 traces tagged "wiener"
FrameSaver: q=2.15489e+06 n=353408 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36572 traces, 36572 summary
Retagger: tagging trace set: gauss with 35328 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 35328 traces tagged "gauss"
FrameSaver: q=2.56244e+06 n=543576 tag=gauss
wclsFrameSaver: saving 36572 traces tagged "wiener"
FrameSaver: q=2.73766e+06 n=497848 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 30824 traces, 30824 summary
Retagger: tagging trace set: gauss with 30341 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 30341 traces tagged "gauss"
FrameSaver: q=1.7017e+06 n=409224 tag=gauss
wclsFrameSaver: saving 30824 traces tagged "wiener"
FrameSaver: q=1.829e+06 n=367006 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 32906 traces, 32906 summary
Retagger: tagging trace set: gauss with 31794 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31794 traces tagged "gauss"
FrameSaver: q=1.98989e+06 n=436478 tag=gauss
wclsFrameSaver: saving 32906 traces tagged "wiener"
FrameSaver: q=2.13691e+06 n=389325 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  hitpdune
  pandora
  pandora2Shower
  pandora2Track
  pandora2cali
  pandora2calipid
  pandora2pid
  pandoraShower
  pandoraTrack
  pandoraWriter
  pandoracalipid
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
23-Jan-2026 04:38:11 GMT  Initiating request to open input file "prod_mpvmpr_protodunesp_1_293907_101_1769141118_gen_g4_stage1_detsim.root"
23-Jan-2026 04:38:12 GMT  Opened input file "prod_mpvmpr_protodunesp_1_293907_101_1769141118_gen_g4_stage1_detsim.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 33070 traces, 33070 summary
Retagger: tagging trace set: gauss with 31926 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31926 traces tagged "gauss"
FrameSaver: q=1.91535e+06 n=445335 tag=gauss
wclsFrameSaver: saving 33070 traces tagged "wiener"
FrameSaver: q=2.05223e+06 n=399308 tag=wiener
23-Jan-2026 04:39:55 GMT  Opened output file with pattern "prod_mpvmpr_protodunesp_1_293907_101_1769141118_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 45197 traces, 45197 summary
Retagger: tagging trace set: gauss with 40622 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 40622 traces tagged "gauss"
FrameSaver: q=3.68221e+06 n=593072 tag=gauss
wclsFrameSaver: saving 45197 traces tagged "wiener"
FrameSaver: q=3.97822e+06 n=522375 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 34563 traces, 34563 summary
Retagger: tagging trace set: gauss with 34100 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34100 traces tagged "gauss"
FrameSaver: q=2.25303e+06 n=501311 tag=gauss
wclsFrameSaver: saving 34563 traces tagged "wiener"
FrameSaver: q=2.38258e+06 n=455065 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 35345 traces, 35345 summary
Retagger: tagging trace set: gauss with 33701 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 33701 traces tagged "gauss"
FrameSaver: q=2.22718e+06 n=466698 tag=gauss
wclsFrameSaver: saving 35345 traces tagged "wiener"
FrameSaver: q=2.39908e+06 n=417780 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 47028 traces, 47028 summary
Retagger: tagging trace set: gauss with 40460 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 40460 traces tagged "gauss"
FrameSaver: q=3.14786e+06 n=552690 tag=gauss
wclsFrameSaver: saving 47028 traces tagged "wiener"
FrameSaver: q=3.5318e+06 n=483517 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36646 traces, 36646 summary
Retagger: tagging trace set: gauss with 33809 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 33809 traces tagged "gauss"
FrameSaver: q=2.23231e+06 n=446082 tag=gauss
wclsFrameSaver: saving 36646 traces tagged "wiener"
FrameSaver: q=2.44158e+06 n=390399 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 44154 traces, 44154 summary
Retagger: tagging trace set: gauss with 38948 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38948 traces tagged "gauss"
FrameSaver: q=2.6856e+06 n=551004 tag=gauss
wclsFrameSaver: saving 44154 traces tagged "wiener"
FrameSaver: q=2.91835e+06 n=484848 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 35796 traces, 35796 summary
Retagger: tagging trace set: gauss with 31828 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31828 traces tagged "gauss"
FrameSaver: q=1.9481e+06 n=409572 tag=gauss
wclsFrameSaver: saving 35796 traces tagged "wiener"
FrameSaver: q=2.15489e+06 n=353408 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36572 traces, 36572 summary
Retagger: tagging trace set: gauss with 35328 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 35328 traces tagged "gauss"
FrameSaver: q=2.56244e+06 n=543576 tag=gauss
wclsFrameSaver: saving 36572 traces tagged "wiener"
FrameSaver: q=2.73766e+06 n=497848 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 30824 traces, 30824 summary
Retagger: tagging trace set: gauss with 30341 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 30341 traces tagged "gauss"
FrameSaver: q=1.7017e+06 n=409224 tag=gauss
wclsFrameSaver: saving 30824 traces tagged "wiener"
FrameSaver: q=1.829e+06 n=367006 tag=wiener
RawFrameSource: got 15360 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 32906 traces, 32906 summary
Retagger: tagging trace set: gauss with 31794 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31794 traces tagged "gauss"
FrameSaver: q=1.98989e+06 n=436478 tag=gauss
wclsFrameSaver: saving 32906 traces tagged "wiener"
FrameSaver: q=2.13691e+06 n=389325 tag=wiener

ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
    "dune.campaign": "sp-protodune-spine-reco",
    "core.start_time": 1769141117.0,
    "core.end_time": 1769144383.0
  }
}
justIN time: 2026-02-04 04:29:25 UTC       justIN version: 01.06.00