justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272899.0@dunegpschedd02.fnal.gov

Jobsub ID272899.0@dunegpschedd02.fnal.gov
Workflow ID11943
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-15 02:59:22
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_klomp_multicore
Last heartbeat2026-01-15 03:35:38
From worker nodeHostnamewn-pijl-030.farm.nikhef.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-15 03:00:56
Input filesmonte-carlo-011943-000353
JobscriptExit code0
Real time34m (2056s)
CPU time34m (2053s = 99%)
Max RSS bytes1917288448 (1828 MiB)
Outputting started2026-01-15 03:35:14
Output files
Finished2026-01-15 03:35:38
Saved logsjustin-logs:272899.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gauss"
FrameSaver: q=2.77416e+06 n=245012 tag=gauss
wclsFrameSaver: saving 19238 traces tagged "wiener"
FrameSaver: q=2.97694e+06 n=227388 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 19813 traces, 19813 summary
Retagger: tagging trace set: gauss with 15533 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 15533 traces tagged "gauss"
FrameSaver: q=3.1171e+06 n=256088 tag=gauss
wclsFrameSaver: saving 19813 traces tagged "wiener"
FrameSaver: q=3.32515e+06 n=234293 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodunevd_v5_ggd.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16484 nodes/ 2469 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
15-Jan-2026 04:19:43 CET  Initiating request to open input file "prod_muon_400MeV_protodunevd_1_272899_0_1768446068_gen_g4_stage1_g4_stage2_detsim.root"
15-Jan-2026 04:19:44 CET  Opened input file "prod_muon_400MeV_protodunevd_1_272899_0_1768446068_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 19913 traces, 19913 summary
Retagger: tagging trace set: gauss with 15058 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 15058 traces tagged "gauss"
FrameSaver: q=4.12676e+06 n=269402 tag=gauss
wclsFrameSaver: saving 19913 traces tagged "wiener"
FrameSaver: q=4.38672e+06 n=246212 tag=wiener
15-Jan-2026 04:20:53 CET  Opened output file with pattern "prod_muon_400MeV_protodunevd_1_272899_0_1768446068_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 14502 traces, 14502 summary
Retagger: tagging trace set: gauss with 11711 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 11711 traces tagged "gauss"
FrameSaver: q=3.00637e+06 n=192363 tag=gauss
wclsFrameSaver: saving 14502 traces tagged "wiener"
FrameSaver: q=3.2633e+06 n=179884 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 16780 traces, 16780 summary
Retagger: tagging trace set: gauss with 13877 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 13877 traces tagged "gauss"
FrameSaver: q=2.38531e+06 n=220813 tag=gauss
wclsFrameSaver: saving 16780 traces tagged "wiener"
FrameSaver: q=2.5347e+06 n=205639 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 14569 traces, 14569 summary
Retagger: tagging trace set: gauss with 11803 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 11803 traces tagged "gauss"
FrameSaver: q=1.80441e+06 n=177409 tag=gauss
wclsFrameSaver: saving 14569 traces tagged "wiener"
FrameSaver: q=1.95549e+06 n=162328 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 18021 traces, 18021 summary
Retagger: tagging trace set: gauss with 14522 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 14522 traces tagged "gauss"
FrameSaver: q=2.2579e+06 n=221891 tag=gauss
wclsFrameSaver: saving 18021 traces tagged "wiener"
FrameSaver: q=2.40852e+06 n=202168 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 29106 traces, 29106 summary
Retagger: tagging trace set: gauss with 21423 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 21423 traces tagged "gauss"
FrameSaver: q=4.7262e+06 n=356596 tag=gauss
wclsFrameSaver: saving 29106 traces tagged "wiener"
FrameSaver: q=5.08596e+06 n=322845 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 14755 traces, 14755 summary
Retagger: tagging trace set: gauss with 12290 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 12290 traces tagged "gauss"
FrameSaver: q=2.24237e+06 n=214986 tag=gauss
wclsFrameSaver: saving 14755 traces tagged "wiener"
FrameSaver: q=2.36961e+06 n=204553 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 23165 traces, 23165 summary
Retagger: tagging trace set: gauss with 17541 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 17541 traces tagged "gauss"
FrameSaver: q=2.94425e+06 n=254164 tag=gauss
wclsFrameSaver: saving 23165 traces tagged "wiener"
FrameSaver: q=3.16657e+06 n=222550 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 25003 traces, 25003 summary
Retagger: tagging trace set: gauss with 19007 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 19007 traces tagged "gauss"
FrameSaver: q=4.507e+06 n=312277 tag=gauss
wclsFrameSaver: saving 25003 traces tagged "wiener"
FrameSaver: q=4.75642e+06 n=280836 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 28652 traces, 28652 summary
Retagger: tagging trace set: gauss with 21655 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 21655 traces tagged "gauss"
FrameSaver: q=4.34671e+06 n=331881 tag=gauss
wclsFrameSaver: saving 28652 traces tagged "wiener"
FrameSaver: q=4.68267e+06 n=296443 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 28107 traces, 28107 summary
Retagger: tagging trace set: gauss with 21225 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 21225 traces tagged "gauss"
FrameSaver: q=4.13787e+06 n=343873 tag=gauss
wclsFrameSaver: saving 28107 traces tagged "wiener"
FrameSaver: q=4.49594e+06 n=316194 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 17917 traces, 17917 summary
Retagger: tagging trace set: gauss with 15476 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 15476 traces tagged "gauss"
FrameSaver: q=2.65234e+06 n=257004 tag=gauss
wclsFrameSaver: saving 17917 traces tagged "wiener"
FrameSaver: q=2.78061e+06 n=244376 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 13342 traces, 13342 summary
Retagger: tagging trace set: gauss with 11899 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 11899 traces tagged "gauss"
FrameSaver: q=2.00742e+06 n=203118 tag=gauss
wclsFrameSaver: saving 13342 traces tagged "wiener"
FrameSaver: q=2.09695e+06 n=197145 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 19238 traces, 19238 summary
Retagger: tagging trace set: gauss with 14523 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 14523 traces tagged "gauss"
FrameSaver: q=2.77416e+06 n=245012 tag=gauss
wclsFrameSaver: saving 19238 traces tagged "wiener"
FrameSaver: q=2.97694e+06 n=227388 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 19813 traces, 19813 summary
Retagger: tagging trace set: gauss with 15533 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 15533 traces tagged "gauss"
FrameSaver: q=3.1171e+06 n=256088 tag=gauss
wclsFrameSaver: saving 19813 traces tagged "wiener"
FrameSaver: q=3.32515e+06 n=234293 tag=wiener

ls: cannot access *larcvpdvd*.root: No such file or directory
Output files:
 larcv files: 
formed
{
  "name": "",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_16_00d00",
    "core.data_tier": "root-tuple-virtual",
    "core.runs": null,
    "core.runs_subruns": null,
    "dune.config_file": "run_supera_pdunevd_mpvmpr_doublets.fcl",
    "dune.campaign": "vd-protodune-spine-reco",
    "core.start_time": 1768446067.0,
    "core.end_time": 1768448110.0
  }
}
justIN time: 2026-02-04 09:14:18 UTC       justIN version: 01.06.00