justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273015.1@dunegpschedd01.fnal.gov

Jobsub ID273015.1@dunegpschedd01.fnal.gov
Workflow ID11585
Stage ID1
User namenivedpm@fnal.gov
RequestedProcessors4
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-09 16:54:29
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_dissel_multicore
Last heartbeat2026-01-09 19:58:41
From worker nodeHostnamewn-pijl-024.farm.nikhef.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors4
RSS bytes7864320000 (7500 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-09 16:55:41
Input filesmonte-carlo-011585-000010
JobscriptExit code0
Real time3h (10953s)
CPU time3h (10861s = 99%)
Max RSS bytes2555297792 (2436 MiB)
Outputting started2026-01-09 19:58:15
Output files
Finished2026-01-09 19:58:41
Saved logsjustin-logs:273015.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 46884 traces, 46884 summary
Retagger: tagging trace set: gauss with 43780 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43780 traces tagged "gauss"
FrameSaver: q=678770 n=553920 tag=gauss
wclsFrameSaver: saving 46884 traces tagged "wiener"
FrameSaver: q=941440 n=569312 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42217 traces, 42217 summary
Retagger: tagging trace set: gauss with 38935 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38935 traces tagged "gauss"
FrameSaver: q=596150 n=492367 tag=gauss
wclsFrameSaver: saving 42217 traces tagged "wiener"
FrameSaver: q=797901 n=522721 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41010 traces, 41010 summary
Retagger: tagging trace set: gauss with 39279 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 39279 traces tagged "gauss"
FrameSaver: q=481909 n=479168 tag=gauss
wclsFrameSaver: saving 41010 traces tagged "wiener"
FrameSaver: q=660035 n=505902 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 45484 traces, 45484 summary
Retagger: tagging trace set: gauss with 42477 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42477 traces tagged "gauss"
FrameSaver: q=612770 n=492895 tag=gauss
wclsFrameSaver: saving 45484 traces tagged "wiener"
FrameSaver: q=925834 n=521861 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42849 traces, 42849 summary
Retagger: tagging trace set: gauss with 40409 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 40409 traces tagged "gauss"
FrameSaver: q=648106 n=479941 tag=gauss
wclsFrameSaver: saving 42849 traces tagged "wiener"
FrameSaver: q=868569 n=503564 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48312 traces, 48312 summary
Retagger: tagging trace set: gauss with 43929 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43929 traces tagged "gauss"
FrameSaver: q=822098 n=578685 tag=gauss
wclsFrameSaver: saving 48312 traces tagged "wiener"
FrameSaver: q=1.03621e+06 n=585170 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48139 traces, 48139 summary
Retagger: tagging trace set: gauss with 43999 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43999 traces tagged "gauss"
FrameSaver: q=771789 n=586357 tag=gauss
wclsFrameSaver: saving 48139 traces tagged "wiener"
FrameSaver: q=967497 n=596037 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 46452 traces, 46452 summary
Retagger: tagging trace set: gauss with 42429 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42429 traces tagged "gauss"
FrameSaver: q=659241 n=551905 tag=gauss
wclsFrameSaver: saving 46452 traces tagged "wiener"
FrameSaver: q=862426 n=562995 tag=wiener

Error in processing reco
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  crtreco
  crttag
  emtrkmichelid
  hitpdune
  pandora
  pandora2GnocchiCalo
  pandora2GnocchiCalonosce
  pandora2Shower
  pandora2Showercalo
  pandora2Showercalonosce
  pandora2Track
  pandora2calo
  pandora2calonosce
  pandoraGnocchiCalo
  pandoraGnocchiCalonosce
  pandoraShower
  pandoraShowercalo
  pandoraShowercalonosce
  pandoraTrack
  pandoraWriter
  pandoracalo
  pandoracalonosce
  pandorapid
  reco3d
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_15_00d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25610 nodes/ 5537 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
09-Jan-2026 20:44:26 CET  Initiating request to open input file "prod_muon_400MeV_protodunevd_1_273015_1_1767977767_gen_g4_stage1_detsim.root"
09-Jan-2026 20:44:27 CET  Opened input file "prod_muon_400MeV_protodunevd_1_273015_1_1767977767_gen_g4_stage1_detsim.root"
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 44403 traces, 44403 summary
Retagger: tagging trace set: gauss with 41507 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 41507 traces tagged "gauss"
FrameSaver: q=679602 n=545620 tag=gauss
wclsFrameSaver: saving 44403 traces tagged "wiener"
FrameSaver: q=856246 n=558636 tag=wiener
09-Jan-2026 20:45:32 CET  Opened output file with pattern "prod_muon_400MeV_protodunevd_1_273015_1_1767977767_gen_g4_stage1_detsim_reco.root"
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 46884 traces, 46884 summary
Retagger: tagging trace set: gauss with 43780 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43780 traces tagged "gauss"
FrameSaver: q=678770 n=553920 tag=gauss
wclsFrameSaver: saving 46884 traces tagged "wiener"
FrameSaver: q=941440 n=569312 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42217 traces, 42217 summary
Retagger: tagging trace set: gauss with 38935 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38935 traces tagged "gauss"
FrameSaver: q=596150 n=492367 tag=gauss
wclsFrameSaver: saving 42217 traces tagged "wiener"
FrameSaver: q=797901 n=522721 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 41010 traces, 41010 summary
Retagger: tagging trace set: gauss with 39279 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 39279 traces tagged "gauss"
FrameSaver: q=481909 n=479168 tag=gauss
wclsFrameSaver: saving 41010 traces tagged "wiener"
FrameSaver: q=660035 n=505902 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 45484 traces, 45484 summary
Retagger: tagging trace set: gauss with 42477 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42477 traces tagged "gauss"
FrameSaver: q=612770 n=492895 tag=gauss
wclsFrameSaver: saving 45484 traces tagged "wiener"
FrameSaver: q=925834 n=521861 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42849 traces, 42849 summary
Retagger: tagging trace set: gauss with 40409 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 40409 traces tagged "gauss"
FrameSaver: q=648106 n=479941 tag=gauss
wclsFrameSaver: saving 42849 traces tagged "wiener"
FrameSaver: q=868569 n=503564 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48312 traces, 48312 summary
Retagger: tagging trace set: gauss with 43929 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43929 traces tagged "gauss"
FrameSaver: q=822098 n=578685 tag=gauss
wclsFrameSaver: saving 48312 traces tagged "wiener"
FrameSaver: q=1.03621e+06 n=585170 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48139 traces, 48139 summary
Retagger: tagging trace set: gauss with 43999 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 43999 traces tagged "gauss"
FrameSaver: q=771789 n=586357 tag=gauss
wclsFrameSaver: saving 48139 traces tagged "wiener"
FrameSaver: q=967497 n=596037 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 46452 traces, 46452 summary
Retagger: tagging trace set: gauss with 42429 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42429 traces tagged "gauss"
FrameSaver: q=659241 n=551905 tag=gauss
wclsFrameSaver: saving 46452 traces tagged "wiener"
FrameSaver: q=862426 n=562995 tag=wiener

ls: cannot access *_g4_stage2.root: No such file or directory
ls: cannot access *_reco.root: No such file or directory
Forming g4 metadata
Forming reco metadata
justIN time: 2026-02-06 15:46:46 UTC       justIN version: 01.06.00