justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.19@dunegpschedd02.fnal.gov

Jobsub ID297921.19@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:39:45
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:21
Input filesvd-protodune:np02vd_raw_run042433_0463_df-s04-d0_dw_0_20260203T213806.hdf5
JobscriptExit code0
Real time52m (3122s)
CPU time47m (2874s = 92%)
Max RSS bytes3349155840 (3194 MiB)
Outputting started2026-02-04 00:37:24
Output files
Finished2026-02-04 00:39:45
Saved logsjustin-logs:297921.19-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x58e091a0, 0x576fa2f0
Caught exception in looking up space point ptr... 0x576fa2f0, 0x34cf9eb0
Caught exception in looking up space point ptr... 0x34cf9eb0, 0x34cfbdc0
Caught exception in looking up space point ptr... 0x576f37a0, 0x34cfa710
Caught exception in looking up space point ptr... 0x34cfa710, 0x576f3cf0
Caught exception in looking up space point ptr... 0x576f3cf0, 0x58e091a0
Caught exception in looking up space point ptr... 0x467f1260, 0xf72b180
Caught exception in looking up space point ptr... 0xf72b180, 0x522cc870
Caught exception in looking up space point ptr... 0x522cc870, 0x646a4a50
Caught exception in looking up space point ptr... 0x3c3dfed0, 0x3c3df030
Caught exception in looking up space point ptr... 0x3c3df030, 0x3c3def90
Caught exception in looking up space point ptr... 0x43b890c0, 0x5ac42b60
Caught exception in looking up space point ptr... 0x5ac42b60, 0x43b890c0
Caught exception in looking up space point ptr... 0x2d2f5de0, 0x6e0a8ea0
Caught exception in looking up space point ptr... 0x2d2f68c0, 0x2d2f5de0
Caught exception in looking up space point ptr... 0x2ca9aca0, 0x75a77f30
Caught exception in looking up space point ptr... 0x2ca9b190, 0x2ca9aca0
Caught exception in looking up space point ptr... 0x4997ec30, 0x15f885f0
Caught exception in looking up space point ptr... 0x15f885f0, 0x4997ec30
Caught exception in looking up space point ptr... 0x4997ec30, 0x382c85d0
Caught exception in looking up space point ptr... 0x382c85d0, 0x4997ec30
Caught exception in looking up space point ptr... 0x4997ec30, 0xf563210
Caught exception in looking up space point ptr... 0xf563210, 0x4997ec30
Caught exception in looking up space point ptr... 0x7ee09de0, 0x7ee0a180
Caught exception in looking up space point ptr... 0x89e2460, 0x7ee09de0
Caught exception in looking up space point ptr... 0x9394990, 0x9394b90
Caught exception in looking up space point ptr... 0x9394b90, 0x9394990
Caught exception in looking up space point ptr... 0x5f45fde0, 0x387d71f0
Caught exception in looking up space point ptr... 0x387d6e00, 0x5f45fde0
Caught exception in looking up space point ptr... 0x66316390, 0x31334b90
Caught exception in looking up space point ptr... 0x31334b90, 0x66314f60
Caught exception in looking up space point ptr... 0x3687c520, 0x57d15550
Caught exception in looking up space point ptr... 0x57d15550, 0x6b7b59f0
Caught exception in looking up space point ptr... 0x46bdb140, 0x3c5a62c0
Caught exception in looking up space point ptr... 0x3c5a62c0, 0x64327180
Caught exception in looking up space point ptr... 0x10595fa0, 0x46bdb140
Caught exception in looking up space point ptr... 0x35232010, 0x480a1390
Caught exception in looking up space point ptr... 0x480a1390, 0x35232010
Caught exception in looking up space point ptr... 0x35232010, 0x480a10a0
Caught exception in looking up space point ptr... 0x480a10a0, 0x35232010
Caught exception in looking up space point ptr... 0x668c2990, 0x668c28f0
Caught exception in looking up space point ptr... 0x668c28f0, 0x3ac78940
Caught exception in looking up space point ptr... 0x61711bf0, 0x7ee0b5f0
Caught exception in looking up space point ptr... 0x7ee0b5f0, 0x7ee0b4f0
Caught exception in looking up space point ptr... 0x61711df0, 0x61711bf0
Caught exception in looking up space point ptr... 0x3571f5e0, 0x3d5e9410
Caught exception in looking up space point ptr... 0x3d5e9410, 0x2e6989d0
Caught exception in looking up space point ptr... 0x173b16e0, 0x173b3860
Caught exception in looking up space point ptr... 0x173b3860, 0x173b16e0
Caught exception in looking up space point ptr... 0x5975e360, 0x642d1620
Caught exception in looking up space point ptr... 0x665f43e0, 0x5975e360
Caught exception in looking up space point ptr... 0x193a0ec0, 0x193a0a10
Caught exception in looking up space point ptr... 0x193a0a10, 0x193a0ba0
Caught exception in looking up space point ptr... 0x655c9fc0, 0x655c9b00
Caught exception in looking up space point ptr... 0x655c9b00, 0x655c9e90
Caught exception in looking up space point ptr... 0xa150540, 0x52830010
Caught exception in looking up space point ptr... 0x52830010, 0xa150670
Caught exception in looking up space point ptr... 0x514cc5d0, 0x5944a8b0
Caught exception in looking up space point ptr... 0x5944a8b0, 0x736a2fe0
Caught exception in looking up space point ptr... 0x76ae5700, 0x76ae63d0
Caught exception in looking up space point ptr... 0x76ae63d0, 0x76ae5830
Caught exception in looking up space point ptr... 0x6b561b70, 0x2e203200
Caught exception in looking up space point ptr... 0x2e203200, 0x6b561b70
Caught exception in looking up space point ptr... 0x18760b90, 0x6b1743a0
Caught exception in looking up space point ptr... 0x6b1743a0, 0x18760b90
Caught exception in looking up space point ptr... 0x3adb1420, 0x65781e50
Caught exception in looking up space point ptr... 0x65781e50, 0x3adb1420
++++>>>> total num hits: 56242, num free: 54182
04-Feb-2026 00:37:16 GMT  Closed output file "np02vd_raw_run042433_0463_df-s04-d0_dw_0_20260203T213806_reco_stage0_20260204T003716_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         150.054       160.343       175.926       160.919       6.83082        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.6821e-05    0.000123533   0.000229283   0.000104531   4.18118e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.1643       13.7328       14.2263       13.7899      0.284637        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.524113      0.539921      0.565568      0.540193      0.0122504       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0176302     0.0194915     0.0380635     0.0179494    0.00482811       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176292     0.017889      0.0182687     0.0178342    0.000168548      19     
produce:beamevent:BeamEvent                      0.000119012   0.000226571   0.00109694    0.000189802   0.000207804      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.59466       3.22209       3.69187       3.25285      0.265532        19     
produce:ophit:OpHitFinder                         0.0349414     0.041416      0.045596      0.0413711    0.00261997       19     
produce:wclsdatavd:WireCellToolkit                 109.964       118.506       125.848       120.239       4.92553        19     
produce:gaushit:GausHitFinder                     0.881681       1.41083       2.06806       1.38134      0.338571        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000201223   0.00032202    0.000640609   0.000313125   9.97889e-05      19     
produce:reco3d:SpacePointSolver                    10.2355       14.2836       19.4717       15.0951       2.41166        19     
produce:hitpdune:DisambigFromSpacePoints          0.145773      0.219027      0.342509      0.204728      0.0566679       19     
produce:cluster3d:Cluster3D                        1.56287       2.73151       4.4841        2.56935      0.815074        19     
[art]:TriggerResults:TriggerResultInserter       3.3201e-05    4.55587e-05   0.000102211    4.042e-05    1.54822e-05      19     
end_path:out1:RootOutput                          7.83e-06     1.08733e-05    2.923e-05     9.77e-06     4.43417e-06      19     
end_path:out1:RootOutput(write)                    4.22335       5.55687       6.49481       5.84036      0.688773        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5348.21 MB
  Peak resident set size usage (VmHWM): 3349.16 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0463_df-s04-d0_dw_0_20260203T213806_reco_stage0_20260204T003716_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0463_df-s04-d0_dw_0_20260203T213806_reco_stage0_20260204T003716_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0463_df-s04-d0_dw_0_20260203T213806_reco_stage0_20260204T003716_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162334.0,
    "core.end_time": 1770165438.0,
    "core.events": [
      52792,
      52798,
      52804,
      52810,
      52816,
      52822,
      52828,
      52834,
      52840,
      52846,
      52852,
      52858,
      52864,
      52870,
      52876,
      52882,
      52888,
      52894,
      52900
    ],
    "core.event_count": 19,
    "core.first_event_number": 52792,
    "core.last_event_number": 52900,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0463_df-s04-d0_dw_0_20260203T213806.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:38 UTC       justIN version: 01.06.00