justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303654.140@dunegpschedd01.fnal.gov

Jobsub ID303654.140@dunegpschedd01.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:52:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:17:55
From worker nodeHostnameheplnc148.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:09:46
Input filesvd-protodune:np02vd_raw_run042433_0190_df-s03-d1_dw_0_20260203T195421.hdf5
JobscriptExit code0
Real time1h (3941s)
CPU time58m (3517s = 89%)
Max RSS bytes3359752192 (3204 MiB)
Outputting started2026-02-04 01:15:29
Output files
Finished2026-02-04 01:17:55
Saved logsjustin-logs:303654.140-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

space point ptr... 0x6d9b58c0, 0x6d9b4820
Caught exception in looking up space point ptr... 0x6d9b4820, 0x6d9b58c0
Caught exception in looking up space point ptr... 0xf5aec10, 0x680aa0c0
Caught exception in looking up space point ptr... 0x680aa0c0, 0xf5aec10
Caught exception in looking up space point ptr... 0x438e41f0, 0x438e3fe0
Caught exception in looking up space point ptr... 0x438e3fe0, 0x438e41f0
Caught exception in looking up space point ptr... 0xe525370, 0xe513330
Caught exception in looking up space point ptr... 0xe513330, 0xe50ebd0
Caught exception in looking up space point ptr... 0xe50ebd0, 0xe513660
Caught exception in looking up space point ptr... 0x8c67e140, 0x8c68f810
Caught exception in looking up space point ptr... 0x8c68f280, 0x8c67e140
Caught exception in looking up space point ptr... 0x43962ce0, 0x43963190
Caught exception in looking up space point ptr... 0x43963190, 0x43962ce0
Caught exception in looking up space point ptr... 0x43962ce0, 0x43963000
Caught exception in looking up space point ptr... 0x43963000, 0x43962ce0
Caught exception in looking up space point ptr... 0x6d42e460, 0x6d42e260
Caught exception in looking up space point ptr... 0x6d42e260, 0x6d42e460
Caught exception in looking up space point ptr... 0x402d7d40, 0x402d8060
Caught exception in looking up space point ptr... 0x402d8060, 0x402d7d40
Caught exception in looking up space point ptr... 0x9dc4760, 0x9dd5740
Caught exception in looking up space point ptr... 0x9dd2b20, 0x9dc4760
Caught exception in looking up space point ptr... 0x3ec9f5d0, 0x15559980
Caught exception in looking up space point ptr... 0x15559980, 0x3ec9f5d0
Caught exception in looking up space point ptr... 0x3c369c90, 0x3c36a1e0
Caught exception in looking up space point ptr... 0x3c36a1e0, 0x3c369e70
Caught exception in looking up space point ptr... 0x433098d0, 0x43309780
Caught exception in looking up space point ptr... 0x43309780, 0x433098d0
Caught exception in looking up space point ptr... 0x9ef8fb0, 0x9ef43b0
Caught exception in looking up space point ptr... 0x9ef43b0, 0x9ef8670
Caught exception in looking up space point ptr... 0x3b854850, 0x3ec89d10
Caught exception in looking up space point ptr... 0x3ec89d10, 0x31ef75a0
Caught exception in looking up space point ptr... 0x439706d0, 0x46b32570
Caught exception in looking up space point ptr... 0x46b32570, 0x439706d0
Caught exception in looking up space point ptr... 0x3debbd30, 0x46edb2d0
Caught exception in looking up space point ptr... 0x46edb2d0, 0x3debbd30
Caught exception in looking up space point ptr... 0x2d9f1140, 0x2d9f12d0
Caught exception in looking up space point ptr... 0x2d9f12d0, 0x2d9f1140
Caught exception in looking up space point ptr... 0x390bc0f0, 0x390bb6c0
Caught exception in looking up space point ptr... 0x390bb6c0, 0x390bb440
Caught exception in looking up space point ptr... 0x37bd64c0, 0x37bd66c0
Caught exception in looking up space point ptr... 0x37bd66c0, 0x37bd64c0
Caught exception in looking up space point ptr... 0x1591de60, 0x159216b0
Caught exception in looking up space point ptr... 0x159216b0, 0x1591b940
Caught exception in looking up space point ptr... 0x402e8ef0, 0x55df2720
Caught exception in looking up space point ptr... 0x55df2720, 0x402e8ef0
Caught exception in looking up space point ptr... 0x402e8ef0, 0x57ff7210
Caught exception in looking up space point ptr... 0x57ff7210, 0x402e8ef0
Caught exception in looking up space point ptr... 0x65de97b0, 0x65de9520
Caught exception in looking up space point ptr... 0x65de9520, 0x65de97b0
Caught exception in looking up space point ptr... 0xf5e5d70, 0xf5d6d80
Caught exception in looking up space point ptr... 0xf5d6d80, 0xf5e5d70
Caught exception in looking up space point ptr... 0x32507b70, 0x6ee23b50
Caught exception in looking up space point ptr... 0x6ee23b50, 0x32507b70
Caught exception in looking up space point ptr... 0x84589450, 0x84594ca0
Caught exception in looking up space point ptr... 0x84594ca0, 0x84589450
Caught exception in looking up space point ptr... 0x2fdfafc0, 0x2fdfad00
Caught exception in looking up space point ptr... 0x2fdfad00, 0x2fdfafc0
Caught exception in looking up space point ptr... 0x580149a0, 0x58014cc0
Caught exception in looking up space point ptr... 0x58014cc0, 0x580149a0
Caught exception in looking up space point ptr... 0x19670bb0, 0x10cc2c10
Caught exception in looking up space point ptr... 0x10cc2c10, 0x19670bb0
Caught exception in looking up space point ptr... 0x55e668d0, 0x55e3d350
Caught exception in looking up space point ptr... 0x55e3d350, 0x55e668d0
Caught exception in looking up space point ptr... 0x32506e80, 0x2d826440
Caught exception in looking up space point ptr... 0x2d826440, 0x32506e80
Caught exception in looking up space point ptr... 0x189dee30, 0x189deca0
Caught exception in looking up space point ptr... 0x189deca0, 0x189df150
++++>>>> total num hits: 158892, num free: 153105
04-Feb-2026 01:15:19 GMT  Closed output file "np02vd_raw_run042433_0190_df-s03-d1_dw_0_20260203T195421_reco_stage0_20260204T011519_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         179.186       200.888       222.238       199.79        12.5691        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        9.04e-05     0.000146146   0.000483262   0.000119805   8.63579e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                15.1806       18.0789       20.1142       18.6745       1.22441        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.520157      0.552298      0.643143      0.541208      0.0317121       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175155     0.0187485     0.0335615     0.017961     0.00349895       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0173522     0.0178486     0.0181247     0.0179202    0.000201917      19     
produce:beamevent:BeamEvent                      0.000115147   0.000218439    0.0012151    0.000158449   0.000236376      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.57331       3.79012       4.04147       3.75946      0.140361        19     
produce:ophit:OpHitFinder                         0.0330247     0.0435034     0.0501199     0.0446569    0.00389835       19     
produce:wclsdatavd:WireCellToolkit                 131.205       141.267       151.584       140.055       7.02315        19     
produce:gaushit:GausHitFinder                      1.19484       1.8231        2.46321       1.84393      0.292278        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000386439   0.000564391   0.00101636    0.000551751   0.000139434      19     
produce:reco3d:SpacePointSolver                    14.4518       24.6786       37.0619       23.5131       6.74367        19     
produce:hitpdune:DisambigFromSpacePoints          0.126112      0.281823       0.4435       0.285016      0.0761519       19     
produce:cluster3d:Cluster3D                        2.00553       3.51698       5.39609       3.35819       1.04973        19     
[art]:TriggerResults:TriggerResultInserter       2.9084e-05    4.4322e-05    0.000141186   3.8643e-05    2.35924e-05      19     
end_path:out1:RootOutput                          7.043e-06    1.21033e-05   3.5737e-05     1.06e-05     6.36864e-06      19     
end_path:out1:RootOutput(write)                    5.20092       6.77313       7.91504       7.05932       0.74496        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5353.14 MB
  Peak resident set size usage (VmHWM): 3359.75 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0190_df-s03-d1_dw_0_20260203T195421_reco_stage0_20260204T011519_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0190_df-s03-d1_dw_0_20260203T195421_reco_stage0_20260204T011519_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0190_df-s03-d1_dw_0_20260203T195421_reco_stage0_20260204T011519_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163808.0,
    "core.end_time": 1770167721.0,
    "core.events": [
      21669,
      21675,
      21681,
      21687,
      21693,
      21699,
      21705,
      21711,
      21717,
      21723,
      21729,
      21735,
      21741,
      21747,
      21753,
      21759,
      21765,
      21771,
      21777
    ],
    "core.event_count": 19,
    "core.first_event_number": 21669,
    "core.last_event_number": 21777,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0190_df-s03-d1_dw_0_20260203T195421.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:37:41 UTC       justIN version: 01.06.00