justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.26@dunegpschedd02.fnal.gov

Jobsub ID297921.26@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:39:56
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:03
Input filesvd-protodune:np02vd_raw_run042433_0076_df-s02-d1_dw_0_20260203T191100.hdf5
JobscriptExit code0
Real time52m (3148s)
CPU time48m (2888s = 91%)
Max RSS bytes3340795904 (3186 MiB)
Outputting started2026-02-04 00:37:33
Output files
Finished2026-02-04 00:39:56
Saved logsjustin-logs:297921.26-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

in looking up space point ptr... 0x4f9ea010, 0x4f9ef420
Caught exception in looking up space point ptr... 0x33765420, 0x58f682e0
Caught exception in looking up space point ptr... 0x58f682e0, 0x33765420
Caught exception in looking up space point ptr... 0x109f9db0, 0x109f9630
Caught exception in looking up space point ptr... 0x109f9630, 0x109f9db0
Caught exception in looking up space point ptr... 0x19382fb0, 0x1937ffe0
Caught exception in looking up space point ptr... 0x1937ffe0, 0x19382b30
Caught exception in looking up space point ptr... 0x78be6a20, 0x78be6e10
Caught exception in looking up space point ptr... 0x78be6e10, 0x78be6a20
Caught exception in looking up space point ptr... 0x78be6a20, 0x78be6cc0
Caught exception in looking up space point ptr... 0x78be6cc0, 0x78be6a20
Caught exception in looking up space point ptr... 0x134c6ee0, 0x134c5560
Caught exception in looking up space point ptr... 0x134c5560, 0x134c6ee0
Caught exception in looking up space point ptr... 0x3749f950, 0x374a12d0
Caught exception in looking up space point ptr... 0x374a12d0, 0x374a06a0
Caught exception in looking up space point ptr... 0x18bd85b0, 0x18bd8290
Caught exception in looking up space point ptr... 0x18bd8290, 0x18bd85b0
Caught exception in looking up space point ptr... 0x374a1070, 0x374a07b0
Caught exception in looking up space point ptr... 0x374a07b0, 0x374a37f0
Caught exception in looking up space point ptr... 0x145ead90, 0x55db36c0
Caught exception in looking up space point ptr... 0x55db36c0, 0x55d9ac00
Caught exception in looking up space point ptr... 0xe865bc0, 0xe8659e0
Caught exception in looking up space point ptr... 0xe8659e0, 0xe865bc0
Caught exception in looking up space point ptr... 0x71cba240, 0x6e1cec00
Caught exception in looking up space point ptr... 0x6e1cec00, 0x71cba240
Caught exception in looking up space point ptr... 0x71cba240, 0x331d51a0
Caught exception in looking up space point ptr... 0x331d51a0, 0x71cba240
Caught exception in looking up space point ptr... 0x18270780, 0x1826feb0
Caught exception in looking up space point ptr... 0x1826feb0, 0x1826f860
Caught exception in looking up space point ptr... 0x661ef2b0, 0x11ab9a90
Caught exception in looking up space point ptr... 0x11ab9a90, 0xe0b6120
Caught exception in looking up space point ptr... 0x3cb4b0d0, 0x3cb4c440
Caught exception in looking up space point ptr... 0x3cb4c440, 0x3cb4b0d0
Caught exception in looking up space point ptr... 0x3cb4b0d0, 0x3cb4af30
Caught exception in looking up space point ptr... 0x3cb4af30, 0x3cb4b0d0
Caught exception in looking up space point ptr... 0x846be120, 0x846be490
Caught exception in looking up space point ptr... 0x846be490, 0x846be120
Caught exception in looking up space point ptr... 0x6df998f0, 0x770c5fc0
Caught exception in looking up space point ptr... 0x770c5fc0, 0x6d641b0
Caught exception in looking up space point ptr... 0x67bf8750, 0x67bf8600
Caught exception in looking up space point ptr... 0x67bf8600, 0x67bf8750
Caught exception in looking up space point ptr... 0x58f8a1a0, 0x58f8b540
Caught exception in looking up space point ptr... 0x58f8b540, 0x58f8a1a0
Caught exception in looking up space point ptr... 0x3ae6b5b0, 0x3ae6b100
Caught exception in looking up space point ptr... 0x3ae6b100, 0x3ae6b5b0
Caught exception in looking up space point ptr... 0x15d20d90, 0x15d36d90
Caught exception in looking up space point ptr... 0x15d36d90, 0xf51ee00
Caught exception in looking up space point ptr... 0xe09acc0, 0xe09b030
Caught exception in looking up space point ptr... 0xe09b030, 0xe09acc0
Caught exception in looking up space point ptr... 0x18be6a80, 0x18be6bd0
Caught exception in looking up space point ptr... 0x18be6bd0, 0x18be6a80
Caught exception in looking up space point ptr... 0x82af3e50, 0x82af3d10
Caught exception in looking up space point ptr... 0x82af3d10, 0x82af3c70
Caught exception in looking up space point ptr... 0x5901d0f0, 0x5901cf50
Caught exception in looking up space point ptr... 0x5901cf50, 0x5901d0f0
Caught exception in looking up space point ptr... 0x68204c10, 0x68204da0
Caught exception in looking up space point ptr... 0x68204da0, 0x68204f30
Caught exception in looking up space point ptr... 0x17709150, 0x17708fc0
Caught exception in looking up space point ptr... 0x17708fc0, 0x17709150
Caught exception in looking up space point ptr... 0x17709150, 0x17708e30
Caught exception in looking up space point ptr... 0x17708e30, 0x17709150
Caught exception in looking up space point ptr... 0x17709150, 0x193875c0
Caught exception in looking up space point ptr... 0x193875c0, 0x17709150
Caught exception in looking up space point ptr... 0x17710a10, 0x846f8720
Caught exception in looking up space point ptr... 0x846f8720, 0x17710a10
Caught exception in looking up space point ptr... 0x5c546780, 0x846d6c80
Caught exception in looking up space point ptr... 0x846d6c80, 0x5c546780
++++>>>> total num hits: 122099, num free: 116960
04-Feb-2026 00:37:25 GMT  Closed output file "np02vd_raw_run042433_0076_df-s02-d1_dw_0_20260203T191100_reco_stage0_20260204T003725_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         141.173       161.289       178.86        162.233       9.36642        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.8991e-05    0.000130275   0.000308084   9.5502e-05    5.96761e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                12.9292       13.8738       17.8979       13.6979      0.980698        19     
produce:triggerrawdecoder:PDVDTriggerReader4       0.52899      0.551789      0.578145      0.547042      0.017195        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0176663     0.0189586     0.0293491     0.0180441     0.0028958       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176513     0.0179619     0.0184791     0.0179633    0.000177022      19     
produce:beamevent:BeamEvent                      0.000118012   0.000217656   0.000985753   0.000176612   0.00018343       19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.36368       3.18492       3.61582       3.2593       0.292281        19     
produce:ophit:OpHitFinder                         0.030706       0.04026      0.0501391     0.0419496    0.00474905       19     
produce:wclsdatavd:WireCellToolkit                 96.5833       114.299       124.912       114.439       6.30849        19     
produce:gaushit:GausHitFinder                     0.638995       1.62484       2.58446       1.66265      0.443714        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000177403   0.000425185   0.000914242   0.000389085   0.000153036      19     
produce:reco3d:SpacePointSolver                    9.5245        18.4857       29.2428       17.9465       5.51183        19     
produce:hitpdune:DisambigFromSpacePoints          0.106624      0.223448      0.344498       0.23943      0.0723178       19     
produce:cluster3d:Cluster3D                        1.62831       2.97905       4.40112       3.35294      0.932059        19     
[art]:TriggerResults:TriggerResultInserter        2.98e-05     4.41704e-05   0.000122341    4.028e-05    1.91145e-05      19     
end_path:out1:RootOutput                          4.17e-06     8.53332e-06   2.9981e-05     7.79e-06     5.31366e-06      19     
end_path:out1:RootOutput(write)                    4.35683       5.93418       6.54524       6.2176       0.597691        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5332.66 MB
  Peak resident set size usage (VmHWM): 3340.8 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0076_df-s02-d1_dw_0_20260203T191100_reco_stage0_20260204T003725_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0076_df-s02-d1_dw_0_20260203T191100_reco_stage0_20260204T003725_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0076_df-s02-d1_dw_0_20260203T191100_reco_stage0_20260204T003725_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162315.0,
    "core.end_time": 1770165447.0,
    "core.events": [
      8671,
      8677,
      8683,
      8689,
      8695,
      8701,
      8707,
      8713,
      8719,
      8725,
      8731,
      8737,
      8743,
      8749,
      8755,
      8761,
      8767,
      8773,
      8779
    ],
    "core.event_count": 19,
    "core.first_event_number": 8671,
    "core.last_event_number": 8779,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0076_df-s02-d1_dw_0_20260203T191100.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:40 UTC       justIN version: 01.06.00