justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.33@dunegpschedd01.fnal.gov

Jobsub ID303656.33@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:06:20
From worker nodeHostnameheplnc147.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:11:37
Input filesvd-protodune:np02vd_raw_run042421_0164_df-s04-d1_dw_0_20260203T145944.hdf5
JobscriptExit code0
Real time52m (3141s)
CPU time45m (2720s = 86%)
Max RSS bytes2605395968 (2484 MiB)
Outputting started2026-02-04 01:03:59
Output files
Finished2026-02-04 01:06:20
Saved logsjustin-logs:303656.33-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

70, 0x13f29aa0
Caught exception in looking up space point ptr... 0x13f29aa0, 0x13f295f0
Caught exception in looking up space point ptr... 0x13f295f0, 0x5b41fb30
Caught exception in looking up space point ptr... 0x5b41fb30, 0x3c039fe0
Caught exception in looking up space point ptr... 0x3c039fe0, 0x3c03a280
Caught exception in looking up space point ptr... 0x3c03a280, 0x5b427910
Caught exception in looking up space point ptr... 0x127b6f00, 0x127b6d70
Caught exception in looking up space point ptr... 0x12748420, 0x1282f090
Caught exception in looking up space point ptr... 0x1282f090, 0x12837ed0
Caught exception in looking up space point ptr... 0x1277e8d0, 0x12772d30
Caught exception in looking up space point ptr... 0x12772d30, 0x127754d0
Caught exception in looking up space point ptr... 0x127754d0, 0x1277b390
Caught exception in looking up space point ptr... 0x1277b390, 0x1277e8d0
Caught exception in looking up space point ptr... 0xd2a9cf0, 0x3a32bf90
Caught exception in looking up space point ptr... 0x3a32bf90, 0x8341410
Caught exception in looking up space point ptr... 0x8341410, 0x33e4e150
Caught exception in looking up space point ptr... 0x33e4e150, 0x344bcfc0
Caught exception in looking up space point ptr... 0x344bcfc0, 0x14dd9410
Caught exception in looking up space point ptr... 0x14dd9410, 0x30cd2a90
Caught exception in looking up space point ptr... 0x30cd2a90, 0x6241cfb0
Caught exception in looking up space point ptr... 0x46804b20, 0x342cb210
Caught exception in looking up space point ptr... 0x342cb210, 0x32065ac0
Caught exception in looking up space point ptr... 0x4728eb20, 0x2657fd80
Caught exception in looking up space point ptr... 0x2657fd80, 0x548e8d60
Caught exception in looking up space point ptr... 0x29c8c900, 0x29c8c7b0
Caught exception in looking up space point ptr... 0x29c8c7b0, 0x29c8c900
Caught exception in looking up space point ptr... 0x29c8c900, 0x29c8bfc0
Caught exception in looking up space point ptr... 0x29c8bfc0, 0x29c8c900
Caught exception in looking up space point ptr... 0x627b18f0, 0x46732e90
Caught exception in looking up space point ptr... 0x46732e90, 0x297b0670
Caught exception in looking up space point ptr... 0x255e1770, 0x255dff40
Caught exception in looking up space point ptr... 0x255dff40, 0x255e0370
Caught exception in looking up space point ptr... 0x1449b560, 0x3a2c2c40
Caught exception in looking up space point ptr... 0xd4e7d70, 0x1449b560
Caught exception in looking up space point ptr... 0x85464c0, 0x85463e0
Caught exception in looking up space point ptr... 0x5000d2e0, 0x85464c0
Caught exception in looking up space point ptr... 0x7fec890, 0x13fbf240
Caught exception in looking up space point ptr... 0x13fbf240, 0x7fe9370
Caught exception in looking up space point ptr... 0x7fe9370, 0x7fed0f0
Caught exception in looking up space point ptr... 0x7fed0f0, 0x7fec890
Caught exception in looking up space point ptr... 0x2e9d1c00, 0x8cdbc00
Caught exception in looking up space point ptr... 0x8cdbc00, 0x5778f5a0
Caught exception in looking up space point ptr... 0x2502f8f0, 0x3fd07160
Caught exception in looking up space point ptr... 0x3fd07160, 0x2502f8f0
Caught exception in looking up space point ptr... 0x127f1570, 0x61c66ca0
Caught exception in looking up space point ptr... 0x61c66ca0, 0x127f1570
Caught exception in looking up space point ptr... 0x2bafe930, 0x12adb4e0
Caught exception in looking up space point ptr... 0x12adb4e0, 0x34453870
Caught exception in looking up space point ptr... 0x5000e950, 0x5000e870
Caught exception in looking up space point ptr... 0x5000e870, 0x5000e950
Caught exception in looking up space point ptr... 0x13f3d5e0, 0x13f3d500
Caught exception in looking up space point ptr... 0x13f3d500, 0x13f3d5e0
Caught exception in looking up space point ptr... 0x14b488a0, 0x2b3693d0
Caught exception in looking up space point ptr... 0x2b3693d0, 0x14b488a0
Caught exception in looking up space point ptr... 0x30bb4f60, 0x4e2fe930
Caught exception in looking up space point ptr... 0x4e2fe930, 0x30bb50f0
Caught exception in looking up space point ptr... 0x41e86200, 0x397df7a0
Caught exception in looking up space point ptr... 0x397df7a0, 0x41e86200
Caught exception in looking up space point ptr... 0x454018d0, 0x45402360
Caught exception in looking up space point ptr... 0x45402360, 0x13f3df50
Caught exception in looking up space point ptr... 0x344f2ab0, 0x4f6a2260
Caught exception in looking up space point ptr... 0x4f6a2260, 0x344f2ab0
Caught exception in looking up space point ptr... 0x12b11830, 0x8fc16b0
Caught exception in looking up space point ptr... 0x8fc16b0, 0x2f120ea0
Caught exception in looking up space point ptr... 0x2f120ea0, 0x8fc16b0
Caught exception in looking up space point ptr... 0x8fc16b0, 0xc3418a0
++++>>>> total num hits: 40089, num free: 38670
04-Feb-2026 01:03:51 GMT  Closed output file "np02vd_raw_run042421_0164_df-s04-d1_dw_0_20260203T145944_reco_stage0_20260204T010351_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         91.9946       112.509       136.137       112.203       10.0484        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       6.8189e-05    0.000123679   0.000351704   9.5901e-05    6.14252e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                11.4865       13.7756       16.2817       14.1827       1.19424        27     
produce:triggerrawdecoder:PDVDTriggerReader4       0.5269       0.548064      0.571872      0.547125      0.0121254       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0176272     0.0179761     0.0185873     0.0179733    0.000198075      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.087196      0.0889194      0.10562      0.0883104    0.00332815       27     
produce:beamevent:BeamEvent                      9.0491e-05    0.000202566   0.00172505    0.000147679   0.000300359      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.41923       3.84938       4.21902       3.85358      0.190964        27     
produce:ophit:OpHitFinder                         0.0292198     0.0411031     0.0503019     0.0403419    0.00412875       27     
produce:wclsdatavd:WireCellToolkit                 59.8832       74.1671       88.6084       73.6019       6.83403        27     
produce:gaushit:GausHitFinder                     0.699363       1.11703       1.73747       1.09377      0.280217        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000234392    0.0003887    0.00098529    0.000370419   0.000141617      27     
produce:reco3d:SpacePointSolver                    6.58104       12.0912       20.3929       12.409        3.90645        27     
produce:hitpdune:DisambigFromSpacePoints          0.0752919     0.122018      0.229932      0.115074      0.0357954       27     
produce:cluster3d:Cluster3D                        1.10583       2.17128       4.23411       2.07779      0.744138        27     
[art]:TriggerResults:TriggerResultInserter       2.2082e-05    4.41974e-05   0.000290057   3.5597e-05    4.88602e-05      27     
end_path:out1:RootOutput                          3.767e-06    7.84733e-06   3.9965e-05     6.442e-06    6.72784e-06      27     
end_path:out1:RootOutput(write)                    3.43673       4.4887        5.9039        4.59959      0.669524        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4560.33 MB
  Peak resident set size usage (VmHWM): 2605.4 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0164_df-s04-d1_dw_0_20260203T145944_reco_stage0_20260204T010351_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0164_df-s04-d1_dw_0_20260203T145944_reco_stage0_20260204T010351_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0164_df-s04-d1_dw_0_20260203T145944_reco_stage0_20260204T010351_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163908.0,
    "core.end_time": 1770167033.0,
    "core.events": [
      26651,
      26657,
      26663,
      26669,
      26675,
      26681,
      26687,
      26693,
      26699,
      26705,
      26711,
      26717,
      26723,
      26729,
      26735,
      26741,
      26747,
      26753,
      26759,
      26765,
      26771,
      26777,
      26783,
      26789,
      26795,
      26801,
      26807
    ],
    "core.event_count": 27,
    "core.first_event_number": 26651,
    "core.last_event_number": 26807,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0164_df-s04-d1_dw_0_20260203T145944.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:33:22 UTC       justIN version: 01.06.00