justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.14@dunegpschedd02.fnal.gov

Jobsub ID297921.14@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:53:16
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:40
Input filesvd-protodune:np02vd_raw_run042433_0423_df-s04-d0_dw_0_20260203T212253.hdf5
JobscriptExit code0
Real time1h (3894s)
CPU time1h (3635s = 93%)
Max RSS bytes3571077120 (3405 MiB)
Outputting started2026-02-04 00:50:37
Output files
Finished2026-02-04 00:53:16
Saved logsjustin-logs:297921.14-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

space point ptr... 0x3f585300, 0x3f58e7b0
Caught exception in looking up space point ptr... 0x39439760, 0x5df590e0
Caught exception in looking up space point ptr... 0x5df590e0, 0x39439760
Caught exception in looking up space point ptr... 0x19d44a40, 0xb5c99e0
Caught exception in looking up space point ptr... 0xb5c99e0, 0x19d44a40
Caught exception in looking up space point ptr... 0x7a3bc4b0, 0x15502a00
Caught exception in looking up space point ptr... 0x15502a00, 0x39e186a0
Caught exception in looking up space point ptr... 0xd91b1b0, 0xd92d2c0
Caught exception in looking up space point ptr... 0xd92d5e0, 0xd91b1b0
Caught exception in looking up space point ptr... 0x19d51fa0, 0x19d521f0
Caught exception in looking up space point ptr... 0x19d521f0, 0x19d55280
Caught exception in looking up space point ptr... 0xeba3e80, 0xeba4190
Caught exception in looking up space point ptr... 0x51612440, 0xebb8e10
Caught exception in looking up space point ptr... 0xebb8e10, 0xeba3e80
Caught exception in looking up space point ptr... 0x625c8270, 0x891ac9e0
Caught exception in looking up space point ptr... 0x625c80e0, 0x625c8270
Caught exception in looking up space point ptr... 0x6faa7aa0, 0x6fa967a0
Caught exception in looking up space point ptr... 0x6fa967a0, 0x6faa7780
Caught exception in looking up space point ptr... 0x3910f860, 0x3e3058e0
Caught exception in looking up space point ptr... 0x3e3058e0, 0x3910f860
Caught exception in looking up space point ptr... 0xa136760, 0x1aa36eb0
Caught exception in looking up space point ptr... 0x1aa36eb0, 0x1aa36d20
Caught exception in looking up space point ptr... 0x6737c6e0, 0x4ea526c0
Caught exception in looking up space point ptr... 0x13b85d40, 0x3e317a50
Caught exception in looking up space point ptr... 0x3e317a50, 0x53480cd0
Caught exception in looking up space point ptr... 0x4285aa10, 0x6737c6e0
Caught exception in looking up space point ptr... 0x6fe30cb0, 0x88cf7370
Caught exception in looking up space point ptr... 0x6fe30f10, 0x6fe31040
Caught exception in looking up space point ptr... 0x6fe31040, 0x6fe30cb0
Caught exception in looking up space point ptr... 0x79e35ee0, 0x79e47ff0
Caught exception in looking up space point ptr... 0x79e47ff0, 0x3b0831c0
Caught exception in looking up space point ptr... 0x3b0831c0, 0x79e48310
Caught exception in looking up space point ptr... 0x79e48310, 0x79e35ee0
Caught exception in looking up space point ptr... 0x55d82a90, 0x55d82840
Caught exception in looking up space point ptr... 0x55d858d0, 0x55d82a90
Caught exception in looking up space point ptr... 0x34716d10, 0x3f65a580
Caught exception in looking up space point ptr... 0x3f65a580, 0x34716d10
Caught exception in looking up space point ptr... 0x51709090, 0x516e5fa0
Caught exception in looking up space point ptr... 0x516e5fa0, 0x516d47f0
Caught exception in looking up space point ptr... 0x9529310, 0x633630a0
Caught exception in looking up space point ptr... 0x633630a0, 0x6337a6e0
Caught exception in looking up space point ptr... 0x77abf3f0, 0x14d003e0
Caught exception in looking up space point ptr... 0x77ac0750, 0x77abf3f0
Caught exception in looking up space point ptr... 0x7ff29fc0, 0x7ff29d30
Caught exception in looking up space point ptr... 0x154c5260, 0x7ff29fc0
Caught exception in looking up space point ptr... 0xe6a5150, 0x7ff3e150
Caught exception in looking up space point ptr... 0x7ff3e150, 0xe69e890
Caught exception in looking up space point ptr... 0x14d091b0, 0x51825000
Caught exception in looking up space point ptr... 0x51825000, 0x14d09400
Caught exception in looking up space point ptr... 0x3703b460, 0x5ba93c60
Caught exception in looking up space point ptr... 0x5ba93c60, 0x3703b460
Caught exception in looking up space point ptr... 0x61c8cab0, 0x61c8cc40
Caught exception in looking up space point ptr... 0x61c7a680, 0x61c8cab0
Caught exception in looking up space point ptr... 0x1291cdb0, 0x1292ef10
Caught exception in looking up space point ptr... 0x1290ab10, 0x1291cdb0
Caught exception in looking up space point ptr... 0x57aa0a40, 0x319a1de0
Caught exception in looking up space point ptr... 0x319a1de0, 0x57aa0a40
Caught exception in looking up space point ptr... 0x37631e10, 0xe879f40
Caught exception in looking up space point ptr... 0x37631fa0, 0x37631e10
Caught exception in looking up space point ptr... 0x87c28e40, 0xde78fc0
Caught exception in looking up space point ptr... 0xde78fc0, 0xde79150
Caught exception in looking up space point ptr... 0x145ba890, 0x3b3075a0
Caught exception in looking up space point ptr... 0x3b3075a0, 0x145ba890
Caught exception in looking up space point ptr... 0x11297370, 0x52856630
Caught exception in looking up space point ptr... 0x52856630, 0x5ce436e0
Caught exception in looking up space point ptr... 0x1a49bb00, 0x68b7e790
Caught exception in looking up space point ptr... 0x68b7e790, 0x1a49bb00
++++>>>> total num hits: 517480, num free: 509347
04-Feb-2026 00:50:28 GMT  Closed output file "np02vd_raw_run042433_0423_df-s04-d0_dw_0_20260203T212253_reco_stage0_20260204T005028_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         149.493       200.921       234.077       205.587       26.6855        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.0541e-05    0.000148976   0.000257773   0.000137171   4.77634e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.1592       13.6083       14.5594       13.5712      0.325689        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.514954      0.538239      0.573708       0.53389      0.0142554       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175538     0.018381      0.0231342     0.0179255    0.00123803       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176124     0.0178795     0.0184085     0.017863     0.000177354      19     
produce:beamevent:BeamEvent                      0.000122862   0.000204066    0.0010676    0.000154342   0.000204698      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.51137       3.17868       3.49369       3.29649      0.280541        19     
produce:ophit:OpHitFinder                         0.0344313     0.0397873     0.0454193     0.0388494    0.00321994       19     
produce:wclsdatavd:WireCellToolkit                 98.4866       116.262       129.364       116.768       7.81974        19     
produce:gaushit:GausHitFinder                     0.966733       2.12661       2.9614        2.07711      0.478038        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000228003   0.000424116   0.00076013    0.000424976   0.000119095      19     
produce:reco3d:SpacePointSolver                    13.319        19.6581       28.6335       19.2198       4.62271        19     
produce:hitpdune:DisambigFromSpacePoints          0.203328      0.296712      0.448916      0.271309      0.0785123       19     
produce:cluster3d:Cluster3D                        2.3244        38.5015       64.5527       56.4285       27.0341        19     
[art]:TriggerResults:TriggerResultInserter       2.7311e-05    4.41058e-05   0.000126352    3.644e-05    2.11858e-05      19     
end_path:out1:RootOutput                          3.38e-06     8.54811e-06    3.202e-05      7.2e-06     5.82834e-06      19     
end_path:out1:RootOutput(write)                    4.52928       6.61968       8.04633       6.73949       1.06322        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5549.14 MB
  Peak resident set size usage (VmHWM): 3571.08 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0423_df-s04-d0_dw_0_20260203T212253_reco_stage0_20260204T005028_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0423_df-s04-d0_dw_0_20260203T212253_reco_stage0_20260204T005028_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0423_df-s04-d0_dw_0_20260203T212253_reco_stage0_20260204T005028_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162352.0,
    "core.end_time": 1770166229.0,
    "core.events": [
      48232,
      48238,
      48244,
      48250,
      48256,
      48262,
      48268,
      48274,
      48280,
      48286,
      48292,
      48298,
      48304,
      48310,
      48316,
      48322,
      48328,
      48334,
      48340
    ],
    "core.event_count": 19,
    "core.first_event_number": 48232,
    "core.last_event_number": 48340,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0423_df-s04-d0_dw_0_20260203T212253.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:46 UTC       justIN version: 01.06.00