justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.125@dunegpschedd02.fnal.gov

Jobsub ID297923.125@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 02:14:57
From worker nodeHostnameheplnc137.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:03:12
Input filesvd-protodune:np02vd_raw_run042433_0474_df-s03-d0_dw_0_20260203T214216.hdf5
JobscriptExit code0
Real time2h (7734s)
CPU time2h (7298s = 94%)
Max RSS bytes6848307200 (6531 MiB)
Outputting started2026-02-04 02:12:10
Output files
Finished2026-02-04 02:14:57
Saved logsjustin-logs:297923.125-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

space point ptr... 0x7357ffa0, 0x120713c0
Caught exception in looking up space point ptr... 0x2dca50b0, 0x2dca53d0
Caught exception in looking up space point ptr... 0x2dca53d0, 0x2dca50b0
Caught exception in looking up space point ptr... 0xec3cc70, 0x471c07f0
Caught exception in looking up space point ptr... 0x471c07f0, 0x3ffd0150
Caught exception in looking up space point ptr... 0x7f290980, 0x3a23c450
Caught exception in looking up space point ptr... 0x3a23c450, 0x7f290980
Caught exception in looking up space point ptr... 0x3584cea0, 0x3584cd10
Caught exception in looking up space point ptr... 0x3584cd10, 0x3584cea0
Caught exception in looking up space point ptr... 0x3584cea0, 0x9cc9c90
Caught exception in looking up space point ptr... 0x9cc9c90, 0x3584cea0
Caught exception in looking up space point ptr... 0x11be4940, 0xa56e520
Caught exception in looking up space point ptr... 0xa56e520, 0x8930a670
Caught exception in looking up space point ptr... 0x2c277f50, 0xa4da8f0
Caught exception in looking up space point ptr... 0xa4da8f0, 0x2c277f50
Caught exception in looking up space point ptr... 0x66c640f0, 0x145dc800
Caught exception in looking up space point ptr... 0x145dc800, 0x7eb20010
Caught exception in looking up space point ptr... 0x76c39550, 0x2de3f1b0
Caught exception in looking up space point ptr... 0x58e2df90, 0x76c39550
Caught exception in looking up space point ptr... 0x1645b220, 0x16007250
Caught exception in looking up space point ptr... 0x16007250, 0x1645b220
Caught exception in looking up space point ptr... 0x392af0c0, 0xa5b7580
Caught exception in looking up space point ptr... 0xa5b7580, 0x83200990
Caught exception in looking up space point ptr... 0x83200990, 0x5288a6b0
Caught exception in looking up space point ptr... 0xde8d240, 0x16482010
Caught exception in looking up space point ptr... 0x16482010, 0x872508e0
Caught exception in looking up space point ptr... 0x4fde7550, 0x358a4b20
Caught exception in looking up space point ptr... 0x358a4b20, 0x4fe314b0
Caught exception in looking up space point ptr... 0x59a4a570, 0x411bdec0
Caught exception in looking up space point ptr... 0x411bdec0, 0x6ee9d540
Caught exception in looking up space point ptr... 0x6ee9d540, 0x411be050
Caught exception in looking up space point ptr... 0x15fe3f70, 0x9f29460
Caught exception in looking up space point ptr... 0x9f29460, 0x8416ed0
Caught exception in looking up space point ptr... 0x6a11b980, 0x35d40240
Caught exception in looking up space point ptr... 0x35d40240, 0x6a11b980
Caught exception in looking up space point ptr... 0x3f3cc9a0, 0x4f50ede0
Caught exception in looking up space point ptr... 0x4f50ede0, 0x3ed315a0
Caught exception in looking up space point ptr... 0x34c69190, 0x34c69000
Caught exception in looking up space point ptr... 0x34c69000, 0x34c69190
Caught exception in looking up space point ptr... 0x34c69190, 0x800d19a0
Caught exception in looking up space point ptr... 0x800d19a0, 0x34c69190
Caught exception in looking up space point ptr... 0x39f1bc20, 0x4f7762e0
Caught exception in looking up space point ptr... 0x4f7762e0, 0x39f1bc20
Caught exception in looking up space point ptr... 0x6f4cc0c0, 0x813033c0
Caught exception in looking up space point ptr... 0x813033c0, 0x34612190
Caught exception in looking up space point ptr... 0x15d8b520, 0x6603cde0
Caught exception in looking up space point ptr... 0x6603cde0, 0x15d8b520
Caught exception in looking up space point ptr... 0x9ca34c0, 0x9ca3380
Caught exception in looking up space point ptr... 0x9ca3380, 0x9ca34c0
Caught exception in looking up space point ptr... 0x6a146650, 0x35d17370
Caught exception in looking up space point ptr... 0x35d17370, 0x6a146650
Caught exception in looking up space point ptr... 0x549fd4d0, 0x463fe340
Caught exception in looking up space point ptr... 0x463fe340, 0x7dc3970
Caught exception in looking up space point ptr... 0x83a96490, 0x18704850
Caught exception in looking up space point ptr... 0x18704850, 0x83a96490
Caught exception in looking up space point ptr... 0x53bdd140, 0x2f356ab0
Caught exception in looking up space point ptr... 0x2f356ab0, 0x53bddbf0
Caught exception in looking up space point ptr... 0x2e55f210, 0x4b3fb210
Caught exception in looking up space point ptr... 0x4b3fb210, 0x6591eca0
Caught exception in looking up space point ptr... 0x86bf9b0, 0x86d0b90
Caught exception in looking up space point ptr... 0x86d0b90, 0x86bf9b0
Caught exception in looking up space point ptr... 0x13c183a0, 0x13c18210
Caught exception in looking up space point ptr... 0x13c18210, 0x13c18080
Caught exception in looking up space point ptr... 0x2ddbec80, 0x2e47d360
Caught exception in looking up space point ptr... 0x2e47d360, 0x2ddbec80
Caught exception in looking up space point ptr... 0x4fcff600, 0x17759200
Caught exception in looking up space point ptr... 0x17759200, 0x35977af0
++++>>>> total num hits: 1180301, num free: 1138200
04-Feb-2026 02:11:54 GMT  Closed output file "np02vd_raw_run042433_0474_df-s03-d0_dw_0_20260203T214216_reco_stage0_20260204T021154_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         225.38        396.396       1151.67       277.818       252.852        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.0554e-05    0.000154405   0.000311908   0.000133554   5.80834e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                16.4333       17.1522       17.9478       17.1704      0.406205        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.528072      0.558601      0.637103      0.550692      0.0270348       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177909     0.0203288     0.035397      0.0179494    0.00552844       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.017741      0.0181501      0.02137      0.0179606    0.000781379      19     
produce:beamevent:BeamEvent                      0.000144227   0.000268369   0.00152026    0.000202811   0.000298038      19     
produce:pdvddaphne:DAPHNEReaderPDVD                6.35974       8.02098       9.74736       8.04139      0.822243        19     
produce:ophit:OpHitFinder                         0.0448914     0.0722668     0.0999544     0.0728141     0.0130927       19     
produce:wclsdatavd:WireCellToolkit                 171.029       199.91        246.569       195.654       20.2542        19     
produce:gaushit:GausHitFinder                      1.32819       4.04144       18.7259       2.29051       4.11241        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000410103   0.000849035   0.00157363    0.000639553   0.000390407      19     
produce:reco3d:SpacePointSolver                    15.3501       33.0451       70.8029       22.7383       19.5413        19     
produce:hitpdune:DisambigFromSpacePoints          0.210685      0.961928       3.90758      0.389587       1.09278        19     
produce:cluster3d:Cluster3D                        1.80658       119.562       820.382       3.72522       225.812        19     
[art]:TriggerResults:TriggerResultInserter       3.0375e-05    5.74647e-05   0.000156248    5.203e-05    2.88707e-05      19     
end_path:out1:RootOutput                          6.536e-06    2.54805e-05   0.000225539   1.1968e-05    4.80108e-05      19     
end_path:out1:RootOutput(write)                    7.00234       12.9582       29.283        11.253        6.35778        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9149.42 MB
  Peak resident set size usage (VmHWM): 6848.31 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0474_df-s03-d0_dw_0_20260203T214216_reco_stage0_20260204T021154_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0474_df-s03-d0_dw_0_20260203T214216_reco_stage0_20260204T021154_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0474_df-s03-d0_dw_0_20260203T214216_reco_stage0_20260204T021154_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163423.0,
    "core.end_time": 1770171120.0,
    "core.events": [
      54044,
      54050,
      54056,
      54062,
      54068,
      54074,
      54080,
      54086,
      54092,
      54098,
      54104,
      54110,
      54116,
      54122,
      54128,
      54134,
      54140,
      54146,
      54152
    ],
    "core.event_count": 19,
    "core.first_event_number": 54044,
    "core.last_event_number": 54152,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0474_df-s03-d0_dw_0_20260203T214216.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:29:41 UTC       justIN version: 01.06.00