justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303650.185@dunegpschedd01.fnal.gov

Jobsub ID303650.185@dunegpschedd01.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:48:28
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod00
Last heartbeat2026-02-04 00:55:24
From worker nodeHostnamewh01.grid.hep.ph.ic.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:16:04
Input filesvd-protodune:np02vd_raw_run042433_0269_df-s03-d1_dw_0_20260203T202421.hdf5
JobscriptExit code0
Real time1h (5773s)
CPU time1h (4765s = 82%)
Max RSS bytes3416711168 (3258 MiB)
Outputting started2026-02-04 00:52:22
Output files
Finished2026-02-04 00:55:24
Saved logsjustin-logs:303650.185-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

... 0x3b21fc10, 0x3b21ff30
Caught exception in looking up space point ptr... 0x3b21ff30, 0x3b21fc10
Caught exception in looking up space point ptr... 0x4c0f8c60, 0x4c0f8ad0
Caught exception in looking up space point ptr... 0x4c0f3ea0, 0x4c0f8c60
Caught exception in looking up space point ptr... 0x195d8010, 0x195d85e0
Caught exception in looking up space point ptr... 0x195d85e0, 0x195d7990
Caught exception in looking up space point ptr... 0x2c8890e0, 0x2c888c20
Caught exception in looking up space point ptr... 0x2c888c20, 0x2c8890e0
Caught exception in looking up space point ptr... 0x53355410, 0x53348e30
Caught exception in looking up space point ptr... 0x53348f30, 0x53349080
Caught exception in looking up space point ptr... 0x53349080, 0x533577d0
Caught exception in looking up space point ptr... 0x5335e250, 0x53355410
Caught exception in looking up space point ptr... 0x2c938c80, 0x597a1dd0
Caught exception in looking up space point ptr... 0x597a1dd0, 0x2c938c80
Caught exception in looking up space point ptr... 0x1780ae60, 0x1780ac90
Caught exception in looking up space point ptr... 0x1780ac90, 0x1780ef00
Caught exception in looking up space point ptr... 0x564c2f50, 0x564c31f0
Caught exception in looking up space point ptr... 0x564c31f0, 0x564c3490
Caught exception in looking up space point ptr... 0x3c4bd890, 0x3c4bded0
Caught exception in looking up space point ptr... 0x3c4bded0, 0x3c4bdbb0
Caught exception in looking up space point ptr... 0x3c4bdbb0, 0x3c4b56d0
Caught exception in looking up space point ptr... 0x19918870, 0x1993b4f0
Caught exception in looking up space point ptr... 0x1992abe0, 0x19918870
Caught exception in looking up space point ptr... 0x1a5161d0, 0x1a5160d0
Caught exception in looking up space point ptr... 0x1a5160d0, 0x1a5162d0
Caught exception in looking up space point ptr... 0x5507f350, 0x5507d590
Caught exception in looking up space point ptr... 0x5507fcd0, 0x5507f350
Caught exception in looking up space point ptr... 0x2a598ca0, 0x2a598eb0
Caught exception in looking up space point ptr... 0x1942f460, 0x2a598ca0
Caught exception in looking up space point ptr... 0x482a8e20, 0x396aa9f0
Caught exception in looking up space point ptr... 0x396aa470, 0x482a8e20
Caught exception in looking up space point ptr... 0x56c96310, 0xb1e83f0
Caught exception in looking up space point ptr... 0xb1e83f0, 0x56c96310
Caught exception in looking up space point ptr... 0x4c101910, 0x3fdd2090
Caught exception in looking up space point ptr... 0x3fdd2090, 0x4c101910
Caught exception in looking up space point ptr... 0x176dffb0, 0x176d9c30
Caught exception in looking up space point ptr... 0x176d9c30, 0x176d5970
Caught exception in looking up space point ptr... 0x3b1ebe00, 0x3b1eb960
Caught exception in looking up space point ptr... 0x3b1eb960, 0x3b1eb710
Caught exception in looking up space point ptr... 0x78544e80, 0x78544d80
Caught exception in looking up space point ptr... 0x78544d80, 0x78538f10
Caught exception in looking up space point ptr... 0x48441fd0, 0x5538d4a0
Caught exception in looking up space point ptr... 0x5538d4a0, 0x48441fd0
Caught exception in looking up space point ptr... 0x48441fd0, 0x484421b0
Caught exception in looking up space point ptr... 0x484421b0, 0x48441fd0
Caught exception in looking up space point ptr... 0x8502b730, 0x8502b900
Caught exception in looking up space point ptr... 0x8502b900, 0x8502b730
Caught exception in looking up space point ptr... 0x678a55d0, 0x3881bb80
Caught exception in looking up space point ptr... 0x3881bb80, 0x678a55d0
Caught exception in looking up space point ptr... 0x708c32d0, 0x708c3460
Caught exception in looking up space point ptr... 0x708c3460, 0x708c32d0
Caught exception in looking up space point ptr... 0x6fb48800, 0x6fb48aa0
Caught exception in looking up space point ptr... 0x6fb48aa0, 0x6fb48800
Caught exception in looking up space point ptr... 0x55272870, 0x55272210
Caught exception in looking up space point ptr... 0x552724e0, 0x55272870
Caught exception in looking up space point ptr... 0x3307d720, 0x3307f380
Caught exception in looking up space point ptr... 0x3307f380, 0x3307d720
Caught exception in looking up space point ptr... 0x3c3b0a90, 0x5987e820
Caught exception in looking up space point ptr... 0x5987e820, 0x3b054e10
Caught exception in looking up space point ptr... 0x5505dd00, 0x5505e020
Caught exception in looking up space point ptr... 0x5505e020, 0x5505dd00
Caught exception in looking up space point ptr... 0x70162e50, 0x701632a0
Caught exception in looking up space point ptr... 0x701632a0, 0x70162e50
Caught exception in looking up space point ptr... 0x11b41080, 0x11b2e810
Caught exception in looking up space point ptr... 0x11b2e810, 0x11b41080
Caught exception in looking up space point ptr... 0x11b41080, 0x12b30310
Caught exception in looking up space point ptr... 0x12b30310, 0x11b41080
++++>>>> total num hits: 70512, num free: 67799
04-Feb-2026 00:52:03 GMT  Closed output file "np02vd_raw_run042433_0269_df-s03-d1_dw_0_20260203T202421_reco_stage0_20260204T005203_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         234.044       298.668       327.218       309.878       27.7657        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       5.8533e-05    0.000120468   0.000232212   9.6359e-05    5.11921e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                46.3112       48.4262       53.0821       48.1872       1.62975        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.515802      0.572897      0.664048      0.564076      0.0434257       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0165567     0.0193609     0.0288952     0.0168203    0.00359976       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0164857     0.0166755     0.0169341     0.0166642    0.000113531      19     
produce:beamevent:BeamEvent                      0.000107924   0.000197097   0.00112558    0.000140198   0.000220179      19     
produce:pdvddaphne:DAPHNEReaderPDVD                7.08453       8.12342       8.9977        8.02038      0.478636        19     
produce:ophit:OpHitFinder                         0.0289235     0.0457981     0.0548032     0.0480063    0.00652296       19     
produce:wclsdatavd:WireCellToolkit                 145.718       197.825       217.63        209.102       22.6179        19     
produce:gaushit:GausHitFinder                      1.6381        2.86535       3.91583       2.85966      0.552018        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000428514   0.000591346   0.000752845   0.000603752   8.29542e-05      19     
produce:reco3d:SpacePointSolver                    15.6202       23.7943       33.3684       21.3994       5.88634        19     
produce:hitpdune:DisambigFromSpacePoints          0.218468      0.339914       0.45191      0.358473      0.0689234       19     
produce:cluster3d:Cluster3D                        3.02524       4.8494        7.74692        4.641        1.31472        19     
[art]:TriggerResults:TriggerResultInserter       2.7839e-05    4.01906e-05   9.4806e-05    3.5327e-05    1.43342e-05      19     
end_path:out1:RootOutput                          5.247e-06    1.15835e-05    3.168e-05    1.1455e-05    5.97025e-06      19     
end_path:out1:RootOutput(write)                    7.82075       11.7573       13.9658       12.068        1.57937        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5317.33 MB
  Peak resident set size usage (VmHWM): 3416.71 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0269_df-s03-d1_dw_0_20260203T202421_reco_stage0_20260204T005203_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0269_df-s03-d1_dw_0_20260203T202421_reco_stage0_20260204T005203_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0269_df-s03-d1_dw_0_20260203T202421_reco_stage0_20260204T005203_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770160577.0,
    "core.end_time": 1770166328.0,
    "core.events": [
      30675,
      30681,
      30687,
      30693,
      30699,
      30705,
      30711,
      30717,
      30723,
      30729,
      30735,
      30741,
      30747,
      30753,
      30759,
      30765,
      30771,
      30777,
      30783
    ],
    "core.event_count": 19,
    "core.first_event_number": 30675,
    "core.last_event_number": 30783,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0269_df-s03-d1_dw_0_20260203T202421.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:00:35 UTC       justIN version: 01.06.00