justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.122@dunegpschedd02.fnal.gov

Jobsub ID297923.122@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:32:03
From worker nodeHostnameheplnc137.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:03:50
Input filesvd-protodune:np02vd_raw_run042433_0540_df-s04-d1_dw_0_20260203T220722.hdf5
JobscriptExit code0
Real time1h (5139s)
CPU time1h (4748s = 92%)
Max RSS bytes3381620736 (3224 MiB)
Outputting started2026-02-04 01:29:31
Output files
Finished2026-02-04 01:32:03
Saved logsjustin-logs:297923.122-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

space point ptr... 0x5ecaaa30, 0x34183a10
Caught exception in looking up space point ptr... 0x553d9110, 0x553d9610
Caught exception in looking up space point ptr... 0x553d9610, 0x553d9110
Caught exception in looking up space point ptr... 0x7909ec70, 0x7909eb40
Caught exception in looking up space point ptr... 0x7909eb40, 0x7909ec70
Caught exception in looking up space point ptr... 0x325a7020, 0x325a6e40
Caught exception in looking up space point ptr... 0x325a6e40, 0x325a7020
Caught exception in looking up space point ptr... 0x2f188620, 0x866e9e40
Caught exception in looking up space point ptr... 0x866e9e40, 0x866e6370
Caught exception in looking up space point ptr... 0x2cb5bb50, 0x2cb5be40
Caught exception in looking up space point ptr... 0x2cb5be40, 0x2cb5bb50
Caught exception in looking up space point ptr... 0x496925a0, 0x496928c0
Caught exception in looking up space point ptr... 0x496928c0, 0x496925a0
Caught exception in looking up space point ptr... 0xe049dd0, 0xe049930
Caught exception in looking up space point ptr... 0xe049930, 0xe0496e0
Caught exception in looking up space point ptr... 0x395bf930, 0x471df0f0
Caught exception in looking up space point ptr... 0x471df0f0, 0x395bf930
Caught exception in looking up space point ptr... 0x661fbd90, 0x661e2dc0
Caught exception in looking up space point ptr... 0x661e4b20, 0x661fbd90
Caught exception in looking up space point ptr... 0x75408e80, 0x63339530
Caught exception in looking up space point ptr... 0x63339530, 0x6333ad30
Caught exception in looking up space point ptr... 0x4691a410, 0x4691a560
Caught exception in looking up space point ptr... 0x4691a560, 0x4691a880
Caught exception in looking up space point ptr... 0x553bffd0, 0x553c8f50
Caught exception in looking up space point ptr... 0x553c8f50, 0x553cfbd0
Caught exception in looking up space point ptr... 0x9d8e2d0, 0x9d8e180
Caught exception in looking up space point ptr... 0x9d8e180, 0x9d8e2d0
Caught exception in looking up space point ptr... 0x3783fcd0, 0x3783ade0
Caught exception in looking up space point ptr... 0xcf40600, 0x3783fcd0
Caught exception in looking up space point ptr... 0x49344860, 0x49344de0
Caught exception in looking up space point ptr... 0x49344de0, 0x49345260
Caught exception in looking up space point ptr... 0xaaf3a90, 0x130185b0
Caught exception in looking up space point ptr... 0x130185b0, 0xaaf3a90
Caught exception in looking up space point ptr... 0x2cba9a10, 0x2cba7f30
Caught exception in looking up space point ptr... 0x2cba7f30, 0x2cba9ba0
Caught exception in looking up space point ptr... 0x88bd5c0, 0x88bd380
Caught exception in looking up space point ptr... 0x88bd380, 0x88bd5c0
Caught exception in looking up space point ptr... 0x666ba5c0, 0x3e7f89f0
Caught exception in looking up space point ptr... 0x6d4e16e0, 0x666ba5c0
Caught exception in looking up space point ptr... 0x3b76fdb0, 0x3b76fa90
Caught exception in looking up space point ptr... 0x3b76fa90, 0x3b76fc20
Caught exception in looking up space point ptr... 0x5eac1410, 0x5eac1e50
Caught exception in looking up space point ptr... 0x5eac1e50, 0x5eac1410
Caught exception in looking up space point ptr... 0x12c3f8f0, 0x146e5cb0
Caught exception in looking up space point ptr... 0x146e5cb0, 0x12c3f8f0
Caught exception in looking up space point ptr... 0x59fd0e90, 0x59fcf2b0
Caught exception in looking up space point ptr... 0x59fcf2b0, 0x59fd0e90
Caught exception in looking up space point ptr... 0x4b1f9a70, 0x37a79890
Caught exception in looking up space point ptr... 0x37a79890, 0x4b1f9a70
Caught exception in looking up space point ptr... 0x866ff9a0, 0x867075f0
Caught exception in looking up space point ptr... 0x867075f0, 0x866ff9a0
Caught exception in looking up space point ptr... 0x6a388130, 0x6a3886f0
Caught exception in looking up space point ptr... 0x6a388410, 0x6a388130
Caught exception in looking up space point ptr... 0x56a6f260, 0x3ba89e70
Caught exception in looking up space point ptr... 0x3ba89e70, 0x66f6d850
Caught exception in looking up space point ptr... 0xb2b83e0, 0xb2b4ec0
Caught exception in looking up space point ptr... 0xb2b4ec0, 0xb2b83e0
Caught exception in looking up space point ptr... 0x36fd55f0, 0x36fd5ea0
Caught exception in looking up space point ptr... 0x36fd5ea0, 0x36fd60c0
Caught exception in looking up space point ptr... 0x13fae010, 0x13fae330
Caught exception in looking up space point ptr... 0x13fae330, 0x13fae010
Caught exception in looking up space point ptr... 0x2cb3a8c0, 0x2cb3ae10
Caught exception in looking up space point ptr... 0x2cb3ae10, 0x2cb3a8c0
Caught exception in looking up space point ptr... 0x8117e270, 0x8117edd0
Caught exception in looking up space point ptr... 0x8117edd0, 0x8117e270
Caught exception in looking up space point ptr... 0x38026800, 0x380256c0
Caught exception in looking up space point ptr... 0x380256c0, 0x38026800
++++>>>> total num hits: 124514, num free: 119660
04-Feb-2026 01:29:10 GMT  Closed output file "np02vd_raw_run042433_0540_df-s04-d1_dw_0_20260203T220722_reco_stage0_20260204T012910_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         220.482       261.642       341.804       255.13        32.2169        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.00011043    0.000166727   0.000307744   0.000120223   6.35668e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                16.6766       17.5705       18.9326       17.5189      0.489586        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.529028      0.573234      0.673138       0.56444      0.0349105       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.017842      0.0217065     0.048158      0.0181515    0.00741184       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177726     0.0179555     0.0183592     0.017938     0.000127531      19     
produce:beamevent:BeamEvent                      0.000148665   0.000275477   0.00130171    0.000227069   0.000243582      19     
produce:pdvddaphne:DAPHNEReaderPDVD                7.02286       8.04223       10.8584       8.02705      0.939949        19     
produce:ophit:OpHitFinder                         0.0402665     0.0662005     0.0879213     0.0717566     0.0150795       19     
produce:wclsdatavd:WireCellToolkit                 171.709       196.715       249.146       191.82        21.2246        19     
produce:gaushit:GausHitFinder                      1.13507       2.27552       4.07715       2.03634      0.728624        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000397925   0.000675837   0.00133017    0.000652941   0.000202097      19     
produce:reco3d:SpacePointSolver                    10.3705       22.3089       46.7901       20.6348       8.24686        19     
produce:hitpdune:DisambigFromSpacePoints          0.137325      0.344217      0.825439      0.287152      0.156222        19     
produce:cluster3d:Cluster3D                        1.17388       3.86308       9.87314       3.4383        2.09259        19     
[art]:TriggerResults:TriggerResultInserter       2.8932e-05    4.23812e-05   9.8669e-05    3.6839e-05    1.79544e-05      19     
end_path:out1:RootOutput                          4.492e-06    1.07615e-05   2.9715e-05     9.634e-06    5.38877e-06      19     
end_path:out1:RootOutput(write)                    7.02383       9.77681       16.4147       9.26837       2.18344        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5354.12 MB
  Peak resident set size usage (VmHWM): 3381.62 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0540_df-s04-d1_dw_0_20260203T220722_reco_stage0_20260204T012910_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0540_df-s04-d1_dw_0_20260203T220722_reco_stage0_20260204T012910_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0540_df-s04-d1_dw_0_20260203T220722_reco_stage0_20260204T012910_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163445.0,
    "core.end_time": 1770168558.0,
    "core.events": [
      61571,
      61577,
      61583,
      61589,
      61595,
      61601,
      61607,
      61613,
      61619,
      61625,
      61631,
      61637,
      61643,
      61649,
      61655,
      61661,
      61667,
      61673,
      61679
    ],
    "core.event_count": 19,
    "core.first_event_number": 61571,
    "core.last_event_number": 61679,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0540_df-s04-d1_dw_0_20260203T220722.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:32:58 UTC       justIN version: 01.06.00