justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.128@dunegpschedd02.fnal.gov

Jobsub ID297921.128@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2026-02-04 00:55:15
From worker nodeHostnamedune001-1845823.0-lcg2700.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:55:50
Input filesvd-protodune:np02vd_raw_run042433_0401_df-s04-d1_dw_0_20260203T211433.hdf5
JobscriptExit code0
Real time57m (3462s)
CPU time52m (3131s = 90%)
Max RSS bytes3413385216 (3255 MiB)
Outputting started2026-02-04 00:53:33
Output files
Finished2026-02-04 00:55:15
Saved logsjustin-logs:297921.128-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

. 0xa1d70430, 0x8d4e03e0
Caught exception in looking up space point ptr... 0x8d4e03e0, 0xa1d70430
Caught exception in looking up space point ptr... 0xa1d70430, 0x8d4e0980
Caught exception in looking up space point ptr... 0x76756d60, 0xa858e4a0
Caught exception in looking up space point ptr... 0xa858e4a0, 0xb11736d0
Caught exception in looking up space point ptr... 0x97aae220, 0x740b1b10
Caught exception in looking up space point ptr... 0x740b1b10, 0x77a9d2a0
Caught exception in looking up space point ptr... 0x5558c7c0, 0x5558c1d0
Caught exception in looking up space point ptr... 0x5558c1d0, 0x4c27f440
Caught exception in looking up space point ptr... 0x663ed000, 0x663ed370
Caught exception in looking up space point ptr... 0x663ecce0, 0x663ed000
Caught exception in looking up space point ptr... 0x95a5ccf0, 0x95a5caf0
Caught exception in looking up space point ptr... 0x95a5caf0, 0x95a5d2c0
Caught exception in looking up space point ptr... 0x55530530, 0x6f958380
Caught exception in looking up space point ptr... 0x6f958380, 0x55530530
Caught exception in looking up space point ptr... 0x44678da0, 0x44679040
Caught exception in looking up space point ptr... 0x44679040, 0x44679190
Caught exception in looking up space point ptr... 0x4d2f5910, 0x44394be0
Caught exception in looking up space point ptr... 0x44394be0, 0x91ed47b0
Caught exception in looking up space point ptr... 0x7e79a0c0, 0x7e799ba0
Caught exception in looking up space point ptr... 0x7e799ba0, 0x7e79a0c0
Caught exception in looking up space point ptr... 0x44685b20, 0x44685990
Caught exception in looking up space point ptr... 0x44685990, 0x44685b20
Caught exception in looking up space point ptr... 0x54cd4060, 0x54cd0930
Caught exception in looking up space point ptr... 0x54cd0930, 0x54cd0790
Caught exception in looking up space point ptr... 0x938e1a50, 0x938e1cb0
Caught exception in looking up space point ptr... 0x938e1cb0, 0x938e1b80
Caught exception in looking up space point ptr... 0x9d212b00, 0x8d9bfad0
Caught exception in looking up space point ptr... 0x443eab20, 0x9d212b00
Caught exception in looking up space point ptr... 0xa1ae0ff0, 0xa1ae8ba0
Caught exception in looking up space point ptr... 0xa1ae8ba0, 0xa1ae0ff0
Caught exception in looking up space point ptr... 0x74d298b0, 0x74d293b0
Caught exception in looking up space point ptr... 0x74d293b0, 0x74d298b0
Caught exception in looking up space point ptr... 0x74d28cd0, 0x74d289f0
Caught exception in looking up space point ptr... 0x74d289f0, 0x74d28cd0
Caught exception in looking up space point ptr... 0x931a0220, 0x9319fce0
Caught exception in looking up space point ptr... 0x9319fce0, 0x931a0220
Caught exception in looking up space point ptr... 0x931a0220, 0x9319ff80
Caught exception in looking up space point ptr... 0x9319ff80, 0x931a0220
Caught exception in looking up space point ptr... 0xa6941140, 0xa693fbc0
Caught exception in looking up space point ptr... 0xa693fbc0, 0xa6941140
Caught exception in looking up space point ptr... 0xadd3d700, 0xa91c33f0
Caught exception in looking up space point ptr... 0xa91c33f0, 0xadd3d700
Caught exception in looking up space point ptr... 0xa33a4160, 0xa33a1ca0
Caught exception in looking up space point ptr... 0xa33a1ca0, 0xa33a4160
Caught exception in looking up space point ptr... 0xa858e1e0, 0xa858e050
Caught exception in looking up space point ptr... 0xa858e050, 0xa858db00
Caught exception in looking up space point ptr... 0x73f7ce10, 0x74640530
Caught exception in looking up space point ptr... 0x74640530, 0x494196b0
Caught exception in looking up space point ptr... 0xa33b1c00, 0xa33acc90
Caught exception in looking up space point ptr... 0xa33acc90, 0xa33b1c00
Caught exception in looking up space point ptr... 0x6fab72e0, 0x6fab7600
Caught exception in looking up space point ptr... 0x6fab7600, 0x6fab7920
Caught exception in looking up space point ptr... 0x663dc330, 0x663dc690
Caught exception in looking up space point ptr... 0x663dc8e0, 0x663dc330
Caught exception in looking up space point ptr... 0x7ec1b550, 0xb4595700
Caught exception in looking up space point ptr... 0xb4595700, 0x7ec1b550
Caught exception in looking up space point ptr... 0x7ec1b550, 0x931c8010
Caught exception in looking up space point ptr... 0x931c8010, 0x7ec1b550
Caught exception in looking up space point ptr... 0x4eb9d250, 0x89f95ad0
Caught exception in looking up space point ptr... 0x89f95ad0, 0x4eb9d250
Caught exception in looking up space point ptr... 0x9dd95730, 0xbf7c07d0
Caught exception in looking up space point ptr... 0xbf7c07d0, 0x9dd95730
Caught exception in looking up space point ptr... 0xb1181240, 0xb118e7e0
Caught exception in looking up space point ptr... 0xb1183510, 0xb1181240
Caught exception in looking up space point ptr... 0x54ccce60, 0x54cccb70
Caught exception in looking up space point ptr... 0x54cccb70, 0x54ccc9d0
++++>>>> total num hits: 89183, num free: 85988
04-Feb-2026 00:53:24 UTC  Closed output file "np02vd_raw_run042433_0401_df-s04-d1_dw_0_20260203T211433_reco_stage0_20260204T005324_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         162.024       178.026       200.241       178.472       10.0289        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.1728e-05    0.000116014   0.000287282    9.357e-05    5.41922e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                15.5472       17.3905       20.9505       16.8166       1.59953        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.694186      0.758184      0.843017      0.755055      0.0343878       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0227562     0.0276289     0.0449865     0.0232466    0.00626415       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0225624     0.0230949     0.0239432     0.023112     0.000275184      19     
produce:beamevent:BeamEvent                      8.4707e-05    0.000234671   0.00113781    0.000194812   0.000216994      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.93575       3.60444       4.17004       3.62018      0.331711        19     
produce:ophit:OpHitFinder                         0.0309229     0.0410008     0.052762      0.0411167    0.00522393       19     
produce:wclsdatavd:WireCellToolkit                 107.769       119.123        128.4        120.378       5.52586        19     
produce:gaushit:GausHitFinder                     0.963088       1.80981       2.46082       1.87881       0.32606        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000237406   0.00043601    0.000687012   0.000442114   8.0211e-05       19     
produce:reco3d:SpacePointSolver                    17.0207       25.5262       37.0673       26.7064       5.1261         19     
produce:hitpdune:DisambigFromSpacePoints          0.138942      0.266921      0.419762      0.277223      0.0720308       19     
produce:cluster3d:Cluster3D                        1.84051       3.55512       4.96326       3.78837      0.953743        19     
[art]:TriggerResults:TriggerResultInserter       2.9334e-05    4.15787e-05   0.000125589   3.5753e-05    2.07088e-05      19     
end_path:out1:RootOutput                         1.1788e-05    1.51522e-05   3.8909e-05     1.342e-05    5.76285e-06      19     
end_path:out1:RootOutput(write)                    4.27454       5.83822       6.5851        6.20543      0.727634        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5640.42 MB
  Peak resident set size usage (VmHWM): 3413.39 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0401_df-s04-d1_dw_0_20260203T211433_reco_stage0_20260204T005324_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0401_df-s04-d1_dw_0_20260203T211433_reco_stage0_20260204T005324_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0401_df-s04-d1_dw_0_20260203T211433_reco_stage0_20260204T005324_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162962.0,
    "core.end_time": 1770166406.0,
    "core.events": [
      45725,
      45731,
      45737,
      45743,
      45749,
      45755,
      45761,
      45767,
      45773,
      45779,
      45785,
      45791,
      45797,
      45803,
      45809,
      45815,
      45821,
      45827,
      45833
    ],
    "core.event_count": 19,
    "core.first_event_number": 45725,
    "core.last_event_number": 45833,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0401_df-s04-d1_dw_0_20260203T211433.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:40:39 UTC       justIN version: 01.06.00