justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.63@dunegpschedd02.fnal.gov

Jobsub ID297923.63@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:02:44
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:58:51
Input filesvd-protodune:np02vd_raw_run042433_0558_df-s03-d0_dw_0_20260203T221411.hdf5
JobscriptExit code0
Real time1h (3685s)
CPU time55m (3317s = 90%)
Max RSS bytes3335970816 (3181 MiB)
Outputting started2026-02-04 01:00:17
Output files
Finished2026-02-04 01:02:44
Saved logsjustin-logs:297923.63-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

up space point ptr... 0x8f605f0, 0x8f5a110
Caught exception in looking up space point ptr... 0x48eda520, 0x48ed0070
Caught exception in looking up space point ptr... 0x48ed0070, 0x48ecd450
Caught exception in looking up space point ptr... 0xe7693e0, 0xe767f00
Caught exception in looking up space point ptr... 0xe767f00, 0x1a158170
Caught exception in looking up space point ptr... 0x62346800, 0x17a2ee80
Caught exception in looking up space point ptr... 0x17a2ee80, 0x7f3dcd70
Caught exception in looking up space point ptr... 0x35d0e2f0, 0x62d6e860
Caught exception in looking up space point ptr... 0x62d6e860, 0x128ea080
Caught exception in looking up space point ptr... 0x7cc90050, 0x37c36950
Caught exception in looking up space point ptr... 0x37c36950, 0x7cc903e0
Caught exception in looking up space point ptr... 0x197e1ed0, 0x197e1d40
Caught exception in looking up space point ptr... 0x197e1d40, 0x12748160
Caught exception in looking up space point ptr... 0x12748160, 0x1273e870
Caught exception in looking up space point ptr... 0x1273e870, 0x1273e970
Caught exception in looking up space point ptr... 0x1273e970, 0x197e1ed0
Caught exception in looking up space point ptr... 0x5dd95d90, 0x5dd95f20
Caught exception in looking up space point ptr... 0x5dd95f20, 0x5dd95d90
Caught exception in looking up space point ptr... 0x63644820, 0x365614f0
Caught exception in looking up space point ptr... 0x365614f0, 0x7ea17780
Caught exception in looking up space point ptr... 0x424fc890, 0x1416f510
Caught exception in looking up space point ptr... 0x1416f510, 0x424fc890
Caught exception in looking up space point ptr... 0x31404bf0, 0x31409ac0
Caught exception in looking up space point ptr... 0x31409ac0, 0x31405530
Caught exception in looking up space point ptr... 0x39208410, 0x6c86e1b0
Caught exception in looking up space point ptr... 0x6c86e1b0, 0x39208410
Caught exception in looking up space point ptr... 0x7f3ce060, 0x1a605620
Caught exception in looking up space point ptr... 0x7f3ce1f0, 0x7f3ce060
Caught exception in looking up space point ptr... 0x181c2f10, 0x18132850
Caught exception in looking up space point ptr... 0x18132850, 0x562cee00
Caught exception in looking up space point ptr... 0xd271910, 0xd26dbc0
Caught exception in looking up space point ptr... 0xd26dbc0, 0xd271a10
Caught exception in looking up space point ptr... 0x37444e40, 0x37445270
Caught exception in looking up space point ptr... 0x37445270, 0x37444e40
Caught exception in looking up space point ptr... 0xf5f9470, 0xf5fbf20
Caught exception in looking up space point ptr... 0xf5f91d0, 0xf5f9470
Caught exception in looking up space point ptr... 0x54f56230, 0x54f508f0
Caught exception in looking up space point ptr... 0x54f508f0, 0x54f56230
Caught exception in looking up space point ptr... 0x5339f1f0, 0x5339f0c0
Caught exception in looking up space point ptr... 0x5339f0c0, 0x5339f1f0
Caught exception in looking up space point ptr... 0x49576ab0, 0x46012950
Caught exception in looking up space point ptr... 0x46012950, 0x49576ab0
Caught exception in looking up space point ptr... 0x36c778a0, 0x36c77360
Caught exception in looking up space point ptr... 0x36c77360, 0x36c778a0
Caught exception in looking up space point ptr... 0x141cd440, 0x141cbdb0
Caught exception in looking up space point ptr... 0x141cbdb0, 0x141cd440
Caught exception in looking up space point ptr... 0x1a61d540, 0x1a61d2a0
Caught exception in looking up space point ptr... 0x1a61d2a0, 0x1a61d540
Caught exception in looking up space point ptr... 0x7ed14680, 0x7ed14200
Caught exception in looking up space point ptr... 0x7ed14200, 0x7ed14680
Caught exception in looking up space point ptr... 0xb834b90, 0x3c2f7a60
Caught exception in looking up space point ptr... 0x3c2f7a60, 0xb834b90
Caught exception in looking up space point ptr... 0x4b10cb70, 0x4b10c850
Caught exception in looking up space point ptr... 0x4b10c850, 0x4b10cb70
Caught exception in looking up space point ptr... 0x11f27460, 0x11f272d0
Caught exception in looking up space point ptr... 0x11f272d0, 0x11f26fb0
Caught exception in looking up space point ptr... 0x513c6450, 0x513c5e60
Caught exception in looking up space point ptr... 0x513c5e60, 0x4dea7890
Caught exception in looking up space point ptr... 0xb208240, 0xb1fff90
Caught exception in looking up space point ptr... 0xb1fff90, 0xb20b4d0
Caught exception in looking up space point ptr... 0x7ef07830, 0x7ef052c0
Caught exception in looking up space point ptr... 0x7ef052c0, 0x7ef07830
Caught exception in looking up space point ptr... 0xabbb7e0, 0x31406290
Caught exception in looking up space point ptr... 0x31406290, 0x7e9d97d0
Caught exception in looking up space point ptr... 0x5d9e1430, 0x50b12e60
Caught exception in looking up space point ptr... 0x50b12e60, 0x390a2080
Caught exception in looking up space point ptr... 0x390a2080, 0x375954f0
++++>>>> total num hits: 95946, num free: 92154
04-Feb-2026 01:00:08 GMT  Closed output file "np02vd_raw_run042433_0558_df-s03-d0_dw_0_20260203T221411_reco_stage0_20260204T010008_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         163.662       189.204       220.577       186.319       15.6298        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.7865e-05    0.000145758   0.00032076    0.000108253   6.15861e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.4772       16.7891       38.2507       15.5167       5.0832         19     
produce:triggerrawdecoder:PDVDTriggerReader4       0.53084       0.54594      0.569254      0.547924      0.0104109       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0178091     0.0185881     0.0246386     0.018126     0.00152468       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176879     0.0179854     0.0187237     0.0179896    0.000227334      19     
produce:beamevent:BeamEvent                      0.000121718   0.000253336   0.00179692    0.000156854   0.000364853      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.07142       3.81559       4.26339       3.80018      0.258235        19     
produce:ophit:OpHitFinder                         0.0342384     0.0452041     0.0531222     0.0471471     0.005052        19     
produce:wclsdatavd:WireCellToolkit                 117.105       135.986       149.145       136.561       9.68817        19     
produce:gaushit:GausHitFinder                     0.954585       1.75396       2.52984       1.6393       0.456718        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000258565   0.000570367   0.00171726    0.000465882   0.000332938      19     
produce:reco3d:SpacePointSolver                    10.6721       19.6382       26.5925       19.4654       4.61764        19     
produce:hitpdune:DisambigFromSpacePoints          0.0972024     0.245921      0.418544       0.23616      0.0879288       19     
produce:cluster3d:Cluster3D                        1.73249       3.49202       5.38473       3.49171      0.880684        19     
[art]:TriggerResults:TriggerResultInserter       2.9586e-05    5.43836e-05   0.000303899   4.0346e-05    5.95664e-05      19     
end_path:out1:RootOutput                           5.5e-06     2.12161e-05   0.000103303   1.2904e-05    2.36269e-05      19     
end_path:out1:RootOutput(write)                    4.80278       6.80566       7.72074       6.98903      0.781494        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5344.65 MB
  Peak resident set size usage (VmHWM): 3335.97 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0558_df-s03-d0_dw_0_20260203T221411_reco_stage0_20260204T010008_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0558_df-s03-d0_dw_0_20260203T221411_reco_stage0_20260204T010008_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0558_df-s03-d0_dw_0_20260203T221411_reco_stage0_20260204T010008_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163144.0,
    "core.end_time": 1770166810.0,
    "core.events": [
      63620,
      63626,
      63632,
      63638,
      63644,
      63650,
      63656,
      63662,
      63668,
      63674,
      63680,
      63686,
      63692,
      63698,
      63704,
      63710,
      63716,
      63722,
      63728
    ],
    "core.event_count": 19,
    "core.first_event_number": 63620,
    "core.last_event_number": 63728,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0558_df-s03-d0_dw_0_20260203T221411.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:31:36 UTC       justIN version: 01.06.00