justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.71@dunegpschedd02.fnal.gov

Jobsub ID297923.71@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:03:51
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:59:05
Input filesvd-protodune:np02vd_raw_run042433_0479_df-s04-d0_dw_0_20260203T214411.hdf5
JobscriptExit code0
Real time1h (3732s)
CPU time56m (3369s = 90%)
Max RSS bytes3425492992 (3266 MiB)
Outputting started2026-02-04 01:01:18
Output files
Finished2026-02-04 01:03:51
Saved logsjustin-logs:297923.71-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oint ptr... 0x152c97e0, 0x16176770
Caught exception in looking up space point ptr... 0x16176770, 0x152c97e0
Caught exception in looking up space point ptr... 0x86e5a360, 0x86e5a260
Caught exception in looking up space point ptr... 0x86e5a260, 0x161a33a0
Caught exception in looking up space point ptr... 0x77f40200, 0x77f40390
Caught exception in looking up space point ptr... 0x77f3e010, 0x77f40200
Caught exception in looking up space point ptr... 0x19021290, 0x19023280
Caught exception in looking up space point ptr... 0x19023730, 0x19021290
Caught exception in looking up space point ptr... 0x7d9e6730, 0x7d9ea540
Caught exception in looking up space point ptr... 0x7d9ea540, 0x7d9ea3b0
Caught exception in looking up space point ptr... 0x7d9ea3b0, 0x7d9e68c0
Caught exception in looking up space point ptr... 0x5648b680, 0x414bc330
Caught exception in looking up space point ptr... 0x414bc330, 0x56487130
Caught exception in looking up space point ptr... 0x632bc1f0, 0x616684a0
Caught exception in looking up space point ptr... 0x616684a0, 0x632c1a90
Caught exception in looking up space point ptr... 0x41647ff0, 0x416481f0
Caught exception in looking up space point ptr... 0x416481f0, 0x41647ff0
Caught exception in looking up space point ptr... 0x161a5670, 0x16185320
Caught exception in looking up space point ptr... 0x16185320, 0x1619ac80
Caught exception in looking up space point ptr... 0x18032e60, 0x18032cd0
Caught exception in looking up space point ptr... 0x18032cd0, 0x7f033930
Caught exception in looking up space point ptr... 0x7e8b2240, 0x1354f8d0
Caught exception in looking up space point ptr... 0x1354f8d0, 0x15297c50
Caught exception in looking up space point ptr... 0x4ec962c0, 0x4ec89660
Caught exception in looking up space point ptr... 0x4ec89660, 0x4ec897b0
Caught exception in looking up space point ptr... 0x406897f0, 0x54376f40
Caught exception in looking up space point ptr... 0x54376f40, 0x406897f0
Caught exception in looking up space point ptr... 0x8c4b1e20, 0x8c4b1b80
Caught exception in looking up space point ptr... 0x8c4b1b80, 0x8c4b1e20
Caught exception in looking up space point ptr... 0x407f1610, 0x3cac2dd0
Caught exception in looking up space point ptr... 0x3c2cf5b0, 0x407f1610
Caught exception in looking up space point ptr... 0x194ed390, 0x541f40b0
Caught exception in looking up space point ptr... 0x541f40b0, 0x194ed390
Caught exception in looking up space point ptr... 0x48445bc0, 0x48445f10
Caught exception in looking up space point ptr... 0x48445f10, 0x48445bc0
Caught exception in looking up space point ptr... 0x4022a860, 0x13a2a4d0
Caught exception in looking up space point ptr... 0x13a2a4d0, 0x13a2b310
Caught exception in looking up space point ptr... 0x9b037d0, 0x9b03080
Caught exception in looking up space point ptr... 0x9b03080, 0x9b037d0
Caught exception in looking up space point ptr... 0x9b037d0, 0x9b034b0
Caught exception in looking up space point ptr... 0x9b034b0, 0x9b037d0
Caught exception in looking up space point ptr... 0x19157790, 0x1915b120
Caught exception in looking up space point ptr... 0x1915b120, 0x191576a0
Caught exception in looking up space point ptr... 0x4bb15120, 0x4bb14f90
Caught exception in looking up space point ptr... 0x4bb14f90, 0x4bb14e00
Caught exception in looking up space point ptr... 0x14086de0, 0x57391d30
Caught exception in looking up space point ptr... 0x57391d30, 0x14086de0
Caught exception in looking up space point ptr... 0x899b3a40, 0x899b57b0
Caught exception in looking up space point ptr... 0x899b57b0, 0x899b3a40
Caught exception in looking up space point ptr... 0x899b3a40, 0x899b5920
Caught exception in looking up space point ptr... 0x899b5920, 0x899b3a40
Caught exception in looking up space point ptr... 0x1a36a450, 0x1a36a350
Caught exception in looking up space point ptr... 0x1a36a350, 0x1a36a450
Caught exception in looking up space point ptr... 0x40681f60, 0x406822d0
Caught exception in looking up space point ptr... 0x406822d0, 0x40681f60
Caught exception in looking up space point ptr... 0x5638dba0, 0x5638d770
Caught exception in looking up space point ptr... 0x5638d770, 0x5638d390
Caught exception in looking up space point ptr... 0x3bccbf50, 0x56a67b50
Caught exception in looking up space point ptr... 0x56a67b50, 0x56a67ca0
Caught exception in looking up space point ptr... 0x3c1a39b0, 0x4f64f970
Caught exception in looking up space point ptr... 0x4f64f970, 0x4f64f870
Caught exception in looking up space point ptr... 0x3ccfda60, 0xeb16f80
Caught exception in looking up space point ptr... 0xeb16f80, 0x4f4f3580
Caught exception in looking up space point ptr... 0x39e10560, 0x687fdac0
Caught exception in looking up space point ptr... 0x687fdac0, 0x39e10560
Caught exception in looking up space point ptr... 0x39e10560, 0x39e106f0
Caught exception in looking up space point ptr... 0x39e106f0, 0x39e10560
++++>>>> total num hits: 93204, num free: 89058
04-Feb-2026 01:01:10 GMT  Closed output file "np02vd_raw_run042433_0479_df-s04-d0_dw_0_20260203T214411_reco_stage0_20260204T010110_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         161.859       191.913       232.779       191.188       14.6273        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.6661e-05    0.000180251   0.000394829   0.000165189   8.02933e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.7411       16.5973       18.7944       16.3478       1.15965        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.528376      0.593099      0.694342      0.582449      0.0450674       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0178129     0.0192744     0.0290919     0.0181812    0.00278903       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177163     0.0180807      0.01863      0.0180784    0.00022918       19     
produce:beamevent:BeamEvent                      0.000132207   0.000303581   0.00150843    0.000196378   0.000338214      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.16571       3.84694       4.80529       3.84211      0.318108        19     
produce:ophit:OpHitFinder                         0.0379837     0.0466631     0.0522241     0.0470646    0.00370824       19     
produce:wclsdatavd:WireCellToolkit                 124.261       139.557       155.417       141.495       7.94548        19     
produce:gaushit:GausHitFinder                     0.727871       1.72332       2.71528       1.56274      0.519788        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000232024   0.000566346   0.00138043    0.000429485   0.000310119      19     
produce:reco3d:SpacePointSolver                    8.16886       18.3988       31.0644       17.2568       5.94626        19     
produce:hitpdune:DisambigFromSpacePoints          0.096866      0.249718      0.471863      0.250769      0.0958071       19     
produce:cluster3d:Cluster3D                       0.945493       4.18964       23.955        3.2159        4.77234        19     
[art]:TriggerResults:TriggerResultInserter       3.0026e-05    4.77726e-05   0.000148889   3.7921e-05    2.68846e-05      19     
end_path:out1:RootOutput                          6.552e-06    1.38575e-05   4.4904e-05    1.1231e-05    7.97811e-06      19     
end_path:out1:RootOutput(write)                    4.42596       6.61746       8.94184       7.0352        1.20382        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5447.17 MB
  Peak resident set size usage (VmHWM): 3425.49 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0479_df-s04-d0_dw_0_20260203T214411_reco_stage0_20260204T010110_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0479_df-s04-d0_dw_0_20260203T214411_reco_stage0_20260204T010110_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0479_df-s04-d0_dw_0_20260203T214411_reco_stage0_20260204T010110_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163158.0,
    "core.end_time": 1770166872.0,
    "core.events": [
      54616,
      54622,
      54628,
      54634,
      54640,
      54646,
      54652,
      54658,
      54664,
      54670,
      54676,
      54682,
      54688,
      54694,
      54700,
      54706,
      54712,
      54718,
      54724
    ],
    "core.event_count": 19,
    "core.first_event_number": 54616,
    "core.last_event_number": 54724,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0479_df-s04-d0_dw_0_20260203T214411.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:13:00 UTC       justIN version: 01.06.00