justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.18@dunegpschedd02.fnal.gov

Jobsub ID297921.18@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:40:15
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:42
Input filesvd-protodune:np02vd_raw_run042433_0247_df-s04-d1_dw_0_20260203T201600.hdf5
JobscriptExit code0
Real time52m (3126s)
CPU time47m (2868s = 91%)
Max RSS bytes3369033728 (3212 MiB)
Outputting started2026-02-04 00:37:50
Output files
Finished2026-02-04 00:40:15
Saved logsjustin-logs:297921.18-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x508df290, 0x508ddb30
Caught exception in looking up space point ptr... 0x508ddb30, 0x508dd9a0
Caught exception in looking up space point ptr... 0x5090e7a0, 0x5090e700
Caught exception in looking up space point ptr... 0x5090e700, 0x5090cdb0
Caught exception in looking up space point ptr... 0x5891cc40, 0x65f4c740
Caught exception in looking up space point ptr... 0x65f4c740, 0x1567b700
Caught exception in looking up space point ptr... 0x3db3db80, 0x3db3c850
Caught exception in looking up space point ptr... 0x3db3c850, 0x3db3da80
Caught exception in looking up space point ptr... 0x36451160, 0x36451480
Caught exception in looking up space point ptr... 0x36451480, 0x364517a0
Caught exception in looking up space point ptr... 0x364517a0, 0x364512f0
Caught exception in looking up space point ptr... 0x364512f0, 0x36451160
Caught exception in looking up space point ptr... 0x3a9abf60, 0x59bfdab0
Caught exception in looking up space point ptr... 0x59bfdab0, 0x722a0ce0
Caught exception in looking up space point ptr... 0x49509c90, 0x4c3b9df0
Caught exception in looking up space point ptr... 0x4c3b9df0, 0x3bd37ac0
Caught exception in looking up space point ptr... 0x147d2c80, 0x147d3180
Caught exception in looking up space point ptr... 0x147d3180, 0x147d2c80
Caught exception in looking up space point ptr... 0x146b58d0, 0x146b59d0
Caught exception in looking up space point ptr... 0x146b59d0, 0x146b58d0
Caught exception in looking up space point ptr... 0x7495b740, 0x7495b5d0
Caught exception in looking up space point ptr... 0x7495b5d0, 0x7494dce0
Caught exception in looking up space point ptr... 0x7494dce0, 0x7495b5d0
Caught exception in looking up space point ptr... 0x7495b5d0, 0x7495a5f0
Caught exception in looking up space point ptr... 0x7495a5f0, 0x7495b8d0
Caught exception in looking up space point ptr... 0x50ca5ae0, 0x50abfac0
Caught exception in looking up space point ptr... 0x50abfac0, 0x50ca5ae0
Caught exception in looking up space point ptr... 0x55f2c5c0, 0x5a4e3f40
Caught exception in looking up space point ptr... 0x5a4e3f40, 0x55f2c5c0
Caught exception in looking up space point ptr... 0x19c72970, 0x19c740c0
Caught exception in looking up space point ptr... 0x19c740c0, 0x19c72970
Caught exception in looking up space point ptr... 0x5a5966d0, 0x5a59c070
Caught exception in looking up space point ptr... 0x5a59c070, 0x5a59c300
Caught exception in looking up space point ptr... 0x615940b0, 0x61597ee0
Caught exception in looking up space point ptr... 0x61597ee0, 0x6158f7d0
Caught exception in looking up space point ptr... 0x50ad1900, 0x50ad8710
Caught exception in looking up space point ptr... 0x50ad8710, 0x50ada9d0
Caught exception in looking up space point ptr... 0x933f070, 0x933ef10
Caught exception in looking up space point ptr... 0x933ef10, 0x933f070
Caught exception in looking up space point ptr... 0x3a922a50, 0x3a922e50
Caught exception in looking up space point ptr... 0x3a922e50, 0x3a922a50
Caught exception in looking up space point ptr... 0x677a4ef0, 0x677a7b30
Caught exception in looking up space point ptr... 0x677a7b30, 0x677a4ef0
Caught exception in looking up space point ptr... 0x50a9ba80, 0x50a99ba0
Caught exception in looking up space point ptr... 0x50a99ba0, 0x50a9d420
Caught exception in looking up space point ptr... 0x3bd82000, 0x3bd81ce0
Caught exception in looking up space point ptr... 0x3bd81ce0, 0x3bd82000
Caught exception in looking up space point ptr... 0x403ea1a0, 0x166bac50
Caught exception in looking up space point ptr... 0x166bac50, 0x166ba8e0
Caught exception in looking up space point ptr... 0x3dbf59d0, 0x403e9030
Caught exception in looking up space point ptr... 0x403e9030, 0x3dbf59d0
Caught exception in looking up space point ptr... 0x9355620, 0x934b770
Caught exception in looking up space point ptr... 0x9353da0, 0x9355620
Caught exception in looking up space point ptr... 0x3a901c40, 0x14a1c740
Caught exception in looking up space point ptr... 0x14a1c740, 0x3a901c40
Caught exception in looking up space point ptr... 0x937cef0, 0x937e3a0
Caught exception in looking up space point ptr... 0x937e3a0, 0x937cef0
Caught exception in looking up space point ptr... 0x46e42e10, 0x46e45b10
Caught exception in looking up space point ptr... 0x46e45b10, 0x46e49d80
Caught exception in looking up space point ptr... 0x50ca5580, 0x50ca21f0
Caught exception in looking up space point ptr... 0x50ca21f0, 0x50ca5580
Caught exception in looking up space point ptr... 0x50ca5580, 0x50ca8870
Caught exception in looking up space point ptr... 0x50ca8870, 0x50ca5580
Caught exception in looking up space point ptr... 0x133c5870, 0x50b9a090
Caught exception in looking up space point ptr... 0x62ecf9f0, 0x133c5870
Caught exception in looking up space point ptr... 0x3db32d00, 0x55b8fc40
Caught exception in looking up space point ptr... 0x55b8fc40, 0x3db32d00
++++>>>> total num hits: 77894, num free: 74808
04-Feb-2026 00:37:43 GMT  Closed output file "np02vd_raw_run042433_0247_df-s04-d1_dw_0_20260203T201600_reco_stage0_20260204T003743_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         147.245       160.576       177.632       160.722       8.05904        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       6.5531e-05    0.000126172   0.000223343   0.000105121   4.70266e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.5862       14.3564       21.2934       13.9576       1.65607        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.527449      0.547837      0.587349      0.545961      0.0151521       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177562     0.0182979     0.0247577     0.0179368    0.00152938       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177124     0.0179441     0.0182332     0.0179364    0.000126265      19     
produce:beamevent:BeamEvent                      0.000138812   0.000224778   0.00106939    0.000177772   0.000200391      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.86715       3.26437       3.50098       3.30087      0.165892        19     
produce:ophit:OpHitFinder                          0.0282       0.0397215     0.0465812     0.0403881     0.0044579       19     
produce:wclsdatavd:WireCellToolkit                 104.796       115.752       122.568       116.133       4.7898         19     
produce:gaushit:GausHitFinder                     0.734607       1.26316       1.78733       1.25337      0.312674        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000174542   0.000327828   0.000666999   0.000320474   0.000104695      19     
produce:reco3d:SpacePointSolver                    9.85535       16.1166       24.7094       16.6453       3.80477        19     
produce:hitpdune:DisambigFromSpacePoints          0.112974      0.217002      0.324157      0.237869      0.0565544       19     
produce:cluster3d:Cluster3D                        1.28305       3.00316       5.7724        3.18946       1.05433        19     
[art]:TriggerResults:TriggerResultInserter       2.8131e-05    4.30737e-05   0.000117952   4.0101e-05    1.83473e-05      19     
end_path:out1:RootOutput                          6.92e-06     1.24644e-05   3.1371e-05     1.119e-05    6.32331e-06      19     
end_path:out1:RootOutput(write)                    4.05478       5.92526       6.64699       6.16718      0.627774        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5350.85 MB
  Peak resident set size usage (VmHWM): 3369.03 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0247_df-s04-d1_dw_0_20260203T201600_reco_stage0_20260204T003743_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0247_df-s04-d1_dw_0_20260203T201600_reco_stage0_20260204T003743_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0247_df-s04-d1_dw_0_20260203T201600_reco_stage0_20260204T003743_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162354.0,
    "core.end_time": 1770165464.0,
    "core.events": [
      28169,
      28175,
      28181,
      28187,
      28193,
      28199,
      28205,
      28211,
      28217,
      28223,
      28229,
      28235,
      28241,
      28247,
      28253,
      28259,
      28265,
      28271,
      28277
    ],
    "core.event_count": 19,
    "core.first_event_number": 28169,
    "core.last_event_number": 28277,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0247_df-s04-d1_dw_0_20260203T201600.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:34 UTC       justIN version: 01.06.00