justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.24@dunegpschedd02.fnal.gov

Jobsub ID297921.24@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:40:57
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:39
Input filesvd-protodune:np02vd_raw_run042433_0552_df-s03-d0_dw_0_20260203T221155.hdf5
JobscriptExit code0
Real time52m (3175s)
CPU time48m (2932s = 92%)
Max RSS bytes3386798080 (3229 MiB)
Outputting started2026-02-04 00:38:36
Output files
Finished2026-02-04 00:40:57
Saved logsjustin-logs:297921.24-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x55359740, 0x6a7637d0
Caught exception in looking up space point ptr... 0x6a7637d0, 0x72af3060
Caught exception in looking up space point ptr... 0x72af3060, 0x6a7637d0
Caught exception in looking up space point ptr... 0x2b44a200, 0x79e4e4d0
Caught exception in looking up space point ptr... 0x79e4e4d0, 0x2b449e70
Caught exception in looking up space point ptr... 0x5ab472c0, 0x5ab77210
Caught exception in looking up space point ptr... 0x5ab47820, 0x5ab472c0
Caught exception in looking up space point ptr... 0x48e8e7d0, 0x91a8f00
Caught exception in looking up space point ptr... 0x91a8f00, 0x5a9c6e10
Caught exception in looking up space point ptr... 0x6275bc20, 0x6274d4c0
Caught exception in looking up space point ptr... 0x627538a0, 0x6275bc20
Caught exception in looking up space point ptr... 0x62742ce0, 0x62740030
Caught exception in looking up space point ptr... 0x62740030, 0x6273fe50
Caught exception in looking up space point ptr... 0x799f2c20, 0x341c5310
Caught exception in looking up space point ptr... 0x39d6bbc0, 0x799f2c20
Caught exception in looking up space point ptr... 0x91bf390, 0x91bb460
Caught exception in looking up space point ptr... 0x91bb460, 0x91bb3c0
Caught exception in looking up space point ptr... 0x79a064e0, 0x79a06170
Caught exception in looking up space point ptr... 0x79a06170, 0x79a064e0
Caught exception in looking up space point ptr... 0x5ab572f0, 0x5ab57060
Caught exception in looking up space point ptr... 0x5ab57060, 0x5ab68bc0
Caught exception in looking up space point ptr... 0x5b0a82e0, 0x72a77e50
Caught exception in looking up space point ptr... 0x72a77e50, 0x72a77bb0
Caught exception in looking up space point ptr... 0x62f6fd70, 0x10dd9900
Caught exception in looking up space point ptr... 0x10dd9900, 0x392e0270
Caught exception in looking up space point ptr... 0x1a89c780, 0x1a89c5f0
Caught exception in looking up space point ptr... 0x1a89c5f0, 0x1a89c780
Caught exception in looking up space point ptr... 0x534543e0, 0x534540b0
Caught exception in looking up space point ptr... 0x534540b0, 0x13a99aa0
Caught exception in looking up space point ptr... 0x2b496fe0, 0x2b496d90
Caught exception in looking up space point ptr... 0x2b496d90, 0x2b496fe0
Caught exception in looking up space point ptr... 0x5c733590, 0xfc9e550
Caught exception in looking up space point ptr... 0xfc9e550, 0x5c732ae0
Caught exception in looking up space point ptr... 0x8163b610, 0x38f2c8d0
Caught exception in looking up space point ptr... 0x38f2c8d0, 0x8163b610
Caught exception in looking up space point ptr... 0x10f7d920, 0x10f7f1b0
Caught exception in looking up space point ptr... 0x10f7f1b0, 0x10f7d920
Caught exception in looking up space point ptr... 0x1236a8d0, 0x3a0e5980
Caught exception in looking up space point ptr... 0x3a0e5980, 0x1236a8d0
Caught exception in looking up space point ptr... 0x2f1fca60, 0x2f1fc800
Caught exception in looking up space point ptr... 0x2f203980, 0x2f1fca60
Caught exception in looking up space point ptr... 0x62e1b350, 0x62e23e40
Caught exception in looking up space point ptr... 0x62e23e40, 0x62e1d3e0
Caught exception in looking up space point ptr... 0x877c9a90, 0x5e3e6ac0
Caught exception in looking up space point ptr... 0x5e3e6ac0, 0x877c9a90
Caught exception in looking up space point ptr... 0x91b1700, 0x18a87670
Caught exception in looking up space point ptr... 0x18a87670, 0x91b0b70
Caught exception in looking up space point ptr... 0x5a9fc650, 0x5a9e63d0
Caught exception in looking up space point ptr... 0x5a9e63d0, 0x5a9fc650
Caught exception in looking up space point ptr... 0x2b2225b0, 0x2b21e470
Caught exception in looking up space point ptr... 0x2b21e470, 0x2b2225b0
Caught exception in looking up space point ptr... 0x53dbe9e0, 0x53dc1e70
Caught exception in looking up space point ptr... 0x53dc1e70, 0x6ffcc840
Caught exception in looking up space point ptr... 0x5b29c8f0, 0x5b29f220
Caught exception in looking up space point ptr... 0x5b29f220, 0x5b29f480
Caught exception in looking up space point ptr... 0x4d5689b0, 0x35633190
Caught exception in looking up space point ptr... 0x35633190, 0x4d5689b0
Caught exception in looking up space point ptr... 0x48bad560, 0x86b341f0
Caught exception in looking up space point ptr... 0x86b341f0, 0x48bd4ad0
Caught exception in looking up space point ptr... 0x5b1b4690, 0x4f6ab1e0
Caught exception in looking up space point ptr... 0x4f6ab1e0, 0x5b1b4690
Caught exception in looking up space point ptr... 0x4f5a2730, 0x4f5a2830
Caught exception in looking up space point ptr... 0x4f5a2830, 0x6fb17800
Caught exception in looking up space point ptr... 0x610fef30, 0x591554f0
Caught exception in looking up space point ptr... 0x591554f0, 0x610fef30
Caught exception in looking up space point ptr... 0x6035eb00, 0x6035dd90
Caught exception in looking up space point ptr... 0x6035dd90, 0x2b452c40
++++>>>> total num hits: 77885, num free: 75197
04-Feb-2026 00:38:28 GMT  Closed output file "np02vd_raw_run042433_0552_df-s03-d0_dw_0_20260203T221155_reco_stage0_20260204T003828_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         150.515       163.432       181.503       163.69        8.57962        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.6611e-05    0.000129072   0.000257263   0.000100101   4.89458e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.0731       13.678        13.9426       13.707       0.209219        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.515309      0.546046      0.582516      0.542809      0.0145784       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0176388     0.0180438     0.0183838     0.0180564    0.000190514      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176682     0.0180383     0.0185501     0.0180349    0.000227576      19     
produce:beamevent:BeamEvent                      0.000118862   0.000259101   0.00138587    0.000196063   0.000269048      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.77961       3.24326       3.50993       3.28622      0.193064        19     
produce:ophit:OpHitFinder                         0.0343309     0.0417045     0.0466464     0.0413042    0.00311373       19     
produce:wclsdatavd:WireCellToolkit                 99.7484       117.398       129.885       120.115       7.51329        19     
produce:gaushit:GausHitFinder                     0.986083       1.59949       2.09389       1.6648       0.338436        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000252754   0.000368549   0.000559637   0.000359354   6.97237e-05      19     
produce:reco3d:SpacePointSolver                    12.6541       17.4139       24.5455       16.3128       3.00416        19     
produce:hitpdune:DisambigFromSpacePoints          0.134063      0.236557      0.307602      0.243045      0.0518198       19     
produce:cluster3d:Cluster3D                        1.8828        3.2751        5.50258       3.04112      0.986178        19     
[art]:TriggerResults:TriggerResultInserter       3.3921e-05    5.16496e-05   0.000165332   4.1381e-05    2.8897e-05       19     
end_path:out1:RootOutput                          5.84e-06     1.43034e-05    4.614e-05     1.232e-05    8.10574e-06      19     
end_path:out1:RootOutput(write)                    4.2731        5.89708       6.63706       6.16059      0.672917        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5354.99 MB
  Peak resident set size usage (VmHWM): 3386.8 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0552_df-s03-d0_dw_0_20260203T221155_reco_stage0_20260204T003828_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0552_df-s03-d0_dw_0_20260203T221155_reco_stage0_20260204T003828_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0552_df-s03-d0_dw_0_20260203T221155_reco_stage0_20260204T003828_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162352.0,
    "core.end_time": 1770165510.0,
    "core.events": [
      62936,
      62942,
      62948,
      62954,
      62960,
      62966,
      62972,
      62978,
      62984,
      62990,
      62996,
      63002,
      63008,
      63014,
      63020,
      63026,
      63032,
      63038,
      63044
    ],
    "core.event_count": 19,
    "core.first_event_number": 62936,
    "core.last_event_number": 63044,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0552_df-s03-d0_dw_0_20260203T221155.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:42 UTC       justIN version: 01.06.00