justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.27@dunegpschedd02.fnal.gov

Jobsub ID297921.27@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:45:19
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:39
Input filesvd-protodune:np02vd_raw_run042433_0410_df-s02-d1_dw_0_20260203T211757.hdf5
JobscriptExit code0
Real time57m (3434s)
CPU time52m (3168s = 92%)
Max RSS bytes3583037440 (3417 MiB)
Outputting started2026-02-04 00:42:55
Output files
Finished2026-02-04 00:45:19
Saved logsjustin-logs:297921.27-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nt ptr... 0x36524260, 0x3d90fb20
Caught exception in looking up space point ptr... 0x122ad250, 0x122ad150
Caught exception in looking up space point ptr... 0x122ad150, 0x122aeab0
Caught exception in looking up space point ptr... 0x122aefb0, 0x122ad250
Caught exception in looking up space point ptr... 0x6f23b300, 0x6f23ae50
Caught exception in looking up space point ptr... 0x6f23ae50, 0x6f23b300
Caught exception in looking up space point ptr... 0x68cd2110, 0x68cd1fc0
Caught exception in looking up space point ptr... 0x68cd1fc0, 0x68cd2110
Caught exception in looking up space point ptr... 0x15c97ae0, 0x57291810
Caught exception in looking up space point ptr... 0x57291810, 0x2b90f100
Caught exception in looking up space point ptr... 0x57294820, 0x15c97ae0
Caught exception in looking up space point ptr... 0x122439d0, 0xcc3d210
Caught exception in looking up space point ptr... 0xcc3d210, 0x4711f0c0
Caught exception in looking up space point ptr... 0x357683e0, 0x38d550b0
Caught exception in looking up space point ptr... 0x38d550b0, 0x357683e0
Caught exception in looking up space point ptr... 0x46e1d120, 0x3f255e30
Caught exception in looking up space point ptr... 0x3f255e30, 0x6916a7b0
Caught exception in looking up space point ptr... 0x1942a280, 0x4532ce90
Caught exception in looking up space point ptr... 0x3c728d30, 0x1942a280
Caught exception in looking up space point ptr... 0x3148eb90, 0x3148f170
Caught exception in looking up space point ptr... 0x3148f450, 0x3148eb90
Caught exception in looking up space point ptr... 0x39ff1a80, 0x3b7b2c70
Caught exception in looking up space point ptr... 0x3b7b2c70, 0x382a17e0
Caught exception in looking up space point ptr... 0x46e97990, 0x45159c20
Caught exception in looking up space point ptr... 0x45159c20, 0x46e97990
Caught exception in looking up space point ptr... 0x6e0e6e70, 0x6e0e6cb0
Caught exception in looking up space point ptr... 0x6e0e6cb0, 0x6e0e6e70
Caught exception in looking up space point ptr... 0x11c71ce0, 0x12d55790
Caught exception in looking up space point ptr... 0x12d55790, 0x11c71ce0
Caught exception in looking up space point ptr... 0x44def9a0, 0x44def3a0
Caught exception in looking up space point ptr... 0x44def3a0, 0x44def9a0
Caught exception in looking up space point ptr... 0x3b613720, 0x38b3ca70
Caught exception in looking up space point ptr... 0x38b3ca70, 0x4ff37a50
Caught exception in looking up space point ptr... 0x11c1f090, 0x11c1dd00
Caught exception in looking up space point ptr... 0x11c1dd00, 0x11c1dc60
Caught exception in looking up space point ptr... 0x614e3370, 0x69180bb0
Caught exception in looking up space point ptr... 0x69180bb0, 0x614e25a0
Caught exception in looking up space point ptr... 0x45779680, 0x45779380
Caught exception in looking up space point ptr... 0x45779380, 0x45779680
Caught exception in looking up space point ptr... 0x117d9b50, 0x117d9d50
Caught exception in looking up space point ptr... 0x117d9d50, 0x65e525d0
Caught exception in looking up space point ptr... 0x3a9c8c30, 0x3b67d350
Caught exception in looking up space point ptr... 0x3b67d350, 0x3a9c8c30
Caught exception in looking up space point ptr... 0x5d80c6f0, 0x3877f920
Caught exception in looking up space point ptr... 0x3877f920, 0x5d80c6f0
Caught exception in looking up space point ptr... 0x71cfae30, 0x61373ce0
Caught exception in looking up space point ptr... 0x61373ce0, 0x71cfae30
Caught exception in looking up space point ptr... 0xd458b30, 0x17ae3510
Caught exception in looking up space point ptr... 0x17ae3510, 0xd458b30
Caught exception in looking up space point ptr... 0x5d8ea4f0, 0x2c956630
Caught exception in looking up space point ptr... 0x2c956630, 0x5d8ea3f0
Caught exception in looking up space point ptr... 0x13feeaf0, 0x6d0b3300
Caught exception in looking up space point ptr... 0x6d0b3300, 0x6d0b3430
Caught exception in looking up space point ptr... 0x5dda77d0, 0x5dda7960
Caught exception in looking up space point ptr... 0x5dda7960, 0x5dda7c80
Caught exception in looking up space point ptr... 0xeaaeba0, 0xeaae900
Caught exception in looking up space point ptr... 0xeaae900, 0xeaaea50
Caught exception in looking up space point ptr... 0x61495810, 0x61495a70
Caught exception in looking up space point ptr... 0x61495a70, 0x61495810
Caught exception in looking up space point ptr... 0x8614e180, 0x8614e360
Caught exception in looking up space point ptr... 0x8614e360, 0x8614e180
Caught exception in looking up space point ptr... 0x8612cd10, 0x484d1ca0
Caught exception in looking up space point ptr... 0x484d1ca0, 0x8612cd10
Caught exception in looking up space point ptr... 0x59748f10, 0x35272fa0
Caught exception in looking up space point ptr... 0x35272fa0, 0x59748f10
Caught exception in looking up space point ptr... 0x2b848240, 0x384ff190
Caught exception in looking up space point ptr... 0x384ff190, 0x3857ec70
++++>>>> total num hits: 55213, num free: 52791
04-Feb-2026 00:42:47 GMT  Closed output file "np02vd_raw_run042433_0410_df-s02-d1_dw_0_20260203T211757_reco_stage0_20260204T004247_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         155.709       176.681       213.82        170.976       15.1025        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        6.969e-05    0.000142757   0.000316034   0.000122321   5.9006e-05       19     
produce:tpcrawdecoder:PDVDTPCReader                14.0787       14.6539       16.1278       14.6051      0.476544        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.532582      0.569756       0.6502        0.55549      0.0336687       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0179029     0.0198844     0.028598      0.0181463    0.00350352       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0178174     0.0181065     0.0186043     0.0181011    0.000188924      19     
produce:beamevent:BeamEvent                      0.000144402   0.000261116   0.00160186    0.000184253   0.000317521      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.72286       3.23559       3.72073       3.26188      0.255916        19     
produce:ophit:OpHitFinder                         0.033261      0.0394439     0.0430738     0.0400207    0.00259701       19     
produce:wclsdatavd:WireCellToolkit                 104.488       117.513       129.339       117.02        6.95673        19     
produce:gaushit:GausHitFinder                      1.25767       1.91783       3.18718       1.89833      0.481741        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000260264   0.000449057   0.000832901   0.000409025   0.000121191      19     
produce:reco3d:SpacePointSolver                    15.7688       26.4471       47.7732       25.3476       6.98453        19     
produce:hitpdune:DisambigFromSpacePoints          0.185237      0.307958      0.839372      0.274779      0.138378        19     
produce:cluster3d:Cluster3D                        1.70872       5.54102       24.4763       3.56401       5.82327        19     
[art]:TriggerResults:TriggerResultInserter        3.757e-05    4.98358e-05   0.000147532    4.356e-05    2.36948e-05      19     
end_path:out1:RootOutput                          5.651e-06    1.07138e-05   3.9011e-05     9.46e-06     6.79806e-06      19     
end_path:out1:RootOutput(write)                    4.84053       6.35043       7.89657       6.31506      0.572917        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5547.18 MB
  Peak resident set size usage (VmHWM): 3583.04 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0410_df-s02-d1_dw_0_20260203T211757_reco_stage0_20260204T004247_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0410_df-s02-d1_dw_0_20260203T211757_reco_stage0_20260204T004247_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0410_df-s02-d1_dw_0_20260203T211757_reco_stage0_20260204T004247_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162351.0,
    "core.end_time": 1770165769.0,
    "core.events": [
      46747,
      46753,
      46759,
      46765,
      46771,
      46777,
      46783,
      46789,
      46795,
      46801,
      46807,
      46813,
      46819,
      46825,
      46831,
      46837,
      46843,
      46849,
      46855
    ],
    "core.event_count": 19,
    "core.first_event_number": 46747,
    "core.last_event_number": 46855,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0410_df-s02-d1_dw_0_20260203T211757.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:36:54 UTC       justIN version: 01.06.00