justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.96@dunegpschedd02.fnal.gov

Jobsub ID297923.96@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:05:59
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:00:55
Input filesvd-protodune:np02vd_raw_run042433_0124_df-s04-d1_dw_0_20260203T192918.hdf5
JobscriptExit code0
Real time1h (3751s)
CPU time57m (3423s = 91%)
Max RSS bytes3391832064 (3234 MiB)
Outputting started2026-02-04 01:03:27
Output files
Finished2026-02-04 01:05:59
Saved logsjustin-logs:297923.96-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

king up space point ptr... 0xfe5cea0, 0xfe5d3e0
Caught exception in looking up space point ptr... 0xfe5d3e0, 0xbb382a0
Caught exception in looking up space point ptr... 0xbb382a0, 0xbb395c0
Caught exception in looking up space point ptr... 0xbb395c0, 0xfe5d240
Caught exception in looking up space point ptr... 0xf317f60, 0xf3158b0
Caught exception in looking up space point ptr... 0xf3158b0, 0xf317f60
Caught exception in looking up space point ptr... 0x53f24c00, 0x53f24b00
Caught exception in looking up space point ptr... 0x5112d670, 0x53f24c00
Caught exception in looking up space point ptr... 0x34950b00, 0x6413ed90
Caught exception in looking up space point ptr... 0x6413ed90, 0x64140420
Caught exception in looking up space point ptr... 0x38b0d1a0, 0x38b0cfc0
Caught exception in looking up space point ptr... 0x38b0cfc0, 0x38b0ce00
Caught exception in looking up space point ptr... 0x150858a0, 0x62246400
Caught exception in looking up space point ptr... 0x102d3120, 0x5135d5a0
Caught exception in looking up space point ptr... 0x5135d5a0, 0x192a35c0
Caught exception in looking up space point ptr... 0x192a35c0, 0x7dbabc20
Caught exception in looking up space point ptr... 0x7dbabc20, 0x150858a0
Caught exception in looking up space point ptr... 0x39db5910, 0x39db5230
Caught exception in looking up space point ptr... 0x39db5230, 0x39db57e0
Caught exception in looking up space point ptr... 0x85ba4700, 0x85ba44b0
Caught exception in looking up space point ptr... 0x85ba4260, 0x85ba4700
Caught exception in looking up space point ptr... 0x670d36f0, 0x548bb600
Caught exception in looking up space point ptr... 0x548bb600, 0x670d36f0
Caught exception in looking up space point ptr... 0x3b594380, 0x54607e50
Caught exception in looking up space point ptr... 0x54607e50, 0x543f39e0
Caught exception in looking up space point ptr... 0x62242e10, 0x622438e0
Caught exception in looking up space point ptr... 0x622438e0, 0x62243530
Caught exception in looking up space point ptr... 0x58308f50, 0x39e1cff0
Caught exception in looking up space point ptr... 0x39e1cff0, 0x383fe6c0
Caught exception in looking up space point ptr... 0xb2c3990, 0xb2c3850
Caught exception in looking up space point ptr... 0xb2c3850, 0xb2c3990
Caught exception in looking up space point ptr... 0x37d7fa80, 0x33872ff0
Caught exception in looking up space point ptr... 0x33872ff0, 0x37d7fa80
Caught exception in looking up space point ptr... 0x5c785b90, 0x5c785d50
Caught exception in looking up space point ptr... 0x5c785d50, 0xb0736f0
Caught exception in looking up space point ptr... 0x39db3110, 0x39db3070
Caught exception in looking up space point ptr... 0x34950640, 0x39db3110
Caught exception in looking up space point ptr... 0x7c6ad640, 0x7c6ad810
Caught exception in looking up space point ptr... 0x7c6ad810, 0x7c6ad640
Caught exception in looking up space point ptr... 0x47a28c20, 0x3b4fcac0
Caught exception in looking up space point ptr... 0x3b4fcac0, 0x565896d0
Caught exception in looking up space point ptr... 0x552f9570, 0x552f9180
Caught exception in looking up space point ptr... 0x56955190, 0x552f9570
Caught exception in looking up space point ptr... 0x7a5810e0, 0x955aba0
Caught exception in looking up space point ptr... 0x955aba0, 0x7a5810e0
Caught exception in looking up space point ptr... 0x38afee80, 0x38afd6b0
Caught exception in looking up space point ptr... 0x38afd6b0, 0x38afdd50
Caught exception in looking up space point ptr... 0x3f256880, 0x4f349b60
Caught exception in looking up space point ptr... 0x4f349b60, 0x3f256880
Caught exception in looking up space point ptr... 0x7d7ac270, 0x14e25030
Caught exception in looking up space point ptr... 0x14e25030, 0x7d7ac270
Caught exception in looking up space point ptr... 0x63ba43e0, 0x4760ca90
Caught exception in looking up space point ptr... 0x4760ca90, 0x3b4910f0
Caught exception in looking up space point ptr... 0x86565180, 0x16f8e920
Caught exception in looking up space point ptr... 0x16f8e920, 0x86565310
Caught exception in looking up space point ptr... 0x1a691720, 0x3b56a320
Caught exception in looking up space point ptr... 0x3b56a320, 0x1a691720
Caught exception in looking up space point ptr... 0x326b39d0, 0x4cf349a0
Caught exception in looking up space point ptr... 0x4cf349a0, 0x326b39d0
Caught exception in looking up space point ptr... 0x7cea67d0, 0x7cea5310
Caught exception in looking up space point ptr... 0x7cea5310, 0x7cea67d0
Caught exception in looking up space point ptr... 0xf19abd0, 0xf19a980
Caught exception in looking up space point ptr... 0xf19a980, 0xf19a730
Caught exception in looking up space point ptr... 0xf199ff0, 0x51730d00
Caught exception in looking up space point ptr... 0x51730d00, 0xf19a240
Caught exception in looking up space point ptr... 0x3418f470, 0x71e12d40
Caught exception in looking up space point ptr... 0x71e12d40, 0x3418f470
++++>>>> total num hits: 45876, num free: 44033
04-Feb-2026 01:03:19 GMT  Closed output file "np02vd_raw_run042433_0124_df-s04-d1_dw_0_20260203T192918_reco_stage0_20260204T010319_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         165.89        192.605       216.842       194.433       14.6575        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.5599e-05    0.000182722   0.000392455   0.000193302   8.16181e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.587        15.1255       15.7498       15.1032      0.287506        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.527255      0.547904      0.586931      0.545865      0.016974        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.017743      0.019541      0.0458456     0.0180157    0.00620788       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176392     0.0179782     0.018693      0.0179183    0.000231204      19     
produce:beamevent:BeamEvent                      0.000127919   0.000240462   0.00111062    0.000192401   0.000207518      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.40346       3.75129       4.1889        3.66855       0.24055        19     
produce:ophit:OpHitFinder                         0.0396923     0.047595      0.0545168     0.0479703    0.00371873       19     
produce:wclsdatavd:WireCellToolkit                 116.92        136.749       154.559       137.322       12.2138        19     
produce:gaushit:GausHitFinder                      1.03273       1.76646        2.612        1.76033      0.393194        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000351869   0.000618493    0.0020929    0.000472685   0.000455768      19     
produce:reco3d:SpacePointSolver                    14.7429       24.2151       40.6042       23.3382       7.28625        19     
produce:hitpdune:DisambigFromSpacePoints          0.135158      0.252271      0.495101       0.20897      0.101484        19     
produce:cluster3d:Cluster3D                        1.71337       3.51313       8.53134       2.73536       1.8863         19     
[art]:TriggerResults:TriggerResultInserter       2.9364e-05    4.58521e-05   0.00012273    4.1949e-05    2.00082e-05      19     
end_path:out1:RootOutput                          4.018e-06    1.16776e-05   3.7971e-05     9.608e-06    7.6268e-06       19     
end_path:out1:RootOutput(write)                    4.81643       6.53979       8.3031        6.67213       1.02975        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5431.92 MB
  Peak resident set size usage (VmHWM): 3391.83 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0124_df-s04-d1_dw_0_20260203T192918_reco_stage0_20260204T010319_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0124_df-s04-d1_dw_0_20260203T192918_reco_stage0_20260204T010319_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0124_df-s04-d1_dw_0_20260203T192918_reco_stage0_20260204T010319_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163267.0,
    "core.end_time": 1770167000.0,
    "core.events": [
      14147,
      14153,
      14159,
      14165,
      14171,
      14177,
      14183,
      14189,
      14195,
      14201,
      14207,
      14213,
      14219,
      14225,
      14231,
      14237,
      14243,
      14249,
      14255
    ],
    "core.event_count": 19,
    "core.first_event_number": 14147,
    "core.last_event_number": 14255,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0124_df-s04-d1_dw_0_20260203T192918.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:45:32 UTC       justIN version: 01.06.00