justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303647.130@dunegpschedd01.fnal.gov

Jobsub ID303647.130@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:46:24
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce512
Last heartbeat2026-02-03 23:33:43
From worker nodeHostnameb9p10p5730.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 22:47:08
Input filesvd-protodune:np02vd_raw_run042421_0165_df-s04-d0_dw_0_20260203T150014.hdf5
JobscriptExit code0
Real time43m (2639s)
CPU time42m (2561s = 97%)
Max RSS bytes2716217344 (2590 MiB)
Outputting started2026-02-03 23:31:09
Output files
Finished2026-02-03 23:33:43
Saved logsjustin-logs:303647.130-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ht exception in looking up space point ptr... 0x3a925cf0, 0x3a925b60
Caught exception in looking up space point ptr... 0x298cdca0, 0x2bb54d90
Caught exception in looking up space point ptr... 0x2bb54d90, 0x298cdca0
Caught exception in looking up space point ptr... 0x46d7a3f0, 0x46d84480
Caught exception in looking up space point ptr... 0x46d84480, 0x46d826d0
Caught exception in looking up space point ptr... 0x5ec28fe0, 0x5ec16520
Caught exception in looking up space point ptr... 0x46c8b190, 0x5ec28fe0
Caught exception in looking up space point ptr... 0x3d3234b0, 0x58971670
Caught exception in looking up space point ptr... 0x58971670, 0x46af8fc0
Caught exception in looking up space point ptr... 0x5ec1a260, 0x38b77250
Caught exception in looking up space point ptr... 0x5ec2c9b0, 0x5ec1a260
Caught exception in looking up space point ptr... 0x5ec17830, 0x38b74820
Caught exception in looking up space point ptr... 0x5ec29f80, 0x5ec17830
Caught exception in looking up space point ptr... 0x63a1c020, 0x63a1ccc0
Caught exception in looking up space point ptr... 0x63a1ccc0, 0x3e972360
Caught exception in looking up space point ptr... 0x2bbeca50, 0x2bbec8c0
Caught exception in looking up space point ptr... 0x2bbec8c0, 0x306dd330
Caught exception in looking up space point ptr... 0x3e9180c0, 0x3dca6750
Caught exception in looking up space point ptr... 0x3e918250, 0x3e9180c0
Caught exception in looking up space point ptr... 0x4e6a1ef0, 0x4e68ff20
Caught exception in looking up space point ptr... 0x4e68ff20, 0x45da50a0
Caught exception in looking up space point ptr... 0x46d7b650, 0x46d7b470
Caught exception in looking up space point ptr... 0x46d7b470, 0x46d7b650
Caught exception in looking up space point ptr... 0x45940920, 0x58e31860
Caught exception in looking up space point ptr... 0x58e31860, 0x45940920
Caught exception in looking up space point ptr... 0x28bffbc0, 0x28bff9e0
Caught exception in looking up space point ptr... 0x28bff9e0, 0x28bffbc0
Caught exception in looking up space point ptr... 0x369dad30, 0x2d9d02d0
Caught exception in looking up space point ptr... 0x2d9d02d0, 0x369dad30
Caught exception in looking up space point ptr... 0x63332fe0, 0x63333170
Caught exception in looking up space point ptr... 0x63333170, 0x63332fe0
Caught exception in looking up space point ptr... 0x2102d100, 0x4b59e370
Caught exception in looking up space point ptr... 0x4b59e370, 0x2102d100
Caught exception in looking up space point ptr... 0x51d55e30, 0x51d55ca0
Caught exception in looking up space point ptr... 0x51d55ca0, 0x51d55e30
Caught exception in looking up space point ptr... 0x5436f8c0, 0x4c89b5f0
Caught exception in looking up space point ptr... 0x4c89b5f0, 0x5436f8c0
Caught exception in looking up space point ptr... 0x54370ca0, 0x4ad83cf0
Caught exception in looking up space point ptr... 0x4ad83cf0, 0x32d001a0
Caught exception in looking up space point ptr... 0x4d18ed70, 0x4d18c810
Caught exception in looking up space point ptr... 0x4d18c810, 0x4d18ed70
Caught exception in looking up space point ptr... 0xae4a820, 0xae4ab40
Caught exception in looking up space point ptr... 0xae4ab40, 0xae4a820
Caught exception in looking up space point ptr... 0x43a21900, 0x43a22120
Caught exception in looking up space point ptr... 0x43a22120, 0x3e0b5530
Caught exception in looking up space point ptr... 0x379c5b50, 0x379d67c0
Caught exception in looking up space point ptr... 0x379d67c0, 0x379da3f0
Caught exception in looking up space point ptr... 0x46011010, 0x460114c0
Caught exception in looking up space point ptr... 0x460114c0, 0x46011010
Caught exception in looking up space point ptr... 0x56b8afb0, 0x56b8b140
Caught exception in looking up space point ptr... 0x56b8b140, 0x56b8afb0
Caught exception in looking up space point ptr... 0x46d8a0b0, 0x46d897d0
Caught exception in looking up space point ptr... 0x46d897d0, 0x46d898d0
Caught exception in looking up space point ptr... 0x6208fc30, 0x41cfb0a0
Caught exception in looking up space point ptr... 0x41cfb0a0, 0x6208fc30
Caught exception in looking up space point ptr... 0x63330e70, 0x41ced000
Caught exception in looking up space point ptr... 0x41ced000, 0x63330e70
Caught exception in looking up space point ptr... 0x53218130, 0x3dc1f930
Caught exception in looking up space point ptr... 0x3dc1f930, 0x53218130
Caught exception in looking up space point ptr... 0x37355340, 0x2fbc1900
Caught exception in looking up space point ptr... 0x2fbc1900, 0x2fbb33d0
Caught exception in looking up space point ptr... 0x10a6e5a0, 0x3a925520
Caught exception in looking up space point ptr... 0x3a925520, 0x10a6e5a0
Caught exception in looking up space point ptr... 0x10a6e5a0, 0x10a6e730
Caught exception in looking up space point ptr... 0x10a6e730, 0x10a6e5a0
++++>>>> total num hits: 311333, num free: 304239
04-Feb-2026 00:31:01 CET  Closed output file "np02vd_raw_run042421_0165_df-s04-d0_dw_0_20260203T150014_reco_stage0_20260203T233101_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         64.1064       95.6923       124.192       100.04        16.3773        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       3.6261e-05    8.12876e-05   0.000190052    6.946e-05    3.81327e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                3.10961       3.63578       4.01174       3.66933      0.296851        27     
produce:triggerrawdecoder:PDVDTriggerReader4     0.00961933     0.0159636     0.0409602     0.0134123     0.0070047       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000385885   0.000531415   0.00079128    0.000494316   0.000111229      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00144792    0.00187987    0.00294724    0.00182283    0.000373823      27     
produce:beamevent:BeamEvent                       5.51e-05     0.000110504   0.000981062   7.5451e-05    0.000171289      27     
produce:pdvddaphne:DAPHNEReaderPDVD                2.01191       2.46748       3.19384       2.38444      0.320287        27     
produce:ophit:OpHitFinder                         0.020249      0.0321487     0.0417982     0.0322569     0.0044994       27     
produce:wclsdatavd:WireCellToolkit                 50.6258       57.2691       72.2132       56.4102       5.17255        27     
produce:gaushit:GausHitFinder                     0.525299       1.00576       1.81669      0.993261      0.272337        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000143111   0.000235042   0.000551867   0.000227942   7.15186e-05      27     
produce:reco3d:SpacePointSolver                    3.33208       6.12699       10.2246       5.65582       1.83572        27     
produce:hitpdune:DisambigFromSpacePoints          0.0438346      0.14106      0.248644      0.134089      0.0600089       27     
produce:cluster3d:Cluster3D                       0.665039       21.2392       32.9924       28.1687       13.0108        27     
[art]:TriggerResults:TriggerResultInserter        1.945e-05    3.40037e-05   0.000116902   2.9901e-05    1.78571e-05      27     
end_path:out1:RootOutput                          3.87e-06     7.04489e-06    3.304e-05     5.87e-06     5.25803e-06      27     
end_path:out1:RootOutput(write)                    2.66081       3.73767       6.0137        3.72269      0.740275        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4714.12 MB
  Peak resident set size usage (VmHWM): 2716.22 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0165_df-s04-d0_dw_0_20260203T150014_reco_stage0_20260203T233101_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0165_df-s04-d0_dw_0_20260203T150014_reco_stage0_20260203T233101_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0165_df-s04-d0_dw_0_20260203T150014_reco_stage0_20260203T233101_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770158839.0,
    "core.end_time": 1770161462.0,
    "core.events": [
      26806,
      26812,
      26818,
      26824,
      26830,
      26836,
      26842,
      26848,
      26854,
      26860,
      26866,
      26872,
      26878,
      26884,
      26890,
      26896,
      26902,
      26908,
      26914,
      26920,
      26926,
      26932,
      26938,
      26944,
      26950,
      26956,
      26962
    ],
    "core.event_count": 27,
    "core.first_event_number": 26806,
    "core.last_event_number": 26962,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0165_df-s04-d0_dw_0_20260203T150014.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:46:29 UTC       justIN version: 01.06.00