justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297916.115@dunegpschedd02.fnal.gov

Jobsub ID297916.115@dunegpschedd02.fnal.gov
Workflow ID12718
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:46:26
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce515
Last heartbeat2026-02-03 23:29:25
From worker nodeHostnameb9p12p0201.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 22:54:04
Input filesvd-protodune:np02vd_raw_run042422_0010_df-s02-d1_dw_0_20260203T152012.hdf5
JobscriptExit code0
Real time32m (1969s)
CPU time31m (1863s = 94%)
Max RSS bytes2606145536 (2485 MiB)
Outputting started2026-02-03 23:26:53
Output files
Finished2026-02-03 23:29:25
Saved logsjustin-logs:297916.115-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

... 0x23361860, 0x2335e160
Caught exception in looking up space point ptr... 0x5bf92250, 0x5bf920c0
Caught exception in looking up space point ptr... 0x5bf920c0, 0x5bf91b70
Caught exception in looking up space point ptr... 0x4ad919c0, 0x4ad91d30
Caught exception in looking up space point ptr... 0x4ad91d30, 0x4ad91ba0
Caught exception in looking up space point ptr... 0x31af85b0, 0x31af7240
Caught exception in looking up space point ptr... 0x31af7240, 0x50a0c9e0
Caught exception in looking up space point ptr... 0x50a0c9e0, 0x50a0c820
Caught exception in looking up space point ptr... 0x50a0c820, 0x31af85b0
Caught exception in looking up space point ptr... 0x397c05b0, 0x397c06b0
Caught exception in looking up space point ptr... 0x397c06b0, 0x2c159260
Caught exception in looking up space point ptr... 0x495d8c20, 0x495dc9c0
Caught exception in looking up space point ptr... 0x495d8a90, 0x495d8c20
Caught exception in looking up space point ptr... 0x11c48ea0, 0x11c49010
Caught exception in looking up space point ptr... 0x11c49010, 0x11c48ea0
Caught exception in looking up space point ptr... 0x4da70d70, 0x4869b270
Caught exception in looking up space point ptr... 0x4869b270, 0x486963b0
Caught exception in looking up space point ptr... 0x46487be0, 0x4648a910
Caught exception in looking up space point ptr... 0x4648a910, 0x46487be0
Caught exception in looking up space point ptr... 0x263a5200, 0x263a50e0
Caught exception in looking up space point ptr... 0x263a50e0, 0x263a69b0
Caught exception in looking up space point ptr... 0x5d1b55d0, 0x26362d80
Caught exception in looking up space point ptr... 0x26362d80, 0x5d1b55d0
Caught exception in looking up space point ptr... 0x430663b0, 0x43066850
Caught exception in looking up space point ptr... 0x43066850, 0x43066600
Caught exception in looking up space point ptr... 0x299461b0, 0x29946310
Caught exception in looking up space point ptr... 0x299469f0, 0x299461b0
Caught exception in looking up space point ptr... 0x5f8899b0, 0x3798fed0
Caught exception in looking up space point ptr... 0x5f889590, 0x5f8899b0
Caught exception in looking up space point ptr... 0x4c623a10, 0x12666e30
Caught exception in looking up space point ptr... 0x29ef9080, 0x4c623a10
Caught exception in looking up space point ptr... 0xb6db380, 0x2ab0a390
Caught exception in looking up space point ptr... 0x2ab0a390, 0x4ee6bda0
Caught exception in looking up space point ptr... 0x352df810, 0x5240a3d0
Caught exception in looking up space point ptr... 0x5240a3d0, 0x352df810
Caught exception in looking up space point ptr... 0x352df810, 0x3d3e65c0
Caught exception in looking up space point ptr... 0x3d3e65c0, 0x352df810
Caught exception in looking up space point ptr... 0x16303870, 0x35f691d0
Caught exception in looking up space point ptr... 0x35f691d0, 0x162efc60
Caught exception in looking up space point ptr... 0x486acff0, 0x486acdf0
Caught exception in looking up space point ptr... 0x486acdf0, 0x486acff0
Caught exception in looking up space point ptr... 0x53e3dcd0, 0xe4659c0
Caught exception in looking up space point ptr... 0xe4659c0, 0x31096480
Caught exception in looking up space point ptr... 0x31096480, 0xe465b50
Caught exception in looking up space point ptr... 0x3d51bee0, 0x385cd1a0
Caught exception in looking up space point ptr... 0x385cd1a0, 0x3d51b4c0
Caught exception in looking up space point ptr... 0x3d51bd20, 0x3d51be00
Caught exception in looking up space point ptr... 0x3d51be00, 0x3d51bee0
Caught exception in looking up space point ptr... 0x46460a70, 0x46461b70
Caught exception in looking up space point ptr... 0x46461b70, 0x46460a70
Caught exception in looking up space point ptr... 0x2c8eed30, 0x4219e2d0
Caught exception in looking up space point ptr... 0x4219e2d0, 0x2c8eed30
Caught exception in looking up space point ptr... 0x3eece3c0, 0x3eeceb50
Caught exception in looking up space point ptr... 0x3eeceb50, 0x3eece900
Caught exception in looking up space point ptr... 0x3a475040, 0x3a475330
Caught exception in looking up space point ptr... 0x3a475330, 0x3a475040
Caught exception in looking up space point ptr... 0x48696c70, 0x486b25d0
Caught exception in looking up space point ptr... 0x486b25d0, 0x48696c70
Caught exception in looking up space point ptr... 0x2c159640, 0x2c15ff60
Caught exception in looking up space point ptr... 0x2c15ff60, 0x2c15acd0
Caught exception in looking up space point ptr... 0x4ad9d640, 0x4ad90a20
Caught exception in looking up space point ptr... 0x4ad90a20, 0x4ad9d640
Caught exception in looking up space point ptr... 0x42c06950, 0x42c0cc10
Caught exception in looking up space point ptr... 0x42c0cc10, 0x42c06950
Caught exception in looking up space point ptr... 0x263a7c20, 0x263a8e90
Caught exception in looking up space point ptr... 0x263a8e90, 0x263a7c20
++++>>>> total num hits: 63138, num free: 60346
04-Feb-2026 00:26:47 CET  Closed output file "np02vd_raw_run042422_0010_df-s02-d1_dw_0_20260203T152012_reco_stage0_20260203T232647_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         61.909        71.0385       84.5629       69.5842       6.25536        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        3.915e-05    7.20889e-05   0.000183891   6.2231e-05    3.33128e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                3.53405       4.01917       5.10099       3.96276      0.370766        27     
produce:triggerrawdecoder:PDVDTriggerReader4     0.00885842     0.0125124     0.0316523     0.0103135    0.00538016       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000354381   0.000439009   0.000738012   0.000414672   7.92295e-05      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00134781    0.00181552    0.00877309    0.00152627    0.00137231       27     
produce:beamevent:BeamEvent                      5.0631e-05    8.95629e-05   0.000573662    6.657e-05    9.67325e-05      27     
produce:pdvddaphne:DAPHNEReaderPDVD                1.87047       2.31538       3.01805       2.23197      0.293535        27     
produce:ophit:OpHitFinder                         0.0190803     0.0297092     0.0411497     0.0287056    0.00448751       27     
produce:wclsdatavd:WireCellToolkit                 45.6422       51.7131       63.8119       48.6423       5.81688        27     
produce:gaushit:GausHitFinder                     0.450309      0.755507       1.11731      0.757599      0.148854        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000135171   0.000205902   0.000446352   0.000188041   6.88241e-05      27     
produce:reco3d:SpacePointSolver                    4.29918       7.78071       12.9962       7.23079       2.2825         27     
produce:hitpdune:DisambigFromSpacePoints          0.043884      0.0704464     0.117016      0.0675032     0.018381        27     
produce:cluster3d:Cluster3D                        0.67505       1.36361       2.2323        1.35824      0.462043        27     
[art]:TriggerResults:TriggerResultInserter        1.685e-05    2.46407e-05    9.584e-05     2.165e-05    1.46506e-05      27     
end_path:out1:RootOutput                          3.41e-06      6.596e-06     1.928e-05     6.58e-06     3.12682e-06      27     
end_path:out1:RootOutput(write)                    2.66527       2.95995       3.8264        2.87024      0.298872        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4630.73 MB
  Peak resident set size usage (VmHWM): 2606.15 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042422_0010_df-s02-d1_dw_0_20260203T152012_reco_stage0_20260203T232647_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042422_0010_df-s02-d1_dw_0_20260203T152012_reco_stage0_20260203T232647_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042422_0010_df-s02-d1_dw_0_20260203T152012_reco_stage0_20260203T232647_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770159254.0,
    "core.end_time": 1770161208.0,
    "core.events": [
      1621,
      1627,
      1633,
      1639,
      1645,
      1651,
      1657,
      1663,
      1669,
      1675,
      1681,
      1687,
      1693,
      1699,
      1705,
      1711,
      1717,
      1723,
      1729,
      1735,
      1741,
      1747,
      1753,
      1759,
      1765,
      1771,
      1777
    ],
    "core.event_count": 27,
    "core.first_event_number": 1621,
    "core.last_event_number": 1777,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42422
    ],
    "core.runs_subruns": [
      4242200001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042422_0010_df-s02-d1_dw_0_20260203T152012.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:15:15 UTC       justIN version: 01.06.00