justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.92@dunegpschedd01.fnal.gov

Jobsub ID303656.92@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2026-02-04 01:14:37
From worker nodeHostnamet2wn011.physics.ox.ac.uk
cpuinfoAMD EPYC 7713P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:13:37
Input filesvd-protodune:np02vd_raw_run042421_0131_df-s04-d1_dw_0_20260203T144152.hdf5
JobscriptExit code0
Real time57m (3472s)
CPU time50m (3052s = 87%)
Max RSS bytes2546794496 (2428 MiB)
Outputting started2026-02-04 01:11:30
Output files
Finished2026-02-04 01:14:37
Saved logsjustin-logs:303656.92-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

1c590
Caught exception in looking up space point ptr... 0x1461c590, 0x1461afa0
Caught exception in looking up space point ptr... 0x145f97c0, 0x145f7f00
Caught exception in looking up space point ptr... 0x145f7f00, 0x145f9230
Caught exception in looking up space point ptr... 0x13605840, 0x28c8ead0
Caught exception in looking up space point ptr... 0x28c8ead0, 0x28c8ebd0
Caught exception in looking up space point ptr... 0x15738a20, 0x3bef61f0
Caught exception in looking up space point ptr... 0x4c19d710, 0x15738a20
Caught exception in looking up space point ptr... 0x483ae2a0, 0x416a5ad0
Caught exception in looking up space point ptr... 0x416a5ad0, 0x483ae2a0
Caught exception in looking up space point ptr... 0x483ae2a0, 0x4626c830
Caught exception in looking up space point ptr... 0x4626c830, 0x483ae2a0
Caught exception in looking up space point ptr... 0x483ae2a0, 0x21d6eb20
Caught exception in looking up space point ptr... 0x21d6eb20, 0x483ae2a0
Caught exception in looking up space point ptr... 0xe229a60, 0x2974f420
Caught exception in looking up space point ptr... 0x5ebac520, 0xe229a60
Caught exception in looking up space point ptr... 0x11ac6fc0, 0x12f91140
Caught exception in looking up space point ptr... 0x12f91140, 0x11ac6fc0
Caught exception in looking up space point ptr... 0x434ca710, 0x44ed1c60
Caught exception in looking up space point ptr... 0x5bf3cce0, 0x434ca710
Caught exception in looking up space point ptr... 0x2664c040, 0x2664bd60
Caught exception in looking up space point ptr... 0x2664bd60, 0x2664beb0
Caught exception in looking up space point ptr... 0x3714c6c0, 0x523e3510
Caught exception in looking up space point ptr... 0x523e3510, 0x523e3280
Caught exception in looking up space point ptr... 0x2d67ae20, 0x2d67ab40
Caught exception in looking up space point ptr... 0x2d67ab40, 0x2d67ae20
Caught exception in looking up space point ptr... 0x2d67ae20, 0xe010c70
Caught exception in looking up space point ptr... 0xe010c70, 0x2d67ae20
Caught exception in looking up space point ptr... 0x164bbd20, 0x4626b100
Caught exception in looking up space point ptr... 0x4626b100, 0x164bbd20
Caught exception in looking up space point ptr... 0x4724ed80, 0x4724eed0
Caught exception in looking up space point ptr... 0x4724eed0, 0x4724ed80
Caught exception in looking up space point ptr... 0x2d67b2d0, 0x2d67afb0
Caught exception in looking up space point ptr... 0x2d67afb0, 0x2d67b4b0
Caught exception in looking up space point ptr... 0x2b81dcc0, 0x2b81dea0
Caught exception in looking up space point ptr... 0x2b81dea0, 0x2b81e210
Caught exception in looking up space point ptr... 0x2c557530, 0x2c5573a0
Caught exception in looking up space point ptr... 0x2c5573a0, 0x2c5571c0
Caught exception in looking up space point ptr... 0x13af7150, 0x3a339a80
Caught exception in looking up space point ptr... 0x3a339a80, 0x13af7150
Caught exception in looking up space point ptr... 0x4e535220, 0x4e5356c0
Caught exception in looking up space point ptr... 0x4e5356c0, 0x4e535470
Caught exception in looking up space point ptr... 0x15a70b70, 0x53a2ce80
Caught exception in looking up space point ptr... 0x53a2ce80, 0x29912c20
Caught exception in looking up space point ptr... 0x45328b60, 0x50b38b60
Caught exception in looking up space point ptr... 0x50b38b60, 0x45328430
Caught exception in looking up space point ptr... 0x135e3090, 0x135e2f90
Caught exception in looking up space point ptr... 0x135e2f90, 0x135e3090
Caught exception in looking up space point ptr... 0x35512dd0, 0x35512c60
Caught exception in looking up space point ptr... 0x35512c60, 0x35512920
Caught exception in looking up space point ptr... 0x14603c10, 0x146038f0
Caught exception in looking up space point ptr... 0x146038f0, 0x14603c10
Caught exception in looking up space point ptr... 0x14603c10, 0x14603440
Caught exception in looking up space point ptr... 0x14603440, 0x14603c10
Caught exception in looking up space point ptr... 0x14603c10, 0x14603a80
Caught exception in looking up space point ptr... 0x14603a80, 0x14603c10
Caught exception in looking up space point ptr... 0x12e218d0, 0x12e1e590
Caught exception in looking up space point ptr... 0x12e1e590, 0x12e218d0
Caught exception in looking up space point ptr... 0x8d08f00, 0x73ee640
Caught exception in looking up space point ptr... 0x73ee640, 0x8d08f00
Caught exception in looking up space point ptr... 0x8d08f00, 0x16abba00
Caught exception in looking up space point ptr... 0x16abba00, 0x8d08f00
Caught exception in looking up space point ptr... 0x8d08f00, 0x3636cb80
Caught exception in looking up space point ptr... 0x3636cb80, 0x8d08f00
Caught exception in looking up space point ptr... 0x568d6750, 0x4c219380
Caught exception in looking up space point ptr... 0x141dfe50, 0x568d6750
++++>>>> total num hits: 45983, num free: 44535
04-Feb-2026 01:11:20 GMT  Closed output file "np02vd_raw_run042421_0131_df-s04-d1_dw_0_20260203T144152_reco_stage0_20260204T011120_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         106.647       124.732       146.955       122.888       11.3021        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.5735e-05    0.000104688   0.000288986   9.6476e-05    3.76432e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                13.2363       14.9393       17.0811       14.8009       1.25042        27     
produce:triggerrawdecoder:PDVDTriggerReader4       0.57221      0.594847      0.632866       0.59675      0.0121967       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0192988     0.0195538     0.0200433     0.0195176    0.000177461      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0954819     0.0964658     0.100519      0.0962333    0.000910434      27     
produce:beamevent:BeamEvent                      9.2909e-05    0.00015497    0.00118461    0.000113789   0.000202184      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.91042       4.40908       4.8992        4.41442      0.240373        27     
produce:ophit:OpHitFinder                         0.0370779     0.0497167     0.0571874     0.0504288    0.00530229       27     
produce:wclsdatavd:WireCellToolkit                 67.815        81.8531       100.08        79.9081       8.17883        27     
produce:gaushit:GausHitFinder                     0.757299       1.30617       2.15494       1.1889       0.389436        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000205295   0.000326001   0.000601679   0.000314736   9.1772e-05       27     
produce:reco3d:SpacePointSolver                    7.28721       13.8136       25.0083       12.5597       4.83908        27     
produce:hitpdune:DisambigFromSpacePoints          0.0683769     0.128737      0.280794      0.109317      0.0601354       27     
produce:cluster3d:Cluster3D                        1.22887       2.57288       4.99937       2.44951      0.974765        27     
[art]:TriggerResults:TriggerResultInserter       2.1081e-05    3.41821e-05   0.000113208   2.9396e-05    1.74571e-05      27     
end_path:out1:RootOutput                          3.567e-06     7.088e-06    3.6881e-05     4.559e-06    7.43813e-06      27     
end_path:out1:RootOutput(write)                    3.77693       4.91136       6.42936       4.49727      0.919365        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4547.71 MB
  Peak resident set size usage (VmHWM): 2546.79 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0131_df-s04-d1_dw_0_20260203T144152_reco_stage0_20260204T011120_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0131_df-s04-d1_dw_0_20260203T144152_reco_stage0_20260204T011120_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0131_df-s04-d1_dw_0_20260203T144152_reco_stage0_20260204T011120_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770164033.0,
    "core.end_time": 1770167482.0,
    "core.events": [
      21293,
      21299,
      21305,
      21311,
      21317,
      21323,
      21329,
      21335,
      21341,
      21347,
      21353,
      21359,
      21365,
      21371,
      21377,
      21383,
      21389,
      21395,
      21401,
      21407,
      21413,
      21419,
      21425,
      21431,
      21437,
      21443,
      21449
    ],
    "core.event_count": 27,
    "core.first_event_number": 21293,
    "core.last_event_number": 21449,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0131_df-s04-d1_dw_0_20260203T144152.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:31:42 UTC       justIN version: 01.06.00