justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.130@dunegpschedd02.fnal.gov

Jobsub ID297923.130@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2026-02-04 01:12:37
From worker nodeHostnamedune001-1846007.0-lcg2400.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:04:32
Input filesvd-protodune:np02vd_raw_run042433_0200_df-s02-d0_dw_0_20260203T195808.hdf5
JobscriptExit code0
Real time1h (3979s)
CPU time1h (3616s = 90%)
Max RSS bytes3387551744 (3230 MiB)
Outputting started2026-02-04 01:10:53
Output files
Finished2026-02-04 01:12:37
Saved logsjustin-logs:297923.130-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

.. 0x4fc20e50, 0x4fc31c90
Caught exception in looking up space point ptr... 0x60598d30, 0x60598ba0
Caught exception in looking up space point ptr... 0x60598ba0, 0x4fce45c0
Caught exception in looking up space point ptr... 0x53050790, 0x4898b530
Caught exception in looking up space point ptr... 0x4898b530, 0x53050790
Caught exception in looking up space point ptr... 0x46d25fe0, 0x46d24fa0
Caught exception in looking up space point ptr... 0x4fc052d0, 0x46d25fe0
Caught exception in looking up space point ptr... 0x5304e2b0, 0x5304c990
Caught exception in looking up space point ptr... 0x5304c990, 0x5304d2a0
Caught exception in looking up space point ptr... 0x47d1bc70, 0x47d1a260
Caught exception in looking up space point ptr... 0x47d1a260, 0x5da7b850
Caught exception in looking up space point ptr... 0x3d7a1c50, 0x5d418430
Caught exception in looking up space point ptr... 0x5d418430, 0x61f66ca0
Caught exception in looking up space point ptr... 0x5303da50, 0x5303d8c0
Caught exception in looking up space point ptr... 0x87a41d90, 0x5303da50
Caught exception in looking up space point ptr... 0x4be82220, 0x18f19160
Caught exception in looking up space point ptr... 0x4be7d3a0, 0x4be82220
Caught exception in looking up space point ptr... 0x45e52a10, 0x45e54770
Caught exception in looking up space point ptr... 0x45e54770, 0x656e7b40
Caught exception in looking up space point ptr... 0x659de4e0, 0x40250e10
Caught exception in looking up space point ptr... 0x40250e10, 0x7338bff0
Caught exception in looking up space point ptr... 0x4898ea40, 0x4898e8f0
Caught exception in looking up space point ptr... 0x4898e8f0, 0x4898ea40
Caught exception in looking up space point ptr... 0x66367890, 0x66174d80
Caught exception in looking up space point ptr... 0x66174d80, 0x66367890
Caught exception in looking up space point ptr... 0x4fcb8540, 0x4fcbe4c0
Caught exception in looking up space point ptr... 0x4fcbed20, 0x4fcb8540
Caught exception in looking up space point ptr... 0x3e9b82e0, 0x3e9b8680
Caught exception in looking up space point ptr... 0x3e9b8680, 0x3e9b82e0
Caught exception in looking up space point ptr... 0x1a3f9300, 0x1a42df00
Caught exception in looking up space point ptr... 0x1a42df00, 0x1a3f9300
Caught exception in looking up space point ptr... 0xe640b10, 0x5e6119e0
Caught exception in looking up space point ptr... 0x5e6119e0, 0x460457a0
Caught exception in looking up space point ptr... 0x6f7a1450, 0x5351e290
Caught exception in looking up space point ptr... 0x5351e290, 0x6f7a1450
Caught exception in looking up space point ptr... 0x6f783650, 0x6f783290
Caught exception in looking up space point ptr... 0x6f783290, 0x6f783650
Caught exception in looking up space point ptr... 0x733e6d60, 0x47362c40
Caught exception in looking up space point ptr... 0x47362c40, 0x733e6d60
Caught exception in looking up space point ptr... 0x733e6d60, 0x45eb77a0
Caught exception in looking up space point ptr... 0x45eb77a0, 0x733e6d60
Caught exception in looking up space point ptr... 0x53251820, 0x53251070
Caught exception in looking up space point ptr... 0x53251070, 0x53251820
Caught exception in looking up space point ptr... 0x53251820, 0x53251500
Caught exception in looking up space point ptr... 0x53251500, 0x53251820
Caught exception in looking up space point ptr... 0x42986270, 0x42986170
Caught exception in looking up space point ptr... 0x42986170, 0x45fc3dc0
Caught exception in looking up space point ptr... 0x3f6ee6a0, 0x3f6edaf0
Caught exception in looking up space point ptr... 0x3f6edaf0, 0x3f6ee6a0
Caught exception in looking up space point ptr... 0x6c811100, 0x6c811060
Caught exception in looking up space point ptr... 0x6c811060, 0x46d06d70
Caught exception in looking up space point ptr... 0x87a4bba0, 0x5303e540
Caught exception in looking up space point ptr... 0x5303e540, 0x87a4bba0
Caught exception in looking up space point ptr... 0x6c6d16c0, 0x6caca320
Caught exception in looking up space point ptr... 0x3d4b72d0, 0x6c6d16c0
Caught exception in looking up space point ptr... 0x519a4d40, 0x45114f90
Caught exception in looking up space point ptr... 0x45114f90, 0x519a4d40
Caught exception in looking up space point ptr... 0x4d3c8a80, 0x5e577400
Caught exception in looking up space point ptr... 0x5e577400, 0x4d3c8a80
Caught exception in looking up space point ptr... 0x3d46a2a0, 0x734667b0
Caught exception in looking up space point ptr... 0x734667b0, 0x1cf4ba80
Caught exception in looking up space point ptr... 0x4f3a7e30, 0x4f3a80d0
Caught exception in looking up space point ptr... 0x4f3a80d0, 0x4f3a7f80
Caught exception in looking up space point ptr... 0x131b4a40, 0x6b5f20f0
Caught exception in looking up space point ptr... 0x6b5f20f0, 0x131a95d0
Caught exception in looking up space point ptr... 0x44c5a5e0, 0x44d01800
Caught exception in looking up space point ptr... 0x44d01800, 0x44c5a5e0
++++>>>> total num hits: 69185, num free: 65955
04-Feb-2026 01:10:42 UTC  Closed output file "np02vd_raw_run042433_0200_df-s02-d0_dw_0_20260203T195808_reco_stage0_20260204T011042_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         178.356       203.942       233.727       206.494       12.183         19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.6863e-05    0.000161149   0.000400885   0.000121769   7.93664e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                17.4593       20.6271       22.4224       21.4452       1.50725        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.686408      0.714238      0.833985      0.703634      0.0322412       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0229561     0.0244489     0.0331015     0.0235975    0.00241323       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.022872      0.0235106     0.0242816     0.0234744    0.00033179       19     
produce:beamevent:BeamEvent                      0.000118263   0.000288657   0.00119668    0.000231406   0.000236768      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.67524       4.09626       4.80234       4.06426       0.31032        19     
produce:ophit:OpHitFinder                         0.0351687     0.0476681     0.0660478     0.046104     0.00818355       19     
produce:wclsdatavd:WireCellToolkit                 126.621       143.525       154.38        144.362       7.16262        19     
produce:gaushit:GausHitFinder                      1.18496       1.81943       2.80063       1.81368      0.390342        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000369376   0.000509198   0.000933438   0.000478401   0.000127996      19     
produce:reco3d:SpacePointSolver                    13.654        21.7384       31.1834       21.0934       5.53546        19     
produce:hitpdune:DisambigFromSpacePoints          0.169875      0.281576      0.492907      0.288914      0.0708574       19     
produce:cluster3d:Cluster3D                        1.94668       4.2362        18.4354       3.36581       3.54687        19     
[art]:TriggerResults:TriggerResultInserter       2.5759e-05    4.88498e-05   0.000158278   3.8713e-05    3.17379e-05      19     
end_path:out1:RootOutput                          6.202e-06    3.73746e-05   0.000404161   1.2684e-05    8.70166e-05      19     
end_path:out1:RootOutput(write)                    4.99482       6.7412        7.42828       6.83373      0.490987        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5670.09 MB
  Peak resident set size usage (VmHWM): 3387.55 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0200_df-s02-d0_dw_0_20260203T195808_reco_stage0_20260204T011042_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0200_df-s02-d0_dw_0_20260203T195808_reco_stage0_20260204T011042_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0200_df-s02-d0_dw_0_20260203T195808_reco_stage0_20260204T011042_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163486.0,
    "core.end_time": 1770167444.0,
    "core.events": [
      22806,
      22812,
      22818,
      22824,
      22830,
      22836,
      22842,
      22848,
      22854,
      22860,
      22866,
      22872,
      22878,
      22884,
      22890,
      22896,
      22902,
      22908,
      22914
    ],
    "core.event_count": 19,
    "core.first_event_number": 22806,
    "core.last_event_number": 22914,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0200_df-s02-d0_dw_0_20260203T195808.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:45:22 UTC       justIN version: 01.06.00