justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.129@dunegpschedd02.fnal.gov

Jobsub ID297923.129@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2026-02-04 01:24:21
From worker nodeHostnamedune001-1846006.0-lcg2567.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:04:02
Input filesvd-protodune:np02vd_raw_run042433_0275_df-s04-d1_dw_0_20260203T202638.hdf5
JobscriptExit code0
Real time1h (4703s)
CPU time1h (4319s = 91%)
Max RSS bytes3352494080 (3197 MiB)
Outputting started2026-02-04 01:22:27
Output files
Finished2026-02-04 01:24:21
Saved logsjustin-logs:297923.129-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

. 0x4a57a0e0, 0x473b49a0
Caught exception in looking up space point ptr... 0x46901a40, 0x46901940
Caught exception in looking up space point ptr... 0x46901940, 0x4ab6e3b0
Caught exception in looking up space point ptr... 0x6c1b7db0, 0x4ab6d170
Caught exception in looking up space point ptr... 0x4ab6d170, 0x46901a40
Caught exception in looking up space point ptr... 0x8da0f230, 0x49b125b0
Caught exception in looking up space point ptr... 0x49b125b0, 0x7c615020
Caught exception in looking up space point ptr... 0x7c615020, 0x4e76ca40
Caught exception in looking up space point ptr... 0x4e76ca40, 0x9e0803d0
Caught exception in looking up space point ptr... 0x659fab00, 0x659fa920
Caught exception in looking up space point ptr... 0x659fa920, 0x659fa560
Caught exception in looking up space point ptr... 0x44401670, 0x773b6a00
Caught exception in looking up space point ptr... 0x773b6a00, 0x44401670
Caught exception in looking up space point ptr... 0x83174940, 0x83174c60
Caught exception in looking up space point ptr... 0x83174c60, 0x83174570
Caught exception in looking up space point ptr... 0x86847640, 0x86846fc0
Caught exception in looking up space point ptr... 0x86846fc0, 0x86847640
Caught exception in looking up space point ptr... 0x942129e0, 0x7a365ac0
Caught exception in looking up space point ptr... 0x7a365ac0, 0x942129e0
Caught exception in looking up space point ptr... 0x602d74e0, 0x8316fa90
Caught exception in looking up space point ptr... 0x8316fa90, 0xa290abd0
Caught exception in looking up space point ptr... 0x72914950, 0xa8226dd0
Caught exception in looking up space point ptr... 0xa8226dd0, 0x6f9c0ac0
Caught exception in looking up space point ptr... 0x4c829c00, 0x83452040
Caught exception in looking up space point ptr... 0x83452040, 0x4c829c00
Caught exception in looking up space point ptr... 0x6a570670, 0x7b1803a0
Caught exception in looking up space point ptr... 0x7b1803a0, 0x42d9a860
Caught exception in looking up space point ptr... 0x42d9a860, 0x7eff63f0
Caught exception in looking up space point ptr... 0x4b94db30, 0xb6d7d5d0
Caught exception in looking up space point ptr... 0xb6d7d5d0, 0x4b94db30
Caught exception in looking up space point ptr... 0x4b94db30, 0x4e5ddc40
Caught exception in looking up space point ptr... 0x4e5ddc40, 0x4b94db30
Caught exception in looking up space point ptr... 0x79cf0040, 0x8e4d4980
Caught exception in looking up space point ptr... 0x8e4d4980, 0x79cf0040
Caught exception in looking up space point ptr... 0x70f1b8f0, 0x68d619e0
Caught exception in looking up space point ptr... 0x924d4c00, 0x70f1b8f0
Caught exception in looking up space point ptr... 0xb1d95480, 0x7357eaa0
Caught exception in looking up space point ptr... 0x7357eaa0, 0x7357da00
Caught exception in looking up space point ptr... 0x3d2f4110, 0x9f473880
Caught exception in looking up space point ptr... 0x9f473880, 0x3e8f5720
Caught exception in looking up space point ptr... 0x4221a210, 0x6ed448c0
Caught exception in looking up space point ptr... 0x6ed448c0, 0x4221a210
Caught exception in looking up space point ptr... 0x9ee337d0, 0x9ee34e30
Caught exception in looking up space point ptr... 0x9ee34e30, 0x9ee34b40
Caught exception in looking up space point ptr... 0x9ee34b40, 0x8b10c0e0
Caught exception in looking up space point ptr... 0x7102b1a0, 0x70c1f7c0
Caught exception in looking up space point ptr... 0x70c1f7c0, 0xbafd7610
Caught exception in looking up space point ptr... 0x46251c90, 0x6fb34ba0
Caught exception in looking up space point ptr... 0x6fb34ba0, 0x46251c90
Caught exception in looking up space point ptr... 0x708809d0, 0x8913f680
Caught exception in looking up space point ptr... 0x8913f680, 0x76fbea70
Caught exception in looking up space point ptr... 0x462b90a0, 0x462ba430
Caught exception in looking up space point ptr... 0x462ba430, 0x462b90a0
Caught exception in looking up space point ptr... 0x9ee322a0, 0x8c87d0c0
Caught exception in looking up space point ptr... 0x8c87d0c0, 0x9ee322a0
Caught exception in looking up space point ptr... 0xbab35990, 0x88ec58c0
Caught exception in looking up space point ptr... 0x88ec58c0, 0xbab35990
Caught exception in looking up space point ptr... 0x8769a110, 0x9236e830
Caught exception in looking up space point ptr... 0x6ed27660, 0x8769a110
Caught exception in looking up space point ptr... 0x7f06cbd0, 0x88ec5b20
Caught exception in looking up space point ptr... 0x88ec5b20, 0x7f06cbd0
Caught exception in looking up space point ptr... 0x96d918a0, 0x8b280200
Caught exception in looking up space point ptr... 0x8b280200, 0x61353d90
Caught exception in looking up space point ptr... 0x6c696790, 0x42585540
Caught exception in looking up space point ptr... 0x42585540, 0x6c696790
Caught exception in looking up space point ptr... 0x4e6d1330, 0x76c3fbc0
Caught exception in looking up space point ptr... 0x76c3fbc0, 0x4e6d1330
++++>>>> total num hits: 40699, num free: 39272
04-Feb-2026 01:22:13 UTC  Closed output file "np02vd_raw_run042433_0275_df-s04-d1_dw_0_20260203T202638_reco_stage0_20260204T012213_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         222.612       240.787       286.606       237.846       15.552         19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000100851   0.00120614     0.0201707    0.000130971   0.00447037       19     
produce:tpcrawdecoder:PDVDTPCReader                18.6541       21.4351        54.31        19.4765       7.84286        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.691827      0.738833      0.806499      0.733315      0.0351656       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0233909     0.0283539     0.0517255     0.023703     0.00820401       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0233072     0.0237433     0.0254015     0.0234541    0.000646068      19     
produce:beamevent:BeamEvent                      0.000159183   0.00031944    0.00154824    0.000236614   0.000300981      19     
produce:pdvddaphne:DAPHNEReaderPDVD                4.11142       4.60986       5.07116       4.60755      0.276911        19     
produce:ophit:OpHitFinder                         0.0468422     0.0560468     0.0658354     0.056538     0.00557885       19     
produce:wclsdatavd:WireCellToolkit                 153.832        174.1        204.564       175.515       11.3964        19     
produce:gaushit:GausHitFinder                      1.20357       2.01873       2.67868       1.84999      0.431224        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000407986   0.000568977   0.000992656   0.000521378   0.000127465      19     
produce:reco3d:SpacePointSolver                    16.335        25.4002       39.0867       24.9957       5.21743        19     
produce:hitpdune:DisambigFromSpacePoints          0.191058      0.331358      0.422143      0.347103      0.0615164       19     
produce:cluster3d:Cluster3D                        1.54217       3.86566       6.47896       3.89501       1.31351        19     
[art]:TriggerResults:TriggerResultInserter       3.0111e-05    5.39253e-05   0.000199454    3.514e-05    4.02085e-05      19     
end_path:out1:RootOutput                          9.24e-06     3.37474e-05   0.000381216   1.1631e-05    8.21544e-05      19     
end_path:out1:RootOutput(write)                    6.77116       8.09279       9.2329        8.28972      0.663649        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5531.72 MB
  Peak resident set size usage (VmHWM): 3352.49 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0275_df-s04-d1_dw_0_20260203T202638_reco_stage0_20260204T012213_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0275_df-s04-d1_dw_0_20260203T202638_reco_stage0_20260204T012213_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0275_df-s04-d1_dw_0_20260203T202638_reco_stage0_20260204T012213_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163459.0,
    "core.end_time": 1770168136.0,
    "core.events": [
      31361,
      31367,
      31373,
      31379,
      31385,
      31391,
      31397,
      31403,
      31409,
      31415,
      31421,
      31427,
      31433,
      31439,
      31445,
      31451,
      31457,
      31463,
      31469
    ],
    "core.event_count": 19,
    "core.first_event_number": 31361,
    "core.last_event_number": 31469,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0275_df-s04-d1_dw_0_20260203T202638.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:17:31 UTC       justIN version: 01.06.00