justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303717.2@dunegpschedd01.fnal.gov

Jobsub ID303717.2@dunegpschedd01.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-04 00:50:28
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce03
Last heartbeat2026-02-04 01:54:03
From worker nodeHostnamedune001-1535105.0-lcg2452.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:51:24
Input filesvd-protodune:np02vd_raw_run042433_0479_df-s03-d1_dw_0_20260203T214411.hdf5
JobscriptExit code0
Real time1h (3656s)
CPU time54m (3262s = 89%)
Max RSS bytes3386171392 (3229 MiB)
Outputting started2026-02-04 01:52:21
Output files
Finished2026-02-04 01:54:03
Saved logsjustin-logs:303717.2-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

. 0x4e13f7c0, 0x27d94ca0
Caught exception in looking up space point ptr... 0x8a107500, 0x6ca0ca30
Caught exception in looking up space point ptr... 0x6ca0ca30, 0x9a63e620
Caught exception in looking up space point ptr... 0x4e13c680, 0x72587190
Caught exception in looking up space point ptr... 0x72587190, 0x4e13c680
Caught exception in looking up space point ptr... 0x579c7910, 0x579c6860
Caught exception in looking up space point ptr... 0x579c6860, 0x579c7910
Caught exception in looking up space point ptr... 0x4e156440, 0x3020c2b0
Caught exception in looking up space point ptr... 0x3020c2b0, 0x4e156440
Caught exception in looking up space point ptr... 0x671dc0f0, 0x9ac95930
Caught exception in looking up space point ptr... 0x9ac95930, 0x671dc0f0
Caught exception in looking up space point ptr... 0x8e921a50, 0x8e9218c0
Caught exception in looking up space point ptr... 0x8e9218c0, 0x8f2f0900
Caught exception in looking up space point ptr... 0x918ee9a0, 0x918edef0
Caught exception in looking up space point ptr... 0x918edef0, 0x918ee9a0
Caught exception in looking up space point ptr... 0x32456b60, 0x9fd49dc0
Caught exception in looking up space point ptr... 0x9fd49dc0, 0x32456b60
Caught exception in looking up space point ptr... 0x6aab4b70, 0x6aab3df0
Caught exception in looking up space point ptr... 0x6aab3df0, 0x6aab4e70
Caught exception in looking up space point ptr... 0x29f22e80, 0x868d6d60
Caught exception in looking up space point ptr... 0x868d6d60, 0x6a6da170
Caught exception in looking up space point ptr... 0x308f8ee0, 0x76c15220
Caught exception in looking up space point ptr... 0x76c15220, 0x308f8ee0
Caught exception in looking up space point ptr... 0x973ab260, 0x5b7842e0
Caught exception in looking up space point ptr... 0x5b7842e0, 0x973ab260
Caught exception in looking up space point ptr... 0x5a088250, 0x8eefb090
Caught exception in looking up space point ptr... 0x8eefb090, 0x38bde820
Caught exception in looking up space point ptr... 0x307fbe80, 0x307f64d0
Caught exception in looking up space point ptr... 0x307f64d0, 0x307fbe80
Caught exception in looking up space point ptr... 0x60bc3c70, 0x60bc6e60
Caught exception in looking up space point ptr... 0x60bc6e60, 0x8ef0e1a0
Caught exception in looking up space point ptr... 0x8ef0e1a0, 0x96ad2f20
Caught exception in looking up space point ptr... 0x96ad2f20, 0x8ef0d430
Caught exception in looking up space point ptr... 0x9cba6020, 0x9cba5e90
Caught exception in looking up space point ptr... 0x9cba5e90, 0x9cba5d20
Caught exception in looking up space point ptr... 0x96742970, 0x71b736b0
Caught exception in looking up space point ptr... 0x71b736b0, 0x96742970
Caught exception in looking up space point ptr... 0x96742970, 0x285934b0
Caught exception in looking up space point ptr... 0x285934b0, 0x96742970
Caught exception in looking up space point ptr... 0x355ebc10, 0x355ebeb0
Caught exception in looking up space point ptr... 0x355ebeb0, 0x355ebc10
Caught exception in looking up space point ptr... 0x37f497a0, 0x96e7b610
Caught exception in looking up space point ptr... 0x96e7b610, 0x37f497a0
Caught exception in looking up space point ptr... 0x8bacb720, 0x573ad210
Caught exception in looking up space point ptr... 0x573ad210, 0x573ad110
Caught exception in looking up space point ptr... 0x7b4f3280, 0x7b4f2b90
Caught exception in looking up space point ptr... 0x7b4f2b90, 0x7b4f2de0
Caught exception in looking up space point ptr... 0x60c5a190, 0x5a8f6630
Caught exception in looking up space point ptr... 0x5a8f6630, 0x60c5a190
Caught exception in looking up space point ptr... 0x308d6e30, 0x77189970
Caught exception in looking up space point ptr... 0x77189970, 0x3692f100
Caught exception in looking up space point ptr... 0x39ed2170, 0x39ed0ec0
Caught exception in looking up space point ptr... 0x39ed0ec0, 0x39ed2170
Caught exception in looking up space point ptr... 0x72c22b40, 0x51226cb0
Caught exception in looking up space point ptr... 0x51226cb0, 0x686395c0
Caught exception in looking up space point ptr... 0x35e08290, 0x60b21d10
Caught exception in looking up space point ptr... 0x60b21d10, 0x35e08290
Caught exception in looking up space point ptr... 0x35e08290, 0x72662b70
Caught exception in looking up space point ptr... 0x72662b70, 0x35e08290
Caught exception in looking up space point ptr... 0x800ac980, 0x685e8650
Caught exception in looking up space point ptr... 0x685e8650, 0x800ac980
Caught exception in looking up space point ptr... 0x55babaf0, 0x38ead130
Caught exception in looking up space point ptr... 0x38ead130, 0x60e90af0
Caught exception in looking up space point ptr... 0x8e8bf460, 0x8e8bf640
Caught exception in looking up space point ptr... 0x8e8bf640, 0x8e8bf460
Caught exception in looking up space point ptr... 0x5c1040a0, 0x33826d30
Caught exception in looking up space point ptr... 0x33826d30, 0x89811240
++++>>>> total num hits: 75110, num free: 71910
04-Feb-2026 01:52:11 UTC  Closed output file "np02vd_raw_run042433_0479_df-s03-d1_dw_0_20260203T214411_reco_stage0_20260204T015211_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         148.132       187.014       213.203       189.419       20.3316        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.7475e-05    0.000144432   0.000260922   0.000116169   5.73363e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                17.5159       22.1498       25.2647       22.3428       1.75237        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.693492      0.907208       1.24039      0.817477      0.195545        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0228126     0.0284931     0.0470537     0.0253171    0.00741116       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0226446     0.0232093     0.0241161     0.0231679    0.000343609      19     
produce:beamevent:BeamEvent                      8.9258e-05    0.000212987   0.00115332    0.000147809   0.000227658      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.99963       3.99537       4.74802       3.97696      0.440332        19     
produce:ophit:OpHitFinder                         0.0264407     0.0384772     0.0473906     0.038705     0.00540658       19     
produce:wclsdatavd:WireCellToolkit                 106.73        131.578       154.969       132.762       15.5105        19     
produce:gaushit:GausHitFinder                     0.738428       1.53353        2.303        1.55251       0.38904        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000248799   0.000426756   0.000895388   0.000408189   0.000126418      19     
produce:reco3d:SpacePointSolver                    9.52215       17.5135       28.8357       17.1914       3.97458        19     
produce:hitpdune:DisambigFromSpacePoints          0.0977538     0.234207      0.356425      0.235779      0.0624852       19     
produce:cluster3d:Cluster3D                       0.997953       2.80673       5.46729       2.67086       1.05068        19     
[art]:TriggerResults:TriggerResultInserter       2.4886e-05    3.98334e-05   0.000122431   3.5867e-05    2.12384e-05      19     
end_path:out1:RootOutput                          5.109e-06    1.26111e-05   3.4114e-05    1.0389e-05    7.60102e-06      19     
end_path:out1:RootOutput(write)                    4.45523       6.16097       7.53097       6.70359       1.02155        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5571.87 MB
  Peak resident set size usage (VmHWM): 3386.17 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0479_df-s03-d1_dw_0_20260203T214411_reco_stage0_20260204T015211_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0479_df-s03-d1_dw_0_20260203T214411_reco_stage0_20260204T015211_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0479_df-s03-d1_dw_0_20260203T214411_reco_stage0_20260204T015211_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770166298.0,
    "core.end_time": 1770169934.0,
    "core.events": [
      54615,
      54621,
      54627,
      54633,
      54639,
      54645,
      54651,
      54657,
      54663,
      54669,
      54675,
      54681,
      54687,
      54693,
      54699,
      54705,
      54711,
      54717,
      54723
    ],
    "core.event_count": 19,
    "core.first_event_number": 54615,
    "core.last_event_number": 54723,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0479_df-s03-d1_dw_0_20260203T214411.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:06:42 UTC       justIN version: 01.06.00