justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.24@dunegpschedd01.fnal.gov

Jobsub ID303656.24@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-04 02:09:51
From worker nodeHostnamecn106.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:11:35
Input filesvd-protodune:np02vd_raw_run042421_0020_df-s04-d0_dw_0_20260203T134147.hdf5
JobscriptExit code0
Real time1h (6953s)
CPU time53m (3193s = 45%)
Max RSS bytes2626281472 (2504 MiB)
Outputting started2026-02-04 02:07:29
Output files
Finished2026-02-04 02:09:51
Saved logsjustin-logs:303656.24-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ce point ptr... 0x46e9d260, 0x30119600
Caught exception in looking up space point ptr... 0x30119600, 0x28185690
Caught exception in looking up space point ptr... 0x2a6864c0, 0x2a682790
Caught exception in looking up space point ptr... 0x2a682790, 0x2a6965e0
Caught exception in looking up space point ptr... 0x2a6965e0, 0x2a683f40
Caught exception in looking up space point ptr... 0x2a684260, 0x2a690760
Caught exception in looking up space point ptr... 0x2a690760, 0x2a6864c0
Caught exception in looking up space point ptr... 0x369eb7a0, 0x369ef100
Caught exception in looking up space point ptr... 0x369ef100, 0x369ecb00
Caught exception in looking up space point ptr... 0x369ecb00, 0x369e7760
Caught exception in looking up space point ptr... 0x369e7760, 0x369e7d90
Caught exception in looking up space point ptr... 0x369e7d90, 0x369e7640
Caught exception in looking up space point ptr... 0x369e7640, 0x369eb7a0
Caught exception in looking up space point ptr... 0x2848f0a0, 0x55839d30
Caught exception in looking up space point ptr... 0x55845f40, 0x2848ecc0
Caught exception in looking up space point ptr... 0x2848ecc0, 0x5584aa40
Caught exception in looking up space point ptr... 0x5584aa40, 0x2848f0a0
Caught exception in looking up space point ptr... 0x3591b750, 0x3592e250
Caught exception in looking up space point ptr... 0x3592e250, 0x3592e3b0
Caught exception in looking up space point ptr... 0x3592e3b0, 0x36970190
Caught exception in looking up space point ptr... 0x36970190, 0x3696f410
Caught exception in looking up space point ptr... 0x3696f410, 0x3698c910
Caught exception in looking up space point ptr... 0x3698c910, 0x3591b750
Caught exception in looking up space point ptr... 0x224b1040, 0x224b0d20
Caught exception in looking up space point ptr... 0x224b0d20, 0x224b1040
Caught exception in looking up space point ptr... 0x144115d0, 0x1440f810
Caught exception in looking up space point ptr... 0x14412050, 0x14411db0
Caught exception in looking up space point ptr... 0x14411db0, 0x144115d0
Caught exception in looking up space point ptr... 0x5cb2e080, 0x5cb2dee0
Caught exception in looking up space point ptr... 0x5cb2dee0, 0x5cb2e080
Caught exception in looking up space point ptr... 0x369e40d0, 0x369e3f80
Caught exception in looking up space point ptr... 0x369e3f80, 0x369e6580
Caught exception in looking up space point ptr... 0x369e6580, 0x369dc660
Caught exception in looking up space point ptr... 0x369dc660, 0x369e9080
Caught exception in looking up space point ptr... 0x369e85f0, 0x369e6c30
Caught exception in looking up space point ptr... 0x369e6c30, 0x369e40d0
Caught exception in looking up space point ptr... 0x10220400, 0x10220360
Caught exception in looking up space point ptr... 0x10220360, 0x5e1b8b50
Caught exception in looking up space point ptr... 0x3a7486a0, 0x3ab4c650
Caught exception in looking up space point ptr... 0x3ab4c650, 0x3a7486a0
Caught exception in looking up space point ptr... 0x3aa921e0, 0x2c3f3ea0
Caught exception in looking up space point ptr... 0x2c3f3ea0, 0x3aa921e0
Caught exception in looking up space point ptr... 0x372c9400, 0x2a49b510
Caught exception in looking up space point ptr... 0x2a49b510, 0x28975580
Caught exception in looking up space point ptr... 0x2a4a1d30, 0x2a4a21f0
Caught exception in looking up space point ptr... 0x2a4a21f0, 0x372c9400
Caught exception in looking up space point ptr... 0x86348a0, 0x8634470
Caught exception in looking up space point ptr... 0x8634470, 0x86348a0
Caught exception in looking up space point ptr... 0x354514b0, 0x2a35d600
Caught exception in looking up space point ptr... 0x2708d390, 0x354514b0
Caught exception in looking up space point ptr... 0x3de09c60, 0x329493c0
Caught exception in looking up space point ptr... 0x329493c0, 0x3de09e50
Caught exception in looking up space point ptr... 0x10527390, 0x63998240
Caught exception in looking up space point ptr... 0x63998240, 0x639981a0
Caught exception in looking up space point ptr... 0x36971950, 0x369717c0
Caught exception in looking up space point ptr... 0x369717c0, 0x36971950
Caught exception in looking up space point ptr... 0x8625d50, 0x8625a30
Caught exception in looking up space point ptr... 0x8625a30, 0x8625d50
Caught exception in looking up space point ptr... 0x13b683d0, 0x32334e70
Caught exception in looking up space point ptr... 0x13b682a0, 0x13b683d0
Caught exception in looking up space point ptr... 0xeb91930, 0xeb917e0
Caught exception in looking up space point ptr... 0xeb917e0, 0xeb91930
Caught exception in looking up space point ptr... 0xc380b50, 0xc3807f0
Caught exception in looking up space point ptr... 0xc3807f0, 0xc380b50
Caught exception in looking up space point ptr... 0x53702c70, 0x53702a70
Caught exception in looking up space point ptr... 0x53702a70, 0x53702c70
++++>>>> total num hits: 46456, num free: 44752
04-Feb-2026 02:07:21 GMT  Closed output file "np02vd_raw_run042421_0020_df-s04-d0_dw_0_20260203T134147_reco_stage0_20260204T020721_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         208.303       254.134       302.448       246.606       26.9483        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.3896e-05    0.000107538    0.0002232    0.000104763   2.79225e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                90.362        128.845       181.658       124.183       23.7757        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.565287       0.75024       1.09398      0.743711      0.117078        27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0165659     0.0208724     0.0416635     0.0175193    0.00620253       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0814533     0.087731      0.137439      0.0827568     0.011588        27     
produce:beamevent:BeamEvent                      8.7118e-05    0.000169981   0.00115484    0.000126799   0.000194537      27     
produce:pdvddaphne:DAPHNEReaderPDVD                14.9611       19.0976       24.2721       19.4796       2.33217        27     
produce:ophit:OpHitFinder                         0.0320581     0.0551736     0.0793558     0.0587554     0.0147111       27     
produce:wclsdatavd:WireCellToolkit                 66.3088       85.0479       111.232       80.628        15.3752        27     
produce:gaushit:GausHitFinder                     0.617731       1.22604       1.95267       1.21043      0.319518        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000378606   0.000572466   0.00079121    0.000583294   0.000112426      27     
produce:reco3d:SpacePointSolver                    7.60047       12.6765       19.3287       12.5588       3.28317        27     
produce:hitpdune:DisambigFromSpacePoints          0.0825671     0.164076      0.319456       0.15645      0.0581498       27     
produce:cluster3d:Cluster3D                       0.827584       2.09883       6.74019       1.95642       1.0968         27     
[art]:TriggerResults:TriggerResultInserter       1.8304e-05    3.21071e-05   6.8175e-05    2.9617e-05    9.63047e-06      27     
end_path:out1:RootOutput                          2.662e-06    6.36026e-06   1.8402e-05     5.603e-06    2.96538e-06      27     
end_path:out1:RootOutput(write)                    3.03323       4.0289        5.44552       3.63654      0.857489        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4534.72 MB
  Peak resident set size usage (VmHWM): 2626.28 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0020_df-s04-d0_dw_0_20260203T134147_reco_stage0_20260204T020721_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0020_df-s04-d0_dw_0_20260203T134147_reco_stage0_20260204T020721_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0020_df-s04-d0_dw_0_20260203T134147_reco_stage0_20260204T020721_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163904.0,
    "core.end_time": 1770170844.0,
    "core.events": [
      3274,
      3280,
      3286,
      3292,
      3298,
      3304,
      3310,
      3316,
      3322,
      3328,
      3334,
      3340,
      3346,
      3352,
      3358,
      3364,
      3370,
      3376,
      3382,
      3388,
      3394,
      3400,
      3406,
      3412,
      3418,
      3424,
      3430
    ],
    "core.event_count": 27,
    "core.first_event_number": 3274,
    "core.last_event_number": 3430,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0020_df-s04-d0_dw_0_20260203T134147.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:06:53 UTC       justIN version: 01.06.00