justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303649.134@dunegpschedd01.fnal.gov

Jobsub ID303649.134@dunegpschedd01.fnal.gov
Workflow ID12718
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:48:26
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod01
Last heartbeat2026-02-04 00:41:49
From worker nodeHostnamewh17.grid.hep.ph.ic.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:02:47
Input filesvd-protodune:np02vd_raw_run042422_0108_df-s03-d1_dw_0_20260203T161309.hdf5
JobscriptExit code0
Real time1h (5760s)
CPU time1h (4496s = 78%)
Max RSS bytes4111622144 (3921 MiB)
Outputting started2026-02-04 00:38:54
Output files
Finished2026-02-04 00:41:49
Saved logsjustin-logs:303649.134-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

exception in looking up space point ptr... 0x2e253260, 0x3b3b5aa0
Caught exception in looking up space point ptr... 0x45c3d370, 0x45c3d4e0
Caught exception in looking up space point ptr... 0x45c3d4e0, 0x45c3d6c0
Caught exception in looking up space point ptr... 0x3a0b2fa0, 0x3a0b03e0
Caught exception in looking up space point ptr... 0x3a0b03e0, 0x3a0b2fa0
Caught exception in looking up space point ptr... 0x50b4bf70, 0x3247e270
Caught exception in looking up space point ptr... 0x3247e270, 0x4ab49e10
Caught exception in looking up space point ptr... 0x4ab49e10, 0x35664660
Caught exception in looking up space point ptr... 0x2b9ec530, 0x2b9ebf60
Caught exception in looking up space point ptr... 0x2b9ebf60, 0x4f3a2bd0
Caught exception in looking up space point ptr... 0x36676620, 0x50817080
Caught exception in looking up space point ptr... 0x50817080, 0x417c9ea0
Caught exception in looking up space point ptr... 0x521571e0, 0x4fada570
Caught exception in looking up space point ptr... 0x4fada570, 0x521571e0
Caught exception in looking up space point ptr... 0x440cf010, 0x50bbe5b0
Caught exception in looking up space point ptr... 0x50bbe5b0, 0x440cf010
Caught exception in looking up space point ptr... 0x2891ba10, 0x2891bba0
Caught exception in looking up space point ptr... 0x2891bba0, 0x2891b880
Caught exception in looking up space point ptr... 0x2899ac00, 0x2899a8b0
Caught exception in looking up space point ptr... 0x2899a8b0, 0x2899ac00
Caught exception in looking up space point ptr... 0x2d452450, 0x2c5341b0
Caught exception in looking up space point ptr... 0x2c5341b0, 0x2d452450
Caught exception in looking up space point ptr... 0x31dbdbd0, 0x2953c220
Caught exception in looking up space point ptr... 0x2953c220, 0x31dbdbd0
Caught exception in looking up space point ptr... 0x3e6f0720, 0x2c849300
Caught exception in looking up space point ptr... 0x2c849300, 0x3e6f0720
Caught exception in looking up space point ptr... 0x3b92ee80, 0x35377ec0
Caught exception in looking up space point ptr... 0x35377ec0, 0x3b92ef40
Caught exception in looking up space point ptr... 0x3b92ef40, 0x3c3a43b0
Caught exception in looking up space point ptr... 0x39733da0, 0x39733b40
Caught exception in looking up space point ptr... 0x39733b40, 0x39733da0
Caught exception in looking up space point ptr... 0x508b40e0, 0x5045b270
Caught exception in looking up space point ptr... 0x5045b270, 0x3b7cb0d0
Caught exception in looking up space point ptr... 0x5bf6bbb0, 0x5bf6b960
Caught exception in looking up space point ptr... 0x41961e60, 0x5bf6bbb0
Caught exception in looking up space point ptr... 0x31d4e8a0, 0x5046d530
Caught exception in looking up space point ptr... 0x5046d530, 0x3948c750
Caught exception in looking up space point ptr... 0x2e0242c0, 0x2d475290
Caught exception in looking up space point ptr... 0x2d475290, 0x2d477800
Caught exception in looking up space point ptr... 0x2d477800, 0x2d474ab0
Caught exception in looking up space point ptr... 0x2d474ab0, 0x38da68b0
Caught exception in looking up space point ptr... 0x4ea024e0, 0x43b4cc80
Caught exception in looking up space point ptr... 0x43b4cc80, 0x4e869760
Caught exception in looking up space point ptr... 0x2eac4800, 0x50be4ff0
Caught exception in looking up space point ptr... 0x50be4ff0, 0x2eac4800
Caught exception in looking up space point ptr... 0x398bbac0, 0x398bd410
Caught exception in looking up space point ptr... 0x398bd410, 0x398bbac0
Caught exception in looking up space point ptr... 0x40878210, 0x408783f0
Caught exception in looking up space point ptr... 0x408783f0, 0x40878080
Caught exception in looking up space point ptr... 0x35b73ef0, 0x4fe7ad00
Caught exception in looking up space point ptr... 0x4fe7ad00, 0x5062ce70
Caught exception in looking up space point ptr... 0x2e658a60, 0x293e82f0
Caught exception in looking up space point ptr... 0x293e82f0, 0x50d275d0
Caught exception in looking up space point ptr... 0x50cafa20, 0x3d3e5900
Caught exception in looking up space point ptr... 0x3d3e5900, 0x50cafa20
Caught exception in looking up space point ptr... 0x3d42e440, 0x2ba12c10
Caught exception in looking up space point ptr... 0x2ba12c10, 0x50b75450
Caught exception in looking up space point ptr... 0x4ef8ac40, 0x17cdc560
Caught exception in looking up space point ptr... 0x17cdc560, 0x17cdc6f0
Caught exception in looking up space point ptr... 0x36a337b0, 0x3ac48280
Caught exception in looking up space point ptr... 0x3ac48280, 0x3ac47f60
Caught exception in looking up space point ptr... 0x2d5c1730, 0x47ae5bf0
Caught exception in looking up space point ptr... 0x47ae5bf0, 0x2d5c1730
Caught exception in looking up space point ptr... 0x40761440, 0x40761120
Caught exception in looking up space point ptr... 0x40761120, 0x40760d60
++++>>>> total num hits: 53828, num free: 51453
04-Feb-2026 00:38:38 GMT  Closed output file "np02vd_raw_run042422_0108_df-s03-d1_dw_0_20260203T161309_reco_stage0_20260204T003838_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         160.977       208.065       530.977       202.617       66.408         27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       5.8358e-05    8.98088e-05   0.000259371   8.3148e-05    4.09609e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                38.0243       39.6798       43.9535       39.5691       1.35718        27     
produce:triggerrawdecoder:PDVDTriggerReader4       0.49663      0.569084      0.731996      0.546695      0.0613812       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.016212      0.019963      0.0465452     0.0165921    0.00637486       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.080486      0.0828368     0.0973905     0.0810914    0.00503638       27     
produce:beamevent:BeamEvent                       8.749e-05    0.000143621   0.000840994   0.000114663   0.00013872       27     
produce:pdvddaphne:DAPHNEReaderPDVD                6.38633       8.31162       10.0615       8.49657      0.937811        27     
produce:ophit:OpHitFinder                         0.0321637     0.0454461     0.0578093     0.0479279    0.00808601       27     
produce:wclsdatavd:WireCellToolkit                 93.4022       118.08        146.085       116.565       16.4057        27     
produce:gaushit:GausHitFinder                      1.02593       1.94815       8.10263       1.64475       1.29559        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000224049   0.000447148   0.00135249    0.000417387   0.00021116       27     
produce:reco3d:SpacePointSolver                    6.6183        15.8243       31.4817       16.1976       5.98969        27     
produce:hitpdune:DisambigFromSpacePoints          0.0882246     0.228861       1.4093       0.177847      0.242866        27     
produce:cluster3d:Cluster3D                        1.52647       14.8639       317.645       3.27488       59.3917        27     
[art]:TriggerResults:TriggerResultInserter        2.398e-05    3.42466e-05   8.3462e-05    2.9518e-05    1.33022e-05      27     
end_path:out1:RootOutput                          4.173e-06    8.9027e-06    2.1891e-05     8.262e-06    4.08925e-06      27     
end_path:out1:RootOutput(write)                    5.47494       8.38795       17.5596       8.10926       2.19985        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6128.84 MB
  Peak resident set size usage (VmHWM): 4111.62 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042422_0108_df-s03-d1_dw_0_20260203T161309_reco_stage0_20260204T003838_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042422_0108_df-s03-d1_dw_0_20260203T161309_reco_stage0_20260204T003838_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042422_0108_df-s03-d1_dw_0_20260203T161309_reco_stage0_20260204T003838_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770159785.0,
    "core.end_time": 1770165521.0,
    "core.events": [
      17511,
      17517,
      17523,
      17529,
      17535,
      17541,
      17547,
      17553,
      17559,
      17565,
      17571,
      17577,
      17583,
      17589,
      17595,
      17601,
      17607,
      17613,
      17619,
      17625,
      17631,
      17637,
      17643,
      17649,
      17655,
      17661,
      17667
    ],
    "core.event_count": 27,
    "core.first_event_number": 17511,
    "core.last_event_number": 17667,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42422
    ],
    "core.runs_subruns": [
      4242200001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042422_0108_df-s03-d1_dw_0_20260203T161309.hdf5"
    }
  ]
}
justIN time: 2026-02-04 05:59:13 UTC       justIN version: 01.06.00