justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297920.170@dunegpschedd02.fnal.gov

Jobsub ID297920.170@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:54:27
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2026-02-04 03:05:37
From worker nodeHostnamewn1906250.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:43:50
Input filesvd-protodune:np02vd_raw_run042433_0558_df-s04-d0_dw_0_20260203T221412.hdf5
JobscriptExit code0
Real time3h (11926s)
CPU time2h (7217s = 60%)
Max RSS bytes3539034112 (3375 MiB)
Outputting started2026-02-04 03:02:39
Output files
Finished2026-02-04 03:05:37
Saved logsjustin-logs:297920.170-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x5830d080, 0x36e08670
Caught exception in looking up space point ptr... 0x36e08670, 0x36e084e0
Caught exception in looking up space point ptr... 0x6a9508a0, 0x2a9eb590
Caught exception in looking up space point ptr... 0x2a9eb590, 0x6a9508a0
Caught exception in looking up space point ptr... 0x6a9508a0, 0x188847b0
Caught exception in looking up space point ptr... 0x188847b0, 0x6a9508a0
Caught exception in looking up space point ptr... 0x6a9508a0, 0xcccbdc0
Caught exception in looking up space point ptr... 0xcccbdc0, 0x6a9508a0
Caught exception in looking up space point ptr... 0x1729a3d0, 0x644348d0
Caught exception in looking up space point ptr... 0x644348d0, 0x644345b0
Caught exception in looking up space point ptr... 0x33043a30, 0x625b8540
Caught exception in looking up space point ptr... 0x7c4c2f60, 0x33043a30
Caught exception in looking up space point ptr... 0x4ec6a5b0, 0x5263cb70
Caught exception in looking up space point ptr... 0x424574a0, 0x4ec6a750
Caught exception in looking up space point ptr... 0x4ec6a750, 0x17705e10
Caught exception in looking up space point ptr... 0x17705e10, 0x4ec6a5b0
Caught exception in looking up space point ptr... 0x156e2e60, 0x156d0bc0
Caught exception in looking up space point ptr... 0x156d0bc0, 0x156c0860
Caught exception in looking up space point ptr... 0x13719a90, 0x717867d0
Caught exception in looking up space point ptr... 0x85ec000, 0x85ebe70
Caught exception in looking up space point ptr... 0x85ebe70, 0x13719a90
Caught exception in looking up space point ptr... 0x52f5df30, 0x52f4c2d0
Caught exception in looking up space point ptr... 0x52f817f0, 0x52f5df30
Caught exception in looking up space point ptr... 0x14369b80, 0x53975310
Caught exception in looking up space point ptr... 0x53975310, 0x172619d0
Caught exception in looking up space point ptr... 0x15270510, 0x15270380
Caught exception in looking up space point ptr... 0x15250300, 0x15270510
Caught exception in looking up space point ptr... 0x394e4ec0, 0x394d4310
Caught exception in looking up space point ptr... 0x394e51e0, 0x394e4ec0
Caught exception in looking up space point ptr... 0x52cdde60, 0x52cddbd0
Caught exception in looking up space point ptr... 0x52cddbd0, 0x52cdef80
Caught exception in looking up space point ptr... 0xeda8500, 0xed9a720
Caught exception in looking up space point ptr... 0xeda8690, 0xeda8500
Caught exception in looking up space point ptr... 0x7c1b1970, 0x530d1380
Caught exception in looking up space point ptr... 0x530d1380, 0x7c1b1970
Caught exception in looking up space point ptr... 0x852e1f50, 0x852e1d00
Caught exception in looking up space point ptr... 0x852e1d00, 0x852e3420
Caught exception in looking up space point ptr... 0x6f741430, 0x6f6cfe40
Caught exception in looking up space point ptr... 0x6f6cfe40, 0x6f741430
Caught exception in looking up space point ptr... 0x6e041ff0, 0x3d7855b0
Caught exception in looking up space point ptr... 0x3d7855b0, 0x17b869b0
Caught exception in looking up space point ptr... 0x197a34e0, 0xeab4c80
Caught exception in looking up space point ptr... 0xeab4c80, 0x197a34e0
Caught exception in looking up space point ptr... 0x87f7e430, 0x87f8fa50
Caught exception in looking up space point ptr... 0x87f6cc80, 0x87f7e430
Caught exception in looking up space point ptr... 0x17e5baa0, 0x17e49fd0
Caught exception in looking up space point ptr... 0x17e6d250, 0x17e5baa0
Caught exception in looking up space point ptr... 0x42704ce0, 0x1a0703a0
Caught exception in looking up space point ptr... 0x1a0703a0, 0x42704ce0
Caught exception in looking up space point ptr... 0x39013140, 0x3e73fda0
Caught exception in looking up space point ptr... 0x3e73fda0, 0x39013140
Caught exception in looking up space point ptr... 0x39013140, 0x6a871140
Caught exception in looking up space point ptr... 0x6a871140, 0x39013140
Caught exception in looking up space point ptr... 0xc7f1f20, 0x53ad9540
Caught exception in looking up space point ptr... 0x53ad9540, 0x656f0cf0
Caught exception in looking up space point ptr... 0x5bd90c40, 0x65142f50
Caught exception in looking up space point ptr... 0x65142f50, 0x49154310
Caught exception in looking up space point ptr... 0x29d77070, 0x45075290
Caught exception in looking up space point ptr... 0x45075290, 0x29d77070
Caught exception in looking up space point ptr... 0x544a8ba0, 0x38818330
Caught exception in looking up space point ptr... 0x38818330, 0x544a8ba0
Caught exception in looking up space point ptr... 0x189dcb70, 0x44c95a20
Caught exception in looking up space point ptr... 0x44c95a20, 0x189dcb70
Caught exception in looking up space point ptr... 0x4408ce40, 0x14ac0a60
Caught exception in looking up space point ptr... 0x14ac0a60, 0x4408ce40
Caught exception in looking up space point ptr... 0x4e145520, 0x65105940
Caught exception in looking up space point ptr... 0x65105940, 0xd53a1c0
++++>>>> total num hits: 505710, num free: 495522
04-Feb-2026 03:02:13 GMT  Closed output file "np02vd_raw_run042433_0558_df-s04-d0_dw_0_20260203T221412_reco_stage0_20260204T030213_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         517.651       618.925       734.501       629.658       59.7188        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.7553e-05    0.000236793    0.0005999    0.00022617    0.000119335      19     
produce:tpcrawdecoder:PDVDTPCReader                165.464       229.047       330.494       233.215       44.4023        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.768226       1.23644       3.16505       1.0608       0.542315        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0224092     0.0292191     0.060593      0.0245351     0.0109544       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0220367     0.0235743     0.0437562     0.0224498    0.00476096       19     
produce:beamevent:BeamEvent                      0.000123933   0.000317944   0.00136334    0.000248876   0.000271513      19     
produce:pdvddaphne:DAPHNEReaderPDVD                15.8545       22.8568       29.0598       22.4473       3.23987        19     
produce:ophit:OpHitFinder                         0.0460164     0.0756715     0.103709      0.0756539     0.0153655       19     
produce:wclsdatavd:WireCellToolkit                 211.01        257.22        323.957       262.996       27.0147        19     
produce:gaushit:GausHitFinder                      2.15759       3.66544       5.50524       3.70942       1.05654        19     
produce:nhitsfilter:NumberOfHitsFilter           0.00052927    0.000838554   0.00132259    0.000864584   0.000199145      19     
produce:reco3d:SpacePointSolver                    13.727        23.4334       39.4429       22.1529       6.15216        19     
produce:hitpdune:DisambigFromSpacePoints          0.288004      0.559032       1.02304      0.552929      0.193464        19     
produce:cluster3d:Cluster3D                        3.50135       67.6908       124.667       91.7391       47.899         19     
[art]:TriggerResults:TriggerResultInserter       3.8313e-05    8.03731e-05   0.000185368   7.3813e-05    3.13078e-05      19     
end_path:out1:RootOutput                          6.269e-06    1.6121e-05    5.4019e-05     1.216e-05    1.13421e-05      19     
end_path:out1:RootOutput(write)                    7.69473       12.9922       20.2172       12.7055       2.98257        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5533.93 MB
  Peak resident set size usage (VmHWM): 3539.03 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0558_df-s04-d0_dw_0_20260203T221412_reco_stage0_20260204T030213_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0558_df-s04-d0_dw_0_20260203T221412_reco_stage0_20260204T030213_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0558_df-s04-d0_dw_0_20260203T221412_reco_stage0_20260204T030213_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162242.0,
    "core.end_time": 1770174149.0,
    "core.events": [
      63622,
      63628,
      63634,
      63640,
      63646,
      63652,
      63658,
      63664,
      63670,
      63676,
      63682,
      63688,
      63694,
      63700,
      63706,
      63712,
      63718,
      63724,
      63730
    ],
    "core.event_count": 19,
    "core.first_event_number": 63622,
    "core.last_event_number": 63730,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0558_df-s04-d0_dw_0_20260203T221412.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:37:40 UTC       justIN version: 01.06.00