justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.12@dunegpschedd01.fnal.gov

Jobsub ID303656.12@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-04 02:13:06
From worker nodeHostnamecn106.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:11:22
Input filesvd-protodune:np02vd_raw_run042421_0029_df-s04-d1_dw_0_20260203T134643.hdf5
JobscriptExit code0
Real time1h (7168s)
CPU time49m (2954s = 41%)
Max RSS bytes2541166592 (2423 MiB)
Outputting started2026-02-04 02:10:52
Output files
Finished2026-02-04 02:13:06
Saved logsjustin-logs:303656.12-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

space point ptr... 0x54766040, 0xb906a20
Caught exception in looking up space point ptr... 0xb906a20, 0xb907730
Caught exception in looking up space point ptr... 0xb907730, 0x54769130
Caught exception in looking up space point ptr... 0x54769130, 0x5476abe0
Caught exception in looking up space point ptr... 0x5476abe0, 0x5b906800
Caught exception in looking up space point ptr... 0x2eaa4560, 0x47a79ec0
Caught exception in looking up space point ptr... 0x47a79ec0, 0x3fb3a990
Caught exception in looking up space point ptr... 0x35ebd4e0, 0x264122e0
Caught exception in looking up space point ptr... 0x264122e0, 0x264123e0
Caught exception in looking up space point ptr... 0x26e46810, 0xdfb2ea0
Caught exception in looking up space point ptr... 0xdfb2ea0, 0x5c290200
Caught exception in looking up space point ptr... 0x4709ceb0, 0x26e46810
Caught exception in looking up space point ptr... 0x3649dbc0, 0x89dc1d0
Caught exception in looking up space point ptr... 0x2958e510, 0x3649dbc0
Caught exception in looking up space point ptr... 0x2efc3600, 0xa45b840
Caught exception in looking up space point ptr... 0xa45b840, 0x2efc4c60
Caught exception in looking up space point ptr... 0x3f71ec40, 0x3f71eab0
Caught exception in looking up space point ptr... 0x3f7130e0, 0x3f71ec40
Caught exception in looking up space point ptr... 0x3fcfeed0, 0x5b8db470
Caught exception in looking up space point ptr... 0x2bc45270, 0x3fcfeed0
Caught exception in looking up space point ptr... 0xaafa440, 0x334ee6f0
Caught exception in looking up space point ptr... 0x42698960, 0xaafa440
Caught exception in looking up space point ptr... 0x551709b0, 0x551767b0
Caught exception in looking up space point ptr... 0x551767b0, 0x55176490
Caught exception in looking up space point ptr... 0x36a3f270, 0x36a3edc0
Caught exception in looking up space point ptr... 0x36a3edc0, 0x36a3f270
Caught exception in looking up space point ptr... 0x15044210, 0x150440c0
Caught exception in looking up space point ptr... 0x46f7e920, 0x15044210
Caught exception in looking up space point ptr... 0x3557d8d0, 0x2fb85a50
Caught exception in looking up space point ptr... 0x3557c850, 0x3557d8d0
Caught exception in looking up space point ptr... 0x390dd4b0, 0xce5afe0
Caught exception in looking up space point ptr... 0xce5afe0, 0x390dd4b0
Caught exception in looking up space point ptr... 0x20d915e0, 0x35685720
Caught exception in looking up space point ptr... 0x35685720, 0xe005560
Caught exception in looking up space point ptr... 0x15058bd0, 0x15058d60
Caught exception in looking up space point ptr... 0x15058d60, 0x1505f540
Caught exception in looking up space point ptr... 0x3f8fff30, 0x2ffcf3b0
Caught exception in looking up space point ptr... 0x2ffcf3b0, 0xd7a1fb0
Caught exception in looking up space point ptr... 0x8bbe4b0, 0x8bbe320
Caught exception in looking up space point ptr... 0x8bbe320, 0x8bbe4b0
Caught exception in looking up space point ptr... 0x47e2ba70, 0x47e2b8e0
Caught exception in looking up space point ptr... 0xb64b900, 0x47e2ba70
Caught exception in looking up space point ptr... 0x2c79a450, 0x2c799f00
Caught exception in looking up space point ptr... 0x2c799f00, 0x2c79c7a0
Caught exception in looking up space point ptr... 0x4a207ed0, 0x4a208060
Caught exception in looking up space point ptr... 0x4a208060, 0x4a204130
Caught exception in looking up space point ptr... 0x14517600, 0x145174b0
Caught exception in looking up space point ptr... 0x145174b0, 0x14517600
Caught exception in looking up space point ptr... 0x3f749f00, 0x3f744af0
Caught exception in looking up space point ptr... 0x3f744af0, 0x3f749b90
Caught exception in looking up space point ptr... 0x52154c80, 0x3557cf70
Caught exception in looking up space point ptr... 0x3557cf70, 0x52154c80
Caught exception in looking up space point ptr... 0x52154c80, 0x4709d770
Caught exception in looking up space point ptr... 0x4709d770, 0x52154c80
Caught exception in looking up space point ptr... 0x52154c80, 0x35655350
Caught exception in looking up space point ptr... 0x35655350, 0x52154c80
Caught exception in looking up space point ptr... 0x35650e50, 0x35650ce0
Caught exception in looking up space point ptr... 0x35650ce0, 0x35650e50
Caught exception in looking up space point ptr... 0x35650e50, 0x356509f0
Caught exception in looking up space point ptr... 0x356509f0, 0x35650e50
Caught exception in looking up space point ptr... 0x36a3ec30, 0x36a39940
Caught exception in looking up space point ptr... 0x36a39940, 0x36a3ec30
Caught exception in looking up space point ptr... 0x3fd04850, 0x3f756ee0
Caught exception in looking up space point ptr... 0x3f756ee0, 0x3fd05340
Caught exception in looking up space point ptr... 0x35645350, 0x35645c60
Caught exception in looking up space point ptr... 0x35645c60, 0x35645d90
++++>>>> total num hits: 52525, num free: 50932
04-Feb-2026 02:10:43 GMT  Closed output file "np02vd_raw_run042421_0029_df-s04-d1_dw_0_20260203T134643_reco_stage0_20260204T021043_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         229.781       262.113       340.973       257.499       26.1442        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.8881e-05    0.000112833   0.000268451   0.000102131   4.46548e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                109.173       144.106       205.031       141.196       25.5309        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.593558      0.884956       4.13386      0.774339      0.646295        27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0166587     0.0544447     0.647342      0.0231289      0.11885        27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0816914     0.0839554     0.0988445     0.0829397     0.0037624       27     
produce:beamevent:BeamEvent                      9.6026e-05    0.000167176   0.00100242    0.000129094   0.000166045      27     
produce:pdvddaphne:DAPHNEReaderPDVD                15.8856       20.1923       28.1936       19.7312       2.93234        27     
produce:ophit:OpHitFinder                         0.0296292     0.0492953     0.0679418     0.0549239     0.0127922       27     
produce:wclsdatavd:WireCellToolkit                 65.7933       81.8224       110.305       77.876        14.2126        27     
produce:gaushit:GausHitFinder                     0.568665       1.08837       1.68079       1.12429      0.258572        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000320224   0.000446643   0.000667719   0.000461123   7.13752e-05      27     
produce:reco3d:SpacePointSolver                    4.60727       8.37241        11.99        8.6415        1.79944        27     
produce:hitpdune:DisambigFromSpacePoints          0.0625377     0.119666      0.205426      0.124485      0.0325594       27     
produce:cluster3d:Cluster3D                       0.552233       1.52471       3.02999       1.36555       0.62949        27     
[art]:TriggerResults:TriggerResultInserter       2.1145e-05    3.30185e-05   9.3991e-05    3.0014e-05    1.3413e-05       27     
end_path:out1:RootOutput                          3.492e-06    6.24941e-06   2.4002e-05     4.581e-06    4.15939e-06      27     
end_path:out1:RootOutput(write)                    2.96265       3.78871       5.25214       3.48856      0.770847        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4499.64 MB
  Peak resident set size usage (VmHWM): 2541.17 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0029_df-s04-d1_dw_0_20260203T134643_reco_stage0_20260204T021043_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0029_df-s04-d1_dw_0_20260203T134643_reco_stage0_20260204T021043_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0029_df-s04-d1_dw_0_20260203T134643_reco_stage0_20260204T021043_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163890.0,
    "core.end_time": 1770171045.0,
    "core.events": [
      4751,
      4757,
      4763,
      4769,
      4775,
      4781,
      4787,
      4793,
      4799,
      4805,
      4811,
      4817,
      4823,
      4829,
      4835,
      4841,
      4847,
      4853,
      4859,
      4865,
      4871,
      4877,
      4883,
      4889,
      4895,
      4901,
      4907
    ],
    "core.event_count": 27,
    "core.first_event_number": 4751,
    "core.last_event_number": 4907,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0029_df-s04-d1_dw_0_20260203T134643.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:36:52 UTC       justIN version: 01.06.00