justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.177@dunegpschedd02.fnal.gov

Jobsub ID297921.177@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 00:58:36
From worker nodeHostnameheplnc167.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:55:47
Input filesvd-protodune:np02vd_raw_run042433_0375_df-s04-d0_dw_0_20260203T210439.hdf5
JobscriptExit code0
Real time1h (3622s)
CPU time55m (3340s = 92%)
Max RSS bytes3385589760 (3228 MiB)
Outputting started2026-02-04 00:56:10
Output files
Finished2026-02-04 00:58:36
Saved logsjustin-logs:297921.177-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

looking up space point ptr... 0x6b5ab2c0, 0x6b5a52b0
Caught exception in looking up space point ptr... 0xbcd1a40, 0x3567bd30
Caught exception in looking up space point ptr... 0x3567bd30, 0x5accebe0
Caught exception in looking up space point ptr... 0x86d48f0, 0x86d45d0
Caught exception in looking up space point ptr... 0x86d45d0, 0x86d4760
Caught exception in looking up space point ptr... 0x5ea23020, 0x5ea22e40
Caught exception in looking up space point ptr... 0x5ea22e40, 0x5ea23020
Caught exception in looking up space point ptr... 0x426e0380, 0x426dccb0
Caught exception in looking up space point ptr... 0x426dccb0, 0x426e0380
Caught exception in looking up space point ptr... 0xbdcdd00, 0x7e8997e0
Caught exception in looking up space point ptr... 0x7e8997e0, 0xf6386b0
Caught exception in looking up space point ptr... 0x6889e4c0, 0x6889e5c0
Caught exception in looking up space point ptr... 0x6889e5c0, 0x6889e4c0
Caught exception in looking up space point ptr... 0x5487d530, 0x548763e0
Caught exception in looking up space point ptr... 0x548763e0, 0x5487d530
Caught exception in looking up space point ptr... 0x5487d530, 0x5487b070
Caught exception in looking up space point ptr... 0x5487b070, 0x5487d530
Caught exception in looking up space point ptr... 0x462c3e20, 0x462c5190
Caught exception in looking up space point ptr... 0x462c5190, 0x462c3e20
Caught exception in looking up space point ptr... 0x33b3f9b0, 0x1784c620
Caught exception in looking up space point ptr... 0x1784c620, 0x33b3f9b0
Caught exception in looking up space point ptr... 0x33b3f9b0, 0x33b3f5f0
Caught exception in looking up space point ptr... 0x33b3f5f0, 0x33b3f9b0
Caught exception in looking up space point ptr... 0x8575d40, 0x857cd90
Caught exception in looking up space point ptr... 0x857cd90, 0x8576180
Caught exception in looking up space point ptr... 0x788d4c10, 0x788d48f0
Caught exception in looking up space point ptr... 0x788d48f0, 0x788d4c10
Caught exception in looking up space point ptr... 0x3d866ca0, 0x78260510
Caught exception in looking up space point ptr... 0x78260510, 0x3d864030
Caught exception in looking up space point ptr... 0x3a0c14f0, 0x7a1e4e00
Caught exception in looking up space point ptr... 0x7a1e4e00, 0x3a0c14f0
Caught exception in looking up space point ptr... 0x46823960, 0x109a0de0
Caught exception in looking up space point ptr... 0x109a0de0, 0x46823960
Caught exception in looking up space point ptr... 0x5486f410, 0x5486f210
Caught exception in looking up space point ptr... 0x5486f210, 0x5486f660
Caught exception in looking up space point ptr... 0x59cd8fb0, 0x34a33fe0
Caught exception in looking up space point ptr... 0x34a33fe0, 0x59cd8fb0
Caught exception in looking up space point ptr... 0x4909a990, 0x43097b70
Caught exception in looking up space point ptr... 0x43097b70, 0x43097d50
Caught exception in looking up space point ptr... 0x58ac6cf0, 0x6ba8f830
Caught exception in looking up space point ptr... 0x6ba8f830, 0x48f4a480
Caught exception in looking up space point ptr... 0x8590f20, 0x85644f0
Caught exception in looking up space point ptr... 0x85644f0, 0x8590f20
Caught exception in looking up space point ptr... 0x8590f20, 0x8565380
Caught exception in looking up space point ptr... 0x8565380, 0x8590f20
Caught exception in looking up space point ptr... 0x382b2610, 0x3c4caa70
Caught exception in looking up space point ptr... 0x3c4caa70, 0x48028550
Caught exception in looking up space point ptr... 0x61124d30, 0x61124b70
Caught exception in looking up space point ptr... 0x61124b70, 0x61124d30
Caught exception in looking up space point ptr... 0x47455650, 0x47445a20
Caught exception in looking up space point ptr... 0x47445a20, 0x47455650
Caught exception in looking up space point ptr... 0xfe2a850, 0xfe2a700
Caught exception in looking up space point ptr... 0xfe2a700, 0xfe2ace0
Caught exception in looking up space point ptr... 0xfe2ace0, 0xfe2ab50
Caught exception in looking up space point ptr... 0xfe2ab50, 0xfe2a850
Caught exception in looking up space point ptr... 0xf4dc0e0, 0x6b3443c0
Caught exception in looking up space point ptr... 0x6b3443c0, 0xf4dc0e0
Caught exception in looking up space point ptr... 0x18aad9a0, 0x18aaf720
Caught exception in looking up space point ptr... 0x18ab0170, 0x18aad9a0
Caught exception in looking up space point ptr... 0x4768bde0, 0x639b79a0
Caught exception in looking up space point ptr... 0x639b79a0, 0x4768bde0
Caught exception in looking up space point ptr... 0x16a20070, 0x16a20570
Caught exception in looking up space point ptr... 0x16a20570, 0x16a20070
Caught exception in looking up space point ptr... 0x2bade390, 0x3817f1a0
Caught exception in looking up space point ptr... 0x3817f1a0, 0x2bade390
Caught exception in looking up space point ptr... 0x13d92030, 0x13d91d40
Caught exception in looking up space point ptr... 0x13d91d40, 0x13d92030
++++>>>> total num hits: 104562, num free: 98786
04-Feb-2026 00:56:02 GMT  Closed output file "np02vd_raw_run042433_0375_df-s04-d0_dw_0_20260203T210439_reco_stage0_20260204T005602_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         167.739       186.45        211.837       186.216       11.1204        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        9.64e-05     0.000151979   0.000266319   0.000134492   4.69228e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.738        14.3833       14.948        14.4286      0.309239        19     
produce:triggerrawdecoder:PDVDTriggerReader4       0.52388      0.534092      0.552215      0.533804     0.00701271       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175912     0.0178886     0.0190042     0.0177885    0.000292064      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0175615     0.0178322      0.01841      0.0178102    0.000194588      19     
produce:beamevent:BeamEvent                      0.000108573   0.000228574   0.00150162    0.000163346   0.000301084      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.36017       3.77474       4.16625       3.82121      0.230343        19     
produce:ophit:OpHitFinder                         0.0337031     0.0447716     0.0550079     0.0462597    0.00542713       19     
produce:wclsdatavd:WireCellToolkit                 107.96        130.606       146.317       132.079       9.42812        19     
produce:gaushit:GausHitFinder                      1.01063       1.87226       2.86501       1.93076      0.414111        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000301364   0.000480889   0.000667971   0.000478455   9.10147e-05      19     
produce:reco3d:SpacePointSolver                    14.0658       25.0145       35.3355       26.0323       5.7906         19     
produce:hitpdune:DisambigFromSpacePoints          0.114067      0.250921      0.446236      0.264615      0.0813141       19     
produce:cluster3d:Cluster3D                        1.55419       3.28658       5.63454       3.36636       0.98798        19     
[art]:TriggerResults:TriggerResultInserter       2.6019e-05    3.89232e-05   8.3385e-05    3.5166e-05    1.34278e-05      19     
end_path:out1:RootOutput                          5.22e-06     1.22742e-05   2.6169e-05    1.0279e-05    5.96695e-06      19     
end_path:out1:RootOutput(write)                    4.89779       6.59498       7.95564       7.29271       1.12787        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5424.84 MB
  Peak resident set size usage (VmHWM): 3385.59 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0375_df-s04-d0_dw_0_20260203T210439_reco_stage0_20260204T005602_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0375_df-s04-d0_dw_0_20260203T210439_reco_stage0_20260204T005602_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0375_df-s04-d0_dw_0_20260203T210439_reco_stage0_20260204T005602_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162958.0,
    "core.end_time": 1770166564.0,
    "core.events": [
      42760,
      42766,
      42772,
      42778,
      42784,
      42790,
      42796,
      42802,
      42808,
      42814,
      42820,
      42826,
      42832,
      42838,
      42844,
      42850,
      42856,
      42862,
      42868
    ],
    "core.event_count": 19,
    "core.first_event_number": 42760,
    "core.last_event_number": 42868,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0375_df-s04-d0_dw_0_20260203T210439.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:17:22 UTC       justIN version: 01.06.00