justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.169@dunegpschedd02.fnal.gov

Jobsub ID297921.169@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 00:59:41
From worker nodeHostnameheplnc167.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:56:14
Input filesvd-protodune:np02vd_raw_run042433_0630_df-s02-d1_dw_0_20260203T224133.hdf5
JobscriptExit code0
Real time1h (3660s)
CPU time56m (3384s = 92%)
Max RSS bytes3409723392 (3251 MiB)
Outputting started2026-02-04 00:57:15
Output files
Finished2026-02-04 00:59:41
Saved logsjustin-logs:297921.169-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ce point ptr... 0x47340ca0, 0x47362760
Caught exception in looking up space point ptr... 0x32b3bdf0, 0x146b5f00
Caught exception in looking up space point ptr... 0x146b5f00, 0x4b391220
Caught exception in looking up space point ptr... 0x4b17d740, 0x32b3bdf0
Caught exception in looking up space point ptr... 0x3ece2410, 0x38715ac0
Caught exception in looking up space point ptr... 0x38715ac0, 0x3ece2410
Caught exception in looking up space point ptr... 0x1675a020, 0x16759b70
Caught exception in looking up space point ptr... 0x16759b70, 0x1675a020
Caught exception in looking up space point ptr... 0x57c82b20, 0x19d872f0
Caught exception in looking up space point ptr... 0x57c82800, 0x57c82b20
Caught exception in looking up space point ptr... 0x11bc6a20, 0x3ed04c50
Caught exception in looking up space point ptr... 0x3ed04c50, 0x67715d20
Caught exception in looking up space point ptr... 0x1ad81ea0, 0x1ad8db10
Caught exception in looking up space point ptr... 0x1ad8db10, 0x1ad81f90
Caught exception in looking up space point ptr... 0x598106a0, 0x59810450
Caught exception in looking up space point ptr... 0x59807d80, 0x598106a0
Caught exception in looking up space point ptr... 0xacb6fb0, 0xacb77a0
Caught exception in looking up space point ptr... 0xacb5e90, 0xacb9380
Caught exception in looking up space point ptr... 0xacb9380, 0xacb6fb0
Caught exception in looking up space point ptr... 0x620bb370, 0x5ebddc90
Caught exception in looking up space point ptr... 0x5ebddc90, 0x620bb370
Caught exception in looking up space point ptr... 0x4f920840, 0x4f9211c0
Caught exception in looking up space point ptr... 0x4f920aa0, 0x4f920840
Caught exception in looking up space point ptr... 0x53a78bd0, 0x5977bb50
Caught exception in looking up space point ptr... 0x5977bb50, 0x53a7a560
Caught exception in looking up space point ptr... 0x87aee890, 0x87aeeb80
Caught exception in looking up space point ptr... 0x87aeeb80, 0x87aee890
Caught exception in looking up space point ptr... 0x3e1451a0, 0x3e1457b0
Caught exception in looking up space point ptr... 0x3e1457b0, 0x3e145900
Caught exception in looking up space point ptr... 0x151ea510, 0x151e97c0
Caught exception in looking up space point ptr... 0x151e97c0, 0x11c4d680
Caught exception in looking up space point ptr... 0x1ad76d20, 0x1ad73600
Caught exception in looking up space point ptr... 0x1ad73600, 0x1ad73560
Caught exception in looking up space point ptr... 0xe732410, 0xe7321b0
Caught exception in looking up space point ptr... 0xe7321b0, 0xe732080
Caught exception in looking up space point ptr... 0x1ad99f60, 0x1ad9a2f0
Caught exception in looking up space point ptr... 0x1ad9a2f0, 0x1ada5590
Caught exception in looking up space point ptr... 0x3c4fc290, 0x38920e00
Caught exception in looking up space point ptr... 0x38920e00, 0x3c4fc290
Caught exception in looking up space point ptr... 0x12977ff0, 0x1806c750
Caught exception in looking up space point ptr... 0x1806c750, 0x12977ff0
Caught exception in looking up space point ptr... 0x87af0140, 0x87af0320
Caught exception in looking up space point ptr... 0x87af0320, 0x87af0140
Caught exception in looking up space point ptr... 0x16740810, 0x16740910
Caught exception in looking up space point ptr... 0x16740910, 0x16740b10
Caught exception in looking up space point ptr... 0x67be9e00, 0x67be9f00
Caught exception in looking up space point ptr... 0x67be9f00, 0x67be9e00
Caught exception in looking up space point ptr... 0x1ad977b0, 0x1ad9ad20
Caught exception in looking up space point ptr... 0x1ad9ad20, 0x1ad9a190
Caught exception in looking up space point ptr... 0xac9d930, 0xac9de90
Caught exception in looking up space point ptr... 0xac9d270, 0xac9d930
Caught exception in looking up space point ptr... 0x393701b0, 0x5ed3be30
Caught exception in looking up space point ptr... 0x5ed3be30, 0x393701b0
Caught exception in looking up space point ptr... 0x393701b0, 0x37742340
Caught exception in looking up space point ptr... 0x37742340, 0x393701b0
Caught exception in looking up space point ptr... 0x18045b80, 0x18045860
Caught exception in looking up space point ptr... 0x18045860, 0x18045b80
Caught exception in looking up space point ptr... 0x446b1750, 0x446b1650
Caught exception in looking up space point ptr... 0x446b1650, 0x446b1750
Caught exception in looking up space point ptr... 0x16625630, 0x2bdfbcd0
Caught exception in looking up space point ptr... 0x2bdfbcd0, 0x5b2b4840
Caught exception in looking up space point ptr... 0x58268d20, 0x58268a30
Caught exception in looking up space point ptr... 0x58268a30, 0x58268d20
Caught exception in looking up space point ptr... 0x36393b20, 0x129a6d40
Caught exception in looking up space point ptr... 0x129a6d40, 0x36393b20
Caught exception in looking up space point ptr... 0x3ecdd760, 0x3ecdd8f0
Caught exception in looking up space point ptr... 0x3ecdd8f0, 0x3ecdd760
++++>>>> total num hits: 76490, num free: 73866
04-Feb-2026 00:57:06 GMT  Closed output file "np02vd_raw_run042433_0630_df-s02-d1_dw_0_20260203T224133_reco_stage0_20260204T005706_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         148.11        188.112       224.568       191.56         19.24         19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.1984e-05    0.000127662   0.000251501   0.000102051   4.90844e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                12.9659       13.614        15.1923       13.542       0.458134        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.503953      0.529533      0.559572      0.527146      0.0115506       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175139     0.0177979     0.0180987     0.0178059    0.000167711      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0174942     0.0177854     0.0180617     0.0178484    0.000178459      19     
produce:beamevent:BeamEvent                      0.000142958   0.000244725   0.00153553    0.000175639   0.000304677      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.32341       3.72378       4.26479       3.69983        0.198         19     
produce:ophit:OpHitFinder                         0.0260216     0.0457354     0.0530933     0.0467133    0.00527064       19     
produce:wclsdatavd:WireCellToolkit                 107.528       129.85        152.648       129.555       10.3453        19     
produce:gaushit:GausHitFinder                     0.714796       1.81128       2.97497       1.99139      0.589767        19     
produce:nhitsfilter:NumberOfHitsFilter           0.00030463    0.000491748   0.000937615   0.000497853   0.000150642      19     
produce:reco3d:SpacePointSolver                    14.9272       27.6718       46.2018       29.6257       8.39899        19     
produce:hitpdune:DisambigFromSpacePoints           0.11733      0.283217      0.480156      0.303104      0.0984282       19     
produce:cluster3d:Cluster3D                        1.13208       3.54513       7.88112       3.57403       1.3879         19     
[art]:TriggerResults:TriggerResultInserter       2.9315e-05    4.48631e-05   0.000148338   3.7691e-05    2.66634e-05      19     
end_path:out1:RootOutput                          4.709e-06    1.04538e-05   6.3219e-05     7.815e-06    1.26186e-05      19     
end_path:out1:RootOutput(write)                    4.62424       6.95477       8.41148       7.20908       1.01692        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5393.34 MB
  Peak resident set size usage (VmHWM): 3409.72 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0630_df-s02-d1_dw_0_20260203T224133_reco_stage0_20260204T005706_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0630_df-s02-d1_dw_0_20260203T224133_reco_stage0_20260204T005706_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0630_df-s02-d1_dw_0_20260203T224133_reco_stage0_20260204T005706_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162987.0,
    "core.end_time": 1770166627.0,
    "core.events": [
      71827,
      71833,
      71839,
      71845,
      71851,
      71857,
      71863,
      71869,
      71875,
      71881,
      71887,
      71893,
      71899,
      71905,
      71911,
      71917,
      71923,
      71929,
      71935
    ],
    "core.event_count": 19,
    "core.first_event_number": 71827,
    "core.last_event_number": 71935,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0630_df-s02-d1_dw_0_20260203T224133.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:05 UTC       justIN version: 01.06.00