justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.124@dunegpschedd02.fnal.gov

Jobsub ID297923.124@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:32:00
From worker nodeHostnameheplnc137.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:03:23
Input filesvd-protodune:np02vd_raw_run042433_0445_df-s03-d1_dw_0_20260203T213115.hdf5
JobscriptExit code0
Real time1h (5174s)
CPU time1h (4780s = 92%)
Max RSS bytes3381608448 (3224 MiB)
Outputting started2026-02-04 01:29:38
Output files
Finished2026-02-04 01:32:00
Saved logsjustin-logs:297923.124-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

... 0x2bdaecd0, 0x2bdada10
Caught exception in looking up space point ptr... 0x2bdada10, 0x2bdaecd0
Caught exception in looking up space point ptr... 0x2d7b4060, 0x32db2c00
Caught exception in looking up space point ptr... 0x32db2c00, 0x2d7b4060
Caught exception in looking up space point ptr... 0x5a0670f0, 0x5a066fc0
Caught exception in looking up space point ptr... 0x5a066fc0, 0x1ae25b90
Caught exception in looking up space point ptr... 0x3a2a0330, 0x2a843830
Caught exception in looking up space point ptr... 0x2a843830, 0x100296d0
Caught exception in looking up space point ptr... 0x2d772f10, 0x334bc5a0
Caught exception in looking up space point ptr... 0x334bc5a0, 0x2d772e40
Caught exception in looking up space point ptr... 0x2d772e40, 0x16889ef0
Caught exception in looking up space point ptr... 0x6085b920, 0x3a5689a0
Caught exception in looking up space point ptr... 0x3a5689a0, 0x1ae25930
Caught exception in looking up space point ptr... 0x425dc740, 0x425dc380
Caught exception in looking up space point ptr... 0x425dc380, 0x425dc740
Caught exception in looking up space point ptr... 0x47a948e0, 0x47a93ad0
Caught exception in looking up space point ptr... 0x47a93ad0, 0x47a93ce0
Caught exception in looking up space point ptr... 0x38787aa0, 0x38787d90
Caught exception in looking up space point ptr... 0x38787d90, 0x6203f100
Caught exception in looking up space point ptr... 0x68030b60, 0x4296e970
Caught exception in looking up space point ptr... 0x4296e970, 0x3c384ea0
Caught exception in looking up space point ptr... 0x38d2ba90, 0x38d2b960
Caught exception in looking up space point ptr... 0x38d2e810, 0x38d2ba90
Caught exception in looking up space point ptr... 0x4f967540, 0x7162e6b0
Caught exception in looking up space point ptr... 0x7162e6b0, 0x4f967540
Caught exception in looking up space point ptr... 0x326aed60, 0x326af080
Caught exception in looking up space point ptr... 0x326af080, 0x326aec60
Caught exception in looking up space point ptr... 0x396213e0, 0x39621670
Caught exception in looking up space point ptr... 0x39621670, 0x39621b90
Caught exception in looking up space point ptr... 0x817afec0, 0x6334eb40
Caught exception in looking up space point ptr... 0x817afc20, 0x817afec0
Caught exception in looking up space point ptr... 0x328320a0, 0x3282c720
Caught exception in looking up space point ptr... 0x3282c720, 0x328320a0
Caught exception in looking up space point ptr... 0x697c46f0, 0x7ec28890
Caught exception in looking up space point ptr... 0x7ec28890, 0x697c46f0
Caught exception in looking up space point ptr... 0x115b93d0, 0x7aa173a0
Caught exception in looking up space point ptr... 0x7aa173a0, 0x115b93d0
Caught exception in looking up space point ptr... 0x3190b9e0, 0x3190c620
Caught exception in looking up space point ptr... 0x3190c620, 0x3190b9e0
Caught exception in looking up space point ptr... 0x721654b0, 0x10027ae0
Caught exception in looking up space point ptr... 0x10027ae0, 0x2d377730
Caught exception in looking up space point ptr... 0x695fe780, 0x695f1e20
Caught exception in looking up space point ptr... 0x695f1e20, 0x695fe780
Caught exception in looking up space point ptr... 0x39489060, 0x39489190
Caught exception in looking up space point ptr... 0x39488f30, 0x39489060
Caught exception in looking up space point ptr... 0x2ac755b0, 0x4c564f40
Caught exception in looking up space point ptr... 0x4c564f40, 0x4c565040
Caught exception in looking up space point ptr... 0x3a6e7ed0, 0x3a6e8120
Caught exception in looking up space point ptr... 0x3a6e7c80, 0x3a6e7ed0
Caught exception in looking up space point ptr... 0x2ad17e70, 0x2ad179c0
Caught exception in looking up space point ptr... 0x2ad179c0, 0x2ad17e70
Caught exception in looking up space point ptr... 0x695fd5d0, 0x3945eee0
Caught exception in looking up space point ptr... 0x3945eee0, 0x3945f500
Caught exception in looking up space point ptr... 0x63350780, 0x633505f0
Caught exception in looking up space point ptr... 0x633505f0, 0x63350780
Caught exception in looking up space point ptr... 0x695d6600, 0x695d6150
Caught exception in looking up space point ptr... 0x695d6150, 0x695d6600
Caught exception in looking up space point ptr... 0x4f732470, 0xe41d160
Caught exception in looking up space point ptr... 0xe41d160, 0x4f732470
Caught exception in looking up space point ptr... 0x4f732470, 0x3beeb810
Caught exception in looking up space point ptr... 0x3beeb810, 0x4f732470
Caught exception in looking up space point ptr... 0x4f732470, 0x37ffe070
Caught exception in looking up space point ptr... 0x37ffe070, 0x4f732470
Caught exception in looking up space point ptr... 0x32bdd540, 0x38fa4b40
Caught exception in looking up space point ptr... 0x38fa4b40, 0x58ac43e0
Caught exception in looking up space point ptr... 0x64578d50, 0x3961dbc0
Caught exception in looking up space point ptr... 0x3961dbc0, 0x695e5570
++++>>>> total num hits: 71616, num free: 69017
04-Feb-2026 01:29:24 GMT  Closed output file "np02vd_raw_run042433_0445_df-s03-d1_dw_0_20260203T213115_reco_stage0_20260204T012924_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         221.305       262.362       328.411       260.48        26.3335        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.7716e-05    0.000154558   0.000302697   0.000129231   6.65645e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                16.7504       17.4045       18.8524       17.3342      0.514923        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.513997      0.545337      0.579158      0.546842      0.0158778       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0176643     0.0199243     0.0384389     0.0179623    0.00579758       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176583     0.0179318     0.0187278     0.0179022    0.00021162       19     
produce:beamevent:BeamEvent                      0.00012763    0.000254396   0.00140216    0.000199901   0.000272008      19     
produce:pdvddaphne:DAPHNEReaderPDVD                6.71708       7.66144       8.93423       7.44879      0.601344        19     
produce:ophit:OpHitFinder                         0.0395656     0.0640598     0.086314      0.0697873     0.0141993       19     
produce:wclsdatavd:WireCellToolkit                 162.014       196.745       252.286       195.546       20.3522        19     
produce:gaushit:GausHitFinder                      1.51044       2.23368       3.69984       2.0164       0.585189        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000474888   0.000686335   0.00117525    0.000696664   0.000160269      19     
produce:reco3d:SpacePointSolver                    16.7033       23.7692       40.5002       21.6969       5.88731        19     
produce:hitpdune:DisambigFromSpacePoints          0.162223      0.343297       0.73115      0.270417      0.139442        19     
produce:cluster3d:Cluster3D                        2.27559       4.18951       9.35735       3.2907        2.09149        19     
[art]:TriggerResults:TriggerResultInserter       2.9317e-05    4.38646e-05   0.000141782   3.8962e-05    2.38923e-05      19     
end_path:out1:RootOutput                          4.028e-06    1.04278e-05   3.3014e-05     9.478e-06    6.00304e-06      19     
end_path:out1:RootOutput(write)                    7.03536       9.30248       12.9835       9.22587       1.66634        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5371.97 MB
  Peak resident set size usage (VmHWM): 3381.61 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0445_df-s03-d1_dw_0_20260203T213115_reco_stage0_20260204T012924_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0445_df-s03-d1_dw_0_20260203T213115_reco_stage0_20260204T012924_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0445_df-s03-d1_dw_0_20260203T213115_reco_stage0_20260204T012924_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163423.0,
    "core.end_time": 1770168571.0,
    "core.events": [
      50739,
      50745,
      50751,
      50757,
      50763,
      50769,
      50775,
      50781,
      50787,
      50793,
      50799,
      50805,
      50811,
      50817,
      50823,
      50829,
      50835,
      50841,
      50847
    ],
    "core.event_count": 19,
    "core.first_event_number": 50739,
    "core.last_event_number": 50847,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0445_df-s03-d1_dw_0_20260203T213115.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:45:29 UTC       justIN version: 01.06.00