justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.123@dunegpschedd02.fnal.gov

Jobsub ID297923.123@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:32:48
From worker nodeHostnameheplnc137.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:03:27
Input filesvd-protodune:np02vd_raw_run042433_0622_df-s04-d0_dw_0_20260203T223832.hdf5
JobscriptExit code0
Real time1h (5220s)
CPU time1h (4816s = 92%)
Max RSS bytes3384311808 (3227 MiB)
Outputting started2026-02-04 01:30:28
Output files
Finished2026-02-04 01:32:48
Saved logsjustin-logs:297923.123-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x5fbedb90, 0x1448ab20
Caught exception in looking up space point ptr... 0x1448ab20, 0x5fbedb90
Caught exception in looking up space point ptr... 0x32573600, 0x32579210
Caught exception in looking up space point ptr... 0x32579210, 0x3258ab10
Caught exception in looking up space point ptr... 0x3258ab10, 0x32588fb0
Caught exception in looking up space point ptr... 0x32588fb0, 0x3acae270
Caught exception in looking up space point ptr... 0x3acae4c0, 0x32573600
Caught exception in looking up space point ptr... 0x5a773b70, 0x5a773d50
Caught exception in looking up space point ptr... 0x5a773d50, 0x5a774250
Caught exception in looking up space point ptr... 0x5769d510, 0x5769d830
Caught exception in looking up space point ptr... 0x5769d830, 0x5769d510
Caught exception in looking up space point ptr... 0x526b6630, 0x2c54eb00
Caught exception in looking up space point ptr... 0x2c54eb00, 0x526b6630
Caught exception in looking up space point ptr... 0x3719eca0, 0x5a3526c0
Caught exception in looking up space point ptr... 0x5a3526c0, 0x3719eca0
Caught exception in looking up space point ptr... 0x505bc810, 0x637de920
Caught exception in looking up space point ptr... 0x637de920, 0x6ef77c20
Caught exception in looking up space point ptr... 0x60116d40, 0x60117240
Caught exception in looking up space point ptr... 0x60117240, 0x60116d40
Caught exception in looking up space point ptr... 0x5f1b2650, 0x6edc10a0
Caught exception in looking up space point ptr... 0x6edc10a0, 0x6edc0f50
Caught exception in looking up space point ptr... 0x37420c40, 0x5270c550
Caught exception in looking up space point ptr... 0x5270c550, 0x3743d9b0
Caught exception in looking up space point ptr... 0x5a76c0f0, 0x5a76d8e0
Caught exception in looking up space point ptr... 0x5a76d8e0, 0x5a76c0f0
Caught exception in looking up space point ptr... 0x68964a50, 0x168f6c00
Caught exception in looking up space point ptr... 0x168f6c00, 0x68964a50
Caught exception in looking up space point ptr... 0x11867520, 0x1186bd00
Caught exception in looking up space point ptr... 0x1186bd00, 0x11867520
Caught exception in looking up space point ptr... 0x6dacaeb0, 0x1ae89940
Caught exception in looking up space point ptr... 0x1ae89940, 0x6d9cbee0
Caught exception in looking up space point ptr... 0x5a773420, 0x5a7735c0
Caught exception in looking up space point ptr... 0x5a7735c0, 0x5a773280
Caught exception in looking up space point ptr... 0x3743cbb0, 0x3743ca60
Caught exception in looking up space point ptr... 0x3743ca60, 0x3743cbb0
Caught exception in looking up space point ptr... 0x3743cbb0, 0x37436b40
Caught exception in looking up space point ptr... 0x37436b40, 0x3743cbb0
Caught exception in looking up space point ptr... 0x3743cbb0, 0x37430150
Caught exception in looking up space point ptr... 0x37430150, 0x3743cbb0
Caught exception in looking up space point ptr... 0x363aaad0, 0x4aa515b0
Caught exception in looking up space point ptr... 0x4aa515b0, 0x363aaad0
Caught exception in looking up space point ptr... 0xef33300, 0x3735d2f0
Caught exception in looking up space point ptr... 0x3735d2f0, 0xef33300
Caught exception in looking up space point ptr... 0x169c8900, 0x169c8400
Caught exception in looking up space point ptr... 0x169c8400, 0x169c85e0
Caught exception in looking up space point ptr... 0x5a770fb0, 0x5a770130
Caught exception in looking up space point ptr... 0x5a770130, 0x5a770fb0
Caught exception in looking up space point ptr... 0x43796ba0, 0x747c5d20
Caught exception in looking up space point ptr... 0x747c5d20, 0x43796ba0
Caught exception in looking up space point ptr... 0x42f8aa30, 0x3aba0340
Caught exception in looking up space point ptr... 0x3aba0340, 0x42f8aa30
Caught exception in looking up space point ptr... 0x1ae66590, 0x1ae84980
Caught exception in looking up space point ptr... 0x1ae84980, 0x1ae66590
Caught exception in looking up space point ptr... 0xb39a230, 0xa04f680
Caught exception in looking up space point ptr... 0xa04f680, 0xb39a230
Caught exception in looking up space point ptr... 0x54f14c40, 0x54f14960
Caught exception in looking up space point ptr... 0x54f14960, 0x54f158e0
Caught exception in looking up space point ptr... 0x5a774570, 0x5a7748e0
Caught exception in looking up space point ptr... 0x5a7748e0, 0x5a774a70
Caught exception in looking up space point ptr... 0x63ef4330, 0x16078650
Caught exception in looking up space point ptr... 0x16078650, 0xe472e60
Caught exception in looking up space point ptr... 0x6cc91c10, 0x5a5aeec0
Caught exception in looking up space point ptr... 0x5a5aeec0, 0x519a3700
Caught exception in looking up space point ptr... 0x3a1b6cb0, 0x3a1b6e40
Caught exception in looking up space point ptr... 0x3a1b6e40, 0x3a1b6cb0
Caught exception in looking up space point ptr... 0xddb2010, 0xddb0d10
Caught exception in looking up space point ptr... 0xddb0d10, 0xddb2010
++++>>>> total num hits: 89992, num free: 86688
04-Feb-2026 01:30:12 GMT  Closed output file "np02vd_raw_run042433_0622_df-s04-d0_dw_0_20260203T223832_reco_stage0_20260204T013012_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         235.276       264.691       330.866       256.967       23.6157        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.0001115    0.000155152   0.000286558   0.000133187   5.50221e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                16.3816       17.192        18.7249       17.1425      0.562858        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.520214      0.557668      0.683591      0.546827      0.0367192       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177451     0.0211622     0.0411649     0.0180686    0.00719594       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0176317     0.0179417     0.0184343     0.0179222    0.000150503      19     
produce:beamevent:BeamEvent                      0.000136397   0.000325524   0.00139729    0.000262032   0.00026308       19     
produce:pdvddaphne:DAPHNEReaderPDVD                6.99297       8.0373        9.12788       8.09872      0.518405        19     
produce:ophit:OpHitFinder                         0.0443653     0.0723169     0.0896087     0.0746593     0.0105953       19     
produce:wclsdatavd:WireCellToolkit                 171.495       195.974       250.953       191.841       19.7336        19     
produce:gaushit:GausHitFinder                      1.49087       2.39791       3.23382       2.51858      0.527569        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000479766   0.000727923   0.00120469    0.000740399   0.000163801      19     
produce:reco3d:SpacePointSolver                     14.51        25.0884       43.0366       23.3938       6.53233        19     
produce:hitpdune:DisambigFromSpacePoints          0.222824      0.368968      0.880701      0.322789      0.142319        19     
produce:cluster3d:Cluster3D                        2.19696       3.80605       6.60999       3.69187       1.19336        19     
[art]:TriggerResults:TriggerResultInserter       3.0366e-05    4.83784e-05   0.000157094   3.8737e-05    2.7074e-05       19     
end_path:out1:RootOutput                          6.902e-06    1.12193e-05   3.6366e-05     9.834e-06    6.27278e-06      19     
end_path:out1:RootOutput(write)                    7.06362       11.0792       28.4254       10.1733       4.51912        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5352.04 MB
  Peak resident set size usage (VmHWM): 3384.31 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0622_df-s04-d0_dw_0_20260203T223832_reco_stage0_20260204T013012_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0622_df-s04-d0_dw_0_20260203T223832_reco_stage0_20260204T013012_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0622_df-s04-d0_dw_0_20260203T223832_reco_stage0_20260204T013012_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163424.0,
    "core.end_time": 1770168620.0,
    "core.events": [
      70918,
      70924,
      70930,
      70936,
      70942,
      70948,
      70954,
      70960,
      70966,
      70972,
      70978,
      70984,
      70990,
      70996,
      71002,
      71008,
      71014,
      71020,
      71026
    ],
    "core.event_count": 19,
    "core.first_event_number": 70918,
    "core.last_event_number": 71026,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0622_df-s04-d0_dw_0_20260203T223832.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:50:07 UTC       justIN version: 01.06.00