justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.26@dunegpschedd01.fnal.gov

Jobsub ID303656.26@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:29:16
From worker nodeHostnameheplnc148.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:11:05
Input filesvd-protodune:np02vd_raw_run042421_0049_df-s04-d1_dw_0_20260203T135732.hdf5
JobscriptExit code0
Real time1h (4523s)
CPU time1h (4062s = 89%)
Max RSS bytes2711924736 (2586 MiB)
Outputting started2026-02-04 01:26:31
Output files
Finished2026-02-04 01:29:16
Saved logsjustin-logs:303656.26-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ace point ptr... 0x128ec430, 0x128ec330
Caught exception in looking up space point ptr... 0x128ec330, 0x128ec230
Caught exception in looking up space point ptr... 0x655a0ad0, 0x5632db50
Caught exception in looking up space point ptr... 0x5632db50, 0x369ea930
Caught exception in looking up space point ptr... 0x3c4b7840, 0x3c4b7a40
Caught exception in looking up space point ptr... 0x3c4b7a40, 0x3c4b7840
Caught exception in looking up space point ptr... 0x4a6474f0, 0x2716e130
Caught exception in looking up space point ptr... 0x2716e130, 0x4099b900
Caught exception in looking up space point ptr... 0x367d0a00, 0x77dbd30
Caught exception in looking up space point ptr... 0x77dbd30, 0x367d0a00
Caught exception in looking up space point ptr... 0x2fe81850, 0x16fc5500
Caught exception in looking up space point ptr... 0x16fc5500, 0x2fe81850
Caught exception in looking up space point ptr... 0x184a2d20, 0x184a2c20
Caught exception in looking up space point ptr... 0x184a2c20, 0x184a2d20
Caught exception in looking up space point ptr... 0x9617f80, 0x4080e7a0
Caught exception in looking up space point ptr... 0x16bd3550, 0x9617f80
Caught exception in looking up space point ptr... 0x2fb3b470, 0x2fb3b600
Caught exception in looking up space point ptr... 0x2fb3b600, 0x2fb3b470
Caught exception in looking up space point ptr... 0xe0fcc80, 0x13116840
Caught exception in looking up space point ptr... 0x13116840, 0x131166b0
Caught exception in looking up space point ptr... 0x155301d0, 0x508d8e50
Caught exception in looking up space point ptr... 0x508d8e50, 0x36ab7330
Caught exception in looking up space point ptr... 0x65607cf0, 0x65607bc0
Caught exception in looking up space point ptr... 0x65607bc0, 0x65607cf0
Caught exception in looking up space point ptr... 0xaa4fdd0, 0xaa4ff60
Caught exception in looking up space point ptr... 0xaa4ff60, 0x65cffb10
Caught exception in looking up space point ptr... 0x41159a40, 0x41158ad0
Caught exception in looking up space point ptr... 0x48444a60, 0x41159a40
Caught exception in looking up space point ptr... 0x1867f3a0, 0x5a3016d0
Caught exception in looking up space point ptr... 0x5a3016d0, 0x1867f3a0
Caught exception in looking up space point ptr... 0x47514920, 0xd3215c0
Caught exception in looking up space point ptr... 0xd3215c0, 0xaa4f470
Caught exception in looking up space point ptr... 0x42f53f30, 0x1897f910
Caught exception in looking up space point ptr... 0x1897f910, 0x2d88e140
Caught exception in looking up space point ptr... 0x643df440, 0x643df150
Caught exception in looking up space point ptr... 0x643df150, 0x643df440
Caught exception in looking up space point ptr... 0x643df440, 0x643df2f0
Caught exception in looking up space point ptr... 0x643df2f0, 0x643df440
Caught exception in looking up space point ptr... 0x2bfec140, 0x13324740
Caught exception in looking up space point ptr... 0x13324740, 0x2bfec140
Caught exception in looking up space point ptr... 0x29da49b0, 0x30b9d5f0
Caught exception in looking up space point ptr... 0x30b9d5f0, 0x29da49b0
Caught exception in looking up space point ptr... 0x655244c0, 0x655242e0
Caught exception in looking up space point ptr... 0x655242e0, 0x655244c0
Caught exception in looking up space point ptr... 0x5b1284c0, 0x3d147e10
Caught exception in looking up space point ptr... 0x3d147e10, 0x5b1284c0
Caught exception in looking up space point ptr... 0x5a64b490, 0x65598810
Caught exception in looking up space point ptr... 0x146b6cc0, 0x5a64b490
Caught exception in looking up space point ptr... 0x2bba2140, 0x2bba3320
Caught exception in looking up space point ptr... 0x2bba3320, 0x2bba3660
Caught exception in looking up space point ptr... 0x55fc1030, 0x1ad0da30
Caught exception in looking up space point ptr... 0x1ad0da30, 0x35f50d90
Caught exception in looking up space point ptr... 0x2e2bb800, 0x2e2bbf20
Caught exception in looking up space point ptr... 0x2e2bbf20, 0x2e2bb800
Caught exception in looking up space point ptr... 0xb7e1a00, 0x2a540d10
Caught exception in looking up space point ptr... 0x2a540d10, 0xb7e0f20
Caught exception in looking up space point ptr... 0x2c8b2220, 0x2c8b23b0
Caught exception in looking up space point ptr... 0x2c8b23b0, 0x2c8b2220
Caught exception in looking up space point ptr... 0x2d88cf50, 0x2ff35a30
Caught exception in looking up space point ptr... 0x2ff35a30, 0x2d88cf50
Caught exception in looking up space point ptr... 0xd1c7ac0, 0xd0e7340
Caught exception in looking up space point ptr... 0xd0e7340, 0xd1c7ac0
Caught exception in looking up space point ptr... 0x160d4080, 0x6252d090
Caught exception in looking up space point ptr... 0x6252d090, 0x160d4080
Caught exception in looking up space point ptr... 0x4b8a5240, 0x15846760
Caught exception in looking up space point ptr... 0x15846760, 0x2a43b980
++++>>>> total num hits: 44869, num free: 42739
04-Feb-2026 01:26:20 GMT  Closed output file "np02vd_raw_run042421_0049_df-s04-d1_dw_0_20260203T135732_reco_stage0_20260204T012620_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         110.751       163.429       198.988       172.678       25.2538        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        9.556e-05    0.000172357   0.000464456   0.000154011   7.78956e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                12.595        13.1329       14.0591       13.0971      0.326464        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.550867       0.60937      0.694408       0.59349      0.0428344       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0178365     0.0204334     0.0328337     0.018362     0.00451449       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0874709     0.0916425      0.10662      0.0889339    0.00605754       27     
produce:beamevent:BeamEvent                      0.000129154   0.000276294   0.00170527    0.000204806   0.000289548      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.37093       3.91512       4.58446       3.93045      0.252725        27     
produce:ophit:OpHitFinder                         0.0315371     0.0435935     0.0532108     0.0442739    0.00514245       27     
produce:wclsdatavd:WireCellToolkit                 72.9758       88.1779       99.8287       88.7044       7.48008        27     
produce:gaushit:GausHitFinder                     0.698202       1.72263       2.83547       1.8252        0.45717        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000298263   0.000481581   0.000965381   0.000458114   0.000117343      27     
produce:reco3d:SpacePointSolver                    7.30869       13.0371       29.5815       11.9827       4.5886         27     
produce:hitpdune:DisambigFromSpacePoints          0.0844896     0.247784      0.366154      0.263713      0.0845423       27     
produce:cluster3d:Cluster3D                        1.45401       36.6542       63.5487       49.8923       24.4706        27     
[art]:TriggerResults:TriggerResultInserter       2.7252e-05    5.80584e-05   0.000174149    4.788e-05    3.29714e-05      27     
end_path:out1:RootOutput                          4.559e-06    2.09253e-05   0.000240764    9.568e-06    4.40715e-05      27     
end_path:out1:RootOutput(write)                    3.5354        5.7289        7.01993       5.91724      0.973922        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4694.36 MB
  Peak resident set size usage (VmHWM): 2711.92 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0049_df-s04-d1_dw_0_20260203T135732_reco_stage0_20260204T012620_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0049_df-s04-d1_dw_0_20260203T135732_reco_stage0_20260204T012620_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0049_df-s04-d1_dw_0_20260203T135732_reco_stage0_20260204T012620_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163880.0,
    "core.end_time": 1770168382.0,
    "core.events": [
      7991,
      7997,
      8003,
      8009,
      8015,
      8021,
      8027,
      8033,
      8039,
      8045,
      8051,
      8057,
      8063,
      8069,
      8075,
      8081,
      8087,
      8093,
      8099,
      8105,
      8111,
      8117,
      8123,
      8129,
      8135,
      8141,
      8147
    ],
    "core.event_count": 27,
    "core.first_event_number": 7991,
    "core.last_event_number": 8147,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0049_df-s04-d1_dw_0_20260203T135732.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:06:50 UTC       justIN version: 01.06.00