justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297968.1@dunegpschedd02.fnal.gov

Jobsub ID297968.1@dunegpschedd02.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-04 00:36:28
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-02-04 03:00:34
From worker nodeHostnamewn1907241.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:36:49
Input filesvd-protodune:np02vd_raw_run042421_0110_df-s02-d1_dw_0_20260203T143029.hdf5
JobscriptExit code0
Real time2h (8464s)
CPU time1h (3885s = 45%)
Max RSS bytes2550296576 (2432 MiB)
Outputting started2026-02-04 02:57:54
Output files
Finished2026-02-04 03:00:34
Saved logsjustin-logs:297968.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ught exception in looking up space point ptr... 0x49229010, 0x4e243d30
Caught exception in looking up space point ptr... 0x4e243d30, 0x4e243b50
Caught exception in looking up space point ptr... 0x4e243b50, 0x638493f0
Caught exception in looking up space point ptr... 0x638493f0, 0x2188f520
Caught exception in looking up space point ptr... 0x2188f520, 0xdd598f0
Caught exception in looking up space point ptr... 0x62cae660, 0x4e1371e0
Caught exception in looking up space point ptr... 0x4e1371e0, 0x62cae660
Caught exception in looking up space point ptr... 0x224b1e70, 0x29865be0
Caught exception in looking up space point ptr... 0x29865be0, 0xf1ed820
Caught exception in looking up space point ptr... 0x20af2cc0, 0x36665bf0
Caught exception in looking up space point ptr... 0x36665bf0, 0x20af2cc0
Caught exception in looking up space point ptr... 0x4eb8b440, 0x4eb8b2b0
Caught exception in looking up space point ptr... 0x4eb8b2b0, 0x4eb8b440
Caught exception in looking up space point ptr... 0x3fd5a0c0, 0x3da1d310
Caught exception in looking up space point ptr... 0x3da1d310, 0x3bcf3290
Caught exception in looking up space point ptr... 0x499ef8a0, 0x499ef3f0
Caught exception in looking up space point ptr... 0x499ef3f0, 0x499ef8a0
Caught exception in looking up space point ptr... 0x114658b0, 0x39751a30
Caught exception in looking up space point ptr... 0x39751a30, 0x114658b0
Caught exception in looking up space point ptr... 0x114658b0, 0x24d16150
Caught exception in looking up space point ptr... 0x24d16150, 0x114658b0
Caught exception in looking up space point ptr... 0x2d797ad0, 0x2d797940
Caught exception in looking up space point ptr... 0x2d797940, 0x647428f0
Caught exception in looking up space point ptr... 0x568b3b00, 0x3772e520
Caught exception in looking up space point ptr... 0x3772e520, 0x3772e420
Caught exception in looking up space point ptr... 0x55d59a60, 0x42796940
Caught exception in looking up space point ptr... 0x42796940, 0x55d59a60
Caught exception in looking up space point ptr... 0x4c8228e0, 0x514fb280
Caught exception in looking up space point ptr... 0x514fb280, 0x4c8228e0
Caught exception in looking up space point ptr... 0x4c55c9c0, 0x29b24eb0
Caught exception in looking up space point ptr... 0x29b24eb0, 0x3a3d5720
Caught exception in looking up space point ptr... 0x3a3d5720, 0x4c55c7b0
Caught exception in looking up space point ptr... 0x4c55c7b0, 0x4c55c9c0
Caught exception in looking up space point ptr... 0x41f50e30, 0x5e393ab0
Caught exception in looking up space point ptr... 0x5e393ab0, 0x4e8f6f90
Caught exception in looking up space point ptr... 0x4c9e0e50, 0x55eff250
Caught exception in looking up space point ptr... 0x55eff250, 0x4c9e0e50
Caught exception in looking up space point ptr... 0x6272ae70, 0x6272abd0
Caught exception in looking up space point ptr... 0x6272abd0, 0x6272ae70
Caught exception in looking up space point ptr... 0x11699c00, 0x3a502e40
Caught exception in looking up space point ptr... 0x3a502e40, 0xe466840
Caught exception in looking up space point ptr... 0x3faa9810, 0x3f93cdb0
Caught exception in looking up space point ptr... 0x3f93cdb0, 0x3f93cf40
Caught exception in looking up space point ptr... 0x3322d560, 0x3322d6b0
Caught exception in looking up space point ptr... 0x3322d6b0, 0x3322e5a0
Caught exception in looking up space point ptr... 0x146f07c0, 0x11459ab0
Caught exception in looking up space point ptr... 0x11459ab0, 0x146f07c0
Caught exception in looking up space point ptr... 0x39784fa0, 0x39784e40
Caught exception in looking up space point ptr... 0x39784e40, 0x3977f0f0
Caught exception in looking up space point ptr... 0x5e9abdc0, 0x5e9ae480
Caught exception in looking up space point ptr... 0x5e9ae480, 0x5e9abdc0
Caught exception in looking up space point ptr... 0x35e83100, 0x35e87050
Caught exception in looking up space point ptr... 0x35e87050, 0x35e825d0
Caught exception in looking up space point ptr... 0x36141440, 0x15646840
Caught exception in looking up space point ptr... 0x15646840, 0x36141440
Caught exception in looking up space point ptr... 0x3cf8fbf0, 0x4de05300
Caught exception in looking up space point ptr... 0x4de05300, 0x3342c570
Caught exception in looking up space point ptr... 0x4f0969c0, 0x5e9a63a0
Caught exception in looking up space point ptr... 0x5e9a63a0, 0x4f0969c0
Caught exception in looking up space point ptr... 0x1499f5f0, 0x29627760
Caught exception in looking up space point ptr... 0x29627760, 0x1595b700
Caught exception in looking up space point ptr... 0x62aaabb0, 0x5873a400
Caught exception in looking up space point ptr... 0x5873a400, 0x28f7f9c0
Caught exception in looking up space point ptr... 0x49b18630, 0x428b1130
Caught exception in looking up space point ptr... 0x428b1130, 0x49b18630
++++>>>> total num hits: 42094, num free: 40354
04-Feb-2026 02:57:20 GMT  Closed output file "np02vd_raw_run042421_0110_df-s02-d1_dw_0_20260203T143029_reco_stage0_20260204T025720_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         259.405       306.97        379.955       307.948       25.7428        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.1916e-05    0.000152322   0.000358934   0.000110507   7.37081e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                116.633       153.811       199.854       157.006       19.593         27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.754114      0.992894       1.75871      0.944199      0.223913        27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0220232     0.0299314     0.0693522     0.0251696     0.011862        27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.108725      0.117579      0.153296      0.109764      0.0121803       27     
produce:beamevent:BeamEvent                      0.000116676   0.000254337   0.00144649    0.000206902   0.000241548      27     
produce:pdvddaphne:DAPHNEReaderPDVD                17.6015       22.2077       26.7183       22.2138       2.34122        27     
produce:ophit:OpHitFinder                         0.0354494     0.062238       0.07942      0.0646951     0.0126434       27     
produce:wclsdatavd:WireCellToolkit                 88.552        106.61        129.433       104.966       10.1497        27     
produce:gaushit:GausHitFinder                     0.871768       1.53279       2.25564       1.53514      0.344526        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000307174   0.000507294   0.000855425   0.000507747   0.000120096      27     
produce:reco3d:SpacePointSolver                    6.08187       13.0329       20.1829       12.885        3.13197        27     
produce:hitpdune:DisambigFromSpacePoints          0.073378      0.162532      0.384581      0.153816      0.0710592       27     
produce:cluster3d:Cluster3D                       0.916904       2.69869        10.19        2.3445        1.71969        27     
[art]:TriggerResults:TriggerResultInserter       2.8567e-05    4.1323e-05    0.000128963   3.7683e-05    1.86527e-05      27     
end_path:out1:RootOutput                          4.467e-06    1.14714e-05   3.7517e-05     9.818e-06    6.38784e-06      27     
end_path:out1:RootOutput(write)                    4.15631       5.67628       7.0643        5.64425       0.78061        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4518.62 MB
  Peak resident set size usage (VmHWM): 2550.3 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0110_df-s02-d1_dw_0_20260203T143029_reco_stage0_20260204T025720_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0110_df-s02-d1_dw_0_20260203T143029_reco_stage0_20260204T025720_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0110_df-s02-d1_dw_0_20260203T143029_reco_stage0_20260204T025720_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770165425.0,
    "core.end_time": 1770173867.0,
    "core.events": [
      17881,
      17887,
      17893,
      17899,
      17905,
      17911,
      17917,
      17923,
      17929,
      17935,
      17941,
      17947,
      17953,
      17959,
      17965,
      17971,
      17977,
      17983,
      17989,
      17995,
      18001,
      18007,
      18013,
      18019,
      18025,
      18031,
      18037
    ],
    "core.event_count": 27,
    "core.first_event_number": 17881,
    "core.last_event_number": 18037,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0110_df-s02-d1_dw_0_20260203T143029.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:17:57 UTC       justIN version: 01.06.00