justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.57@dunegpschedd02.fnal.gov

Jobsub ID297923.57@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-02-04 02:59:00
From worker nodeHostnamewn1907251.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:59:08
Input filesvd-protodune:np02vd_raw_run042433_0433_df-s04-d1_dw_0_20260203T212641.hdf5
JobscriptExit code0
Real time2h (10629s)
CPU time1h (5425s = 51%)
Max RSS bytes3283857408 (3131 MiB)
Outputting started2026-02-04 02:56:19
Output files
Finished2026-02-04 02:59:00
Saved logsjustin-logs:297923.57-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

p space point ptr... 0x34686f10, 0xe955020
Caught exception in looking up space point ptr... 0xe955020, 0x89da9e0
Caught exception in looking up space point ptr... 0x89da9e0, 0x31951040
Caught exception in looking up space point ptr... 0x8b282ea0, 0x36cd8590
Caught exception in looking up space point ptr... 0x36cd8590, 0x7eaaf060
Caught exception in looking up space point ptr... 0x1a2336b0, 0x1a230ad0
Caught exception in looking up space point ptr... 0x1a230ad0, 0x1a2336b0
Caught exception in looking up space point ptr... 0x3adc0d50, 0x3adc09e0
Caught exception in looking up space point ptr... 0x3adc09e0, 0x3adc1070
Caught exception in looking up space point ptr... 0x5c6c2710, 0x5c6c2a80
Caught exception in looking up space point ptr... 0x5c6c2a80, 0x5c6c28f0
Caught exception in looking up space point ptr... 0x5c6c28f0, 0x5c6c2530
Caught exception in looking up space point ptr... 0x5c6c2530, 0x5c6c2c10
Caught exception in looking up space point ptr... 0x60094980, 0x2f621070
Caught exception in looking up space point ptr... 0x2f621070, 0x60094980
Caught exception in looking up space point ptr... 0x1520d850, 0x1520bca0
Caught exception in looking up space point ptr... 0x1520bca0, 0x15204ae0
Caught exception in looking up space point ptr... 0x628e36c0, 0x628e3360
Caught exception in looking up space point ptr... 0x628e3b60, 0x628e36c0
Caught exception in looking up space point ptr... 0x5b0d00d0, 0x16f3a1e0
Caught exception in looking up space point ptr... 0x16f3a1e0, 0xef1f150
Caught exception in looking up space point ptr... 0x7c2e27e0, 0x7c2e28c0
Caught exception in looking up space point ptr... 0x38b350f0, 0x7c2e27e0
Caught exception in looking up space point ptr... 0x186b6df0, 0x341da3b0
Caught exception in looking up space point ptr... 0x341da3b0, 0x186b6df0
Caught exception in looking up space point ptr... 0x126007d0, 0x12602920
Caught exception in looking up space point ptr... 0x12605170, 0x126007d0
Caught exception in looking up space point ptr... 0x61bad950, 0x61bad4a0
Caught exception in looking up space point ptr... 0x61bad4a0, 0x61bad950
Caught exception in looking up space point ptr... 0x40325410, 0x403252c0
Caught exception in looking up space point ptr... 0x403252c0, 0x40325410
Caught exception in looking up space point ptr... 0x34848540, 0x18a7cb20
Caught exception in looking up space point ptr... 0x18a7cb20, 0x18a7e8a0
Caught exception in looking up space point ptr... 0x61bad310, 0x61bace60
Caught exception in looking up space point ptr... 0x61bace60, 0x61bad310
Caught exception in looking up space point ptr... 0x3e066c00, 0x3e066ea0
Caught exception in looking up space point ptr... 0x3e066ea0, 0x3e066d50
Caught exception in looking up space point ptr... 0x97c0070, 0x97bf090
Caught exception in looking up space point ptr... 0x97bf090, 0x1a4dc450
Caught exception in looking up space point ptr... 0x72aff130, 0x2ee42940
Caught exception in looking up space point ptr... 0x2ee42940, 0x72b09830
Caught exception in looking up space point ptr... 0x47617760, 0x6cbadf70
Caught exception in looking up space point ptr... 0x6cbadf70, 0x47613110
Caught exception in looking up space point ptr... 0x7c7c5380, 0x7c7c5110
Caught exception in looking up space point ptr... 0x7c7c5110, 0x7c7c0740
Caught exception in looking up space point ptr... 0x6a37d080, 0x727d4560
Caught exception in looking up space point ptr... 0x727d4560, 0x6a37d080
Caught exception in looking up space point ptr... 0x571b2ba0, 0x571b2a40
Caught exception in looking up space point ptr... 0x571b2a40, 0x571b2ba0
Caught exception in looking up space point ptr... 0x118cfcc0, 0x118cfa20
Caught exception in looking up space point ptr... 0x118cfa20, 0x118cfcc0
Caught exception in looking up space point ptr... 0xf357d00, 0x37588910
Caught exception in looking up space point ptr... 0x37588910, 0xf357d00
Caught exception in looking up space point ptr... 0x760557b0, 0x76058d60
Caught exception in looking up space point ptr... 0x76058d60, 0x760557b0
Caught exception in looking up space point ptr... 0x52128350, 0x2979adb0
Caught exception in looking up space point ptr... 0x2979adb0, 0x52128350
Caught exception in looking up space point ptr... 0xdebaa90, 0x61f26d70
Caught exception in looking up space point ptr... 0x61f26d70, 0x61f27c90
Caught exception in looking up space point ptr... 0xde8f890, 0xde8f600
Caught exception in looking up space point ptr... 0xde8f600, 0xde8fb20
Caught exception in looking up space point ptr... 0x7c6359f0, 0x12ff7b50
Caught exception in looking up space point ptr... 0x19cb4410, 0x7c6359f0
Caught exception in looking up space point ptr... 0x6be201b0, 0x6be1fe40
Caught exception in looking up space point ptr... 0x6be1fe40, 0x6be201b0
Caught exception in looking up space point ptr... 0xb28ecd0, 0xb28e9e0
Caught exception in looking up space point ptr... 0xb28e9e0, 0xb28ecd0
++++>>>> total num hits: 84765, num free: 80768
04-Feb-2026 02:56:00 GMT  Closed output file "np02vd_raw_run042433_0433_df-s04-d1_dw_0_20260203T212641_reco_stage0_20260204T025600_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         446.959       549.291       675.121       548.87        52.0487        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000113063    0.0001684    0.00031712    0.000127922   7.14522e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                170.125       254.577       367.945       256.698       44.0523        19     
produce:triggerrawdecoder:PDVDTriggerReader4       0.97847       1.24571       1.83272       1.18507       0.22801        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0221247     0.0327259     0.0911528     0.0274214     0.0170291       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0222024     0.0240524     0.0527268     0.0224489    0.00676037       19     
produce:beamevent:BeamEvent                      0.000187675   0.000289881    0.0014109    0.000223987   0.000265548      19     
produce:pdvddaphne:DAPHNEReaderPDVD                16.9569       24.869        30.8028        24.48        3.38982        19     
produce:ophit:OpHitFinder                         0.0447615     0.0690188     0.0794129     0.072981      0.0105359       19     
produce:wclsdatavd:WireCellToolkit                 197.804       228.846       272.19        228.087       19.4072        19     
produce:gaushit:GausHitFinder                      1.59201       2.5153        3.16601       2.58313      0.394768        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000465815   0.000714399   0.00111584    0.000701565    0.0001442       19     
produce:reco3d:SpacePointSolver                    15.0065       22.7367       34.1943       22.0746       4.00218        19     
produce:hitpdune:DisambigFromSpacePoints          0.178824      0.429866      0.593097      0.443678      0.106919        19     
produce:cluster3d:Cluster3D                        1.54797       4.24237       5.94226       4.18627      0.988402        19     
[art]:TriggerResults:TriggerResultInserter       3.7059e-05    5.68351e-05   0.000144225   5.2845e-05    2.30621e-05      19     
end_path:out1:RootOutput                          6.581e-06    1.40192e-05   3.9659e-05    1.3288e-05    6.77178e-06      19     
end_path:out1:RootOutput(write)                    7.31439       9.62448       11.2878       9.81269      0.994054        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5328.19 MB
  Peak resident set size usage (VmHWM): 3283.86 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0433_df-s04-d1_dw_0_20260203T212641_reco_stage0_20260204T025600_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0433_df-s04-d1_dw_0_20260203T212641_reco_stage0_20260204T025600_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0433_df-s04-d1_dw_0_20260203T212641_reco_stage0_20260204T025600_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163171.0,
    "core.end_time": 1770173769.0,
    "core.events": [
      49373,
      49379,
      49385,
      49391,
      49397,
      49403,
      49409,
      49415,
      49421,
      49427,
      49433,
      49439,
      49445,
      49451,
      49457,
      49463,
      49469,
      49475,
      49481
    ],
    "core.event_count": 19,
    "core.first_event_number": 49373,
    "core.last_event_number": 49481,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0433_df-s04-d1_dw_0_20260203T212641.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:48:30 UTC       justIN version: 01.06.00