justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.144@dunegpschedd02.fnal.gov

Jobsub ID297923.144@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2026-02-04 02:09:10
From worker nodeHostnamet2wn175.physics.ox.ac.uk
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:07:15
Input filesvd-protodune:np02vd_raw_run042433_0094_df-s03-d1_dw_0_20260203T191751.hdf5
JobscriptExit code0
Real time1h (7146s)
CPU time41m (2470s = 34%)
Max RSS bytes3202707456 (3054 MiB)
Outputting started2026-02-04 02:06:22
Output files
Finished2026-02-04 02:09:10
Saved logsjustin-logs:297923.144-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t ptr... 0x520b4a60, 0x520b4dd0
Caught exception in looking up space point ptr... 0x41b5b810, 0x41582590
Caught exception in looking up space point ptr... 0x41582590, 0x41b5b810
Caught exception in looking up space point ptr... 0x5ec01450, 0x5ebf4500
Caught exception in looking up space point ptr... 0x5ebf4500, 0x5ebfa2b0
Caught exception in looking up space point ptr... 0x55a7f240, 0x3c2282b0
Caught exception in looking up space point ptr... 0x3c2282b0, 0x55a7f240
Caught exception in looking up space point ptr... 0x7ad9de80, 0x19934a60
Caught exception in looking up space point ptr... 0x19934a60, 0x7ad9de80
Caught exception in looking up space point ptr... 0x4f2b7e10, 0x4f2b7010
Caught exception in looking up space point ptr... 0x4f2b7010, 0x4f2b7e10
Caught exception in looking up space point ptr... 0x357aef50, 0x35797fc0
Caught exception in looking up space point ptr... 0x35797fc0, 0x357aef50
Caught exception in looking up space point ptr... 0x19686010, 0x19685ec0
Caught exception in looking up space point ptr... 0x19685ec0, 0x19686010
Caught exception in looking up space point ptr... 0x1965b5f0, 0x3e696530
Caught exception in looking up space point ptr... 0x3e696530, 0x1965b5f0
Caught exception in looking up space point ptr... 0x7e637b0, 0x7e63620
Caught exception in looking up space point ptr... 0x7e63620, 0x7e637b0
Caught exception in looking up space point ptr... 0x6ec097c0, 0x6752bd50
Caught exception in looking up space point ptr... 0x6752bd50, 0x6ec097c0
Caught exception in looking up space point ptr... 0x72a3f8b0, 0x49c2f770
Caught exception in looking up space point ptr... 0x49c2f770, 0x72a3f8b0
Caught exception in looking up space point ptr... 0x3b437d70, 0x49ebc180
Caught exception in looking up space point ptr... 0x49ebc180, 0x3ca9c580
Caught exception in looking up space point ptr... 0x48c7fb10, 0x39eee420
Caught exception in looking up space point ptr... 0x39eee420, 0x48c7fb10
Caught exception in looking up space point ptr... 0x4dcf5f00, 0x7bb46b70
Caught exception in looking up space point ptr... 0x7bb46b70, 0x4dcf5f00
Caught exception in looking up space point ptr... 0x13172f60, 0x13172e10
Caught exception in looking up space point ptr... 0x13172e10, 0x13172f60
Caught exception in looking up space point ptr... 0x12786bb0, 0x3dc0a3f0
Caught exception in looking up space point ptr... 0x3dc0a3f0, 0x12786bb0
Caught exception in looking up space point ptr... 0x38688ec0, 0x3b498cd0
Caught exception in looking up space point ptr... 0x3b498cd0, 0x2feb37e0
Caught exception in looking up space point ptr... 0x177ed5f0, 0x4e37e770
Caught exception in looking up space point ptr... 0x4e37e770, 0x177ed5f0
Caught exception in looking up space point ptr... 0x177ed5f0, 0x3ea2dff0
Caught exception in looking up space point ptr... 0x3ea2dff0, 0x177ed5f0
Caught exception in looking up space point ptr... 0x61a2d070, 0x16a20060
Caught exception in looking up space point ptr... 0x16a20060, 0x61a2d070
Caught exception in looking up space point ptr... 0x61a2d070, 0x3b40cc80
Caught exception in looking up space point ptr... 0x3b40cc80, 0x61a2d070
Caught exception in looking up space point ptr... 0x64df3c60, 0x64df3a80
Caught exception in looking up space point ptr... 0x64df3a80, 0x64df3df0
Caught exception in looking up space point ptr... 0x7bb79d40, 0x7bb79bb0
Caught exception in looking up space point ptr... 0x7bb79bb0, 0x7bb79d40
Caught exception in looking up space point ptr... 0x39391770, 0x39391900
Caught exception in looking up space point ptr... 0x39391900, 0x7929e610
Caught exception in looking up space point ptr... 0x43599360, 0x469f65d0
Caught exception in looking up space point ptr... 0x3971f0b0, 0x43599360
Caught exception in looking up space point ptr... 0x4f2b7600, 0x4f2b73a0
Caught exception in looking up space point ptr... 0x4f2b73a0, 0x4f2b7600
Caught exception in looking up space point ptr... 0x10df8600, 0x4a988210
Caught exception in looking up space point ptr... 0x4a988210, 0x10df8600
Caught exception in looking up space point ptr... 0x64df4430, 0x64df45c0
Caught exception in looking up space point ptr... 0x64df45c0, 0x64df42a0
Caught exception in looking up space point ptr... 0x19562c40, 0x19562ac0
Caught exception in looking up space point ptr... 0x19562ac0, 0x5ebf9380
Caught exception in looking up space point ptr... 0x5ebfc160, 0x5ebfc290
Caught exception in looking up space point ptr... 0x5ebfc290, 0x19562c40
Caught exception in looking up space point ptr... 0x3b2e0be0, 0x4e37fe30
Caught exception in looking up space point ptr... 0x4e37fe30, 0x3b2dfd80
Caught exception in looking up space point ptr... 0x3c797b50, 0xfe1b300
Caught exception in looking up space point ptr... 0xfe1b300, 0xfe1b440
Caught exception in looking up space point ptr... 0x18c480c0, 0x61ee4c90
Caught exception in looking up space point ptr... 0x61ee4c90, 0x18c480c0
++++>>>> total num hits: 90950, num free: 87460
04-Feb-2026 02:06:14 GMT  Closed output file "np02vd_raw_run042433_0094_df-s03-d1_dw_0_20260203T191751_reco_stage0_20260204T020614_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         297.283       372.334       452.915       369.183       45.2875        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.1709e-05    0.000110282   0.000259804   9.1388e-05    4.30728e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                158.472       231.574       323.032       226.774       42.9712        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.692669       1.03142       1.38808      0.958777      0.226063        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0196086     0.0244104     0.0362056     0.023353     0.00432665       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0192198     0.0203792     0.0267496     0.0196888    0.00197871       19     
produce:beamevent:BeamEvent                      0.000108785   0.000195103   0.00100773    0.000151079   0.000192581      19     
produce:pdvddaphne:DAPHNEReaderPDVD                12.2899       18.814        25.2784       18.3173       3.8822         19     
produce:ophit:OpHitFinder                         0.024697      0.036448       0.04153      0.0376044    0.00415382       19     
produce:wclsdatavd:WireCellToolkit                 83.9084       96.0451       105.06        95.415        5.50681        19     
produce:gaushit:GausHitFinder                     0.878279       1.2873        1.79738       1.24139      0.254915        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000208066   0.00029172    0.000479728   0.000294577   6.58514e-05      19     
produce:reco3d:SpacePointSolver                    9.27212       15.7484       25.1692       14.2644       4.60408        19     
produce:hitpdune:DisambigFromSpacePoints          0.0804522     0.194744      0.281049      0.219097      0.0541381       19     
produce:cluster3d:Cluster3D                        1.10245       2.55263       5.14641       2.47575       1.03126        19     
[art]:TriggerResults:TriggerResultInserter       2.9034e-05    4.21606e-05   0.000117328   3.7807e-05    1.8405e-05       19     
end_path:out1:RootOutput                          5.188e-06    1.06657e-05   3.9961e-05     8.483e-06    7.69333e-06      19     
end_path:out1:RootOutput(write)                    4.27275       4.94976       5.42199       5.0385       0.329691        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5412.37 MB
  Peak resident set size usage (VmHWM): 3202.71 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0094_df-s03-d1_dw_0_20260203T191751_reco_stage0_20260204T020614_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0094_df-s03-d1_dw_0_20260203T191751_reco_stage0_20260204T020614_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0094_df-s03-d1_dw_0_20260203T191751_reco_stage0_20260204T020614_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163646.0,
    "core.end_time": 1770170776.0,
    "core.events": [
      10725,
      10731,
      10737,
      10743,
      10749,
      10755,
      10761,
      10767,
      10773,
      10779,
      10785,
      10791,
      10797,
      10803,
      10809,
      10815,
      10821,
      10827,
      10833
    ],
    "core.event_count": 19,
    "core.first_event_number": 10725,
    "core.last_event_number": 10833,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0094_df-s03-d1_dw_0_20260203T191751.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:50:03 UTC       justIN version: 01.06.00