justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.69@dunegpschedd02.fnal.gov

Jobsub ID297923.69@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:05:17
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:59:44
Input filesvd-protodune:np02vd_raw_run042433_0524_df-s03-d1_dw_0_20260203T220116.hdf5
JobscriptExit code0
Real time1h (3783s)
CPU time57m (3441s = 90%)
Max RSS bytes3383906304 (3227 MiB)
Outputting started2026-02-04 01:02:49
Output files
Finished2026-02-04 01:05:17
Saved logsjustin-logs:297923.69-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t ptr... 0x81e22f40, 0x81e22e40
Caught exception in looking up space point ptr... 0x4de85b50, 0x4de85830
Caught exception in looking up space point ptr... 0x4de85830, 0x4de85b50
Caught exception in looking up space point ptr... 0x55d484f0, 0x55d47e40
Caught exception in looking up space point ptr... 0x55d47e40, 0x55d484f0
Caught exception in looking up space point ptr... 0x880bc40, 0x880cb10
Caught exception in looking up space point ptr... 0x880cb10, 0x880bc40
Caught exception in looking up space point ptr... 0x4e132c70, 0x64f2aa90
Caught exception in looking up space point ptr... 0x64f2aa90, 0x4e132e50
Caught exception in looking up space point ptr... 0x67ca2070, 0x3b7c5de0
Caught exception in looking up space point ptr... 0x3b7c5de0, 0x67c302e0
Caught exception in looking up space point ptr... 0x4cebd240, 0x4cebd560
Caught exception in looking up space point ptr... 0x4cebd560, 0x4cebd6f0
Caught exception in looking up space point ptr... 0x4de7c780, 0x4de7c910
Caught exception in looking up space point ptr... 0x4de7c910, 0x4de7c780
Caught exception in looking up space point ptr... 0x91c3de0, 0x91c6230
Caught exception in looking up space point ptr... 0x91c6230, 0x373a76c0
Caught exception in looking up space point ptr... 0x674aa810, 0x6393a150
Caught exception in looking up space point ptr... 0x6393a150, 0x674aa810
Caught exception in looking up space point ptr... 0x1166ada0, 0x4de684c0
Caught exception in looking up space point ptr... 0x4de684c0, 0x1166b910
Caught exception in looking up space point ptr... 0x2f794e70, 0x4df32230
Caught exception in looking up space point ptr... 0x4df32230, 0x2f7947b0
Caught exception in looking up space point ptr... 0x3348e210, 0x3348e3a0
Caught exception in looking up space point ptr... 0x3348e3a0, 0x367fa7a0
Caught exception in looking up space point ptr... 0x49f80050, 0x49f7fba0
Caught exception in looking up space point ptr... 0x49f7fba0, 0x49f7fd30
Caught exception in looking up space point ptr... 0x5551ff90, 0x5551fda0
Caught exception in looking up space point ptr... 0x5551fda0, 0x5551ff90
Caught exception in looking up space point ptr... 0x63813620, 0x49732e10
Caught exception in looking up space point ptr... 0x49732e10, 0x66487f60
Caught exception in looking up space point ptr... 0x4dde3830, 0x4dde3a90
Caught exception in looking up space point ptr... 0x4dde3a90, 0x4dde24d0
Caught exception in looking up space point ptr... 0x779a77c0, 0x779a7ae0
Caught exception in looking up space point ptr... 0x779a7ae0, 0x779a77c0
Caught exception in looking up space point ptr... 0x779a77c0, 0x779a7c70
Caught exception in looking up space point ptr... 0x779a7c70, 0x779a77c0
Caught exception in looking up space point ptr... 0x33dba120, 0x33db9eb0
Caught exception in looking up space point ptr... 0x33db9eb0, 0x33dba120
Caught exception in looking up space point ptr... 0x2ef8f520, 0x109120d0
Caught exception in looking up space point ptr... 0x109120d0, 0x2ef8f520
Caught exception in looking up space point ptr... 0x4e242950, 0x4e242aa0
Caught exception in looking up space point ptr... 0x4e242aa0, 0x4e242950
Caught exception in looking up space point ptr... 0x7ac831d0, 0x3e2a0a80
Caught exception in looking up space point ptr... 0x3e2a0a80, 0x7ac6af80
Caught exception in looking up space point ptr... 0x779c7fc0, 0x779c81a0
Caught exception in looking up space point ptr... 0x779c81a0, 0x779c8510
Caught exception in looking up space point ptr... 0x4e179240, 0x4e179110
Caught exception in looking up space point ptr... 0x4e179110, 0x4e179240
Caught exception in looking up space point ptr... 0x81c06840, 0x81c069d0
Caught exception in looking up space point ptr... 0x81c069d0, 0x81c06840
Caught exception in looking up space point ptr... 0x3e2bf6c0, 0x3e2d4790
Caught exception in looking up space point ptr... 0x3e2d4790, 0x3e2bf6c0
Caught exception in looking up space point ptr... 0x82b80100, 0x2f3286a0
Caught exception in looking up space point ptr... 0x2f326e80, 0x82b80100
Caught exception in looking up space point ptr... 0x363e1f90, 0x363e42a0
Caught exception in looking up space point ptr... 0x363e42a0, 0x363e1f90
Caught exception in looking up space point ptr... 0x5513ac30, 0x5513b420
Caught exception in looking up space point ptr... 0x5513b420, 0x5513ac30
Caught exception in looking up space point ptr... 0x473956b0, 0x4c635310
Caught exception in looking up space point ptr... 0x4c635310, 0x473956b0
Caught exception in looking up space point ptr... 0x4a0ebd20, 0x4a0ebf00
Caught exception in looking up space point ptr... 0x4a0ebf00, 0x4a0eb9f0
Caught exception in looking up space point ptr... 0x37aca480, 0x775a08e0
Caught exception in looking up space point ptr... 0x775a08e0, 0x37aca480
Caught exception in looking up space point ptr... 0x4de12490, 0x4de12750
Caught exception in looking up space point ptr... 0x4de12750, 0x4de12880
++++>>>> total num hits: 98594, num free: 94239
04-Feb-2026 01:02:39 GMT  Closed output file "np02vd_raw_run042433_0524_df-s03-d1_dw_0_20260203T220116_reco_stage0_20260204T010239_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         169.807       194.698       230.582       195.515       13.2741        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.9287e-05    0.000154757   0.000467675   0.00011763    9.00893e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.813        15.3702       16.0644       15.3047       0.32057        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.526283      0.567873      0.609783      0.564043      0.0206933       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177109     0.0188291     0.025367      0.0181853    0.00190669       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177506     0.018106      0.0187274     0.0179976    0.000273729      19     
produce:beamevent:BeamEvent                      0.000108152   0.000225841   0.00104805    0.000178976   0.000198927      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.31582       3.71546       4.28474       3.63489      0.249764        19     
produce:ophit:OpHitFinder                         0.0286709     0.0454461     0.0691726     0.0444187    0.00774648       19     
produce:wclsdatavd:WireCellToolkit                 121.682       138.269       150.515       140.561       8.04319        19     
produce:gaushit:GausHitFinder                     0.930628       1.83151       2.45669       1.90077      0.488386        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000295894   0.000735213   0.00231516    0.000495668   0.000595546      19     
produce:reco3d:SpacePointSolver                    10.2639       22.5349       36.0357       22.8802       6.36243        19     
produce:hitpdune:DisambigFromSpacePoints          0.0891856     0.260978      0.373539      0.254683      0.0781418       19     
produce:cluster3d:Cluster3D                        1.49582       5.47106       23.4317       3.6889        5.94943        19     
[art]:TriggerResults:TriggerResultInserter       2.5928e-05    4.02379e-05    9.126e-05    3.9173e-05    1.3228e-05       19     
end_path:out1:RootOutput                          3.687e-06    8.67253e-06    2.655e-05     7.304e-06    5.27403e-06      19     
end_path:out1:RootOutput(write)                    4.73411       6.53263       8.02844       6.64818      0.984826        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5408.15 MB
  Peak resident set size usage (VmHWM): 3383.91 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0524_df-s03-d1_dw_0_20260203T220116_reco_stage0_20260204T010239_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0524_df-s03-d1_dw_0_20260203T220116_reco_stage0_20260204T010239_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0524_df-s03-d1_dw_0_20260203T220116_reco_stage0_20260204T010239_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163200.0,
    "core.end_time": 1770166961.0,
    "core.events": [
      59745,
      59751,
      59757,
      59763,
      59769,
      59775,
      59781,
      59787,
      59793,
      59799,
      59805,
      59811,
      59817,
      59823,
      59829,
      59835,
      59841,
      59847,
      59853
    ],
    "core.event_count": 19,
    "core.first_event_number": 59745,
    "core.last_event_number": 59853,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0524_df-s03-d1_dw_0_20260203T220116.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:32 UTC       justIN version: 01.06.00