justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.72@dunegpschedd02.fnal.gov

Jobsub ID297923.72@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:02:30
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:59:10
Input filesvd-protodune:np02vd_raw_run042433_0197_df-s04-d1_dw_0_20260203T195700.hdf5
JobscriptExit code0
Real time1h (3641s)
CPU time55m (3318s = 91%)
Max RSS bytes3329568768 (3175 MiB)
Outputting started2026-02-04 00:59:52
Output files
Finished2026-02-04 01:02:30
Saved logsjustin-logs:297923.72-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

... 0x5ef28f30, 0x5ef32b20
Caught exception in looking up space point ptr... 0x5ef32b20, 0x5ef28f30
Caught exception in looking up space point ptr... 0x15012570, 0x38520310
Caught exception in looking up space point ptr... 0x38520310, 0x15012570
Caught exception in looking up space point ptr... 0x83f02dd0, 0x83f029e0
Caught exception in looking up space point ptr... 0x83f029e0, 0x83edccb0
Caught exception in looking up space point ptr... 0x83edcfd0, 0x83f02dd0
Caught exception in looking up space point ptr... 0x1190d330, 0x11914e00
Caught exception in looking up space point ptr... 0x11914e00, 0x1190f140
Caught exception in looking up space point ptr... 0x487edb70, 0x487ee660
Caught exception in looking up space point ptr... 0x487ee660, 0x487edb70
Caught exception in looking up space point ptr... 0x4d549cf0, 0x4d5495b0
Caught exception in looking up space point ptr... 0x4d5495b0, 0x4d549cf0
Caught exception in looking up space point ptr... 0x390e8df0, 0x390e9090
Caught exception in looking up space point ptr... 0x390e9090, 0x390e8df0
Caught exception in looking up space point ptr... 0x38571440, 0x38572830
Caught exception in looking up space point ptr... 0x38572830, 0x38571440
Caught exception in looking up space point ptr... 0x2c151e70, 0x2d6acab0
Caught exception in looking up space point ptr... 0x2d6acab0, 0x49f13eb0
Caught exception in looking up space point ptr... 0x57dff870, 0x536e46d0
Caught exception in looking up space point ptr... 0x536e46d0, 0x2d2b85b0
Caught exception in looking up space point ptr... 0x49059660, 0x4905ed70
Caught exception in looking up space point ptr... 0x4905ed70, 0x49059660
Caught exception in looking up space point ptr... 0x7c0a02d0, 0x43341b30
Caught exception in looking up space point ptr... 0x43341b30, 0x7c0a5900
Caught exception in looking up space point ptr... 0x1380aa70, 0x362a3cf0
Caught exception in looking up space point ptr... 0x362a3cf0, 0x62d27a40
Caught exception in looking up space point ptr... 0x47e0e090, 0x47e11ca0
Caught exception in looking up space point ptr... 0x47e11ca0, 0x47e12030
Caught exception in looking up space point ptr... 0x6da0b160, 0x6da0aef0
Caught exception in looking up space point ptr... 0x6da0aef0, 0x6da10d50
Caught exception in looking up space point ptr... 0x6da10d50, 0x6da13ea0
Caught exception in looking up space point ptr... 0x6da13ea0, 0x6da0b160
Caught exception in looking up space point ptr... 0x42887ef0, 0x42887b50
Caught exception in looking up space point ptr... 0x42887b50, 0x428877b0
Caught exception in looking up space point ptr... 0x3d2de520, 0x3d2dcab0
Caught exception in looking up space point ptr... 0x3d2dcab0, 0x3d2de520
Caught exception in looking up space point ptr... 0x2ec2ceb0, 0x2ec2d040
Caught exception in looking up space point ptr... 0x2ec2d040, 0x2ec2ceb0
Caught exception in looking up space point ptr... 0x6da08a70, 0x6da03610
Caught exception in looking up space point ptr... 0x6da11bb0, 0x6da08a70
Caught exception in looking up space point ptr... 0xf3e34b0, 0xf3dba40
Caught exception in looking up space point ptr... 0xf3dba40, 0xf3e34b0
Caught exception in looking up space point ptr... 0x49edcef0, 0x30fc6b50
Caught exception in looking up space point ptr... 0x30fc6b50, 0x3bb50590
Caught exception in looking up space point ptr... 0x1190f9b0, 0x1190eef0
Caught exception in looking up space point ptr... 0x1190eef0, 0x1190c0c0
Caught exception in looking up space point ptr... 0x7e665a00, 0x7e665af0
Caught exception in looking up space point ptr... 0x7e665af0, 0x7e665820
Caught exception in looking up space point ptr... 0x5ef5d300, 0x56fd49b0
Caught exception in looking up space point ptr... 0x56fd49b0, 0x5ef5d300
Caught exception in looking up space point ptr... 0x491fd1a0, 0x6c985430
Caught exception in looking up space point ptr... 0x6c985430, 0x491fd1a0
Caught exception in looking up space point ptr... 0x4336c420, 0x4336c740
Caught exception in looking up space point ptr... 0x4336c740, 0x4336c420
Caught exception in looking up space point ptr... 0x3bfd3a00, 0x3bfd3be0
Caught exception in looking up space point ptr... 0x3bfd3be0, 0x3bfd3d70
Caught exception in looking up space point ptr... 0x49fbb990, 0x49fbd120
Caught exception in looking up space point ptr... 0x49fbd120, 0x49fbb990
Caught exception in looking up space point ptr... 0x49fbb990, 0x49fbd580
Caught exception in looking up space point ptr... 0x49fbd580, 0x49fbb990
Caught exception in looking up space point ptr... 0x5ef31160, 0x5ef26060
Caught exception in looking up space point ptr... 0x5ef26060, 0x5ef31340
Caught exception in looking up space point ptr... 0x2edc63b0, 0x2edc7d30
Caught exception in looking up space point ptr... 0x2edc7d30, 0x2edc63b0
Caught exception in looking up space point ptr... 0x36e1dd70, 0x2ec1f230
Caught exception in looking up space point ptr... 0x2ec1f230, 0x2ec1f5d0
++++>>>> total num hits: 107437, num free: 101537
04-Feb-2026 00:59:42 GMT  Closed output file "np02vd_raw_run042433_0197_df-s04-d1_dw_0_20260203T195700_reco_stage0_20260204T005942_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         166.071       186.856       214.115       184.111       14.3599        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000103524   0.00018201    0.000372888   0.000141565   8.55376e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.4336       15.0578       16.3265       15.102       0.578974        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.534686      0.555065      0.598001      0.548483      0.018381        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0178125     0.0182009     0.0189276     0.0181407    0.000247415      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0178235     0.0181314     0.0186362     0.0180583    0.000252223      19     
produce:beamevent:BeamEvent                      0.000101249   0.000276461   0.00158275    0.000206767   0.000311172      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.43384        3.775        4.15097       3.74299      0.221597        19     
produce:ophit:OpHitFinder                         0.0343045     0.0452193     0.0520118     0.0462499    0.00390838       19     
produce:wclsdatavd:WireCellToolkit                 118.649       136.42        150.866       136.507       9.43869        19     
produce:gaushit:GausHitFinder                      0.90859       1.72609       2.86511       1.62789       0.53404        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000318145   0.000511261   0.00110001    0.000484868   0.00019662       19     
produce:reco3d:SpacePointSolver                    11.0492       18.9622       29.6983       17.164        5.19901        19     
produce:hitpdune:DisambigFromSpacePoints          0.0980387     0.241713      0.451814      0.231172      0.0841809       19     
produce:cluster3d:Cluster3D                        1.44991       3.63989       6.53068       3.28162       1.37133        19     
[art]:TriggerResults:TriggerResultInserter       2.9124e-05    4.84537e-05   0.000160832   3.9954e-05    2.85864e-05      19     
end_path:out1:RootOutput                          6.242e-06    1.43927e-05   4.6066e-05    1.1341e-05    8.96484e-06      19     
end_path:out1:RootOutput(write)                    4.65082       6.33419       8.39192       6.67951       1.15282        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5356.57 MB
  Peak resident set size usage (VmHWM): 3329.57 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0197_df-s04-d1_dw_0_20260203T195700_reco_stage0_20260204T005942_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0197_df-s04-d1_dw_0_20260203T195700_reco_stage0_20260204T005942_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0197_df-s04-d1_dw_0_20260203T195700_reco_stage0_20260204T005942_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163165.0,
    "core.end_time": 1770166784.0,
    "core.events": [
      22469,
      22475,
      22481,
      22487,
      22493,
      22499,
      22505,
      22511,
      22517,
      22523,
      22529,
      22535,
      22541,
      22547,
      22553,
      22559,
      22565,
      22571,
      22577
    ],
    "core.event_count": 19,
    "core.first_event_number": 22469,
    "core.last_event_number": 22577,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0197_df-s04-d1_dw_0_20260203T195700.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:50 UTC       justIN version: 01.06.00