justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.22@dunegpschedd01.fnal.gov

Jobsub ID303656.22@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:05:27
From worker nodeHostnameheplnc147.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:11:06
Input filesvd-protodune:np02vd_raw_run042421_0042_df-s03-d1_dw_0_20260203T135344.hdf5
JobscriptExit code0
Real time52m (3121s)
CPU time44m (2669s = 85%)
Max RSS bytes2542972928 (2425 MiB)
Outputting started2026-02-04 01:03:08
Output files
Finished2026-02-04 01:05:27
Saved logsjustin-logs:303656.22-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

.. 0x5ee26570, 0x5ee267a0
Caught exception in looking up space point ptr... 0x5ee267a0, 0x5ee26570
Caught exception in looking up space point ptr... 0x4ae2c930, 0x4ae2c7a0
Caught exception in looking up space point ptr... 0x4ae2c7a0, 0xe74a7c0
Caught exception in looking up space point ptr... 0x29ae5580, 0x29ae5420
Caught exception in looking up space point ptr... 0x29ae5420, 0x29ae5580
Caught exception in looking up space point ptr... 0x29e20ba0, 0x336a52c0
Caught exception in looking up space point ptr... 0x336a52c0, 0x272a2bc0
Caught exception in looking up space point ptr... 0x371c36c0, 0x371bfbc0
Caught exception in looking up space point ptr... 0x371baf90, 0x371c36c0
Caught exception in looking up space point ptr... 0x36614a20, 0x36614590
Caught exception in looking up space point ptr... 0x36614590, 0x36612c60
Caught exception in looking up space point ptr... 0x62bbfef0, 0x62bc0670
Caught exception in looking up space point ptr... 0x62bc0670, 0x62bbfef0
Caught exception in looking up space point ptr... 0x62bbfef0, 0x62bc0090
Caught exception in looking up space point ptr... 0x62bc0090, 0x62bbfef0
Caught exception in looking up space point ptr... 0x4e0ca960, 0x4e0ca7d0
Caught exception in looking up space point ptr... 0x4e0ca7d0, 0x4e0cff80
Caught exception in looking up space point ptr... 0x2b8a8960, 0x50448ee0
Caught exception in looking up space point ptr... 0x50448ee0, 0x2b8a8960
Caught exception in looking up space point ptr... 0x625b1200, 0x625b3f70
Caught exception in looking up space point ptr... 0x625b3f70, 0x3eb64910
Caught exception in looking up space point ptr... 0x3eb64910, 0x3eb64680
Caught exception in looking up space point ptr... 0x3eb643f0, 0x14ed7a30
Caught exception in looking up space point ptr... 0x14ed7a30, 0x625b1200
Caught exception in looking up space point ptr... 0x461c7570, 0x461c7850
Caught exception in looking up space point ptr... 0x58e862b0, 0x461c7570
Caught exception in looking up space point ptr... 0x2c16a850, 0x5041e220
Caught exception in looking up space point ptr... 0x5041e220, 0x2c16a850
Caught exception in looking up space point ptr... 0x4f6b6460, 0x4f6b8200
Caught exception in looking up space point ptr... 0x4f6b8200, 0x4f6b8380
Caught exception in looking up space point ptr... 0x4f6b8380, 0x4f6b66c0
Caught exception in looking up space point ptr... 0x373dbd60, 0x373e6840
Caught exception in looking up space point ptr... 0x373e6840, 0x373dbd60
Caught exception in looking up space point ptr... 0x373dbd60, 0x373e7b50
Caught exception in looking up space point ptr... 0x373e7b50, 0x373dbd60
Caught exception in looking up space point ptr... 0x22712b90, 0x22719c80
Caught exception in looking up space point ptr... 0x22719c80, 0x2271bf70
Caught exception in looking up space point ptr... 0x226fb250, 0x29e1e8f0
Caught exception in looking up space point ptr... 0x29e1e8f0, 0x226fb250
Caught exception in looking up space point ptr... 0x3f865530, 0x3f865630
Caught exception in looking up space point ptr... 0x3f865630, 0x3f865530
Caught exception in looking up space point ptr... 0x5edfaba0, 0x5edf3c90
Caught exception in looking up space point ptr... 0x5edf3c90, 0x5edfaba0
Caught exception in looking up space point ptr... 0x26f53c60, 0xfe4ad50
Caught exception in looking up space point ptr... 0xfe4ad50, 0x5ede9d20
Caught exception in looking up space point ptr... 0x625613b0, 0x62561680
Caught exception in looking up space point ptr... 0x62561680, 0x62561910
Caught exception in looking up space point ptr... 0x37d85c20, 0x13951770
Caught exception in looking up space point ptr... 0x13951770, 0x37d85c20
Caught exception in looking up space point ptr... 0x3fa919b0, 0x3fa91620
Caught exception in looking up space point ptr... 0x3fa91620, 0x3fa919b0
Caught exception in looking up space point ptr... 0x1396bb10, 0x13966690
Caught exception in looking up space point ptr... 0x13966690, 0x1396bb10
Caught exception in looking up space point ptr... 0x4ad7c620, 0x2af044f0
Caught exception in looking up space point ptr... 0x2af044f0, 0x4ad7c620
Caught exception in looking up space point ptr... 0x459f1110, 0x459f1610
Caught exception in looking up space point ptr... 0x459f1610, 0x459f1110
Caught exception in looking up space point ptr... 0x12cdb710, 0x12cdb510
Caught exception in looking up space point ptr... 0x12cdb510, 0x12cdb710
Caught exception in looking up space point ptr... 0x15b668f0, 0x15b665d0
Caught exception in looking up space point ptr... 0x15b665d0, 0x15b66760
Caught exception in looking up space point ptr... 0x26f4ca50, 0x26f4d5c0
Caught exception in looking up space point ptr... 0x26f4d5c0, 0x26f49ee0
Caught exception in looking up space point ptr... 0x37d91fc0, 0x37d91d80
Caught exception in looking up space point ptr... 0x37d91d80, 0x37d91fc0
++++>>>> total num hits: 56278, num free: 54539
04-Feb-2026 01:02:57 GMT  Closed output file "np02vd_raw_run042421_0042_df-s03-d1_dw_0_20260203T135344_reco_stage0_20260204T010257_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         94.7043       111.169       132.886       110.017       10.2119        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.7486e-05    0.000138265   0.000493381   0.000106862   8.62497e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                11.9026       14.2883       15.3464       14.7692       1.07984        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.525728       0.5475       0.616263      0.546376      0.0175607       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177603     0.0181459     0.022026      0.0179626    0.000787822      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0868159     0.0897291     0.105791      0.0883867    0.00459668       27     
produce:beamevent:BeamEvent                      0.000101763   0.000225571   0.00173022    0.00015909    0.000303855      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.33231       3.92149       4.25865       3.98311      0.257525        27     
produce:ophit:OpHitFinder                         0.0310452     0.0435576     0.0502796     0.0450719    0.00501002       27     
produce:wclsdatavd:WireCellToolkit                 61.3983       74.073        87.5573       73.1768       7.00873        27     
produce:gaushit:GausHitFinder                     0.624439      0.984134       1.79721      0.866467      0.335759        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000202923   0.000371429   0.000900038   0.000332107   0.000145385      27     
produce:reco3d:SpacePointSolver                    5.58639       10.605        20.6837       9.63393       3.68347        27     
produce:hitpdune:DisambigFromSpacePoints          0.0603387     0.118494      0.258505      0.102036      0.0472081       27     
produce:cluster3d:Cluster3D                       0.952792       2.02374       4.4586        1.54345       1.04261        27     
[art]:TriggerResults:TriggerResultInserter       2.3545e-05    3.48767e-05   9.8596e-05    3.0698e-05    1.38349e-05      27     
end_path:out1:RootOutput                          4.529e-06    1.68989e-05   0.000246555    7.884e-06    4.52075e-05      27     
end_path:out1:RootOutput(write)                    3.43223       4.42342       5.52169       4.58181      0.698445        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4524.99 MB
  Peak resident set size usage (VmHWM): 2542.97 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0042_df-s03-d1_dw_0_20260203T135344_reco_stage0_20260204T010257_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0042_df-s03-d1_dw_0_20260203T135344_reco_stage0_20260204T010257_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0042_df-s03-d1_dw_0_20260203T135344_reco_stage0_20260204T010257_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163879.0,
    "core.end_time": 1770166980.0,
    "core.events": [
      6855,
      6861,
      6867,
      6873,
      6879,
      6885,
      6891,
      6897,
      6903,
      6909,
      6915,
      6921,
      6927,
      6933,
      6939,
      6945,
      6951,
      6957,
      6963,
      6969,
      6975,
      6981,
      6987,
      6993,
      6999,
      7005,
      7011
    ],
    "core.event_count": 27,
    "core.first_event_number": 6855,
    "core.last_event_number": 7011,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0042_df-s03-d1_dw_0_20260203T135344.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:37:11 UTC       justIN version: 01.06.00