justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.20@dunegpschedd01.fnal.gov

Jobsub ID303656.20@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:06:42
From worker nodeHostnameheplnc147.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:12:00
Input filesvd-protodune:np02vd_raw_run042421_0014_df-s03-d1_dw_0_20260203T133833.hdf5
JobscriptExit code0
Real time52m (3138s)
CPU time44m (2698s = 85%)
Max RSS bytes2577371136 (2457 MiB)
Outputting started2026-02-04 01:04:19
Output files
Finished2026-02-04 01:06:42
Saved logsjustin-logs:303656.20-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

up space point ptr... 0xf2fda20, 0xf2fdbb0
Caught exception in looking up space point ptr... 0xf2fdbb0, 0xf2fcfa0
Caught exception in looking up space point ptr... 0xf2fcfa0, 0xf867080
Caught exception in looking up space point ptr... 0x488220e0, 0x48821f50
Caught exception in looking up space point ptr... 0x48821f50, 0x48821dc0
Caught exception in looking up space point ptr... 0x48821dc0, 0x48823b00
Caught exception in looking up space point ptr... 0x48823b00, 0x3d819340
Caught exception in looking up space point ptr... 0x3d819340, 0x90f8c20
Caught exception in looking up space point ptr... 0x90f8c20, 0x488220e0
Caught exception in looking up space point ptr... 0x47a8a630, 0x17226a90
Caught exception in looking up space point ptr... 0xe051020, 0x2e7ff210
Caught exception in looking up space point ptr... 0x2e7ff210, 0x47a8a630
Caught exception in looking up space point ptr... 0x52699730, 0x30ebd490
Caught exception in looking up space point ptr... 0x30ebd490, 0xd75e410
Caught exception in looking up space point ptr... 0xd75e410, 0xbe5eeb0
Caught exception in looking up space point ptr... 0x5a0ccac0, 0x52699730
Caught exception in looking up space point ptr... 0x268c6b40, 0x30eda8d0
Caught exception in looking up space point ptr... 0x30eda8d0, 0x4547b790
Caught exception in looking up space point ptr... 0x312cbef0, 0x268c6b40
Caught exception in looking up space point ptr... 0xb1b8310, 0x44b0f2f0
Caught exception in looking up space point ptr... 0x44b0f480, 0x44b0f610
Caught exception in looking up space point ptr... 0x44b0f610, 0xb1b8610
Caught exception in looking up space point ptr... 0xb1b8610, 0xb1b8310
Caught exception in looking up space point ptr... 0x12253ee0, 0x59619e20
Caught exception in looking up space point ptr... 0x59619e20, 0x34e6a480
Caught exception in looking up space point ptr... 0x2693a4f0, 0x2693a200
Caught exception in looking up space point ptr... 0x2693a200, 0x2693a4f0
Caught exception in looking up space point ptr... 0x167456c0, 0x167457a0
Caught exception in looking up space point ptr... 0x167457a0, 0x167456c0
Caught exception in looking up space point ptr... 0x4f4235d0, 0x4f4238f0
Caught exception in looking up space point ptr... 0x4f4238f0, 0x4f421b40
Caught exception in looking up space point ptr... 0x3bd1c8e0, 0xdd975f0
Caught exception in looking up space point ptr... 0xdd975f0, 0x4853ee70
Caught exception in looking up space point ptr... 0x4853ee70, 0x4853ef10
Caught exception in looking up space point ptr... 0x4853ef10, 0x4853f000
Caught exception in looking up space point ptr... 0x4853f000, 0x312ff5a0
Caught exception in looking up space point ptr... 0x312ff5a0, 0x3bd1c8e0
Caught exception in looking up space point ptr... 0x1476ddc0, 0x3251e350
Caught exception in looking up space point ptr... 0x3251e350, 0x1476ddc0
Caught exception in looking up space point ptr... 0x4234a290, 0x42349ff0
Caught exception in looking up space point ptr... 0x42349ea0, 0x4234a290
Caught exception in looking up space point ptr... 0x35175a00, 0x35175b60
Caught exception in looking up space point ptr... 0x35175b60, 0x2bc6c3f0
Caught exception in looking up space point ptr... 0x29c0c380, 0x15f5ccb0
Caught exception in looking up space point ptr... 0x15f5ccb0, 0x5fa3a290
Caught exception in looking up space point ptr... 0x50f01260, 0x50f00c70
Caught exception in looking up space point ptr... 0x50f00c70, 0x50f01260
Caught exception in looking up space point ptr... 0x327e7cd0, 0x327e7af0
Caught exception in looking up space point ptr... 0x2693ad70, 0x3251d000
Caught exception in looking up space point ptr... 0x3251d000, 0x42289360
Caught exception in looking up space point ptr... 0x42289360, 0x327e7cd0
Caught exception in looking up space point ptr... 0x4471f7c0, 0xdb4f2b0
Caught exception in looking up space point ptr... 0xdb4f2b0, 0x4471f7c0
Caught exception in looking up space point ptr... 0x4471f7c0, 0x32028fe0
Caught exception in looking up space point ptr... 0x32028fe0, 0x4471f7c0
Caught exception in looking up space point ptr... 0x3c681660, 0x3c681390
Caught exception in looking up space point ptr... 0x3c681390, 0x3c681940
Caught exception in looking up space point ptr... 0x163faba0, 0x47aa2c80
Caught exception in looking up space point ptr... 0x47aa2c80, 0xd8dbb20
Caught exception in looking up space point ptr... 0xd8dbb20, 0x15f46110
Caught exception in looking up space point ptr... 0x15f46110, 0x37c86cb0
Caught exception in looking up space point ptr... 0x34f3f0f0, 0x15f79ad0
Caught exception in looking up space point ptr... 0x15f79ad0, 0x3ea04fd0
Caught exception in looking up space point ptr... 0x3c681240, 0x352d1f80
Caught exception in looking up space point ptr... 0x352d1f80, 0x352d1ce0
Caught exception in looking up space point ptr... 0x352d1ce0, 0x352d1be0
++++>>>> total num hits: 30798, num free: 29286
04-Feb-2026 01:04:12 GMT  Closed output file "np02vd_raw_run042421_0014_df-s03-d1_dw_0_20260203T133833_reco_stage0_20260204T010412_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         95.2512       112.511       135.082       112.276       10.8018        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.6493e-05    0.000104276   0.000308472   9.4909e-05    4.10276e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                11.7816       14.4905       15.5574       14.8505       1.00997        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.527426      0.550637      0.603094      0.550312      0.0152237       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175826     0.0180081     0.0184777     0.018027     0.000187762      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0875867     0.0936569     0.106766      0.0887497     0.0078298       27     
produce:beamevent:BeamEvent                      0.000112452   0.00021466     0.0013436    0.000169319   0.000223377      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.47706       3.9498        4.28145       3.97991      0.177724        27     
produce:ophit:OpHitFinder                         0.0333174     0.0423799     0.0519299     0.0416731    0.00478006       27     
produce:wclsdatavd:WireCellToolkit                 60.4813       74.9824       88.0306       74.7502       8.12265        27     
produce:gaushit:GausHitFinder                     0.477079       1.06387       1.59406      0.999614      0.326106        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000228972   0.00035911    0.00101504    0.000342416   0.000142441      27     
produce:reco3d:SpacePointSolver                    5.77979       10.851        20.3201       9.50375       3.85675        27     
produce:hitpdune:DisambigFromSpacePoints          0.0502641     0.117609      0.188915      0.113033      0.0360857       27     
produce:cluster3d:Cluster3D                       0.769024       2.14439       4.61172       2.00459      0.877589        27     
[art]:TriggerResults:TriggerResultInserter       2.4456e-05    3.84919e-05   0.000133602   3.0928e-05    2.32035e-05      27     
end_path:out1:RootOutput                          3.266e-06    8.6373e-06    3.1549e-05     7.614e-06    5.06181e-06      27     
end_path:out1:RootOutput(write)                    3.27116       4.17481       5.24483       3.76504      0.686769        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4524.05 MB
  Peak resident set size usage (VmHWM): 2577.37 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0014_df-s03-d1_dw_0_20260203T133833_reco_stage0_20260204T010412_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0014_df-s03-d1_dw_0_20260203T133833_reco_stage0_20260204T010412_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0014_df-s03-d1_dw_0_20260203T133833_reco_stage0_20260204T010412_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163933.0,
    "core.end_time": 1770167053.0,
    "core.events": [
      2301,
      2307,
      2313,
      2319,
      2325,
      2331,
      2337,
      2343,
      2349,
      2355,
      2361,
      2367,
      2373,
      2379,
      2385,
      2391,
      2397,
      2403,
      2409,
      2415,
      2421,
      2427,
      2433,
      2439,
      2445,
      2451,
      2457
    ],
    "core.event_count": 27,
    "core.first_event_number": 2301,
    "core.last_event_number": 2457,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0014_df-s03-d1_dw_0_20260203T133833.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:28:21 UTC       justIN version: 01.06.00