justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303647.171@dunegpschedd01.fnal.gov

Jobsub ID303647.171@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:46:24
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce506
Last heartbeat2026-02-03 23:41:43
From worker nodeHostnameb9p06p2822.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 22:46:54
Input filesvd-protodune:np02vd_raw_run042421_0014_df-s04-d1_dw_0_20260203T133834.hdf5
JobscriptExit code0
Real time51m (3114s)
CPU time49m (2987s = 95%)
Max RSS bytes2744651776 (2617 MiB)
Outputting started2026-02-03 23:38:50
Output files
Finished2026-02-03 23:41:43
Saved logsjustin-logs:303647.171-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nt ptr... 0x5b4af4c0, 0x485d0660
Caught exception in looking up space point ptr... 0x485d0660, 0x5b4af4c0
Caught exception in looking up space point ptr... 0x5b4af4c0, 0x97af910
Caught exception in looking up space point ptr... 0x97af910, 0x5b4af4c0
Caught exception in looking up space point ptr... 0x28a3d360, 0x5236a710
Caught exception in looking up space point ptr... 0x5fef4420, 0x28a3d360
Caught exception in looking up space point ptr... 0x390d6700, 0x390d6570
Caught exception in looking up space point ptr... 0x5bb043e0, 0x390d6700
Caught exception in looking up space point ptr... 0x3690dff0, 0x13d37de0
Caught exception in looking up space point ptr... 0x1467e380, 0x3690dff0
Caught exception in looking up space point ptr... 0x45e0a100, 0x2f1147c0
Caught exception in looking up space point ptr... 0x2f1147c0, 0x45e0a100
Caught exception in looking up space point ptr... 0x31cfc940, 0x24081940
Caught exception in looking up space point ptr... 0x24081940, 0x333fef10
Caught exception in looking up space point ptr... 0x4d6f7de0, 0x3719fc90
Caught exception in looking up space point ptr... 0x3719fc90, 0x531f6400
Caught exception in looking up space point ptr... 0xf1cb0e0, 0x142ad7b0
Caught exception in looking up space point ptr... 0x142ad7b0, 0x33d73110
Caught exception in looking up space point ptr... 0x5b9b4d80, 0x385d9e60
Caught exception in looking up space point ptr... 0x385d9e60, 0x47ee4f70
Caught exception in looking up space point ptr... 0x115f42a0, 0x8188a00
Caught exception in looking up space point ptr... 0x115f3f80, 0x115f42a0
Caught exception in looking up space point ptr... 0x5b4adad0, 0x37f5e8f0
Caught exception in looking up space point ptr... 0x37f5e8f0, 0x5b4adad0
Caught exception in looking up space point ptr... 0x5f9326a0, 0x2461aa10
Caught exception in looking up space point ptr... 0x2254bd70, 0x5f9326a0
Caught exception in looking up space point ptr... 0x37d383f0, 0x531eda20
Caught exception in looking up space point ptr... 0x531eda20, 0x531ed890
Caught exception in looking up space point ptr... 0x46372350, 0xce149f0
Caught exception in looking up space point ptr... 0xce149f0, 0x549a5080
Caught exception in looking up space point ptr... 0x16395860, 0x16395a60
Caught exception in looking up space point ptr... 0x16395a60, 0x16395860
Caught exception in looking up space point ptr... 0x3c8d3d90, 0x12a82190
Caught exception in looking up space point ptr... 0x3c8c3710, 0x3c8d3d90
Caught exception in looking up space point ptr... 0x5a1d8690, 0x56029910
Caught exception in looking up space point ptr... 0x5a1e6920, 0x5a1d8690
Caught exception in looking up space point ptr... 0x5088f6e0, 0x10814b80
Caught exception in looking up space point ptr... 0x10814b80, 0x5088f6e0
Caught exception in looking up space point ptr... 0x2a9a86a0, 0x4d9756d0
Caught exception in looking up space point ptr... 0x2a9a8830, 0x2a9a86a0
Caught exception in looking up space point ptr... 0x4607b6d0, 0x21d9ce00
Caught exception in looking up space point ptr... 0x307e9390, 0x4607b6d0
Caught exception in looking up space point ptr... 0x57f9e840, 0x57f9eb60
Caught exception in looking up space point ptr... 0x57f9eb60, 0x57f9e840
Caught exception in looking up space point ptr... 0x10ee7cc0, 0x2738cca0
Caught exception in looking up space point ptr... 0x2738cca0, 0x2738ce30
Caught exception in looking up space point ptr... 0x3689c120, 0x44ced4e0
Caught exception in looking up space point ptr... 0x3689c2b0, 0x3689c120
Caught exception in looking up space point ptr... 0x85dbad0, 0x85c9510
Caught exception in looking up space point ptr... 0xeac8350, 0x85dbad0
Caught exception in looking up space point ptr... 0x32c83c00, 0x32c75010
Caught exception in looking up space point ptr... 0x58b3ef20, 0x32c83c00
Caught exception in looking up space point ptr... 0x1463fd50, 0x1462d790
Caught exception in looking up space point ptr... 0x1462d790, 0x1461b1d0
Caught exception in looking up space point ptr... 0x450b6220, 0x2c615680
Caught exception in looking up space point ptr... 0x2c615680, 0x450b6220
Caught exception in looking up space point ptr... 0x32e8c950, 0x32e8a740
Caught exception in looking up space point ptr... 0x32e8a740, 0x32e8c950
Caught exception in looking up space point ptr... 0x56fdf060, 0x24e453f0
Caught exception in looking up space point ptr... 0x24e453f0, 0x56fdf060
Caught exception in looking up space point ptr... 0xe8570f0, 0x37696c20
Caught exception in looking up space point ptr... 0x37696c20, 0x9d7ebc0
Caught exception in looking up space point ptr... 0x4d053700, 0x4d0533e0
Caught exception in looking up space point ptr... 0x4d0533e0, 0x4d053700
Caught exception in looking up space point ptr... 0x4acde0c0, 0x442ca4f0
Caught exception in looking up space point ptr... 0x442ca4f0, 0x4acde0c0
++++>>>> total num hits: 315397, num free: 308798
04-Feb-2026 00:38:41 CET  Closed output file "np02vd_raw_run042421_0014_df-s04-d1_dw_0_20260203T133834_reco_stage0_20260203T233841_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         78.3205       113.205       134.938       119.093       17.6603        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        4.794e-05    9.5683e-05    0.000200171    8.563e-05    3.62323e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                3.53089       4.12699       5.13468       4.08711       0.33587        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.0104575     0.0170587      0.05924      0.0135641     0.0105072       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000414942   0.000822458   0.00889161    0.000496922    0.0015841       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00146279    0.00185703    0.00333742    0.00181691    0.000332363      27     
produce:beamevent:BeamEvent                      5.8811e-05    0.000124059   0.000757934    9.736e-05    0.000126124      27     
produce:pdvddaphne:DAPHNEReaderPDVD                2.0525        2.68222       3.18706       2.6518       0.326496        27     
produce:ophit:OpHitFinder                         0.0239219     0.0340093     0.0429508     0.0345267     0.0046951       27     
produce:wclsdatavd:WireCellToolkit                 56.7657       65.5324       76.4772       66.2267       5.09296        27     
produce:gaushit:GausHitFinder                     0.531011       1.17168       1.7189        1.20066      0.283151        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000176331   0.000271342   0.000449122   0.000255361   6.00119e-05      27     
produce:reco3d:SpacePointSolver                    3.82648       7.25521       17.3562       6.77517       2.65682        27     
produce:hitpdune:DisambigFromSpacePoints          0.0396793     0.180067      0.346009      0.169889      0.0808472       27     
produce:cluster3d:Cluster3D                       0.570155       28.0567       44.0696       34.5879       14.6568        27     
[art]:TriggerResults:TriggerResultInserter        2.34e-05     3.52799e-05   9.6791e-05     3.234e-05    1.3227e-05       27     
end_path:out1:RootOutput                          3.93e-06     7.32296e-06    2.803e-05     6.76e-06     4.32488e-06      27     
end_path:out1:RootOutput(write)                    2.90035       4.12143       5.6497        3.88995      0.722022        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4744.48 MB
  Peak resident set size usage (VmHWM): 2744.65 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0014_df-s04-d1_dw_0_20260203T133834_reco_stage0_20260203T233841_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0014_df-s04-d1_dw_0_20260203T133834_reco_stage0_20260203T233841_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0014_df-s04-d1_dw_0_20260203T133834_reco_stage0_20260203T233841_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770158823.0,
    "core.end_time": 1770161922.0,
    "core.events": [
      2303,
      2309,
      2315,
      2321,
      2327,
      2333,
      2339,
      2345,
      2351,
      2357,
      2363,
      2369,
      2375,
      2381,
      2387,
      2393,
      2399,
      2405,
      2411,
      2417,
      2423,
      2429,
      2435,
      2441,
      2447,
      2453,
      2459
    ],
    "core.event_count": 27,
    "core.first_event_number": 2303,
    "core.last_event_number": 2459,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0014_df-s04-d1_dw_0_20260203T133834.hdf5"
    }
  ]
}
justIN time: 2026-02-04 05:58:41 UTC       justIN version: 01.06.00