justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303647.111@dunegpschedd01.fnal.gov

Jobsub ID303647.111@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:46:24
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce512
Last heartbeat2026-02-03 23:23:43
From worker nodeHostnameb9p10p5730.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 22:47:13
Input filesvd-protodune:np02vd_raw_run042421_0083_df-s04-d0_dw_0_20260203T141553.hdf5
JobscriptExit code0
Real time34m (2059s)
CPU time33m (1983s = 96%)
Max RSS bytes2678435840 (2554 MiB)
Outputting started2026-02-03 23:21:33
Output files
Finished2026-02-03 23:23:43
Saved logsjustin-logs:303647.111-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0
Caught exception in looking up space point ptr... 0x3c99e910, 0x3c99ecf0
Caught exception in looking up space point ptr... 0x37ad49b0, 0x37ad4820
Caught exception in looking up space point ptr... 0x37ad4820, 0x37ad4cd0
Caught exception in looking up space point ptr... 0x3c9c54e0, 0x3c9bf780
Caught exception in looking up space point ptr... 0x3c9bf780, 0x3c9c54e0
Caught exception in looking up space point ptr... 0x3beefdc0, 0x3bef7510
Caught exception in looking up space point ptr... 0x3bef7510, 0x3beefdc0
Caught exception in looking up space point ptr... 0x258b6c40, 0x258b6b10
Caught exception in looking up space point ptr... 0x258b6b10, 0x3adb0fc0
Caught exception in looking up space point ptr... 0x245c87a0, 0x245c8980
Caught exception in looking up space point ptr... 0x245c8980, 0x245c87a0
Caught exception in looking up space point ptr... 0x2b422720, 0x42b3a0a0
Caught exception in looking up space point ptr... 0x42b3a0a0, 0x42b3a140
Caught exception in looking up space point ptr... 0x6404fa10, 0x6404f7c0
Caught exception in looking up space point ptr... 0x6404f7c0, 0x6404fa10
Caught exception in looking up space point ptr... 0x87a53d0, 0x87b0c30
Caught exception in looking up space point ptr... 0x87b0c30, 0x87a53d0
Caught exception in looking up space point ptr... 0x3ab90410, 0x3ab905a0
Caught exception in looking up space point ptr... 0x3ab905a0, 0x3ab8ff60
Caught exception in looking up space point ptr... 0x2c80d490, 0x2c80d100
Caught exception in looking up space point ptr... 0x26cbd680, 0x2c80d490
Caught exception in looking up space point ptr... 0x2b248ae0, 0x2b248a40
Caught exception in looking up space point ptr... 0x2b247190, 0x2b248ae0
Caught exception in looking up space point ptr... 0x4f05ca30, 0x5d6c9bd0
Caught exception in looking up space point ptr... 0x5d6c9bd0, 0x4f05ca30
Caught exception in looking up space point ptr... 0x21294970, 0x21294b00
Caught exception in looking up space point ptr... 0x21294b00, 0x21294e20
Caught exception in looking up space point ptr... 0x258aba60, 0x258ab880
Caught exception in looking up space point ptr... 0x24422920, 0x258aba60
Caught exception in looking up space point ptr... 0x42f8d8a0, 0x42f8d6e0
Caught exception in looking up space point ptr... 0x42f8d6e0, 0x2e9cad00
Caught exception in looking up space point ptr... 0x3ab8da00, 0x3ab8dad0
Caught exception in looking up space point ptr... 0x3ab8dad0, 0x3ab8dec0
Caught exception in looking up space point ptr... 0x2b4016b0, 0x2b4015d0
Caught exception in looking up space point ptr... 0x2b4015d0, 0x2b401790
Caught exception in looking up space point ptr... 0x42b586f0, 0x42b58550
Caught exception in looking up space point ptr... 0x42b58550, 0x42b57970
Caught exception in looking up space point ptr... 0x2f2cf290, 0x12a67d80
Caught exception in looking up space point ptr... 0x12a67d80, 0x12a67fe0
Caught exception in looking up space point ptr... 0x40163db0, 0x40163b50
Caught exception in looking up space point ptr... 0x40163b50, 0x40163a20
Caught exception in looking up space point ptr... 0x2b23d790, 0x2b23d530
Caught exception in looking up space point ptr... 0x2b23d530, 0x2b23d660
Caught exception in looking up space point ptr... 0x16479e30, 0x258ccf30
Caught exception in looking up space point ptr... 0x258ccf30, 0x258cd2c0
Caught exception in looking up space point ptr... 0x31b17820, 0x46d05c00
Caught exception in looking up space point ptr... 0x46d05c00, 0x31b17900
Caught exception in looking up space point ptr... 0x45cb1730, 0x29917dd0
Caught exception in looking up space point ptr... 0x29917dd0, 0x45cb1730
Caught exception in looking up space point ptr... 0xe488530, 0xe488680
Caught exception in looking up space point ptr... 0xe488680, 0xe488530
Caught exception in looking up space point ptr... 0x32404510, 0x324041f0
Caught exception in looking up space point ptr... 0x324041f0, 0x32404060
Caught exception in looking up space point ptr... 0x4d2f5140, 0x4d2f5010
Caught exception in looking up space point ptr... 0x4d2f5010, 0x4d2f5140
Caught exception in looking up space point ptr... 0x40158860, 0x40153bf0
Caught exception in looking up space point ptr... 0x40153bf0, 0x40158310
Caught exception in looking up space point ptr... 0x42b3ec90, 0x42b3e4f0
Caught exception in looking up space point ptr... 0x42b3e4f0, 0x42b3ec90
Caught exception in looking up space point ptr... 0x2c852190, 0x2c6ef860
Caught exception in looking up space point ptr... 0x2c6ef860, 0x2c852190
Caught exception in looking up space point ptr... 0x2c852190, 0x4d375dd0
Caught exception in looking up space point ptr... 0x4d375dd0, 0x2c852190
Caught exception in looking up space point ptr... 0x29a90770, 0x29a90a90
Caught exception in looking up space point ptr... 0x29a90a90, 0x29a90770
++++>>>> total num hits: 94813, num free: 92335
04-Feb-2026 00:21:27 CET  Closed output file "np02vd_raw_run042421_0083_df-s04-d0_dw_0_20260203T141553_reco_stage0_20260203T232127_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         60.3516       74.237        88.7206       73.502        6.97778        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       3.9571e-05    6.90128e-05   0.000172652   5.8321e-05    3.17369e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                3.2265        3.87345       4.78561       3.86067       0.33176        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.0107674     0.015026      0.0301035     0.0127938    0.00499051       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000369135   0.000589321   0.00188789    0.000457755   0.000353825      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00140608    0.00195652    0.00577451    0.00178459    0.000787073      27     
produce:beamevent:BeamEvent                      4.6181e-05    9.16796e-05   0.000753479   6.7231e-05    0.000130538      27     
produce:pdvddaphne:DAPHNEReaderPDVD                2.00834       2.4745        2.97897       2.42938      0.293722        27     
produce:ophit:OpHitFinder                         0.0211298     0.0326245     0.0428933     0.0320885    0.00564393       27     
produce:wclsdatavd:WireCellToolkit                 46.2553       52.3355       59.5525       50.9879       3.58709        27     
produce:gaushit:GausHitFinder                     0.387446      0.776014       1.45516      0.686707      0.277878        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000129142   0.000231938   0.000632128   0.000210392   9.65006e-05      27     
produce:reco3d:SpacePointSolver                    3.5351        9.90674       18.3372       9.63097       4.62518        27     
produce:hitpdune:DisambigFromSpacePoints          0.0270985     0.081556      0.134761      0.0804305     0.028287        27     
produce:cluster3d:Cluster3D                       0.407226       1.5668        3.12557       1.46805      0.691797        27     
[art]:TriggerResults:TriggerResultInserter       1.5541e-05    2.57856e-05   8.7901e-05     2.293e-05    1.29909e-05      27     
end_path:out1:RootOutput                          2.98e-06     6.31489e-06   2.6891e-05     4.11e-06     4.92744e-06      27     
end_path:out1:RootOutput(write)                    2.62797       3.15756       4.04945       3.10719      0.393735        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4640.39 MB
  Peak resident set size usage (VmHWM): 2678.44 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0083_df-s04-d0_dw_0_20260203T141553_reco_stage0_20260203T232127_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0083_df-s04-d0_dw_0_20260203T141553_reco_stage0_20260203T232127_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0083_df-s04-d0_dw_0_20260203T141553_reco_stage0_20260203T232127_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770158844.0,
    "core.end_time": 1770160888.0,
    "core.events": [
      13504,
      13510,
      13516,
      13522,
      13528,
      13534,
      13540,
      13546,
      13552,
      13558,
      13564,
      13570,
      13576,
      13582,
      13588,
      13594,
      13600,
      13606,
      13612,
      13618,
      13624,
      13630,
      13636,
      13642,
      13648,
      13654,
      13660
    ],
    "core.event_count": 27,
    "core.first_event_number": 13504,
    "core.last_event_number": 13660,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0083_df-s04-d0_dw_0_20260203T141553.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:42:59 UTC       justIN version: 01.06.00