justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297919.126@dunegpschedd02.fnal.gov

Jobsub ID297919.126@dunegpschedd02.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:54:25
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2026-02-04 00:12:33
From worker nodeHostnameb9p02p6824.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:32:59
Input filesvd-protodune:np02vd_raw_run042421_0035_df-s03-d1_dw_0_20260203T134957.hdf5
JobscriptExit code0
Real time37m (2237s)
CPU time35m (2146s = 95%)
Max RSS bytes2653253632 (2530 MiB)
Outputting started2026-02-04 00:10:17
Output files
Finished2026-02-04 00:12:33
Saved logsjustin-logs:297919.126-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e point ptr... 0x47f55d40, 0xa464900
Caught exception in looking up space point ptr... 0xa464900, 0x47f55d40
Caught exception in looking up space point ptr... 0x47f55d40, 0x47f55ba0
Caught exception in looking up space point ptr... 0x47f55ba0, 0x47f55d40
Caught exception in looking up space point ptr... 0x56a813d0, 0x56a7b8e0
Caught exception in looking up space point ptr... 0x12848bf0, 0x56a813d0
Caught exception in looking up space point ptr... 0x3346b950, 0x339b1120
Caught exception in looking up space point ptr... 0x339b1120, 0x76560c0
Caught exception in looking up space point ptr... 0x4e950910, 0x508b17c0
Caught exception in looking up space point ptr... 0x508b17c0, 0x4b4bd930
Caught exception in looking up space point ptr... 0x4b4bd930, 0x4e943700
Caught exception in looking up space point ptr... 0x4af82ef0, 0x4af82bd0
Caught exception in looking up space point ptr... 0x4af87c70, 0x4af82ef0
Caught exception in looking up space point ptr... 0x3399a410, 0x4afe69c0
Caught exception in looking up space point ptr... 0x4afe69c0, 0x55f38ff0
Caught exception in looking up space point ptr... 0x350c7820, 0x350c6380
Caught exception in looking up space point ptr... 0x350c6380, 0x350c34b0
Caught exception in looking up space point ptr... 0x350c8cc0, 0x350c7820
Caught exception in looking up space point ptr... 0x43e4e910, 0x43e4e810
Caught exception in looking up space point ptr... 0x43e4e810, 0x43e4e910
Caught exception in looking up space point ptr... 0x47f48920, 0x47f42700
Caught exception in looking up space point ptr... 0x47f42700, 0x47f48920
Caught exception in looking up space point ptr... 0x62ee3f0, 0x62d2bc0
Caught exception in looking up space point ptr... 0x62d2bc0, 0x62ed8d0
Caught exception in looking up space point ptr... 0x5d0f6610, 0x5858b5f0
Caught exception in looking up space point ptr... 0x5858b5f0, 0x5d0f6610
Caught exception in looking up space point ptr... 0x56a70c90, 0x56a70a30
Caught exception in looking up space point ptr... 0x56a70a30, 0x56a70b60
Caught exception in looking up space point ptr... 0x606c8750, 0x606c43e0
Caught exception in looking up space point ptr... 0x606c43e0, 0x606c8750
Caught exception in looking up space point ptr... 0x3849d6e0, 0x3848b970
Caught exception in looking up space point ptr... 0x3848b970, 0x384912b0
Caught exception in looking up space point ptr... 0x4af84790, 0x5107f0a0
Caught exception in looking up space point ptr... 0x29a9c150, 0x4af84790
Caught exception in looking up space point ptr... 0xa465d80, 0xa465370
Caught exception in looking up space point ptr... 0xa465370, 0xa465d80
Caught exception in looking up space point ptr... 0x44ab2160, 0x44ab1f00
Caught exception in looking up space point ptr... 0x44ab1f00, 0x44ab2160
Caught exception in looking up space point ptr... 0x44ab2160, 0x44ab2030
Caught exception in looking up space point ptr... 0x44ab2030, 0x44ab2160
Caught exception in looking up space point ptr... 0x56a65020, 0x56a64f40
Caught exception in looking up space point ptr... 0x56a64f40, 0x56a65020
Caught exception in looking up space point ptr... 0x2fe070d0, 0x2fe08630
Caught exception in looking up space point ptr... 0x2fe08630, 0x2fe08950
Caught exception in looking up space point ptr... 0x31c28f50, 0x2707a7f0
Caught exception in looking up space point ptr... 0x2707a7f0, 0x2a2da950
Caught exception in looking up space point ptr... 0x102fc720, 0x4616b990
Caught exception in looking up space point ptr... 0x4616b990, 0x102fc720
Caught exception in looking up space point ptr... 0x102fc720, 0x393e68c0
Caught exception in looking up space point ptr... 0x393e68c0, 0x102fc720
Caught exception in looking up space point ptr... 0x352f9670, 0x2d8f5d30
Caught exception in looking up space point ptr... 0x2d8f5d30, 0x352f9670
Caught exception in looking up space point ptr... 0x9892c10, 0x98927e0
Caught exception in looking up space point ptr... 0x98927e0, 0x9892c10
Caught exception in looking up space point ptr... 0x60dc5ed0, 0x31bee380
Caught exception in looking up space point ptr... 0x31bee380, 0x60dc5ed0
Caught exception in looking up space point ptr... 0x2fdfdb00, 0x2fdfd7e0
Caught exception in looking up space point ptr... 0x2fdfd7e0, 0x2fdfdb00
Caught exception in looking up space point ptr... 0x349d91d0, 0x349da940
Caught exception in looking up space point ptr... 0x349da940, 0x349d91d0
Caught exception in looking up space point ptr... 0x56a64770, 0x56a64610
Caught exception in looking up space point ptr... 0x56a64610, 0x56a64770
Caught exception in looking up space point ptr... 0x4af8e120, 0x4af907c0
Caught exception in looking up space point ptr... 0x4af907c0, 0x4af8e120
Caught exception in looking up space point ptr... 0x4c9d50a0, 0x4c9d53c0
Caught exception in looking up space point ptr... 0x4c9d53c0, 0x4c9d50a0
++++>>>> total num hits: 62485, num free: 59669
04-Feb-2026 01:10:10 CET  Closed output file "np02vd_raw_run042421_0035_df-s03-d1_dw_0_20260203T134957_reco_stage0_20260204T001010_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         66.5726       80.521        97.4645       79.4766       7.66279        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       4.4541e-05    9.42774e-05   0.000353553   7.6211e-05    6.35534e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                3.77334       4.30605       4.77662       4.3073       0.260222        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.0102919     0.0138538     0.0431278     0.012343     0.00618292       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000390343   0.000501601   0.000777226   0.000475614   8.76507e-05      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00146443    0.00197524    0.00316689    0.00181683    0.000456502      27     
produce:beamevent:BeamEvent                       5.004e-05    0.000116737   0.000959488   7.6691e-05    0.000168024      27     
produce:pdvddaphne:DAPHNEReaderPDVD                2.14019       2.79751       3.23152       2.87582      0.317515        27     
produce:ophit:OpHitFinder                         0.0249184     0.037656      0.0455803     0.0378441    0.00497565       27     
produce:wclsdatavd:WireCellToolkit                 49.7924       57.1747       68.0515       56.7612       4.25511        27     
produce:gaushit:GausHitFinder                     0.494602      0.929463       1.71147      0.856015      0.322514        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000137871   0.000240186   0.000431244   0.000234902   6.92083e-05      27     
produce:reco3d:SpacePointSolver                    3.89244       10.1553       18.6688       10.1884       3.6441         27     
produce:hitpdune:DisambigFromSpacePoints          0.0478261     0.092294      0.221597      0.0759566     0.042848        27     
produce:cluster3d:Cluster3D                       0.684307       1.6761        3.8972        1.52584      0.753959        27     
[art]:TriggerResults:TriggerResultInserter        1.643e-05    2.98054e-05   0.000115241    2.527e-05    1.79702e-05      27     
end_path:out1:RootOutput                          3.91e-06     7.73044e-06    2.782e-05     6.57e-06     4.62707e-06      27     
end_path:out1:RootOutput(write)                    2.74637       3.31784       4.27633       3.33165      0.401044        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4570.35 MB
  Peak resident set size usage (VmHWM): 2653.25 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0035_df-s03-d1_dw_0_20260203T134957_reco_stage0_20260204T001010_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0035_df-s03-d1_dw_0_20260203T134957_reco_stage0_20260204T001010_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0035_df-s03-d1_dw_0_20260203T134957_reco_stage0_20260204T001010_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770161590.0,
    "core.end_time": 1770163811.0,
    "core.events": [
      5721,
      5727,
      5733,
      5739,
      5745,
      5751,
      5757,
      5763,
      5769,
      5775,
      5781,
      5787,
      5793,
      5799,
      5805,
      5811,
      5817,
      5823,
      5829,
      5835,
      5841,
      5847,
      5853,
      5859,
      5865,
      5871,
      5877
    ],
    "core.event_count": 27,
    "core.first_event_number": 5721,
    "core.last_event_number": 5877,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0035_df-s03-d1_dw_0_20260203T134957.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:45:26 UTC       justIN version: 01.06.00