justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303653.177@dunegpschedd01.fnal.gov

Jobsub ID303653.177@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:52:24
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce509
Last heartbeat2026-02-04 00:21:18
From worker nodeHostnameb9p34p1022.cern.ch
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:48:23
Input filesvd-protodune:np02vd_raw_run042421_0030_df-s04-d1_dw_0_20260203T134716.hdf5
JobscriptExit code0
Real time30m (1841s)
CPU time29m (1766s = 95%)
Max RSS bytes2567020544 (2448 MiB)
Outputting started2026-02-04 00:19:05
Output files
Finished2026-02-04 00:21:18
Saved logsjustin-logs:303653.177-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pace point ptr... 0x2a8211f0, 0x5d3e8ec0
Caught exception in looking up space point ptr... 0x559c0ac0, 0xe59fc90
Caught exception in looking up space point ptr... 0xe59fc90, 0xe59fb50
Caught exception in looking up space point ptr... 0x2ac5e7c0, 0x13eacc40
Caught exception in looking up space point ptr... 0x2ac5e630, 0x2ac5e7c0
Caught exception in looking up space point ptr... 0x1324b630, 0x30742070
Caught exception in looking up space point ptr... 0x30742070, 0x2c897800
Caught exception in looking up space point ptr... 0x2ac40eb0, 0x2ac4fb20
Caught exception in looking up space point ptr... 0x2ac40d20, 0x2ac40eb0
Caught exception in looking up space point ptr... 0x468f7d80, 0xe6fd700
Caught exception in looking up space point ptr... 0x468f7bf0, 0x468f7d80
Caught exception in looking up space point ptr... 0x53d67840, 0x53d67a30
Caught exception in looking up space point ptr... 0x53d67a30, 0xe108d40
Caught exception in looking up space point ptr... 0xaf7b990, 0xaf79ab0
Caught exception in looking up space point ptr... 0xaf79ab0, 0xaf77690
Caught exception in looking up space point ptr... 0x2cc89030, 0x533e6970
Caught exception in looking up space point ptr... 0x533e6970, 0x14e9a150
Caught exception in looking up space point ptr... 0x488adc00, 0x28d39e70
Caught exception in looking up space point ptr... 0x3d8d8b70, 0x488adc00
Caught exception in looking up space point ptr... 0x13ebff90, 0x13ecb3d0
Caught exception in looking up space point ptr... 0x13ebfe00, 0x13ebff90
Caught exception in looking up space point ptr... 0x2953ecb0, 0x52b6e330
Caught exception in looking up space point ptr... 0x2953c980, 0x2953ecb0
Caught exception in looking up space point ptr... 0x534b0f20, 0x155b1c40
Caught exception in looking up space point ptr... 0x534b0d90, 0x534b0f20
Caught exception in looking up space point ptr... 0x36a91880, 0x42b03bf0
Caught exception in looking up space point ptr... 0x42b03bf0, 0x36a91880
Caught exception in looking up space point ptr... 0x36a91880, 0x30cc6320
Caught exception in looking up space point ptr... 0x30cc6320, 0x36a91880
Caught exception in looking up space point ptr... 0xe686b60, 0x4ef10900
Caught exception in looking up space point ptr... 0x4ef10900, 0x560356c0
Caught exception in looking up space point ptr... 0xaf691f0, 0xaf6f570
Caught exception in looking up space point ptr... 0xaf6f570, 0xaf691f0
Caught exception in looking up space point ptr... 0x34e73dd0, 0x34e73600
Caught exception in looking up space point ptr... 0x34e73600, 0x34e73dd0
Caught exception in looking up space point ptr... 0x34e73dd0, 0x34e73790
Caught exception in looking up space point ptr... 0x34e73790, 0x34e73dd0
Caught exception in looking up space point ptr... 0x34e73dd0, 0x34e73c40
Caught exception in looking up space point ptr... 0x34e73c40, 0x34e73dd0
Caught exception in looking up space point ptr... 0x4613e4f0, 0x42727d10
Caught exception in looking up space point ptr... 0x42727d10, 0x4613e4f0
Caught exception in looking up space point ptr... 0x148c3e10, 0x2caf8e70
Caught exception in looking up space point ptr... 0x2caf8e70, 0x38d92db0
Caught exception in looking up space point ptr... 0x38d92db0, 0x38d92f90
Caught exception in looking up space point ptr... 0x38d92f90, 0x148c3e10
Caught exception in looking up space point ptr... 0x304b5530, 0x304ae170
Caught exception in looking up space point ptr... 0x304ae170, 0x128e0440
Caught exception in looking up space point ptr... 0x53e6c680, 0x38d93350
Caught exception in looking up space point ptr... 0x38d93350, 0x2c930910
Caught exception in looking up space point ptr... 0x2c930910, 0x38d93170
Caught exception in looking up space point ptr... 0x4262faf0, 0x4262f860
Caught exception in looking up space point ptr... 0x4262f860, 0x4262ff90
Caught exception in looking up space point ptr... 0x36c6f1a0, 0x32bbc370
Caught exception in looking up space point ptr... 0x32bbc370, 0x36c6f1a0
Caught exception in looking up space point ptr... 0x526c2fa0, 0x4b5b3000
Caught exception in looking up space point ptr... 0x4b5b3000, 0x526cb560
Caught exception in looking up space point ptr... 0x4262bc20, 0x42621e80
Caught exception in looking up space point ptr... 0x42621e80, 0x3c842a20
Caught exception in looking up space point ptr... 0x2a3dce60, 0x2c107710
Caught exception in looking up space point ptr... 0x2c107710, 0x2a3dce60
Caught exception in looking up space point ptr... 0x26b335e0, 0x26b32ef0
Caught exception in looking up space point ptr... 0x26b33390, 0x26b335e0
Caught exception in looking up space point ptr... 0xe593f90, 0xe5940d0
Caught exception in looking up space point ptr... 0xe5940d0, 0xe594210
Caught exception in looking up space point ptr... 0x14c4de20, 0x396b1710
Caught exception in looking up space point ptr... 0x396b1710, 0x14c4de20
++++>>>> total num hits: 64928, num free: 62577
04-Feb-2026 01:18:58 CET  Closed output file "np02vd_raw_run042421_0030_df-s04-d1_dw_0_20260203T134716_reco_stage0_20260204T001858_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         60.975        66.0671       73.9284       65.3225       3.28582        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       6.6139e-05    9.44622e-05   0.000233269   8.1031e-05    3.52765e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                3.49777       4.11355       5.99947       4.10465      0.408107        27     
produce:triggerrawdecoder:PDVDTriggerReader4     0.00895152     0.0351419     0.188177      0.0181943     0.0381147       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000358657   0.00133855    0.00853009    0.000584815   0.00195963       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00118265    0.00202763    0.00420058    0.00170173    0.000835962      27     
produce:beamevent:BeamEvent                      7.7656e-05    0.000169662   0.00151891    0.00012055    0.000265625      27     
produce:pdvddaphne:DAPHNEReaderPDVD                1.82128       2.04826       2.32652       2.05209      0.118265        27     
produce:ophit:OpHitFinder                         0.0242019     0.0327745     0.0404507     0.0337668    0.00430826       27     
produce:wclsdatavd:WireCellToolkit                 43.5468       47.7103       51.2878       47.7469       1.92421        27     
produce:gaushit:GausHitFinder                     0.363409      0.723561       1.05724      0.689227      0.187305        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000128652   0.000178529   0.000346248   0.000173169   4.23687e-05      27     
produce:reco3d:SpacePointSolver                    4.15596       6.33597       10.1225       6.22376       1.50289        27     
produce:hitpdune:DisambigFromSpacePoints          0.0411673     0.0667351     0.112006      0.0663649     0.0196914       27     
produce:cluster3d:Cluster3D                       0.656973       1.80061       8.73556       1.21445       1.75682        27     
[art]:TriggerResults:TriggerResultInserter       2.4186e-05    3.66686e-05   7.8367e-05    3.5513e-05    1.1621e-05       27     
end_path:out1:RootOutput                          5.278e-06    8.05656e-06   2.1112e-05     7.561e-06    2.86195e-06      27     
end_path:out1:RootOutput(write)                    2.59948       3.17132       3.65422       3.17615      0.261622        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4697.62 MB
  Peak resident set size usage (VmHWM): 2567.02 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0030_df-s04-d1_dw_0_20260203T134716_reco_stage0_20260204T001858_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0030_df-s04-d1_dw_0_20260203T134716_reco_stage0_20260204T001858_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0030_df-s04-d1_dw_0_20260203T134716_reco_stage0_20260204T001858_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162511.0,
    "core.end_time": 1770164340.0,
    "core.events": [
      4913,
      4919,
      4925,
      4931,
      4937,
      4943,
      4949,
      4955,
      4961,
      4967,
      4973,
      4979,
      4985,
      4991,
      4997,
      5003,
      5009,
      5015,
      5021,
      5027,
      5033,
      5039,
      5045,
      5051,
      5057,
      5063,
      5069
    ],
    "core.event_count": 27,
    "core.first_event_number": 4913,
    "core.last_event_number": 5069,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0030_df-s04-d1_dw_0_20260203T134716.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:42:55 UTC       justIN version: 01.06.00