justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.79@dunegpschedd02.fnal.gov

Jobsub ID297923.79@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-02-04 00:55:36
From worker nodeHostnameb9p30p1409.cern.ch
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:59:07
Input filesvd-protodune:np02vd_raw_run042433_0503_df-s02-d1_dw_0_20260203T215318.hdf5
JobscriptExit code0
Real time53m (3239s)
CPU time51m (3099s = 95%)
Max RSS bytes3392151552 (3235 MiB)
Outputting started2026-02-04 00:53:08
Output files
Finished2026-02-04 00:55:36
Saved logsjustin-logs:297923.79-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e point ptr... 0x1826f410, 0x18291820
Caught exception in looking up space point ptr... 0x18291820, 0x182808d0
Caught exception in looking up space point ptr... 0x1827f470, 0x1826ec50
Caught exception in looking up space point ptr... 0x38b47fa0, 0x38b48110
Caught exception in looking up space point ptr... 0x38b48110, 0x38b47fa0
Caught exception in looking up space point ptr... 0x82b0ac50, 0x82b0a140
Caught exception in looking up space point ptr... 0x3cece770, 0x82b0ac50
Caught exception in looking up space point ptr... 0x40419470, 0x40419230
Caught exception in looking up space point ptr... 0x40419230, 0x404195b0
Caught exception in looking up space point ptr... 0x71d72040, 0x71d74e30
Caught exception in looking up space point ptr... 0x71d74e30, 0x71d74170
Caught exception in looking up space point ptr... 0x168eb2e0, 0x335b4400
Caught exception in looking up space point ptr... 0x335b4400, 0x7ffa5b40
Caught exception in looking up space point ptr... 0xe032fd0, 0xe0323e0
Caught exception in looking up space point ptr... 0xd0d58c0, 0xe032fd0
Caught exception in looking up space point ptr... 0xe03dc80, 0xe02a190
Caught exception in looking up space point ptr... 0xe02a190, 0xe025520
Caught exception in looking up space point ptr... 0x40571300, 0x62b401a0
Caught exception in looking up space point ptr... 0x62b401a0, 0x40571300
Caught exception in looking up space point ptr... 0xd7ac2c0, 0x304f9910
Caught exception in looking up space point ptr... 0x304f9910, 0xd7ac2c0
Caught exception in looking up space point ptr... 0x59b0c040, 0x59b0c640
Caught exception in looking up space point ptr... 0x59b0c640, 0x59b0c040
Caught exception in looking up space point ptr... 0x80c07630, 0x80c073d0
Caught exception in looking up space point ptr... 0x80c073d0, 0x80c07630
Caught exception in looking up space point ptr... 0x15bcae00, 0x15bca1b0
Caught exception in looking up space point ptr... 0x15bca1b0, 0x15bcae00
Caught exception in looking up space point ptr... 0x4e6546b0, 0x548de690
Caught exception in looking up space point ptr... 0x548de690, 0x4e6546b0
Caught exception in looking up space point ptr... 0x55039a80, 0x55039880
Caught exception in looking up space point ptr... 0x55039880, 0x55039780
Caught exception in looking up space point ptr... 0x85d3b390, 0x85d3b730
Caught exception in looking up space point ptr... 0x85d3b730, 0x3bea85d0
Caught exception in looking up space point ptr... 0x51250480, 0x13330ac0
Caught exception in looking up space point ptr... 0x13330ac0, 0x51250480
Caught exception in looking up space point ptr... 0x6b9d09c0, 0x6b9d0720
Caught exception in looking up space point ptr... 0x6b9d0720, 0x6b9d0870
Caught exception in looking up space point ptr... 0x19dbc310, 0x19dbb500
Caught exception in looking up space point ptr... 0x19dbb500, 0x19dbd560
Caught exception in looking up space point ptr... 0x62b958f0, 0x62b95630
Caught exception in looking up space point ptr... 0x62b95810, 0x62b958f0
Caught exception in looking up space point ptr... 0x38631e10, 0x159c5bb0
Caught exception in looking up space point ptr... 0x159c5bb0, 0x93e2f90
Caught exception in looking up space point ptr... 0x1595b560, 0x1595b1d0
Caught exception in looking up space point ptr... 0x1595b1d0, 0x1595ad10
Caught exception in looking up space point ptr... 0x64534ea0, 0x5e3de860
Caught exception in looking up space point ptr... 0x5e3de860, 0x64534ea0
Caught exception in looking up space point ptr... 0x64522a80, 0x645228f0
Caught exception in looking up space point ptr... 0x645228f0, 0x64522a80
Caught exception in looking up space point ptr... 0xd0aed40, 0x40a5f270
Caught exception in looking up space point ptr... 0x40a5f270, 0xd0aed40
Caught exception in looking up space point ptr... 0x6460d600, 0x6460d270
Caught exception in looking up space point ptr... 0x6460d270, 0x6460d3a0
Caught exception in looking up space point ptr... 0x62b316e0, 0x62b38010
Caught exception in looking up space point ptr... 0x62b38010, 0x62b316e0
Caught exception in looking up space point ptr... 0x80bfdd50, 0x80bfd7f0
Caught exception in looking up space point ptr... 0x80bfd7f0, 0x80bfdd50
Caught exception in looking up space point ptr... 0x6587dfc0, 0x6587de30
Caught exception in looking up space point ptr... 0x6587de30, 0x6587dfc0
Caught exception in looking up space point ptr... 0x5dff2fa0, 0x80ec47f0
Caught exception in looking up space point ptr... 0x80ec47f0, 0x5dff2fa0
Caught exception in looking up space point ptr... 0x6bdcd630, 0x6bdcd730
Caught exception in looking up space point ptr... 0x6bdcd730, 0x6bdcd830
Caught exception in looking up space point ptr... 0x5eaad640, 0x82ad7830
Caught exception in looking up space point ptr... 0x82ad7830, 0x5eaad640
Caught exception in looking up space point ptr... 0xca69320, 0x790212d0
Caught exception in looking up space point ptr... 0x790212d0, 0xca69190
++++>>>> total num hits: 163548, num free: 156930
04-Feb-2026 01:52:58 CET  Closed output file "np02vd_raw_run042433_0503_df-s02-d1_dw_0_20260203T215318_reco_stage0_20260204T005258_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         154.147       166.26        193.341       165.094       9.5575         19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       5.7431e-05    0.000114514   0.000207654   9.6712e-05    4.26948e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                7.0902        8.23927       10.0567       7.93268      0.870442        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.0147525     0.0398065     0.0913829     0.0272728     0.0239339       19     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000505519   0.00162714     0.0180921    0.000681022   0.00388425       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.000469588   0.000594299   0.000854865   0.00056594    0.000115682      19     
produce:beamevent:BeamEvent                      8.2062e-05    0.000169888   0.000900855   0.000111852   0.000178636      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.3682        2.96153       3.31141       2.96433      0.256027        19     
produce:ophit:OpHitFinder                         0.0239346     0.0373563     0.0485445     0.0387678    0.00604238       19     
produce:wclsdatavd:WireCellToolkit                 111.392       125.419       147.281       123.384       9.57992        19     
produce:gaushit:GausHitFinder                     0.862056       1.45035       2.11558       1.50574      0.318027        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000312435   0.000469641   0.000717312   0.000458558   0.000106058      19     
produce:reco3d:SpacePointSolver                    10.6808       19.1698       34.0561       19.1828       5.5645         19     
produce:hitpdune:DisambigFromSpacePoints          0.0908703     0.228221      0.343735      0.220088      0.0683775       19     
produce:cluster3d:Cluster3D                        1.38343       2.97524       4.48328       2.93852      0.897298        19     
[art]:TriggerResults:TriggerResultInserter        2.25e-05      3.302e-05    8.5352e-05    2.9701e-05    1.3035e-05       19     
end_path:out1:RootOutput                          5.23e-06     1.0369e-05     2.406e-05      9.5e-06     4.5269e-06       19     
end_path:out1:RootOutput(write)                    4.39077       5.69817       6.82809       5.51314      0.703515        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5324.05 MB
  Peak resident set size usage (VmHWM): 3392.15 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0503_df-s02-d1_dw_0_20260203T215318_reco_stage0_20260204T005258_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0503_df-s02-d1_dw_0_20260203T215318_reco_stage0_20260204T005258_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0503_df-s02-d1_dw_0_20260203T215318_reco_stage0_20260204T005258_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163161.0,
    "core.end_time": 1770166380.0,
    "core.events": [
      57349,
      57355,
      57361,
      57367,
      57373,
      57379,
      57385,
      57391,
      57397,
      57403,
      57409,
      57415,
      57421,
      57427,
      57433,
      57439,
      57445,
      57451,
      57457
    ],
    "core.event_count": 19,
    "core.first_event_number": 57349,
    "core.last_event_number": 57457,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0503_df-s02-d1_dw_0_20260203T215318.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:38:21 UTC       justIN version: 01.06.00