justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303647.54@dunegpschedd01.fnal.gov

Jobsub ID303647.54@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:46:24
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce514
Last heartbeat2026-02-03 23:32:35
From worker nodeHostnameb9p06p2461.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 22:47:17
Input filesvd-protodune:np02vd_raw_run042421_0072_df-s03-d1_dw_0_20260203T140959.hdf5
JobscriptExit code0
Real time42m (2572s)
CPU time40m (2403s = 93%)
Max RSS bytes2578825216 (2459 MiB)
Outputting started2026-02-03 23:30:10
Output files
Finished2026-02-03 23:32:35
Saved logsjustin-logs:303647.54-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

. 0x4f002350, 0x4f000b40
Caught exception in looking up space point ptr... 0x4f000b40, 0x4f002350
Caught exception in looking up space point ptr... 0x2695ff60, 0x2695fab0
Caught exception in looking up space point ptr... 0x2695fab0, 0x2695fdd0
Caught exception in looking up space point ptr... 0xfdcde70, 0xfdce000
Caught exception in looking up space point ptr... 0xfdce000, 0xfdcde70
Caught exception in looking up space point ptr... 0xaa952e0, 0xaa94670
Caught exception in looking up space point ptr... 0xaa94670, 0xaa952e0
Caught exception in looking up space point ptr... 0xa1bcdf0, 0xa1bad50
Caught exception in looking up space point ptr... 0xa1bad50, 0xa1bcdf0
Caught exception in looking up space point ptr... 0xf1ad150, 0xf1aeee0
Caught exception in looking up space point ptr... 0xf1aeee0, 0xf1aedb0
Caught exception in looking up space point ptr... 0x3e828960, 0x4ae9a950
Caught exception in looking up space point ptr... 0x4ae9a950, 0x3e828960
Caught exception in looking up space point ptr... 0x93a74f0, 0x93a7240
Caught exception in looking up space point ptr... 0x93a7240, 0x93a6fb0
Caught exception in looking up space point ptr... 0x50d07170, 0x42d4b8e0
Caught exception in looking up space point ptr... 0x42d4b8e0, 0x50d07170
Caught exception in looking up space point ptr... 0x3b4b1bf0, 0x361486b0
Caught exception in looking up space point ptr... 0x361486b0, 0x14af9530
Caught exception in looking up space point ptr... 0x4f00b4f0, 0x4f00b140
Caught exception in looking up space point ptr... 0x4f00b140, 0x4f00b4f0
Caught exception in looking up space point ptr... 0x6022c0d0, 0x4d8b5030
Caught exception in looking up space point ptr... 0x6022c220, 0x6022c0d0
Caught exception in looking up space point ptr... 0xef3f780, 0xef3f910
Caught exception in looking up space point ptr... 0xef3f910, 0xef3f780
Caught exception in looking up space point ptr... 0x430beba0, 0x430bef60
Caught exception in looking up space point ptr... 0x430bef60, 0x430bf280
Caught exception in looking up space point ptr... 0x52dc3060, 0x52dc31f0
Caught exception in looking up space point ptr... 0x52dc31f0, 0x52dc3380
Caught exception in looking up space point ptr... 0x52ab0ad0, 0x52aad0b0
Caught exception in looking up space point ptr... 0x52aad0b0, 0x52ab0ad0
Caught exception in looking up space point ptr... 0x1224cf70, 0x334dd850
Caught exception in looking up space point ptr... 0x334dd850, 0x1224cf70
Caught exception in looking up space point ptr... 0x4ae72890, 0x4ae72750
Caught exception in looking up space point ptr... 0x4ae72750, 0x4ae72890
Caught exception in looking up space point ptr... 0xdf555f0, 0x10ec7d90
Caught exception in looking up space point ptr... 0x10ec7d90, 0x10ec64c0
Caught exception in looking up space point ptr... 0x5594dca0, 0x15719a20
Caught exception in looking up space point ptr... 0x15719a20, 0x5594dca0
Caught exception in looking up space point ptr... 0x36120fe0, 0x3611def0
Caught exception in looking up space point ptr... 0x36121230, 0x36120fe0
Caught exception in looking up space point ptr... 0x4fe9e470, 0x4fe9e7e0
Caught exception in looking up space point ptr... 0x4fe9e7e0, 0x4fe9e470
Caught exception in looking up space point ptr... 0x4c0536b0, 0x4c053520
Caught exception in looking up space point ptr... 0x4c053520, 0x4c0536b0
Caught exception in looking up space point ptr... 0x504ccbb0, 0x504cc940
Caught exception in looking up space point ptr... 0x504cc940, 0x504ccbb0
Caught exception in looking up space point ptr... 0x504ccbb0, 0x504cb330
Caught exception in looking up space point ptr... 0x504cb330, 0x504ccbb0
Caught exception in looking up space point ptr... 0x2fbd3fb0, 0x2fbd4f30
Caught exception in looking up space point ptr... 0x2fbd4f30, 0x2fbd3fb0
Caught exception in looking up space point ptr... 0xb0098d0, 0xaad2270
Caught exception in looking up space point ptr... 0xaad2270, 0xaae51b0
Caught exception in looking up space point ptr... 0x52c06070, 0x52c05ee0
Caught exception in looking up space point ptr... 0x52c05ee0, 0x52c06070
Caught exception in looking up space point ptr... 0x407abe20, 0x403237d0
Caught exception in looking up space point ptr... 0x403237d0, 0x407abe20
Caught exception in looking up space point ptr... 0x14a43210, 0x14a432f0
Caught exception in looking up space point ptr... 0x14a432f0, 0x14a43210
Caught exception in looking up space point ptr... 0xdcc26c0, 0x45852390
Caught exception in looking up space point ptr... 0x45852390, 0xdcc26c0
Caught exception in looking up space point ptr... 0x217fb670, 0x10dbfba0
Caught exception in looking up space point ptr... 0x10dbfba0, 0x217fb670
Caught exception in looking up space point ptr... 0x3eccb190, 0x28fbd530
Caught exception in looking up space point ptr... 0x28fbd530, 0x3eccb190
++++>>>> total num hits: 45842, num free: 43780
04-Feb-2026 00:30:00 CET  Closed output file "np02vd_raw_run042421_0072_df-s03-d1_dw_0_20260203T140959_reco_stage0_20260203T233000_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         76.1952       92.457        120.786       93.8378       9.09646        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        5.385e-05    0.000116966   0.000268593   0.000100831   4.79967e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                4.11165       6.50228       22.5395       4.60532       5.12923        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.0101751     0.0149477     0.0326845     0.0130675    0.00466769       27     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000420665   0.00254883     0.0531635    0.000537816   0.00992897       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder          0.00156707    0.00201938    0.00403923    0.00185282    0.000474725      27     
produce:beamevent:BeamEvent                      6.2191e-05    0.00016339    0.00115968    0.000127102   0.000197508      27     
produce:pdvddaphne:DAPHNEReaderPDVD                 2.524        2.9833        3.41606       3.04573      0.261033        27     
produce:ophit:OpHitFinder                         0.0268855     0.0379873     0.0507895     0.038575     0.00569438       27     
produce:wclsdatavd:WireCellToolkit                 54.7728       66.0726       74.4004       68.0395       6.08688        27     
produce:gaushit:GausHitFinder                     0.446348      0.958266       1.61677      0.943965      0.254473        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000173582   0.000271223   0.000461725   0.000256592   7.05705e-05      27     
produce:reco3d:SpacePointSolver                    4.86361       9.94396       15.5942       9.6706        2.78365        27     
produce:hitpdune:DisambigFromSpacePoints          0.0404454     0.103442      0.207533      0.0978205     0.0350256       27     
produce:cluster3d:Cluster3D                        0.60644       1.70989       3.4693        1.68894      0.604873        27     
[art]:TriggerResults:TriggerResultInserter        2.112e-05    3.65549e-05   0.000127882   3.2341e-05    1.92068e-05      27     
end_path:out1:RootOutput                          4.15e-06     8.63159e-06    3.651e-05     7.73e-06     5.66589e-06      27     
end_path:out1:RootOutput(write)                    3.14499       4.10234       5.07925       4.20427      0.542056        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4540.17 MB
  Peak resident set size usage (VmHWM): 2578.83 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0072_df-s03-d1_dw_0_20260203T140959_reco_stage0_20260203T233000_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0072_df-s03-d1_dw_0_20260203T140959_reco_stage0_20260203T233000_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0072_df-s03-d1_dw_0_20260203T140959_reco_stage0_20260203T233000_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770158851.0,
    "core.end_time": 1770161402.0,
    "core.events": [
      11727,
      11733,
      11739,
      11745,
      11751,
      11757,
      11763,
      11769,
      11775,
      11781,
      11787,
      11793,
      11799,
      11805,
      11811,
      11817,
      11823,
      11829,
      11835,
      11841,
      11847,
      11853,
      11859,
      11865,
      11871,
      11877,
      11883
    ],
    "core.event_count": 27,
    "core.first_event_number": 11727,
    "core.last_event_number": 11883,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0072_df-s03-d1_dw_0_20260203T140959.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:07:44 UTC       justIN version: 01.06.00