justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.15@dunegpschedd02.fnal.gov

Jobsub ID297921.15@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 00:40:24
From worker nodeHostnameheplnc171.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:45:01
Input filesvd-protodune:np02vd_raw_run042433_0170_df-s04-d0_dw_0_20260203T194643.hdf5
JobscriptExit code0
Real time52m (3173s)
CPU time48m (2902s = 91%)
Max RSS bytes3366002688 (3210 MiB)
Outputting started2026-02-04 00:37:55
Output files
Finished2026-02-04 00:40:24
Saved logsjustin-logs:297921.15-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x5183b660, 0x5674aa60
Caught exception in looking up space point ptr... 0x6e7ee830, 0xf0f3ee0
Caught exception in looking up space point ptr... 0xf0f3ee0, 0x3a79ede0
Caught exception in looking up space point ptr... 0x3cc705f0, 0x4ce6a630
Caught exception in looking up space point ptr... 0x4ce6a630, 0x3cc705f0
Caught exception in looking up space point ptr... 0x2d93c690, 0x55f2e770
Caught exception in looking up space point ptr... 0x2d93e340, 0x2d93c690
Caught exception in looking up space point ptr... 0x324d0bb0, 0x324d10d0
Caught exception in looking up space point ptr... 0x324d10d0, 0x324d08d0
Caught exception in looking up space point ptr... 0x324d08d0, 0x324d0e40
Caught exception in looking up space point ptr... 0x324d1360, 0x324d0bb0
Caught exception in looking up space point ptr... 0x6dcd96c0, 0x454431b0
Caught exception in looking up space point ptr... 0x454431b0, 0x6dcd96c0
Caught exception in looking up space point ptr... 0x3b861470, 0x3b860ef0
Caught exception in looking up space point ptr... 0x3b860ef0, 0x3f008320
Caught exception in looking up space point ptr... 0x324d36f0, 0x324d35f0
Caught exception in looking up space point ptr... 0x324d35f0, 0x324d36f0
Caught exception in looking up space point ptr... 0x6dcdb150, 0x6dcdac40
Caught exception in looking up space point ptr... 0x6dcdac40, 0x6dcdb150
Caught exception in looking up space point ptr... 0x3acd9e60, 0x6f8e01d0
Caught exception in looking up space point ptr... 0x6f8e01d0, 0x6f8e00d0
Caught exception in looking up space point ptr... 0x5674a690, 0x5674c090
Caught exception in looking up space point ptr... 0x5674c090, 0x5674a690
Caught exception in looking up space point ptr... 0x828db230, 0x828d8150
Caught exception in looking up space point ptr... 0x828d8150, 0x828db230
Caught exception in looking up space point ptr... 0x7deab1c0, 0x3e8f9760
Caught exception in looking up space point ptr... 0x3e8f9760, 0x7deab1c0
Caught exception in looking up space point ptr... 0x420c7e40, 0x324a23e0
Caught exception in looking up space point ptr... 0x324a23e0, 0x420c7e40
Caught exception in looking up space point ptr... 0x29a35470, 0x29a35720
Caught exception in looking up space point ptr... 0x29a35720, 0x29a355f0
Caught exception in looking up space point ptr... 0x3242fb10, 0x3242fc10
Caught exception in looking up space point ptr... 0x3242fc10, 0x3242fb10
Caught exception in looking up space point ptr... 0x29799640, 0x110c2340
Caught exception in looking up space point ptr... 0x110c2340, 0x29799640
Caught exception in looking up space point ptr... 0x82903450, 0x631ee780
Caught exception in looking up space point ptr... 0x631ee780, 0x82903450
Caught exception in looking up space point ptr... 0x82903450, 0x82901b30
Caught exception in looking up space point ptr... 0x82901b30, 0x82903450
Caught exception in looking up space point ptr... 0x37fdb3d0, 0x38bbb2f0
Caught exception in looking up space point ptr... 0x38bbb2f0, 0x37fdb3d0
Caught exception in looking up space point ptr... 0x36e47520, 0x369fa580
Caught exception in looking up space point ptr... 0x369fa580, 0x36e47520
Caught exception in looking up space point ptr... 0x64be6700, 0x64be6460
Caught exception in looking up space point ptr... 0x64be6460, 0x64be6700
Caught exception in looking up space point ptr... 0x44c859f0, 0x44c84b20
Caught exception in looking up space point ptr... 0x44c84b20, 0x44c885b0
Caught exception in looking up space point ptr... 0x534e6a30, 0x77dbfc0
Caught exception in looking up space point ptr... 0x77dbfc0, 0x534e6a30
Caught exception in looking up space point ptr... 0x7718a7c0, 0x7718aa10
Caught exception in looking up space point ptr... 0x7718aa10, 0x7718ab60
Caught exception in looking up space point ptr... 0x3dc2af40, 0x701b0340
Caught exception in looking up space point ptr... 0x701b0340, 0x7dea4f40
Caught exception in looking up space point ptr... 0x324a2640, 0x324a8510
Caught exception in looking up space point ptr... 0x420c82f0, 0x324a2640
Caught exception in looking up space point ptr... 0x3605d5e0, 0x3605d6b0
Caught exception in looking up space point ptr... 0x3605d6b0, 0x3605da80
Caught exception in looking up space point ptr... 0x3605da80, 0x3605d7b0
Caught exception in looking up space point ptr... 0x3605d7b0, 0x3605d5e0
Caught exception in looking up space point ptr... 0x64bff720, 0x64bff500
Caught exception in looking up space point ptr... 0x64bff500, 0x64bff720
Caught exception in looking up space point ptr... 0x9f17850, 0x9f175a0
Caught exception in looking up space point ptr... 0x9f175a0, 0x9f17850
Caught exception in looking up space point ptr... 0x420f5d20, 0x420f57d0
Caught exception in looking up space point ptr... 0x420f57d0, 0x420f5d20
Caught exception in looking up space point ptr... 0x44f01ee0, 0x44f01100
Caught exception in looking up space point ptr... 0x44f01100, 0x44f02e90
++++>>>> total num hits: 77344, num free: 74137
04-Feb-2026 00:37:47 GMT  Closed output file "np02vd_raw_run042433_0170_df-s04-d0_dw_0_20260203T194643_reco_stage0_20260204T003747_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         150.138       162.839       177.251       164.733       7.48383        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.7632e-05    0.000145832   0.000384295   0.000108412   7.11096e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                12.0928       14.7963       16.0808       14.8255      0.749092        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.524177      0.556554      0.825298      0.539124      0.0655325       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177224     0.0179839     0.0185706     0.0179475    0.000201349      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177215     0.0183986     0.0238337     0.0179553    0.00136796       19     
produce:beamevent:BeamEvent                      0.000121282   0.000228548   0.00125393    0.000174743   0.000243099      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.87998       3.28313       3.62579       3.26996      0.205921        19     
produce:ophit:OpHitFinder                         0.0238107     0.0392698     0.0450469     0.0410592    0.00491346       19     
produce:wclsdatavd:WireCellToolkit                 104.446       116.811       126.859       116.45        6.05676        19     
produce:gaushit:GausHitFinder                      1.10267       1.56926       2.14096       1.55209      0.324161        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000244453   0.000389224   0.000788281   0.000376225   0.000107578      19     
produce:reco3d:SpacePointSolver                    10.2093       16.6434       23.112        16.7557       3.70275        19     
produce:hitpdune:DisambigFromSpacePoints           0.12202      0.219358      0.321714      0.222292      0.0540657       19     
produce:cluster3d:Cluster3D                        1.5859        2.88727       4.39942        2.78        0.861049        19     
[art]:TriggerResults:TriggerResultInserter        2.625e-05    4.16183e-05   0.000137571    3.586e-05    2.35034e-05      19     
end_path:out1:RootOutput                          7.56e-06     1.10931e-05     4.2e-05      9.59e-06     7.33448e-06      19     
end_path:out1:RootOutput(write)                    4.12423       5.93741       6.7509        6.09319      0.595686        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5360.91 MB
  Peak resident set size usage (VmHWM): 3366 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0170_df-s04-d0_dw_0_20260203T194643_reco_stage0_20260204T003747_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0170_df-s04-d0_dw_0_20260203T194643_reco_stage0_20260204T003747_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0170_df-s04-d0_dw_0_20260203T194643_reco_stage0_20260204T003747_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162314.0,
    "core.end_time": 1770165469.0,
    "core.events": [
      19390,
      19396,
      19402,
      19408,
      19414,
      19420,
      19426,
      19432,
      19438,
      19444,
      19450,
      19456,
      19462,
      19468,
      19474,
      19480,
      19486,
      19492,
      19498
    ],
    "core.event_count": 19,
    "core.first_event_number": 19390,
    "core.last_event_number": 19498,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0170_df-s04-d0_dw_0_20260203T194643.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:17:44 UTC       justIN version: 01.06.00