justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297920.52@dunegpschedd02.fnal.gov

Jobsub ID297920.52@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:54:27
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod03
Last heartbeat2026-02-04 01:47:40
From worker nodeHostnamewj07.grid.hep.ph.ic.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:38:57
Input filesvd-protodune:np02vd_raw_run042433_0227_df-s03-d1_dw_0_20260203T200824.hdf5
JobscriptExit code0
Real time2h (7552s)
CPU time1h (6397s = 84%)
Max RSS bytes3373031424 (3216 MiB)
Outputting started2026-02-04 01:44:52
Output files
Finished2026-02-04 01:47:40
Saved logsjustin-logs:297920.52-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x3def3690, 0x3def5150
Caught exception in looking up space point ptr... 0x52646e60, 0x52644c40
Caught exception in looking up space point ptr... 0x52644c40, 0x52646e60
Caught exception in looking up space point ptr... 0x56195430, 0x69c38260
Caught exception in looking up space point ptr... 0x69c38260, 0x69c30340
Caught exception in looking up space point ptr... 0x4d258eb0, 0x3744c4d0
Caught exception in looking up space point ptr... 0x3744c4d0, 0x3744c880
Caught exception in looking up space point ptr... 0x106acf30, 0x106acc10
Caught exception in looking up space point ptr... 0x106acc10, 0x106acf30
Caught exception in looking up space point ptr... 0x31812d10, 0x318131c0
Caught exception in looking up space point ptr... 0x318131c0, 0x31813030
Caught exception in looking up space point ptr... 0x6e7aeae0, 0x6e7b8ae0
Caught exception in looking up space point ptr... 0x6e7b8ae0, 0x6e7b8c70
Caught exception in looking up space point ptr... 0x38ad2cb0, 0x38ad23a0
Caught exception in looking up space point ptr... 0x38ad23a0, 0x6f96f350
Caught exception in looking up space point ptr... 0x4f0882f0, 0x4f088160
Caught exception in looking up space point ptr... 0x4f088160, 0x4f0882f0
Caught exception in looking up space point ptr... 0x39f34070, 0x39f33ee0
Caught exception in looking up space point ptr... 0x39f33ee0, 0xdc6b730
Caught exception in looking up space point ptr... 0x68962a20, 0x68962850
Caught exception in looking up space point ptr... 0x68962850, 0x68962a20
Caught exception in looking up space point ptr... 0x68962a20, 0x68962680
Caught exception in looking up space point ptr... 0x68962680, 0x68962a20
Caught exception in looking up space point ptr... 0x12844fe0, 0x12837920
Caught exception in looking up space point ptr... 0x12837920, 0x814f12e0
Caught exception in looking up space point ptr... 0x5131bbf0, 0xe16f2b0
Caught exception in looking up space point ptr... 0x5131be90, 0x5131bbf0
Caught exception in looking up space point ptr... 0x8150fa40, 0x81506030
Caught exception in looking up space point ptr... 0x81506350, 0x8150fa40
Caught exception in looking up space point ptr... 0x52ce21c0, 0x59ee73b0
Caught exception in looking up space point ptr... 0x59ee73b0, 0x52ce21c0
Caught exception in looking up space point ptr... 0x39f41fb0, 0x39f49620
Caught exception in looking up space point ptr... 0x39f49820, 0x39f41fb0
Caught exception in looking up space point ptr... 0x511f6b80, 0x511f9270
Caught exception in looking up space point ptr... 0x51203780, 0x511f6b80
Caught exception in looking up space point ptr... 0x52633eb0, 0x52633b40
Caught exception in looking up space point ptr... 0x52633b40, 0x52633eb0
Caught exception in looking up space point ptr... 0x7dbe770, 0x7dbe8a0
Caught exception in looking up space point ptr... 0x7dbe8a0, 0x7dbe9d0
Caught exception in looking up space point ptr... 0x59349b50, 0x5934adb0
Caught exception in looking up space point ptr... 0x5934adb0, 0x59349b50
Caught exception in looking up space point ptr... 0x6d3bf120, 0x6d3c6ce0
Caught exception in looking up space point ptr... 0x6d3c6ce0, 0x6d3bf120
Caught exception in looking up space point ptr... 0x3757a9e0, 0x3757ab70
Caught exception in looking up space point ptr... 0x3757ab70, 0x3757ae90
Caught exception in looking up space point ptr... 0x7dc0d90, 0x6b65eea0
Caught exception in looking up space point ptr... 0x6b65eea0, 0x11a8c1d0
Caught exception in looking up space point ptr... 0x6d3c68d0, 0x6d3c49e0
Caught exception in looking up space point ptr... 0x6d3c49e0, 0x6d3c68d0
Caught exception in looking up space point ptr... 0x5ae45580, 0x5ae45e80
Caught exception in looking up space point ptr... 0x5ae45e80, 0x5ae4da80
Caught exception in looking up space point ptr... 0xca16c00, 0xca16d50
Caught exception in looking up space point ptr... 0xca16d50, 0xca16ea0
Caught exception in looking up space point ptr... 0x4a7e1ba0, 0x4a7e1d30
Caught exception in looking up space point ptr... 0x4a7e1d30, 0x4a7e1ba0
Caught exception in looking up space point ptr... 0x6f968b00, 0x6f972c80
Caught exception in looking up space point ptr... 0x6f973140, 0x6f968b00
Caught exception in looking up space point ptr... 0x6bbad380, 0x6b4cf970
Caught exception in looking up space point ptr... 0x6b4cf970, 0x547c1450
Caught exception in looking up space point ptr... 0x37500e60, 0x560611a0
Caught exception in looking up space point ptr... 0x560611a0, 0x561c3190
Caught exception in looking up space point ptr... 0x6fc4dff0, 0x6fc4f300
Caught exception in looking up space point ptr... 0x6fc4f300, 0x6fc4f550
Caught exception in looking up space point ptr... 0x5626c7a0, 0x5626c930
Caught exception in looking up space point ptr... 0x5626c930, 0x5626c610
Caught exception in looking up space point ptr... 0x55232e50, 0x552331c0
Caught exception in looking up space point ptr... 0x552331c0, 0x55233030
++++>>>> total num hits: 77630, num free: 74426
04-Feb-2026 01:44:37 GMT  Closed output file "np02vd_raw_run042433_0227_df-s03-d1_dw_0_20260203T200824_reco_stage0_20260204T014437_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         367.157       389.905       422.612       388.233       14.4802        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.6328e-05    0.000135886   0.000284501   0.000112284   4.90875e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                55.5861       58.3018       76.0367       56.9703       4.45173        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.529037      0.593181      0.823096      0.562831      0.0754933       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0163286     0.0237046     0.0743836     0.0217891     0.0125422       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0162819     0.0165394     0.0176485     0.0164954    0.000281521      19     
produce:beamevent:BeamEvent                      0.000114933   0.000198222   0.00111275    0.00014723    0.000216477      19     
produce:pdvddaphne:DAPHNEReaderPDVD                9.28924       9.8279        10.3089       9.77939      0.255072        19     
produce:ophit:OpHitFinder                         0.0565779     0.0647279     0.068071      0.0663342    0.00330755       19     
produce:wclsdatavd:WireCellToolkit                 260.797       269.529       294.272       264.077       8.54017        19     
produce:gaushit:GausHitFinder                      2.08031       3.27521       4.19931       3.35723      0.573652        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000526461   0.00068757    0.00110417    0.00067024    0.000138928      19     
produce:reco3d:SpacePointSolver                    20.4098       29.6606       41.1898       29.3298       5.4713         19     
produce:hitpdune:DisambigFromSpacePoints          0.266623       0.47496      0.623716      0.518084       0.11597        19     
produce:cluster3d:Cluster3D                        2.44311       4.89634       8.6834        4.85112       1.51959        19     
[art]:TriggerResults:TriggerResultInserter        3.475e-05    4.53537e-05   0.000113928   3.9967e-05    1.70952e-05      19     
end_path:out1:RootOutput                          7.909e-06    1.86745e-05   0.000166848    8.645e-06    3.52671e-05      19     
end_path:out1:RootOutput(write)                    11.6242       13.2053       20.6101       12.471        2.11518        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5300.65 MB
  Peak resident set size usage (VmHWM): 3373.03 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0227_df-s03-d1_dw_0_20260203T200824_reco_stage0_20260204T014437_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0227_df-s03-d1_dw_0_20260203T200824_reco_stage0_20260204T014437_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0227_df-s03-d1_dw_0_20260203T200824_reco_stage0_20260204T014437_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770161956.0,
    "core.end_time": 1770169481.0,
    "core.events": [
      25887,
      25893,
      25899,
      25905,
      25911,
      25917,
      25923,
      25929,
      25935,
      25941,
      25947,
      25953,
      25959,
      25965,
      25971,
      25977,
      25983,
      25989,
      25995
    ],
    "core.event_count": 19,
    "core.first_event_number": 25887,
    "core.last_event_number": 25995,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0227_df-s03-d1_dw_0_20260203T200824.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:31:35 UTC       justIN version: 01.06.00