justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.119@dunegpschedd02.fnal.gov

Jobsub ID297923.119@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-04 02:31:01
From worker nodeHostnamecn515.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:02:51
Input filesvd-protodune:np02vd_raw_run042433_0482_df-s04-d1_dw_0_20260203T214519.hdf5
JobscriptExit code0
Real time2h (8751s)
CPU time1h (5037s = 57%)
Max RSS bytes3352002560 (3196 MiB)
Outputting started2026-02-04 02:28:44
Output files
Finished2026-02-04 02:31:01
Saved logsjustin-logs:297923.119-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

space point ptr... 0x4e1a4780, 0x52efcaf0
Caught exception in looking up space point ptr... 0x52efcaf0, 0x919f890
Caught exception in looking up space point ptr... 0x55fc3960, 0x2b5b08c0
Caught exception in looking up space point ptr... 0x2b5b08c0, 0x55fc3960
Caught exception in looking up space point ptr... 0x5b0b41b0, 0x14ea57e0
Caught exception in looking up space point ptr... 0x14ea57e0, 0x5b0b41b0
Caught exception in looking up space point ptr... 0x5b0b41b0, 0x7e4f0800
Caught exception in looking up space point ptr... 0x7e4f0800, 0x5b0b41b0
Caught exception in looking up space point ptr... 0x18175860, 0x414749a0
Caught exception in looking up space point ptr... 0xf552f40, 0x41474af0
Caught exception in looking up space point ptr... 0x41474af0, 0x18175860
Caught exception in looking up space point ptr... 0x5b706730, 0x83aee3f0
Caught exception in looking up space point ptr... 0x83aee3f0, 0x5b7065a0
Caught exception in looking up space point ptr... 0x84e55930, 0x6a75a7f0
Caught exception in looking up space point ptr... 0x6a75a7f0, 0x6a75a660
Caught exception in looking up space point ptr... 0x29c7bbb0, 0x18133b00
Caught exception in looking up space point ptr... 0x18133b00, 0x29c7bad0
Caught exception in looking up space point ptr... 0x29d22710, 0x29d224b0
Caught exception in looking up space point ptr... 0x3e5f5430, 0x29d22710
Caught exception in looking up space point ptr... 0x788eabb0, 0x788ea530
Caught exception in looking up space point ptr... 0x788ea530, 0x788eabb0
Caught exception in looking up space point ptr... 0x788eabb0, 0x788ea890
Caught exception in looking up space point ptr... 0x788ea890, 0x788eabb0
Caught exception in looking up space point ptr... 0xdebd0a0, 0xdebbb40
Caught exception in looking up space point ptr... 0xdebbb40, 0xdebd0a0
Caught exception in looking up space point ptr... 0x475f3230, 0x789c7920
Caught exception in looking up space point ptr... 0x789c7920, 0x475f3230
Caught exception in looking up space point ptr... 0x1a2e2140, 0x49663630
Caught exception in looking up space point ptr... 0x49663630, 0x1a2e2140
Caught exception in looking up space point ptr... 0x58891490, 0x66a1dea0
Caught exception in looking up space point ptr... 0x66a1dea0, 0x58891490
Caught exception in looking up space point ptr... 0xa283f80, 0x7ebf0a80
Caught exception in looking up space point ptr... 0x7ebf0a80, 0x740b4450
Caught exception in looking up space point ptr... 0xacfb630, 0x122ac390
Caught exception in looking up space point ptr... 0x122ac390, 0x45b7a8e0
Caught exception in looking up space point ptr... 0x706fbc30, 0xab492a0
Caught exception in looking up space point ptr... 0xab492a0, 0x533a85a0
Caught exception in looking up space point ptr... 0x3aca8170, 0x706fbc30
Caught exception in looking up space point ptr... 0x80255a60, 0x80255bf0
Caught exception in looking up space point ptr... 0x84b88ae0, 0x80255a60
Caught exception in looking up space point ptr... 0x6065b500, 0x6a8038f0
Caught exception in looking up space point ptr... 0x6a8038f0, 0x6a802780
Caught exception in looking up space point ptr... 0x7422f870, 0x1823d5d0
Caught exception in looking up space point ptr... 0x1823d5d0, 0x7422f870
Caught exception in looking up space point ptr... 0x7422f870, 0x1a762860
Caught exception in looking up space point ptr... 0x1a762860, 0x7422f870
Caught exception in looking up space point ptr... 0x1822bcc0, 0x61b4aa20
Caught exception in looking up space point ptr... 0x61b4aa20, 0xaf349b0
Caught exception in looking up space point ptr... 0xaf349b0, 0x54e95aa0
Caught exception in looking up space point ptr... 0x5bc61100, 0x4f656080
Caught exception in looking up space point ptr... 0x4f656080, 0x5bc61100
Caught exception in looking up space point ptr... 0x59929970, 0xdebe190
Caught exception in looking up space point ptr... 0xdebe190, 0x3e5f66c0
Caught exception in looking up space point ptr... 0x6f2f3990, 0x4d790c30
Caught exception in looking up space point ptr... 0x4d790c30, 0x6f2f3990
Caught exception in looking up space point ptr... 0x139b5ab0, 0x43c9ef80
Caught exception in looking up space point ptr... 0x43c9ef80, 0x83af9d50
Caught exception in looking up space point ptr... 0x4570a5f0, 0x55bb5b00
Caught exception in looking up space point ptr... 0x55bb5b00, 0x55bb5c30
Caught exception in looking up space point ptr... 0x503e66a0, 0x6f13a9f0
Caught exception in looking up space point ptr... 0x6f13a9f0, 0x503e66a0
Caught exception in looking up space point ptr... 0x503e66a0, 0x6f13a170
Caught exception in looking up space point ptr... 0x6f13a170, 0x503e66a0
Caught exception in looking up space point ptr... 0x159f1730, 0x159f01f0
Caught exception in looking up space point ptr... 0x159f01f0, 0x159f1730
Caught exception in looking up space point ptr... 0x788e9a30, 0x788e9b80
Caught exception in looking up space point ptr... 0x788e9b80, 0x788eb070
++++>>>> total num hits: 50553, num free: 48154
04-Feb-2026 02:28:33 GMT  Closed output file "np02vd_raw_run042433_0482_df-s04-d1_dw_0_20260203T214519_reco_stage0_20260204T022833_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         353.49        453.088       562.118       459.722       45.6461        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000102795   0.000139309   0.000309817   0.000127697   4.8719e-05       19     
produce:tpcrawdecoder:PDVDTPCReader                127.666       186.712       260.976       185.68        34.5821        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.645197      0.885572       1.15084      0.845791      0.170475        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0167196     0.0238438     0.0896096     0.018596      0.0161922       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0166776     0.0304885     0.251538      0.0169218     0.0523682       19     
produce:beamevent:BeamEvent                      0.000113145   0.000260485   0.00132563    0.000177409   0.000269958      19     
produce:pdvddaphne:DAPHNEReaderPDVD                13.9858       19.8816       26.2284       20.4344       3.08529        19     
produce:ophit:OpHitFinder                         0.0399988     0.0664745     0.0835349     0.0726343     0.0150871       19     
produce:wclsdatavd:WireCellToolkit                 144.882       205.033       255.046       207.977       37.0539        19     
produce:gaushit:GausHitFinder                      1.29662       2.30489       3.07219       2.29476      0.521057        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000514621   0.000785621   0.00108423    0.000804165   0.000129945      19     
produce:reco3d:SpacePointSolver                    16.0837       26.869        39.4732       27.0466       5.6862         19     
produce:hitpdune:DisambigFromSpacePoints          0.180695      0.360597      0.555153      0.345909      0.105654        19     
produce:cluster3d:Cluster3D                        1.45637       3.41726       6.34083       3.43677       1.29152        19     
[art]:TriggerResults:TriggerResultInserter       3.2685e-05    4.35347e-05   8.5106e-05    4.2219e-05    1.0722e-05       19     
end_path:out1:RootOutput                          7.632e-06    1.01903e-05   2.3946e-05     8.834e-06    3.70008e-06      19     
end_path:out1:RootOutput(write)                    5.46694       7.44131       10.161        6.64834       1.53608        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5334.9 MB
  Peak resident set size usage (VmHWM): 3352 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0482_df-s04-d1_dw_0_20260203T214519_reco_stage0_20260204T022833_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0482_df-s04-d1_dw_0_20260203T214519_reco_stage0_20260204T022833_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0482_df-s04-d1_dw_0_20260203T214519_reco_stage0_20260204T022833_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163393.0,
    "core.end_time": 1770172116.0,
    "core.events": [
      54959,
      54965,
      54971,
      54977,
      54983,
      54989,
      54995,
      55001,
      55007,
      55013,
      55019,
      55025,
      55031,
      55037,
      55043,
      55049,
      55055,
      55061,
      55067
    ],
    "core.event_count": 19,
    "core.first_event_number": 54959,
    "core.last_event_number": 55067,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0482_df-s04-d1_dw_0_20260203T214519.hdf5"
    }
  ]
}
justIN time: 2026-02-04 05:49:05 UTC       justIN version: 01.06.00