justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.136@dunegpschedd02.fnal.gov

Jobsub ID297921.136@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-04 01:23:23
From worker nodeHostnamecn031.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:56:23
Input filesvd-protodune:np02vd_raw_run042433_0273_df-s04-d1_dw_0_20260203T202556.hdf5
JobscriptExit code0
Real time1h (5071s)
CPU time1h (4693s = 92%)
Max RSS bytes3389046784 (3232 MiB)
Outputting started2026-02-04 01:20:55
Output files
Finished2026-02-04 01:23:23
Saved logsjustin-logs:297921.136-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ptr... 0x2d5e9350, 0x3a4cf4a0
Caught exception in looking up space point ptr... 0x3a4cf4a0, 0x3c2648a0
Caught exception in looking up space point ptr... 0x52cce6e0, 0x5b02fcb0
Caught exception in looking up space point ptr... 0x5b02fcb0, 0x54ce1dc0
Caught exception in looking up space point ptr... 0x5400f660, 0x5400f490
Caught exception in looking up space point ptr... 0x5400f490, 0x5400f660
Caught exception in looking up space point ptr... 0x3eff2f80, 0x60fd1b20
Caught exception in looking up space point ptr... 0x60fd1b20, 0x60fd1d80
Caught exception in looking up space point ptr... 0x2c8db040, 0x3a0de860
Caught exception in looking up space point ptr... 0x2c750080, 0x2c8db040
Caught exception in looking up space point ptr... 0x67385020, 0x4c967510
Caught exception in looking up space point ptr... 0x1aa24a90, 0x67385020
Caught exception in looking up space point ptr... 0x67b79ae0, 0x67b7a800
Caught exception in looking up space point ptr... 0x67b7a800, 0x67b7a440
Caught exception in looking up space point ptr... 0x60fba030, 0x60fc1ef0
Caught exception in looking up space point ptr... 0x60fc1ef0, 0x60fc1ba0
Caught exception in looking up space point ptr... 0x61010e80, 0x61010b60
Caught exception in looking up space point ptr... 0x61010b60, 0x61010e80
Caught exception in looking up space point ptr... 0x3aba8480, 0x67200b50
Caught exception in looking up space point ptr... 0x67200b50, 0x3aba8480
Caught exception in looking up space point ptr... 0x6dbe7ca0, 0x697de370
Caught exception in looking up space point ptr... 0x42c03ba0, 0x6dbe7ca0
Caught exception in looking up space point ptr... 0x53f8add0, 0x53f944d0
Caught exception in looking up space point ptr... 0x53f944d0, 0x53f91570
Caught exception in looking up space point ptr... 0x53f91570, 0x53f95220
Caught exception in looking up space point ptr... 0x3a3faa50, 0x3a3fc810
Caught exception in looking up space point ptr... 0x3a3fc810, 0x3a3f9a70
Caught exception in looking up space point ptr... 0x83533480, 0x48c9ec70
Caught exception in looking up space point ptr... 0x48c9ec70, 0x83533480
Caught exception in looking up space point ptr... 0x83533480, 0x43cca380
Caught exception in looking up space point ptr... 0x43cca380, 0x83533480
Caught exception in looking up space point ptr... 0x152cea50, 0x3652bf40
Caught exception in looking up space point ptr... 0x554a6d00, 0x152cea50
Caught exception in looking up space point ptr... 0x3a405350, 0x3a401900
Caught exception in looking up space point ptr... 0x3a401900, 0x3a405350
Caught exception in looking up space point ptr... 0x60f87880, 0x60f9bd90
Caught exception in looking up space point ptr... 0x60f875e0, 0x60f87880
Caught exception in looking up space point ptr... 0x6f53a930, 0x59cc9ef0
Caught exception in looking up space point ptr... 0x59cc9ef0, 0x19f33b00
Caught exception in looking up space point ptr... 0x4ee6c7d0, 0x4ee6c680
Caught exception in looking up space point ptr... 0x4ee6c680, 0x4ee6c7d0
Caught exception in looking up space point ptr... 0x3c3cf480, 0x484e8570
Caught exception in looking up space point ptr... 0x484e8570, 0xe3efb60
Caught exception in looking up space point ptr... 0x17f54860, 0x607bffc0
Caught exception in looking up space point ptr... 0x607bffc0, 0x17f546d0
Caught exception in looking up space point ptr... 0x3a716650, 0x6d565160
Caught exception in looking up space point ptr... 0x6d565160, 0x6d564fd0
Caught exception in looking up space point ptr... 0x67622aa0, 0x43d9a4c0
Caught exception in looking up space point ptr... 0x43d9a4c0, 0x3c3ef6a0
Caught exception in looking up space point ptr... 0x2c74f900, 0x2cb142c0
Caught exception in looking up space point ptr... 0x2cb142c0, 0x4f36d950
Caught exception in looking up space point ptr... 0x54151720, 0x54153870
Caught exception in looking up space point ptr... 0x54153870, 0x54151820
Caught exception in looking up space point ptr... 0x61a5d0f0, 0x61a5d240
Caught exception in looking up space point ptr... 0x61a5d240, 0x61a5d390
Caught exception in looking up space point ptr... 0x1326e600, 0x19c374b0
Caught exception in looking up space point ptr... 0x19c374b0, 0x6fb76490
Caught exception in looking up space point ptr... 0x62057000, 0x47329eb0
Caught exception in looking up space point ptr... 0x47329eb0, 0x62057000
Caught exception in looking up space point ptr... 0x53f55580, 0x540355c0
Caught exception in looking up space point ptr... 0x540355c0, 0x53f55580
Caught exception in looking up space point ptr... 0x3a6a0440, 0x3a6a0590
Caught exception in looking up space point ptr... 0x3a6a0590, 0x3a6a0440
Caught exception in looking up space point ptr... 0x3a3f8a90, 0x3a3f8ea0
Caught exception in looking up space point ptr... 0x3a3f8ea0, 0x3a3f83a0
Caught exception in looking up space point ptr... 0xb6afc40, 0xb2e2710
Caught exception in looking up space point ptr... 0xb2e2710, 0xb6afc40
++++>>>> total num hits: 53179, num free: 50928
04-Feb-2026 01:20:44 GMT  Closed output file "np02vd_raw_run042433_0273_df-s04-d1_dw_0_20260203T202556_reco_stage0_20260204T012044_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         217.647       260.894       289.171       266.144       17.1218        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.0409e-05    0.00010821    0.000244993   9.5213e-05    3.52902e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                15.4963       22.2328       31.2037       23.0632       4.6221         19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.490311      0.508424      0.527929      0.507703      0.0129726       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0165266     0.0175661     0.0316832     0.0167191    0.00333384       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0165528     0.0169192     0.0184391     0.0167204    0.000442496      19     
produce:beamevent:BeamEvent                      0.00012211    0.000206725   0.00113724    0.000156538   0.00022058       19     
produce:pdvddaphne:DAPHNEReaderPDVD                6.44566       7.56566       8.46009       7.70961      0.481757        19     
produce:ophit:OpHitFinder                         0.0371558     0.0608957     0.0815676     0.0609267    0.00868177       19     
produce:wclsdatavd:WireCellToolkit                 160.304       197.08        214.847       200.458       12.9119        19     
produce:gaushit:GausHitFinder                      1.32337       2.10605       2.95816       2.14965      0.373641        19     
produce:nhitsfilter:NumberOfHitsFilter           0.00043534    0.000769045   0.00153445    0.000724605   0.000220686      19     
produce:reco3d:SpacePointSolver                    10.0318       20.2438       33.8708       18.4465       6.39396        19     
produce:hitpdune:DisambigFromSpacePoints          0.191113       0.34471      0.567777      0.351274      0.0934495       19     
produce:cluster3d:Cluster3D                        1.68696       3.13038       4.90386       3.12206      0.965988        19     
[art]:TriggerResults:TriggerResultInserter       3.1721e-05    4.32033e-05   0.000128034   3.8505e-05    2.04289e-05      19     
end_path:out1:RootOutput                          7.253e-06    1.07365e-05   2.8453e-05     9.784e-06    4.51353e-06      19     
end_path:out1:RootOutput(write)                    6.19335       7.53538       9.02644       7.57729      0.568931        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5299.1 MB
  Peak resident set size usage (VmHWM): 3389.05 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0273_df-s04-d1_dw_0_20260203T202556_reco_stage0_20260204T012044_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0273_df-s04-d1_dw_0_20260203T202556_reco_stage0_20260204T012044_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0273_df-s04-d1_dw_0_20260203T202556_reco_stage0_20260204T012044_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162995.0,
    "core.end_time": 1770168048.0,
    "core.events": [
      31133,
      31139,
      31145,
      31151,
      31157,
      31163,
      31169,
      31175,
      31181,
      31187,
      31193,
      31199,
      31205,
      31211,
      31217,
      31223,
      31229,
      31235,
      31241
    ],
    "core.event_count": 19,
    "core.first_event_number": 31133,
    "core.last_event_number": 31241,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0273_df-s04-d1_dw_0_20260203T202556.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:28:40 UTC       justIN version: 01.06.00