justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303654.58@dunegpschedd01.fnal.gov

Jobsub ID303654.58@dunegpschedd01.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:52:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-04 01:07:37
From worker nodeHostnamecn022.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:58:13
Input filesvd-protodune:np02vd_raw_run042433_0200_df-s04-d1_dw_0_20260203T195809.hdf5
JobscriptExit code0
Real time1h (4033s)
CPU time1h (3636s = 90%)
Max RSS bytes3408330752 (3250 MiB)
Outputting started2026-02-04 01:05:28
Output files
Finished2026-02-04 01:07:37
Saved logsjustin-logs:303654.58-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t ptr... 0x3034c120, 0x3034bf90
Caught exception in looking up space point ptr... 0x3034bf90, 0x3034bc20
Caught exception in looking up space point ptr... 0xdaf1040, 0x3fc558f0
Caught exception in looking up space point ptr... 0x3fc558f0, 0x3fc55710
Caught exception in looking up space point ptr... 0x74375410, 0x1b729160
Caught exception in looking up space point ptr... 0x1b729160, 0x1b729340
Caught exception in looking up space point ptr... 0x1b729340, 0x1b728fd0
Caught exception in looking up space point ptr... 0x1b728fd0, 0x1b728df0
Caught exception in looking up space point ptr... 0x1b728df0, 0x74375410
Caught exception in looking up space point ptr... 0x405dcc00, 0x405dd410
Caught exception in looking up space point ptr... 0x405dd410, 0x405d54c0
Caught exception in looking up space point ptr... 0x1b8696d0, 0x1b5f02e0
Caught exception in looking up space point ptr... 0x1b5f02e0, 0x1b8696d0
Caught exception in looking up space point ptr... 0x32ae5560, 0x32ae51c0
Caught exception in looking up space point ptr... 0x32ae51c0, 0x32ae5560
Caught exception in looking up space point ptr... 0x37af91f0, 0x37af8300
Caught exception in looking up space point ptr... 0x37af8300, 0x37af8660
Caught exception in looking up space point ptr... 0x484ad690, 0x484adb40
Caught exception in looking up space point ptr... 0x484adb40, 0x484ad820
Caught exception in looking up space point ptr... 0x6b2bf9f0, 0x37b0cb10
Caught exception in looking up space point ptr... 0x37b0cb10, 0x6b2bf9f0
Caught exception in looking up space point ptr... 0x662080a0, 0x66207c70
Caught exception in looking up space point ptr... 0x66207c70, 0x66207dc0
Caught exception in looking up space point ptr... 0x3ab58be0, 0x3ab58a50
Caught exception in looking up space point ptr... 0x3ab58a50, 0x3ab58be0
Caught exception in looking up space point ptr... 0x560ae820, 0x560b0a70
Caught exception in looking up space point ptr... 0x560b0a70, 0x560ae820
Caught exception in looking up space point ptr... 0xb4cb9e0, 0x30982f30
Caught exception in looking up space point ptr... 0x30982f30, 0xb4cb9e0
Caught exception in looking up space point ptr... 0x3c8e0590, 0x5fcad8b0
Caught exception in looking up space point ptr... 0x5fcad8b0, 0x38ba98b0
Caught exception in looking up space point ptr... 0x405d4620, 0x405d7360
Caught exception in looking up space point ptr... 0x405d7360, 0x405d4620
Caught exception in looking up space point ptr... 0x5602ce90, 0x5602ac30
Caught exception in looking up space point ptr... 0x5602ac30, 0x5602ce90
Caught exception in looking up space point ptr... 0x6b2b9c60, 0x6b2b9660
Caught exception in looking up space point ptr... 0x6b2b9660, 0x6b2b9c60
Caught exception in looking up space point ptr... 0x5c9da010, 0x2a534510
Caught exception in looking up space point ptr... 0x2a534510, 0x5c9da010
Caught exception in looking up space point ptr... 0xfa77c70, 0x90f1fc0
Caught exception in looking up space point ptr... 0x90f1fc0, 0xfa77c70
Caught exception in looking up space point ptr... 0xff82650, 0x499a8d00
Caught exception in looking up space point ptr... 0x1aabfd70, 0xff82650
Caught exception in looking up space point ptr... 0x56034520, 0x5604cfd0
Caught exception in looking up space point ptr... 0x5604cfd0, 0x56050100
Caught exception in looking up space point ptr... 0x11138fa0, 0x4a44d760
Caught exception in looking up space point ptr... 0x4a44d760, 0x4a44caa0
Caught exception in looking up space point ptr... 0x66206530, 0x66206700
Caught exception in looking up space point ptr... 0x66206700, 0x66206530
Caught exception in looking up space point ptr... 0x11611500, 0x302c5f10
Caught exception in looking up space point ptr... 0x302c5f10, 0x11611050
Caught exception in looking up space point ptr... 0x400828b0, 0x40085890
Caught exception in looking up space point ptr... 0x40085890, 0x40083db0
Caught exception in looking up space point ptr... 0x631e2650, 0x631e28b0
Caught exception in looking up space point ptr... 0x631e28b0, 0x631e2650
Caught exception in looking up space point ptr... 0x4889f1c0, 0x782c4560
Caught exception in looking up space point ptr... 0x782c4560, 0x4889f1c0
Caught exception in looking up space point ptr... 0x3ab1fac0, 0x339a3220
Caught exception in looking up space point ptr... 0x339a3220, 0x3ab1fac0
Caught exception in looking up space point ptr... 0x3ab1fac0, 0x339a2d70
Caught exception in looking up space point ptr... 0x339a2d70, 0x3ab1fac0
Caught exception in looking up space point ptr... 0x6a837d30, 0x6a83ce10
Caught exception in looking up space point ptr... 0x6a83ce10, 0x6a837d30
Caught exception in looking up space point ptr... 0x308b07f0, 0x6616d300
Caught exception in looking up space point ptr... 0x6616d300, 0x308b07f0
Caught exception in looking up space point ptr... 0x6b9440e0, 0x5947b430
Caught exception in looking up space point ptr... 0x5949a860, 0x6b9440e0
++++>>>> total num hits: 114029, num free: 109369
04-Feb-2026 01:05:19 GMT  Closed output file "np02vd_raw_run042433_0200_df-s04-d1_dw_0_20260203T195809_reco_stage0_20260204T010519_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         165.538       207.118       245.742       204.781       22.245         19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.8309e-05    0.000120382   0.000311789    0.0001137    4.76166e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                15.6042       23.5114       41.2609       18.6732       8.89916        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.532634      0.624131       1.15764      0.604174      0.132819        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0166581     0.0304476     0.161137      0.0179552     0.0331677       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0164641     0.0168425     0.0177435     0.0167903    0.000302104      19     
produce:beamevent:BeamEvent                      9.6101e-05    0.000210447   0.00138714    0.000148573   0.000278984      19     
produce:pdvddaphne:DAPHNEReaderPDVD                5.80903       6.77453       8.01636       6.64253      0.635194        19     
produce:ophit:OpHitFinder                         0.0270633     0.0460972     0.0710333     0.0418906     0.0123906       19     
produce:wclsdatavd:WireCellToolkit                 122.074       148.256       177.554       150.107       15.6796        19     
produce:gaushit:GausHitFinder                     0.937644       1.76327       3.26704       1.64341      0.518686        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000411582   0.000705192   0.00179542    0.000592072   0.000326252      19     
produce:reco3d:SpacePointSolver                    9.74994       17.7192       31.1295       17.3797       5.41995        19     
produce:hitpdune:DisambigFromSpacePoints           0.13636       0.25742       0.36384      0.250278      0.0596255       19     
produce:cluster3d:Cluster3D                        1.13008       2.51545       4.02851       2.37967       0.71696        19     
[art]:TriggerResults:TriggerResultInserter       2.5164e-05    3.65461e-05   8.1229e-05    3.1099e-05    1.33299e-05      19     
end_path:out1:RootOutput                          5.606e-06    9.00689e-06   2.3379e-05     8.161e-06    3.84236e-06      19     
end_path:out1:RootOutput(write)                    4.18639       5.55834       7.51421       5.04481       1.1369         19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5310.15 MB
  Peak resident set size usage (VmHWM): 3408.33 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0200_df-s04-d1_dw_0_20260203T195809_reco_stage0_20260204T010519_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0200_df-s04-d1_dw_0_20260203T195809_reco_stage0_20260204T010519_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0200_df-s04-d1_dw_0_20260203T195809_reco_stage0_20260204T010519_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163103.0,
    "core.end_time": 1770167122.0,
    "core.events": [
      22811,
      22817,
      22823,
      22829,
      22835,
      22841,
      22847,
      22853,
      22859,
      22865,
      22871,
      22877,
      22883,
      22889,
      22895,
      22901,
      22907,
      22913,
      22919
    ],
    "core.event_count": 19,
    "core.first_event_number": 22811,
    "core.last_event_number": 22919,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0200_df-s04-d1_dw_0_20260203T195809.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:37:09 UTC       justIN version: 01.06.00