justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.120@dunegpschedd02.fnal.gov

Jobsub ID297923.120@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-02-04 02:17:36
From worker nodeHostnamecn541.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:02:51
Input filesvd-protodune:np02vd_raw_run042433_0175_df-s02-d1_dw_0_20260203T194837.hdf5
JobscriptExit code0
Real time2h (7942s)
CPU time1h (4068s = 51%)
Max RSS bytes3376857088 (3220 MiB)
Outputting started2026-02-04 02:15:16
Output files
Finished2026-02-04 02:17:36
Saved logsjustin-logs:297923.120-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ce point ptr... 0x55a30510, 0x55a30800
Caught exception in looking up space point ptr... 0x32e8abc0, 0x32e8b500
Caught exception in looking up space point ptr... 0x32e8b500, 0x32e8abc0
Caught exception in looking up space point ptr... 0x32e8abc0, 0x32e84990
Caught exception in looking up space point ptr... 0x32e84990, 0x32e8abc0
Caught exception in looking up space point ptr... 0x5031c150, 0x50318f10
Caught exception in looking up space point ptr... 0x50318f10, 0x5031d870
Caught exception in looking up space point ptr... 0x3bf41c00, 0x7caefae0
Caught exception in looking up space point ptr... 0x7caefae0, 0x3bf41c00
Caught exception in looking up space point ptr... 0x81f357b0, 0x472e4fc0
Caught exception in looking up space point ptr... 0x472e4fc0, 0x45ed6720
Caught exception in looking up space point ptr... 0x3e43fe00, 0x3e43fb60
Caught exception in looking up space point ptr... 0xfef8200, 0x3e43fe00
Caught exception in looking up space point ptr... 0x4d9a1500, 0x4d978020
Caught exception in looking up space point ptr... 0x4d978020, 0x4d9a1500
Caught exception in looking up space point ptr... 0x6b8aba00, 0x45c04930
Caught exception in looking up space point ptr... 0x45c04930, 0x6b8aba00
Caught exception in looking up space point ptr... 0x6b8aba00, 0xc82fb20
Caught exception in looking up space point ptr... 0xc82fb20, 0x6b8aba00
Caught exception in looking up space point ptr... 0x4935a260, 0x32e774f0
Caught exception in looking up space point ptr... 0x32e774f0, 0x4935a260
Caught exception in looking up space point ptr... 0x16437f20, 0x1644b3d0
Caught exception in looking up space point ptr... 0x1644b3d0, 0x164478b0
Caught exception in looking up space point ptr... 0x45b8f5c0, 0x45b8f750
Caught exception in looking up space point ptr... 0x45b8f750, 0x45b8fa70
Caught exception in looking up space point ptr... 0x50359c20, 0x6abbd800
Caught exception in looking up space point ptr... 0x6abbd800, 0x50359c20
Caught exception in looking up space point ptr... 0x29afb8f0, 0x30223e50
Caught exception in looking up space point ptr... 0x30223e50, 0x29afb8f0
Caught exception in looking up space point ptr... 0x30205f30, 0x30206250
Caught exception in looking up space point ptr... 0x30206250, 0x30206890
Caught exception in looking up space point ptr... 0xfb935a0, 0xfb98540
Caught exception in looking up space point ptr... 0xfb98540, 0xfb9a5a0
Caught exception in looking up space point ptr... 0x17009990, 0x10614500
Caught exception in looking up space point ptr... 0x10614500, 0x17009990
Caught exception in looking up space point ptr... 0xe8c70f0, 0x551992f0
Caught exception in looking up space point ptr... 0x551992f0, 0xe8c70f0
Caught exception in looking up space point ptr... 0x2d3c9ff0, 0x2d39f9a0
Caught exception in looking up space point ptr... 0x2d39f9a0, 0x2d3c9ff0
Caught exception in looking up space point ptr... 0x158cda20, 0x389710b0
Caught exception in looking up space point ptr... 0x389710b0, 0x59183c40
Caught exception in looking up space point ptr... 0x59183c40, 0x426772b0
Caught exception in looking up space point ptr... 0x426772b0, 0x66d71060
Caught exception in looking up space point ptr... 0x130d1c70, 0x130d1e00
Caught exception in looking up space point ptr... 0x130d1e00, 0x130d2120
Caught exception in looking up space point ptr... 0x5628db70, 0x5628e940
Caught exception in looking up space point ptr... 0x5628e940, 0x5628e620
Caught exception in looking up space point ptr... 0x525d08b0, 0x525b9110
Caught exception in looking up space point ptr... 0x525b9110, 0x525d08b0
Caught exception in looking up space point ptr... 0x7c82080, 0x3e78da40
Caught exception in looking up space point ptr... 0x3e78da40, 0x3e78dbd0
Caught exception in looking up space point ptr... 0xfb05ab0, 0xfb05600
Caught exception in looking up space point ptr... 0xfb05600, 0xfb05ab0
Caught exception in looking up space point ptr... 0x29a7c630, 0x29a7c4a0
Caught exception in looking up space point ptr... 0x29a7c4a0, 0x29a7c630
Caught exception in looking up space point ptr... 0x81bebd80, 0x81becc60
Caught exception in looking up space point ptr... 0x81bec940, 0x81bebd80
Caught exception in looking up space point ptr... 0x5acd1ed0, 0x6eb34310
Caught exception in looking up space point ptr... 0x6eb34310, 0x5acd1ed0
Caught exception in looking up space point ptr... 0x81c020c0, 0x81c01f70
Caught exception in looking up space point ptr... 0x81c01f70, 0x81c020c0
Caught exception in looking up space point ptr... 0x502f8340, 0x502f8690
Caught exception in looking up space point ptr... 0x502f8690, 0x502f8340
Caught exception in looking up space point ptr... 0x87ad63e0, 0x87ad6890
Caught exception in looking up space point ptr... 0x87ad6890, 0x87ad6570
Caught exception in looking up space point ptr... 0x52596920, 0x52596e10
Caught exception in looking up space point ptr... 0x52596e10, 0x52596920
++++>>>> total num hits: 98805, num free: 95768
04-Feb-2026 02:15:06 GMT  Closed output file "np02vd_raw_run042433_0175_df-s02-d1_dw_0_20260203T194837_reco_stage0_20260204T021506_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         334.737       413.001       516.792       409.799       46.1408        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.2613e-05    0.000128478   0.000284675   0.000122678   3.97194e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                128.865       195.345       281.629       192.545       36.1442        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.586154      0.967302       1.55647      0.921184      0.230888        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0166001     0.0260086     0.0917004     0.0211184     0.0169924       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0165791     0.0167933     0.017216      0.0167854    0.000150795      19     
produce:beamevent:BeamEvent                      0.000135816   0.000223235   0.00111522    0.000175633   0.000212205      19     
produce:pdvddaphne:DAPHNEReaderPDVD                15.6668       20.2349       28.8873       19.5472       3.61392        19     
produce:ophit:OpHitFinder                         0.0367213     0.0539675     0.104057      0.0453853     0.0184768       19     
produce:wclsdatavd:WireCellToolkit                 135.57        162.279       221.985       152.707       23.005         19     
produce:gaushit:GausHitFinder                     0.955412       1.99758       3.46213       1.81366       0.61969        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000375679   0.000640129   0.00153793    0.000570604   0.000245785      19     
produce:reco3d:SpacePointSolver                    8.63272       22.0291       42.1364       19.2208       8.20107        19     
produce:hitpdune:DisambigFromSpacePoints          0.109988       0.3276       0.695015       0.2763       0.148251        19     
produce:cluster3d:Cluster3D                        1.42084       3.15625       7.75659       2.72315       1.54045        19     
[art]:TriggerResults:TriggerResultInserter       2.7199e-05    3.97697e-05   0.000129217   3.4452e-05    2.19259e-05      19     
end_path:out1:RootOutput                          4.222e-06    9.74511e-06   3.1736e-05     8.113e-06    5.98138e-06      19     
end_path:out1:RootOutput(write)                    5.05143       6.5173        10.948        5.86224       1.60895        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5344.32 MB
  Peak resident set size usage (VmHWM): 3376.86 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0175_df-s02-d1_dw_0_20260203T194837_reco_stage0_20260204T021506_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0175_df-s02-d1_dw_0_20260203T194837_reco_stage0_20260204T021506_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0175_df-s02-d1_dw_0_20260203T194837_reco_stage0_20260204T021506_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163382.0,
    "core.end_time": 1770171309.0,
    "core.events": [
      19957,
      19963,
      19969,
      19975,
      19981,
      19987,
      19993,
      19999,
      20005,
      20011,
      20017,
      20023,
      20029,
      20035,
      20041,
      20047,
      20053,
      20059,
      20065
    ],
    "core.event_count": 19,
    "core.first_event_number": 19957,
    "core.last_event_number": 20065,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0175_df-s02-d1_dw_0_20260203T194837.hdf5"
    }
  ]
}
justIN time: 2026-02-04 10:37:40 UTC       justIN version: 01.06.00