justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303646.58@dunegpschedd01.fnal.gov

Jobsub ID303646.58@dunegpschedd01.fnal.gov
Workflow ID12718
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:44:24
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod00
Last heartbeat2026-02-03 23:45:05
From worker nodeHostnamewl02.grid.hep.ph.ic.ac.uk
cpuinfoAMD EPYC 7742 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 22:46:40
Input filesvd-protodune:np02vd_raw_run042422_0106_df-s04-d1_dw_0_20260203T161204.hdf5
JobscriptExit code0
Real time54m (3258s)
CPU time45m (2703s = 82%)
Max RSS bytes2608205824 (2487 MiB)
Outputting started2026-02-03 23:40:59
Output files
Finished2026-02-03 23:45:05
Saved logsjustin-logs:303646.58-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0
Caught exception in looking up space point ptr... 0x14b39940, 0x31b26370
Caught exception in looking up space point ptr... 0x31b26370, 0x2c7697d0
Caught exception in looking up space point ptr... 0x2c7697d0, 0x2c7a2c20
Caught exception in looking up space point ptr... 0x2c7a2c20, 0x48d9d860
Caught exception in looking up space point ptr... 0x48d9d860, 0x14b39940
Caught exception in looking up space point ptr... 0x10d27150, 0x32ce5330
Caught exception in looking up space point ptr... 0x32ce5330, 0x10d27150
Caught exception in looking up space point ptr... 0x2cb355a0, 0x2cb360f0
Caught exception in looking up space point ptr... 0x2cb360f0, 0x2cb1def0
Caught exception in looking up space point ptr... 0x4a933040, 0x4a9334f0
Caught exception in looking up space point ptr... 0x4a9334f0, 0x4a9331d0
Caught exception in looking up space point ptr... 0x1389fec0, 0x29cbcf70
Caught exception in looking up space point ptr... 0x29cbcf70, 0x29cbe9d0
Caught exception in looking up space point ptr... 0x2906c9a0, 0x2906ca40
Caught exception in looking up space point ptr... 0x2906ca40, 0x2906c9a0
Caught exception in looking up space point ptr... 0x10547ff0, 0x3efda760
Caught exception in looking up space point ptr... 0x3efda760, 0x10547ff0
Caught exception in looking up space point ptr... 0x2ca14ef0, 0x80f44d0
Caught exception in looking up space point ptr... 0x80f44d0, 0x32712350
Caught exception in looking up space point ptr... 0x98248f0, 0x98247a0
Caught exception in looking up space point ptr... 0x98247a0, 0x98248f0
Caught exception in looking up space point ptr... 0x6395a590, 0x6395a770
Caught exception in looking up space point ptr... 0x6395a770, 0x6395a590
Caught exception in looking up space point ptr... 0x6395a590, 0x4c575bb0
Caught exception in looking up space point ptr... 0x4c575bb0, 0x6395a590
Caught exception in looking up space point ptr... 0x286353e0, 0x286356d0
Caught exception in looking up space point ptr... 0x286356d0, 0x286353e0
Caught exception in looking up space point ptr... 0x3a971920, 0x3a971790
Caught exception in looking up space point ptr... 0x3a971790, 0x3a971920
Caught exception in looking up space point ptr... 0x39d212e0, 0x309d71d0
Caught exception in looking up space point ptr... 0x309d71d0, 0x39d212e0
Caught exception in looking up space point ptr... 0x2ec52a80, 0x5784ce70
Caught exception in looking up space point ptr... 0x5784ce70, 0x2ec52a80
Caught exception in looking up space point ptr... 0xcc54bc0, 0x20d80110
Caught exception in looking up space point ptr... 0x20d80110, 0xcc55360
Caught exception in looking up space point ptr... 0x37534540, 0x37534690
Caught exception in looking up space point ptr... 0x37534690, 0x37534540
Caught exception in looking up space point ptr... 0x37534540, 0x375343f0
Caught exception in looking up space point ptr... 0x375343f0, 0x37534540
Caught exception in looking up space point ptr... 0x54e0b5b0, 0x3ff5daf0
Caught exception in looking up space point ptr... 0x4dfeea90, 0x54e0b5b0
Caught exception in looking up space point ptr... 0x63062fc0, 0x20911fc0
Caught exception in looking up space point ptr... 0x20911fc0, 0x63062fc0
Caught exception in looking up space point ptr... 0x2ed28f70, 0x45bf34d0
Caught exception in looking up space point ptr... 0x45bf34d0, 0x2ed28f70
Caught exception in looking up space point ptr... 0x34ea9e80, 0x3f0ada20
Caught exception in looking up space point ptr... 0x3f0ada20, 0x34ea9e80
Caught exception in looking up space point ptr... 0x48954b20, 0x489549d0
Caught exception in looking up space point ptr... 0x489549d0, 0x48954b20
Caught exception in looking up space point ptr... 0x34f97780, 0x14a8d0a0
Caught exception in looking up space point ptr... 0x14a8d0a0, 0x4d9360a0
Caught exception in looking up space point ptr... 0x4d9360a0, 0x34f97680
Caught exception in looking up space point ptr... 0x34f97680, 0x4d9360a0
Caught exception in looking up space point ptr... 0x54e0aa80, 0x54e0a8e0
Caught exception in looking up space point ptr... 0x54e0a8e0, 0x54e0a6e0
Caught exception in looking up space point ptr... 0x3d4be5c0, 0x4d962af0
Caught exception in looking up space point ptr... 0x4d962af0, 0x20c69970
Caught exception in looking up space point ptr... 0x5165be00, 0x5165bea0
Caught exception in looking up space point ptr... 0x26bc7690, 0x5165be00
Caught exception in looking up space point ptr... 0x4d46bce0, 0x4d46bee0
Caught exception in looking up space point ptr... 0x4d46bee0, 0x4d46bde0
Caught exception in looking up space point ptr... 0x6396f6b0, 0x6395c340
Caught exception in looking up space point ptr... 0x6395c340, 0x6396f6b0
Caught exception in looking up space point ptr... 0x6396f6b0, 0x29aa4690
Caught exception in looking up space point ptr... 0x29aa4690, 0x6396f6b0
++++>>>> total num hits: 81017, num free: 78331
03-Feb-2026 23:40:49 GMT  Closed output file "np02vd_raw_run042422_0106_df-s04-d1_dw_0_20260203T161204_reco_stage0_20260203T234049_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         106.143       117.045       133.521       116.32        6.19926        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.3932e-05    0.000108257   0.000380102   9.3674e-05    5.82022e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                11.3079       13.7004       30.5075       12.4013       3.70755        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.485032      0.510746      0.744859      0.501924      0.0479886       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0161249     0.0171872     0.0357966     0.0164798    0.00365491       27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0801636     0.0822763     0.0977377     0.0810523    0.00427841       27     
produce:beamevent:BeamEvent                      7.8612e-05    0.000150526   0.00125963    0.000104333   0.000218941      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.28771       4.12076       5.78565       3.91225      0.659335        27     
produce:ophit:OpHitFinder                         0.0317733     0.0434958     0.0575542     0.0440745     0.0056227       27     
produce:wclsdatavd:WireCellToolkit                 72.5892       75.6605       79.6071       75.3505       1.97334        27     
produce:gaushit:GausHitFinder                      0.64072       1.09027       1.83551       1.02455      0.251415        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000238618   0.000363774   0.000938071   0.000324911   0.00014976       27     
produce:reco3d:SpacePointSolver                    5.46628       12.471        22.6959       10.9942       4.37948        27     
produce:hitpdune:DisambigFromSpacePoints          0.0674358     0.120188      0.250273      0.115152      0.0400523       27     
produce:cluster3d:Cluster3D                        1.52223       2.90116       6.54694       2.80868       1.09567        27     
[art]:TriggerResults:TriggerResultInserter        2.381e-05    3.3621e-05    0.000113184   2.9531e-05    1.63304e-05      27     
end_path:out1:RootOutput                          3.92e-06     1.13273e-05   9.7083e-05     8.321e-06    1.70076e-05      27     
end_path:out1:RootOutput(write)                    5.42264       6.30133       7.40708       6.22164      0.462571        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4519.94 MB
  Peak resident set size usage (VmHWM): 2608.21 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042422_0106_df-s04-d1_dw_0_20260203T161204_reco_stage0_20260203T234049_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042422_0106_df-s04-d1_dw_0_20260203T161204_reco_stage0_20260203T234049_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042422_0106_df-s04-d1_dw_0_20260203T161204_reco_stage0_20260203T234049_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770158814.0,
    "core.end_time": 1770162051.0,
    "core.events": [
      17189,
      17195,
      17201,
      17207,
      17213,
      17219,
      17225,
      17231,
      17237,
      17243,
      17249,
      17255,
      17261,
      17267,
      17273,
      17279,
      17285,
      17291,
      17297,
      17303,
      17309,
      17315,
      17321,
      17327,
      17333,
      17339,
      17345
    ],
    "core.event_count": 27,
    "core.first_event_number": 17189,
    "core.last_event_number": 17345,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42422
    ],
    "core.runs_subruns": [
      4242200001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042422_0106_df-s04-d1_dw_0_20260203T161204.hdf5"
    }
  ]
}
justIN time: 2026-02-04 05:59:19 UTC       justIN version: 01.06.00