justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.94@dunegpschedd01.fnal.gov

Jobsub ID303656.94@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2026-02-04 01:14:12
From worker nodeHostnamet2wn011.physics.ox.ac.uk
cpuinfoAMD EPYC 7713P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:13:46
Input filesvd-protodune:np02vd_raw_run042421_0035_df-s02-d0_dw_0_20260203T134956.hdf5
JobscriptExit code0
Real time57m (3428s)
CPU time49m (2988s = 87%)
Max RSS bytes2578411520 (2458 MiB)
Outputting started2026-02-04 01:10:55
Output files
Finished2026-02-04 01:14:12
Saved logsjustin-logs:303656.94-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t ptr... 0x4e5da5c0, 0x342ce6b0
Caught exception in looking up space point ptr... 0x2a652c00, 0x4c4e4150
Caught exception in looking up space point ptr... 0x4c4e4150, 0x4e5da5c0
Caught exception in looking up space point ptr... 0xe7b08c0, 0xe7b0220
Caught exception in looking up space point ptr... 0xe7b0220, 0xe7afef0
Caught exception in looking up space point ptr... 0x526b73d0, 0x41fa8f70
Caught exception in looking up space point ptr... 0x41fae870, 0x526b7550
Caught exception in looking up space point ptr... 0x526b7550, 0x526b73d0
Caught exception in looking up space point ptr... 0x33377770, 0x3aecf5c0
Caught exception in looking up space point ptr... 0x3aecf5c0, 0x28dd2930
Caught exception in looking up space point ptr... 0x30080160, 0x3aecc360
Caught exception in looking up space point ptr... 0x3aecc360, 0x33392240
Caught exception in looking up space point ptr... 0x527da060, 0x527d7a70
Caught exception in looking up space point ptr... 0x527d7a70, 0x527da060
Caught exception in looking up space point ptr... 0x4a5d5c30, 0x4a5d6630
Caught exception in looking up space point ptr... 0x4a5ce620, 0x4a5d5c30
Caught exception in looking up space point ptr... 0x1b0cfbe0, 0x12eb7f80
Caught exception in looking up space point ptr... 0x12eb7f80, 0x1b0cfbe0
Caught exception in looking up space point ptr... 0x2724bb70, 0x27241150
Caught exception in looking up space point ptr... 0x27241150, 0x2723cb60
Caught exception in looking up space point ptr... 0x1b112e60, 0x1b112af0
Caught exception in looking up space point ptr... 0x1b112af0, 0x1b112e60
Caught exception in looking up space point ptr... 0x2fef44b0, 0x4bfa4730
Caught exception in looking up space point ptr... 0x4bfa4730, 0x2fef44b0
Caught exception in looking up space point ptr... 0x2fef44b0, 0x4c5c3c40
Caught exception in looking up space point ptr... 0x4c5c3c40, 0x2fef44b0
Caught exception in looking up space point ptr... 0x4f470390, 0x4f4723b0
Caught exception in looking up space point ptr... 0x4f471bb0, 0x4f470390
Caught exception in looking up space point ptr... 0xd6c8bf0, 0xd6d3570
Caught exception in looking up space point ptr... 0xd6d3570, 0xd6ca310
Caught exception in looking up space point ptr... 0x2fef17f0, 0x2ff088f0
Caught exception in looking up space point ptr... 0x2ff088f0, 0x2fef17f0
Caught exception in looking up space point ptr... 0x2fef17f0, 0x2fefbde0
Caught exception in looking up space point ptr... 0x2fefbde0, 0x2fef17f0
Caught exception in looking up space point ptr... 0x14696ae0, 0x1468c590
Caught exception in looking up space point ptr... 0x1468c590, 0x14696ae0
Caught exception in looking up space point ptr... 0x1afa4140, 0x1af9b380
Caught exception in looking up space point ptr... 0x1af99c90, 0x1afa4140
Caught exception in looking up space point ptr... 0x4d958570, 0x4d934a50
Caught exception in looking up space point ptr... 0x4d934a50, 0x289bd8a0
Caught exception in looking up space point ptr... 0x134b3640, 0x3c437a70
Caught exception in looking up space point ptr... 0x3c437a70, 0xdc4ae80
Caught exception in looking up space point ptr... 0x28f21270, 0x28f215e0
Caught exception in looking up space point ptr... 0x28f215e0, 0x28f21400
Caught exception in looking up space point ptr... 0x3246c500, 0x13437bd0
Caught exception in looking up space point ptr... 0x13437bd0, 0x4c1d6b70
Caught exception in looking up space point ptr... 0x5bd88f90, 0x5bd91100
Caught exception in looking up space point ptr... 0x5bd91100, 0x5bd98270
Caught exception in looking up space point ptr... 0x5d770120, 0x5d7701c0
Caught exception in looking up space point ptr... 0x5d7701c0, 0x5d772d20
Caught exception in looking up space point ptr... 0x33c91740, 0x33c915b0
Caught exception in looking up space point ptr... 0x33c915b0, 0x33c91740
Caught exception in looking up space point ptr... 0x4c722610, 0x4c7226b0
Caught exception in looking up space point ptr... 0x4c7226b0, 0x4c722610
Caught exception in looking up space point ptr... 0x19bb2c70, 0x19bb59b0
Caught exception in looking up space point ptr... 0x19bb62c0, 0x19bb2c70
Caught exception in looking up space point ptr... 0x2ff17170, 0x2ff17ac0
Caught exception in looking up space point ptr... 0x2ff17ac0, 0x2ff17170
Caught exception in looking up space point ptr... 0x4a5d0750, 0x4a5e1200
Caught exception in looking up space point ptr... 0x4a5e1200, 0x4a5d05b0
Caught exception in looking up space point ptr... 0x41fb2ff0, 0x41fb88e0
Caught exception in looking up space point ptr... 0x41fb88e0, 0x41fb8a70
Caught exception in looking up space point ptr... 0x3337eb10, 0x3337eed0
Caught exception in looking up space point ptr... 0x3337eed0, 0x3337eb10
Caught exception in looking up space point ptr... 0x4d93df30, 0x516a0c20
Caught exception in looking up space point ptr... 0x516a0c20, 0x4d93df30
++++>>>> total num hits: 64628, num free: 62795
04-Feb-2026 01:10:31 GMT  Closed output file "np02vd_raw_run042421_0035_df-s02-d0_dw_0_20260203T134956_reco_stage0_20260204T011031_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         99.4741       122.393       145.618       121.233       9.77474        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.5474e-05    0.000102908   0.000272685   9.5184e-05    3.40546e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                12.8128       15.0003       16.6757       15.4307       1.40785        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.577991      0.602843      0.660643      0.600513      0.0200066       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0193323     0.0197126     0.0209465     0.0196193    0.000362172      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0958676     0.100681      0.116185      0.0973591    0.00737144       27     
produce:beamevent:BeamEvent                      9.0584e-05    0.000153837   0.00110248    0.000118458   0.000186599      27     
produce:pdvddaphne:DAPHNEReaderPDVD                4.07807       4.5318        4.95888        4.533       0.221223        27     
produce:ophit:OpHitFinder                         0.033684      0.0487311     0.0605845     0.0505679    0.00648039       27     
produce:wclsdatavd:WireCellToolkit                 66.0465       80.6745       102.519       79.4614       8.01287        27     
produce:gaushit:GausHitFinder                     0.740795       1.28882       2.02282       1.23313      0.316973        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000165378   0.000322752   0.000631977   0.000293345   0.000111195      27     
produce:reco3d:SpacePointSolver                    6.10025       12.7613       19.2386       13.6112       3.63673        27     
produce:hitpdune:DisambigFromSpacePoints          0.0604076     0.119157       0.26915      0.108147      0.0432462       27     
produce:cluster3d:Cluster3D                       0.946527       2.25788       4.47099       2.09334      0.916446        27     
[art]:TriggerResults:TriggerResultInserter       2.3305e-05    3.29809e-05   0.000128988   2.7593e-05    1.97844e-05      27     
end_path:out1:RootOutput                          3.436e-06    1.33003e-05   0.000207149    4.619e-06    3.84301e-05      27     
end_path:out1:RootOutput(write)                    3.88753       4.95205       6.48808       4.69711      0.830152        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4532.05 MB
  Peak resident set size usage (VmHWM): 2578.41 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0035_df-s02-d0_dw_0_20260203T134956_reco_stage0_20260204T011031_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0035_df-s02-d0_dw_0_20260203T134956_reco_stage0_20260204T011031_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0035_df-s02-d0_dw_0_20260203T134956_reco_stage0_20260204T011031_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770164042.0,
    "core.end_time": 1770167433.0,
    "core.events": [
      5718,
      5724,
      5730,
      5736,
      5742,
      5748,
      5754,
      5760,
      5766,
      5772,
      5778,
      5784,
      5790,
      5796,
      5802,
      5808,
      5814,
      5820,
      5826,
      5832,
      5838,
      5844,
      5850,
      5856,
      5862,
      5868,
      5874
    ],
    "core.event_count": 27,
    "core.first_event_number": 5718,
    "core.last_event_number": 5874,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0035_df-s02-d0_dw_0_20260203T134956.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:06:46 UTC       justIN version: 01.06.00