justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303656.63@dunegpschedd01.fnal.gov

Jobsub ID303656.63@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:56:25
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:27:09
From worker nodeHostnameheplnc133.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:11:57
Input filesvd-protodune:np02vd_raw_run042421_0073_df-s02-d0_dw_0_20260203T141029.hdf5
JobscriptExit code0
Real time1h (4364s)
CPU time1h (3721s = 85%)
Max RSS bytes2549383168 (2431 MiB)
Outputting started2026-02-04 01:24:43
Output files
Finished2026-02-04 01:27:09
Saved logsjustin-logs:303656.63-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ptr... 0x2994d7b0, 0x299458e0
Caught exception in looking up space point ptr... 0x299458e0, 0x2994c330
Caught exception in looking up space point ptr... 0x2994c330, 0x29947b80
Caught exception in looking up space point ptr... 0x29947b80, 0x299504f0
Caught exception in looking up space point ptr... 0x299504f0, 0x29951bc0
Caught exception in looking up space point ptr... 0x29951bc0, 0x2994d8f0
Caught exception in looking up space point ptr... 0x84457f0, 0x8444f00
Caught exception in looking up space point ptr... 0x8444f00, 0x8448280
Caught exception in looking up space point ptr... 0x8448280, 0x84487a0
Caught exception in looking up space point ptr... 0x8445290, 0x84457f0
Caught exception in looking up space point ptr... 0x24e74f60, 0x24e86040
Caught exception in looking up space point ptr... 0x24e86040, 0x24e877f0
Caught exception in looking up space point ptr... 0x84b11c0, 0x84a4910
Caught exception in looking up space point ptr... 0x84a4910, 0x84aea70
Caught exception in looking up space point ptr... 0x84aea70, 0x84b47e0
Caught exception in looking up space point ptr... 0x84b47e0, 0x84b11c0
Caught exception in looking up space point ptr... 0x1105fb50, 0x3af46370
Caught exception in looking up space point ptr... 0x3af46370, 0x91b5570
Caught exception in looking up space point ptr... 0x91b5570, 0x11074050
Caught exception in looking up space point ptr... 0x11074050, 0x3587bab0
Caught exception in looking up space point ptr... 0x21c84470, 0x21c85840
Caught exception in looking up space point ptr... 0x21c83610, 0x21c84470
Caught exception in looking up space point ptr... 0x3fe16ed0, 0x3fe164b0
Caught exception in looking up space point ptr... 0x3fe164b0, 0x3fe108d0
Caught exception in looking up space point ptr... 0x4dcf23d0, 0x4dcf2520
Caught exception in looking up space point ptr... 0x4dcf2520, 0x4dcde380
Caught exception in looking up space point ptr... 0x2375ad40, 0x4f57d7c0
Caught exception in looking up space point ptr... 0x4f57d7c0, 0x2375a470
Caught exception in looking up space point ptr... 0x24e79b00, 0x24e79c90
Caught exception in looking up space point ptr... 0x24e79c90, 0x24e79e20
Caught exception in looking up space point ptr... 0x3fe31e00, 0x3fe34350
Caught exception in looking up space point ptr... 0x3fe34350, 0x3fe34030
Caught exception in looking up space point ptr... 0x2345bf60, 0x2345db50
Caught exception in looking up space point ptr... 0x2345db50, 0x2345e290
Caught exception in looking up space point ptr... 0x409b2550, 0x554d3700
Caught exception in looking up space point ptr... 0x554d3700, 0x37216da0
Caught exception in looking up space point ptr... 0x40920380, 0x44dfe260
Caught exception in looking up space point ptr... 0x44dfe260, 0x5e2447e0
Caught exception in looking up space point ptr... 0x8450d50, 0x8451830
Caught exception in looking up space point ptr... 0x8451830, 0x8450d50
Caught exception in looking up space point ptr... 0x8441850, 0x8441f50
Caught exception in looking up space point ptr... 0x8441f50, 0x8441850
Caught exception in looking up space point ptr... 0xf6a7ae0, 0xf6a79e0
Caught exception in looking up space point ptr... 0xf6a79e0, 0x2af9d360
Caught exception in looking up space point ptr... 0x3005db50, 0x2f524f60
Caught exception in looking up space point ptr... 0x2f524f60, 0x12f1a290
Caught exception in looking up space point ptr... 0x48fa9770, 0x48fa9670
Caught exception in looking up space point ptr... 0x48fa9670, 0x48fad030
Caught exception in looking up space point ptr... 0x23476ab0, 0x234746d0
Caught exception in looking up space point ptr... 0x234746d0, 0x234764c0
Caught exception in looking up space point ptr... 0x611c4c40, 0x611c3b90
Caught exception in looking up space point ptr... 0x611c3b90, 0x611c4ea0
Caught exception in looking up space point ptr... 0x84c1ba0, 0x84c2b00
Caught exception in looking up space point ptr... 0x84c2b00, 0x84c1ba0
Caught exception in looking up space point ptr... 0xd74a930, 0xd74ac50
Caught exception in looking up space point ptr... 0x23479bc0, 0xd74a930
Caught exception in looking up space point ptr... 0x39ed1ed0, 0x44414ca0
Caught exception in looking up space point ptr... 0x44414ca0, 0x39ed1ed0
Caught exception in looking up space point ptr... 0x12656ec0, 0x12657040
Caught exception in looking up space point ptr... 0x12657040, 0x126571c0
Caught exception in looking up space point ptr... 0x2ac4e540, 0x2ac4fee0
Caught exception in looking up space point ptr... 0x2ac4fee0, 0x2ac4fb80
Caught exception in looking up space point ptr... 0x2895b290, 0x32e4ddb0
Caught exception in looking up space point ptr... 0x32e4ddb0, 0x2895b290
Caught exception in looking up space point ptr... 0x4ddfded0, 0x4bb67f80
Caught exception in looking up space point ptr... 0x4bb67f80, 0x8ed9b00
++++>>>> total num hits: 55925, num free: 54515
04-Feb-2026 01:23:16 GMT  Closed output file "np02vd_raw_run042421_0073_df-s02-d0_dw_0_20260203T141029_reco_stage0_20260204T012316_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         124.527       148.948       171.441       150.711       12.4007        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000108332   0.00014483    0.000348675   0.000125673   4.97599e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                12.6535       13.2611       13.755        13.3252      0.308923        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.525304      0.538424      0.572942      0.531236      0.0116615       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177318     0.0178789     0.0181361     0.0178361    0.000109841      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0876729     0.0976635     0.106658      0.104971     0.00870235       27     
produce:beamevent:BeamEvent                      0.000133734   0.00030825    0.00140232    0.000226241   0.000267506      27     
produce:pdvddaphne:DAPHNEReaderPDVD                6.15503       7.99141       14.7894       7.66007       1.74874        27     
produce:ophit:OpHitFinder                         0.0340501     0.0613454     0.0870469     0.0672573     0.0164463       27     
produce:wclsdatavd:WireCellToolkit                 84.6515       101.938       120.415       102.397       10.574         27     
produce:gaushit:GausHitFinder                     0.628722       1.40133       3.1925        1.37032      0.461499        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000288539   0.000466832   0.000847453   0.000448113   0.000125527      27     
produce:reco3d:SpacePointSolver                    6.12356       12.5966       23.5771       11.4216       4.04319        27     
produce:hitpdune:DisambigFromSpacePoints          0.101187      0.184528      0.440796      0.163977      0.0744553       27     
produce:cluster3d:Cluster3D                        1.00515       2.57536       6.65767       2.15565       1.30637        27     
[art]:TriggerResults:TriggerResultInserter       3.0129e-05    4.34623e-05   9.2578e-05     4.352e-05    1.28235e-05      27     
end_path:out1:RootOutput                          4.531e-06    1.80236e-05   0.000213928    9.929e-06    3.88052e-05      27     
end_path:out1:RootOutput(write)                    4.52005       8.24276       22.3999       7.51204       3.5135         27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4503.12 MB
  Peak resident set size usage (VmHWM): 2549.38 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0073_df-s02-d0_dw_0_20260203T141029_reco_stage0_20260204T012316_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0073_df-s02-d0_dw_0_20260203T141029_reco_stage0_20260204T012316_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0073_df-s02-d0_dw_0_20260203T141029_reco_stage0_20260204T012316_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163935.0,
    "core.end_time": 1770168233.0,
    "core.events": [
      11886,
      11892,
      11898,
      11904,
      11910,
      11916,
      11922,
      11928,
      11934,
      11940,
      11946,
      11952,
      11958,
      11964,
      11970,
      11976,
      11982,
      11988,
      11994,
      12000,
      12006,
      12012,
      12018,
      12024,
      12030,
      12036,
      12042
    ],
    "core.event_count": 27,
    "core.first_event_number": 11886,
    "core.last_event_number": 12042,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0073_df-s02-d0_dw_0_20260203T141029.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:31:46 UTC       justIN version: 01.06.00