justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.118@dunegpschedd02.fnal.gov

Jobsub ID297923.118@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:52:21
From worker nodeHostnameheplnc137.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:03:12
Input filesvd-protodune:np02vd_raw_run042433_0088_df-s03-d1_dw_0_20260203T191533.hdf5
JobscriptExit code0
Real time1h (6383s)
CPU time1h (5938s = 93%)
Max RSS bytes3569418240 (3404 MiB)
Outputting started2026-02-04 01:49:38
Output files
Finished2026-02-04 01:52:21
Saved logsjustin-logs:297923.118-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

int ptr... 0x179e2a80, 0x179ea180
Caught exception in looking up space point ptr... 0x2cd82ee0, 0x2cd82c90
Caught exception in looking up space point ptr... 0x2cd82c90, 0x2cd855e0
Caught exception in looking up space point ptr... 0x82161c40, 0x1837a110
Caught exception in looking up space point ptr... 0x1837a110, 0x5a1792c0
Caught exception in looking up space point ptr... 0x199b1660, 0x3089ecf0
Caught exception in looking up space point ptr... 0x3089ecf0, 0x308e7cf0
Caught exception in looking up space point ptr... 0x78701e40, 0x2dfb9d30
Caught exception in looking up space point ptr... 0x2dfb9d30, 0x78702090
Caught exception in looking up space point ptr... 0x57dac7e0, 0x57dac4c0
Caught exception in looking up space point ptr... 0x57dac4c0, 0x57dac330
Caught exception in looking up space point ptr... 0x675943c0, 0x675942f0
Caught exception in looking up space point ptr... 0x638fc3b0, 0x675943c0
Caught exception in looking up space point ptr... 0x6f433a60, 0x6f433960
Caught exception in looking up space point ptr... 0x6f433660, 0x6f433a60
Caught exception in looking up space point ptr... 0x5cc64f20, 0x5cc645e0
Caught exception in looking up space point ptr... 0x5cc62b70, 0x5cc64f20
Caught exception in looking up space point ptr... 0x7a6a23b0, 0xa2e9e30
Caught exception in looking up space point ptr... 0x7a6a2c20, 0x7a6a23b0
Caught exception in looking up space point ptr... 0xf775570, 0x432dfb60
Caught exception in looking up space point ptr... 0x432dfb60, 0xf775250
Caught exception in looking up space point ptr... 0x31db84f0, 0x6d6677a0
Caught exception in looking up space point ptr... 0x6d6677a0, 0x31db81d0
Caught exception in looking up space point ptr... 0x3a1fc3b0, 0x3a1fc110
Caught exception in looking up space point ptr... 0x3a1fc110, 0x3a1fb830
Caught exception in looking up space point ptr... 0x309f0c10, 0x34c569d0
Caught exception in looking up space point ptr... 0x34c569d0, 0x309ef750
Caught exception in looking up space point ptr... 0x17c7e520, 0x17c90630
Caught exception in looking up space point ptr... 0x17c6c0f0, 0x17c7e520
Caught exception in looking up space point ptr... 0x11773180, 0x17a058a0
Caught exception in looking up space point ptr... 0x17999a30, 0x11773180
Caught exception in looking up space point ptr... 0x3a40a300, 0x395b6d10
Caught exception in looking up space point ptr... 0x395b6d10, 0x395b6ea0
Caught exception in looking up space point ptr... 0x5a2fac40, 0x17ea55b0
Caught exception in looking up space point ptr... 0x17ea55b0, 0xf7846b0
Caught exception in looking up space point ptr... 0x308cd2d0, 0x308cd140
Caught exception in looking up space point ptr... 0x308cd140, 0x308bd130
Caught exception in looking up space point ptr... 0x8eee7e0, 0x34a76140
Caught exception in looking up space point ptr... 0x34a76140, 0x34a762d0
Caught exception in looking up space point ptr... 0x17cafee0, 0x17cc2180
Caught exception in looking up space point ptr... 0x17c9dab0, 0x17cafee0
Caught exception in looking up space point ptr... 0xedb2040, 0xeda2320
Caught exception in looking up space point ptr... 0xedc42e0, 0xedb2040
Caught exception in looking up space point ptr... 0x32610b60, 0x32600350
Caught exception in looking up space point ptr... 0x32600350, 0x325ef690
Caught exception in looking up space point ptr... 0x4a625070, 0x4a624270
Caught exception in looking up space point ptr... 0x4a624270, 0x4a625070
Caught exception in looking up space point ptr... 0x4e590b60, 0x720d8a10
Caught exception in looking up space point ptr... 0x720d8a10, 0x3e52fe70
Caught exception in looking up space point ptr... 0x6de6e670, 0x38809080
Caught exception in looking up space point ptr... 0x38809080, 0x6de6e670
Caught exception in looking up space point ptr... 0x5361c720, 0x5361c820
Caught exception in looking up space point ptr... 0x5361c820, 0x5361c720
Caught exception in looking up space point ptr... 0x720c0850, 0xe9df3c0
Caught exception in looking up space point ptr... 0xe9df3c0, 0x720c0850
Caught exception in looking up space point ptr... 0x48a160a0, 0x48a16280
Caught exception in looking up space point ptr... 0x48a16280, 0x48a160a0
Caught exception in looking up space point ptr... 0x48a160a0, 0x44d25e10
Caught exception in looking up space point ptr... 0x44d25e10, 0x48a160a0
Caught exception in looking up space point ptr... 0x62b5fa70, 0x38e42da0
Caught exception in looking up space point ptr... 0x38e42da0, 0x62b5fa70
Caught exception in looking up space point ptr... 0x49b85fc0, 0x49b860c0
Caught exception in looking up space point ptr... 0x49b860c0, 0x49b86d40
Caught exception in looking up space point ptr... 0x38dfa940, 0x38dfa800
Caught exception in looking up space point ptr... 0x38dfa800, 0x38dfa940
Caught exception in looking up space point ptr... 0x52fd7660, 0x52fd81a0
Caught exception in looking up space point ptr... 0x52fd81a0, 0x52fd7660
++++>>>> total num hits: 468646, num free: 458372
04-Feb-2026 01:49:21 GMT  Closed output file "np02vd_raw_run042433_0088_df-s03-d1_dw_0_20260203T191533_reco_stage0_20260204T014921_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         248.926       324.624       386.41        336.755       43.7297        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000117463   0.000185508   0.000369272   0.00014931    6.71467e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                16.6723       17.3138       18.3779       17.2143      0.492565        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.528549      0.557862      0.622163       0.55025      0.0258996       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0177341     0.0180818     0.0210927     0.0179079    0.000717333      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177235     0.0179173     0.0184012     0.0178879    0.000132504      19     
produce:beamevent:BeamEvent                      0.000150064   0.000281048   0.00132991    0.000219405   0.000254187      19     
produce:pdvddaphne:DAPHNEReaderPDVD                6.50826       8.12222       9.50353       8.27883      0.710016        19     
produce:ophit:OpHitFinder                         0.041664      0.0685721     0.0950303     0.0717196     0.0148364       19     
produce:wclsdatavd:WireCellToolkit                 173.55        197.584       245.77        194.127       16.2313        19     
produce:gaushit:GausHitFinder                      1.46892       3.00188       3.87694       3.15336      0.789976        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000470224   0.000704398    0.0012531    0.000674942   0.000179559      19     
produce:reco3d:SpacePointSolver                    13.8787       22.3228       40.7095       20.6103       7.35357        19     
produce:hitpdune:DisambigFromSpacePoints          0.233837      0.509489      0.974116      0.449769      0.238831        19     
produce:cluster3d:Cluster3D                        2.33357       62.6474       124.75        82.6545       41.7579        19     
[art]:TriggerResults:TriggerResultInserter       3.4063e-05    5.24412e-05   9.2507e-05    4.7023e-05    1.47523e-05      19     
end_path:out1:RootOutput                         1.0897e-05    1.65379e-05   4.2906e-05    1.2705e-05    7.8813e-06       19     
end_path:out1:RootOutput(write)                    8.55509       12.3914       18.7653        11.29        2.62551        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5542.65 MB
  Peak resident set size usage (VmHWM): 3569.42 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0088_df-s03-d1_dw_0_20260203T191533_reco_stage0_20260204T014921_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0088_df-s03-d1_dw_0_20260203T191533_reco_stage0_20260204T014921_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0088_df-s03-d1_dw_0_20260203T191533_reco_stage0_20260204T014921_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163423.0,
    "core.end_time": 1770169768.0,
    "core.events": [
      10041,
      10047,
      10053,
      10059,
      10065,
      10071,
      10077,
      10083,
      10089,
      10095,
      10101,
      10107,
      10113,
      10119,
      10125,
      10131,
      10137,
      10143,
      10149
    ],
    "core.event_count": 19,
    "core.first_event_number": 10041,
    "core.last_event_number": 10149,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0088_df-s03-d1_dw_0_20260203T191533.hdf5"
    }
  ]
}
justIN time: 2026-02-04 06:08:21 UTC       justIN version: 01.06.00