justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.61@dunegpschedd02.fnal.gov

Jobsub ID297923.61@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:07:40
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:59:26
Input filesvd-protodune:np02vd_raw_run042433_0208_df-s02-d1_dw_0_20260203T200109.hdf5
JobscriptExit code0
Real time1h (3947s)
CPU time58m (3493s = 88%)
Max RSS bytes3348119552 (3193 MiB)
Outputting started2026-02-04 01:05:14
Output files
Finished2026-02-04 01:07:40
Saved logsjustin-logs:297923.61-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0x10308980, 0x103086e0
Caught exception in looking up space point ptr... 0x2db11b00, 0x107f50f0
Caught exception in looking up space point ptr... 0x107f50f0, 0x3c278ea0
Caught exception in looking up space point ptr... 0x5edbd2b0, 0x5edc7120
Caught exception in looking up space point ptr... 0x5edc7120, 0x5edbd2b0
Caught exception in looking up space point ptr... 0x62fed040, 0x62fe2800
Caught exception in looking up space point ptr... 0x62fe2800, 0x62fed040
Caught exception in looking up space point ptr... 0x72a48c30, 0x72a473c0
Caught exception in looking up space point ptr... 0x72a473c0, 0x72a49fc0
Caught exception in looking up space point ptr... 0x72a49fc0, 0x72a4b280
Caught exception in looking up space point ptr... 0x72a4b280, 0x72a4a870
Caught exception in looking up space point ptr... 0x4b09a820, 0x4b09a720
Caught exception in looking up space point ptr... 0x4b09a720, 0x4b09a820
Caught exception in looking up space point ptr... 0x338ced60, 0x338cdf80
Caught exception in looking up space point ptr... 0x338ce3e0, 0x338ce4c0
Caught exception in looking up space point ptr... 0x338ce4c0, 0x338ce2a0
Caught exception in looking up space point ptr... 0x338ce2a0, 0x338ced60
Caught exception in looking up space point ptr... 0x71bc0dc0, 0x47b7ad40
Caught exception in looking up space point ptr... 0x47b7ad40, 0x47b7b100
Caught exception in looking up space point ptr... 0x68fa7f30, 0x68fa8110
Caught exception in looking up space point ptr... 0x690d8940, 0x690d7a80
Caught exception in looking up space point ptr... 0x690d7a80, 0x690d6760
Caught exception in looking up space point ptr... 0x690d6760, 0x68fa7f30
Caught exception in looking up space point ptr... 0x64de0d70, 0x64de0c20
Caught exception in looking up space point ptr... 0x64de0c20, 0x64de0980
Caught exception in looking up space point ptr... 0x102f0780, 0x102f9ae0
Caught exception in looking up space point ptr... 0x102f9ae0, 0x102dac80
Caught exception in looking up space point ptr... 0x102dac80, 0x102f0da0
Caught exception in looking up space point ptr... 0x102f0da0, 0x102f0780
Caught exception in looking up space point ptr... 0x382a2b40, 0x4f21e110
Caught exception in looking up space point ptr... 0x4f21e110, 0x382a2b40
Caught exception in looking up space point ptr... 0x724dc8f0, 0x724d9c20
Caught exception in looking up space point ptr... 0x724d9c20, 0x724dc8f0
Caught exception in looking up space point ptr... 0x6c499da0, 0x6c49a040
Caught exception in looking up space point ptr... 0x6c49a040, 0x6c49a1d0
Caught exception in looking up space point ptr... 0x3247c980, 0x32479530
Caught exception in looking up space point ptr... 0x32479530, 0x3247c980
Caught exception in looking up space point ptr... 0x3247fe10, 0x3247a450
Caught exception in looking up space point ptr... 0x3247a450, 0x3247cf30
Caught exception in looking up space point ptr... 0x729c2510, 0x5042cf90
Caught exception in looking up space point ptr... 0x5042cf90, 0x371b0130
Caught exception in looking up space point ptr... 0x382849e0, 0x382814d0
Caught exception in looking up space point ptr... 0x382814d0, 0x382849e0
Caught exception in looking up space point ptr... 0x192c5c20, 0x2dbf47c0
Caught exception in looking up space point ptr... 0x2dbf47c0, 0x192c5c20
Caught exception in looking up space point ptr... 0x125d5f50, 0x125df2e0
Caught exception in looking up space point ptr... 0x125df2e0, 0x125dc440
Caught exception in looking up space point ptr... 0x3c4f2a30, 0x3c4f2bd0
Caught exception in looking up space point ptr... 0x3c4f2bd0, 0x3c4f2a30
Caught exception in looking up space point ptr... 0x594f36e0, 0x115c5500
Caught exception in looking up space point ptr... 0x115c5500, 0x594f36e0
Caught exception in looking up space point ptr... 0x6c4aa790, 0x6c4aa4a0
Caught exception in looking up space point ptr... 0x6c4aa4a0, 0x6c4aa640
Caught exception in looking up space point ptr... 0x192c5d50, 0x2dbf41d0
Caught exception in looking up space point ptr... 0x2dbf41d0, 0x192c5d50
Caught exception in looking up space point ptr... 0x48398860, 0x490f64e0
Caught exception in looking up space point ptr... 0x490f64e0, 0x48398860
Caught exception in looking up space point ptr... 0x5c190e40, 0x5c190d10
Caught exception in looking up space point ptr... 0x5c190d10, 0x5c190e40
Caught exception in looking up space point ptr... 0x47827260, 0x47825360
Caught exception in looking up space point ptr... 0x47825360, 0x47827260
Caught exception in looking up space point ptr... 0x5e0f1b80, 0x5e0f1d20
Caught exception in looking up space point ptr... 0x5e0f1d20, 0x5e0f2010
Caught exception in looking up space point ptr... 0x19f04940, 0x19f047f0
Caught exception in looking up space point ptr... 0x19f047f0, 0x19f03200
Caught exception in looking up space point ptr... 0x19f03200, 0x19f03300
Caught exception in looking up space point ptr... 0x19f03300, 0x19f04940
++++>>>> total num hits: 111017, num free: 106797
04-Feb-2026 01:05:04 GMT  Closed output file "np02vd_raw_run042433_0208_df-s02-d1_dw_0_20260203T200109_reco_stage0_20260204T010504_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         175.506       202.763       245.945       200.785       18.4066        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.8735e-05    0.000185404   0.000577972   0.000137989   0.000120224      19     
produce:tpcrawdecoder:PDVDTPCReader                15.259        20.8186       59.146        16.5294       11.1713        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.531206       0.72006       2.08036       0.55366      0.428004        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175686     0.0316259     0.202146      0.0180015     0.0417497       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0174601     0.0180727     0.0207819     0.0179107    0.000687118      19     
produce:beamevent:BeamEvent                      0.000147155   0.000264133   0.00182143    0.000169968   0.000368256      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.2758        3.96948       6.6936        3.82218      0.698981        19     
produce:ophit:OpHitFinder                         0.0360573     0.0462787     0.0518144     0.0479336    0.00429301       19     
produce:wclsdatavd:WireCellToolkit                 119.617       139.255       151.886       138.854       9.34439        19     
produce:gaushit:GausHitFinder                      1.25078       1.82014       2.55876       1.86303      0.374436        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000339777   0.00055328    0.00114848    0.000507601   0.000206982      19     
produce:reco3d:SpacePointSolver                    15.6913       25.4639       35.8296       25.0382       4.94945        19     
produce:hitpdune:DisambigFromSpacePoints          0.145745      0.273133      0.395137      0.282489      0.0713156       19     
produce:cluster3d:Cluster3D                        2.2009        3.65105       6.0377        3.41444      0.971069        19     
[art]:TriggerResults:TriggerResultInserter       2.8443e-05    4.78906e-05   0.00016016    3.8612e-05    2.81042e-05      19     
end_path:out1:RootOutput                          5.26e-06     2.09741e-05   0.000199414    9.578e-06    4.2826e-05       19     
end_path:out1:RootOutput(write)                    4.75223       6.64836       7.9098        6.87242       1.00411        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5353.54 MB
  Peak resident set size usage (VmHWM): 3348.12 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0208_df-s02-d1_dw_0_20260203T200109_reco_stage0_20260204T010504_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0208_df-s02-d1_dw_0_20260203T200109_reco_stage0_20260204T010504_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0208_df-s02-d1_dw_0_20260203T200109_reco_stage0_20260204T010504_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163180.0,
    "core.end_time": 1770167106.0,
    "core.events": [
      23719,
      23725,
      23731,
      23737,
      23743,
      23749,
      23755,
      23761,
      23767,
      23773,
      23779,
      23785,
      23791,
      23797,
      23803,
      23809,
      23815,
      23821,
      23827
    ],
    "core.event_count": 19,
    "core.first_event_number": 23719,
    "core.last_event_number": 23827,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0208_df-s02-d1_dw_0_20260203T200109.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:43:48 UTC       justIN version: 01.06.00