justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.115@dunegpschedd02.fnal.gov

Jobsub ID297923.115@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:50:56
From worker nodeHostnameheplnc137.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:03:35
Input filesvd-protodune:np02vd_raw_run042433_0332_df-s02-d1_dw_0_20260203T204817.hdf5
JobscriptExit code0
Real time1h (6278s)
CPU time1h (5844s = 93%)
Max RSS bytes3572228096 (3406 MiB)
Outputting started2026-02-04 01:48:16
Output files
Finished2026-02-04 01:50:56
Saved logsjustin-logs:297923.115-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tr... 0x66c0d080, 0x47b9be90
Caught exception in looking up space point ptr... 0x4ac9dbe0, 0x1840cb40
Caught exception in looking up space point ptr... 0x1840cb40, 0x4ac9dbe0
Caught exception in looking up space point ptr... 0x3c95fc30, 0x6f946330
Caught exception in looking up space point ptr... 0x6f946330, 0x2ecada90
Caught exception in looking up space point ptr... 0x2ecada90, 0x4ac52d20
Caught exception in looking up space point ptr... 0x47c41490, 0x68a5e2e0
Caught exception in looking up space point ptr... 0x68a5e2e0, 0x3c95fc30
Caught exception in looking up space point ptr... 0x3f741920, 0x3148bb10
Caught exception in looking up space point ptr... 0x3148bb10, 0x3f741920
Caught exception in looking up space point ptr... 0x3bd92440, 0x3bd92310
Caught exception in looking up space point ptr... 0x3bd92310, 0x3bd920b0
Caught exception in looking up space point ptr... 0x56bb69b0, 0x602c7660
Caught exception in looking up space point ptr... 0x602c7660, 0x3600c480
Caught exception in looking up space point ptr... 0x3bde5ff0, 0x4e9bd840
Caught exception in looking up space point ptr... 0x4e9bd840, 0x3bde5ff0
Caught exception in looking up space point ptr... 0x3bde5ff0, 0x3bde5e60
Caught exception in looking up space point ptr... 0x3bde5e60, 0x3bde5ff0
Caught exception in looking up space point ptr... 0x3f729f70, 0x182fc830
Caught exception in looking up space point ptr... 0x546ece40, 0x3f729f70
Caught exception in looking up space point ptr... 0x7dfe49a0, 0x3815c0d0
Caught exception in looking up space point ptr... 0x3815c0d0, 0x7dfe4bf0
Caught exception in looking up space point ptr... 0x38a69b50, 0x4744c640
Caught exception in looking up space point ptr... 0x4744c640, 0x38a69b50
Caught exception in looking up space point ptr... 0x12724000, 0x12723ce0
Caught exception in looking up space point ptr... 0x12723ce0, 0x12724000
Caught exception in looking up space point ptr... 0x7ef3a080, 0x3a158c70
Caught exception in looking up space point ptr... 0x3a158c70, 0x7ef3a080
Caught exception in looking up space point ptr... 0x70a819c0, 0x707326f0
Caught exception in looking up space point ptr... 0x707326f0, 0x689599b0
Caught exception in looking up space point ptr... 0x654857c0, 0x548b9b90
Caught exception in looking up space point ptr... 0x548b9b90, 0x548b9d90
Caught exception in looking up space point ptr... 0x548b9d90, 0x4cb85270
Caught exception in looking up space point ptr... 0x12a6ac30, 0x37f52700
Caught exception in looking up space point ptr... 0x37f52700, 0x37f51e00
Caught exception in looking up space point ptr... 0x73d70140, 0x689e5d80
Caught exception in looking up space point ptr... 0x689e5d80, 0x6f88fe20
Caught exception in looking up space point ptr... 0x3c9923a0, 0x330054e0
Caught exception in looking up space point ptr... 0xe9af680, 0x3c9923a0
Caught exception in looking up space point ptr... 0x35c2f980, 0x35c2f8e0
Caught exception in looking up space point ptr... 0x35bbb2c0, 0x35c2f980
Caught exception in looking up space point ptr... 0x1224ed00, 0x837275b0
Caught exception in looking up space point ptr... 0x837275b0, 0x1224ed00
Caught exception in looking up space point ptr... 0x3f6fdbc0, 0x4d1e4e80
Caught exception in looking up space point ptr... 0x4d1e4e80, 0x4b4dac00
Caught exception in looking up space point ptr... 0x5b873c90, 0x7da7a5e0
Caught exception in looking up space point ptr... 0x7da7a5e0, 0x5b873c90
Caught exception in looking up space point ptr... 0x74f11140, 0x3ad27a20
Caught exception in looking up space point ptr... 0x3ad27a20, 0x330bbf40
Caught exception in looking up space point ptr... 0x4cabcaf0, 0x15637d30
Caught exception in looking up space point ptr... 0x15637d30, 0x155a3e90
Caught exception in looking up space point ptr... 0x3c93e1a0, 0x74ccd2f0
Caught exception in looking up space point ptr... 0x74ccd2f0, 0x3c93e1a0
Caught exception in looking up space point ptr... 0x4d2c8040, 0x7db38e50
Caught exception in looking up space point ptr... 0x7db38e50, 0x4d2c8040
Caught exception in looking up space point ptr... 0x66810320, 0x7a9aef0
Caught exception in looking up space point ptr... 0x7a9aef0, 0x66810320
Caught exception in looking up space point ptr... 0x4ac1ef60, 0x173d9040
Caught exception in looking up space point ptr... 0x173d9040, 0x4ac1ef60
Caught exception in looking up space point ptr... 0x568efda0, 0x13f06b40
Caught exception in looking up space point ptr... 0x13f06b40, 0x568efda0
Caught exception in looking up space point ptr... 0x6f7f2620, 0x63fc09f0
Caught exception in looking up space point ptr... 0x63fc09f0, 0x47c67ce0
Caught exception in looking up space point ptr... 0x4204f7d0, 0x474b5fe0
Caught exception in looking up space point ptr... 0x474b5fe0, 0x4204f7d0
Caught exception in looking up space point ptr... 0x66aa3dd0, 0x66aa3ed0
Caught exception in looking up space point ptr... 0x66aa3ed0, 0xe7f1d70
++++>>>> total num hits: 47570, num free: 45061
04-Feb-2026 01:47:59 GMT  Closed output file "np02vd_raw_run042433_0332_df-s02-d1_dw_0_20260203T204817_reco_stage0_20260204T014759_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         233.959       321.092       416.669       318.741       47.2099        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.3494e-05    0.000154506   0.000262713   0.000142007   4.10902e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                17.2683       18.5411       20.3923       18.4049      0.952603        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.551536      0.580287      0.618924      0.577024      0.0168698       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.017851      0.0224163     0.0298226     0.0210353    0.00431496       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177687      0.01799      0.0188197     0.0179507    0.000222217      19     
produce:beamevent:BeamEvent                      0.000133722   0.000271252   0.00141321    0.000202331   0.000272313      19     
produce:pdvddaphne:DAPHNEReaderPDVD                7.06173       8.28446       12.6741       8.25066       1.12482        19     
produce:ophit:OpHitFinder                         0.0442302     0.0708958     0.0895995     0.0733206     0.0114496       19     
produce:wclsdatavd:WireCellToolkit                 182.677       208.382       239.966       199.658       18.9681        19     
produce:gaushit:GausHitFinder                      1.78873       3.04711       4.50548       3.01573      0.824887        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000535376   0.000808932   0.00154179    0.000768363   0.000218122      19     
produce:reco3d:SpacePointSolver                    13.5686       29.7276       48.8723       32.1882       9.2417         19     
produce:hitpdune:DisambigFromSpacePoints          0.207574      0.536857      0.978803      0.528278      0.201448        19     
produce:cluster3d:Cluster3D                        2.16929       39.9761       96.7225       5.93454       41.7571        19     
[art]:TriggerResults:TriggerResultInserter       3.8083e-05    5.35357e-05   0.000134825   4.7287e-05    2.07072e-05      19     
end_path:out1:RootOutput                          9.478e-06    1.42284e-05   3.1665e-05     1.241e-05    5.31174e-06      19     
end_path:out1:RootOutput(write)                    6.30332       11.8338       17.7905       11.6081       2.94237        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5569.96 MB
  Peak resident set size usage (VmHWM): 3572.23 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0332_df-s02-d1_dw_0_20260203T204817_reco_stage0_20260204T014759_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0332_df-s02-d1_dw_0_20260203T204817_reco_stage0_20260204T014759_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0332_df-s02-d1_dw_0_20260203T204817_reco_stage0_20260204T014759_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163429.0,
    "core.end_time": 1770169687.0,
    "core.events": [
      37855,
      37861,
      37867,
      37873,
      37879,
      37885,
      37891,
      37897,
      37903,
      37909,
      37915,
      37921,
      37927,
      37933,
      37939,
      37945,
      37951,
      37957,
      37963
    ],
    "core.event_count": 19,
    "core.first_event_number": 37855,
    "core.last_event_number": 37963,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0332_df-s02-d1_dw_0_20260203T204817.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:33:37 UTC       justIN version: 01.06.00