justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303654.81@dunegpschedd01.fnal.gov

Jobsub ID303654.81@dunegpschedd01.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:52:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:17:41
From worker nodeHostnameheplnc162.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:02:06
Input filesvd-protodune:np02vd_raw_run042433_0385_df-s02-d0_dw_0_20260203T210827.hdf5
JobscriptExit code0
Real time1h (4362s)
CPU time1h (4018s = 92%)
Max RSS bytes3571068928 (3405 MiB)
Outputting started2026-02-04 01:14:50
Output files
Finished2026-02-04 01:17:41
Saved logsjustin-logs:303654.81-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t ptr... 0x36f2f5a0, 0x328102f0
Caught exception in looking up space point ptr... 0x36f2f3e0, 0x36f2f5a0
Caught exception in looking up space point ptr... 0x38ccdba0, 0xc777630
Caught exception in looking up space point ptr... 0xc777630, 0x38ccdba0
Caught exception in looking up space point ptr... 0x41355820, 0xceacb60
Caught exception in looking up space point ptr... 0xceacb60, 0x3ac8ed40
Caught exception in looking up space point ptr... 0x3ac8ed40, 0x3b97bff0
Caught exception in looking up space point ptr... 0x852b6d90, 0x852b6c00
Caught exception in looking up space point ptr... 0x852b6c00, 0x852b6d90
Caught exception in looking up space point ptr... 0x3a910c90, 0x14ec7430
Caught exception in looking up space point ptr... 0x14ec7430, 0x35de03d0
Caught exception in looking up space point ptr... 0x3bc04000, 0x3bc038a0
Caught exception in looking up space point ptr... 0x3bc038a0, 0x3bc04000
Caught exception in looking up space point ptr... 0x17507b00, 0x58d433e0
Caught exception in looking up space point ptr... 0x58d433e0, 0x17507b00
Caught exception in looking up space point ptr... 0x57b608f0, 0x356f3f60
Caught exception in looking up space point ptr... 0x57b606a0, 0x57b608f0
Caught exception in looking up space point ptr... 0x35f6f3c0, 0x801c4830
Caught exception in looking up space point ptr... 0x801c4830, 0x35f6f3c0
Caught exception in looking up space point ptr... 0x316e4ba0, 0x38c7aaf0
Caught exception in looking up space point ptr... 0x412e3bb0, 0x316e4ba0
Caught exception in looking up space point ptr... 0x18b8aac0, 0x580b4520
Caught exception in looking up space point ptr... 0x580b4520, 0x18b8aac0
Caught exception in looking up space point ptr... 0x3b5d3110, 0x1030d2e0
Caught exception in looking up space point ptr... 0x1030d2e0, 0x6df2ce30
Caught exception in looking up space point ptr... 0x7ba9eef0, 0x7ba9ec50
Caught exception in looking up space point ptr... 0x7ba9ec50, 0x7ba9eef0
Caught exception in looking up space point ptr... 0x6b48cf20, 0x6eeeb190
Caught exception in looking up space point ptr... 0x6eeeb190, 0x7d3cce0
Caught exception in looking up space point ptr... 0x16465ba0, 0x16466190
Caught exception in looking up space point ptr... 0x16466190, 0x16465ba0
Caught exception in looking up space point ptr... 0x41383a40, 0x4092ed10
Caught exception in looking up space point ptr... 0x4092ed10, 0x41383a40
Caught exception in looking up space point ptr... 0x41383a40, 0x36b7ac90
Caught exception in looking up space point ptr... 0x36b7ac90, 0x41383a40
Caught exception in looking up space point ptr... 0x654d7560, 0x63c6f970
Caught exception in looking up space point ptr... 0x63c6f970, 0x654d7560
Caught exception in looking up space point ptr... 0x654d7560, 0x48cba780
Caught exception in looking up space point ptr... 0x48cba780, 0x654d7560
Caught exception in looking up space point ptr... 0x654d7560, 0x7baa0470
Caught exception in looking up space point ptr... 0x7baa0470, 0x654d7560
Caught exception in looking up space point ptr... 0x738dba50, 0x738db8c0
Caught exception in looking up space point ptr... 0x738db8c0, 0x738dba50
Caught exception in looking up space point ptr... 0xfc77ad0, 0x15d746e0
Caught exception in looking up space point ptr... 0x15d746e0, 0xfc77ad0
Caught exception in looking up space point ptr... 0x813d2d60, 0x813d3ed0
Caught exception in looking up space point ptr... 0x813d3ed0, 0x813d35f0
Caught exception in looking up space point ptr... 0x36909d30, 0x3690a1c0
Caught exception in looking up space point ptr... 0x3690a1c0, 0x36909d30
Caught exception in looking up space point ptr... 0x73f78a90, 0x6237bb50
Caught exception in looking up space point ptr... 0x6237bb50, 0x73f78a90
Caught exception in looking up space point ptr... 0x3134f060, 0x3134f160
Caught exception in looking up space point ptr... 0x3134f160, 0x3134fb90
Caught exception in looking up space point ptr... 0x41109bb0, 0x41109ab0
Caught exception in looking up space point ptr... 0x41109ab0, 0x41109bb0
Caught exception in looking up space point ptr... 0x640b9fc0, 0x4b083de0
Caught exception in looking up space point ptr... 0x4b083de0, 0x4085ca10
Caught exception in looking up space point ptr... 0x64901550, 0x64902ce0
Caught exception in looking up space point ptr... 0x64902ce0, 0x64901550
Caught exception in looking up space point ptr... 0x14788690, 0x318a7b10
Caught exception in looking up space point ptr... 0x318a7b10, 0x14788690
Caught exception in looking up space point ptr... 0x6b641290, 0x6b641150
Caught exception in looking up space point ptr... 0x6b641150, 0x6b640fc0
Caught exception in looking up space point ptr... 0x629758b0, 0x85c85da0
Caught exception in looking up space point ptr... 0x85c85da0, 0x4dac0fe0
Caught exception in looking up space point ptr... 0x3b2ee680, 0x3b2ee310
Caught exception in looking up space point ptr... 0x3b2ee310, 0x3b2ee680
++++>>>> total num hits: 66219, num free: 63987
04-Feb-2026 01:14:40 GMT  Closed output file "np02vd_raw_run042433_0385_df-s02-d0_dw_0_20260203T210827_reco_stage0_20260204T011440_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         154.911       224.782       295.573       230.349       37.9443        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.7593e-05    0.00017054    0.000315792   0.000146935   6.74264e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.3742       15.6166       18.1454       15.1412       1.14891        19     
produce:triggerrawdecoder:PDVDTriggerReader4       0.51881      0.542364      0.583544       0.53657      0.0173265       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175822     0.017893      0.0184363     0.0179037    0.00020423       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0174493     0.0178582     0.018495      0.0178275    0.000239717      19     
produce:beamevent:BeamEvent                      0.000112471   0.000221339   0.00122517    0.000172052   0.000238577      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.07579       3.86926       4.77823       3.80412      0.412492        19     
produce:ophit:OpHitFinder                         0.0285558     0.0462334     0.073547      0.0468071    0.00872999       19     
produce:wclsdatavd:WireCellToolkit                 107.904       133.225       158.063       134.379       13.8167        19     
produce:gaushit:GausHitFinder                      1.16242        2.379        3.99295       2.09055      0.763596        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000294292   0.000526469   0.00120106    0.000417713   0.000292464      19     
produce:reco3d:SpacePointSolver                    9.66144       14.9811       21.0693       14.6241       3.60773        19     
produce:hitpdune:DisambigFromSpacePoints          0.178636      0.280055      0.474492       0.2657       0.0624492       19     
produce:cluster3d:Cluster3D                        2.49944       46.0777       81.3347       60.424        30.3191        19     
[art]:TriggerResults:TriggerResultInserter       3.1439e-05    4.96932e-05   0.000140103   4.6156e-05    2.32935e-05      19     
end_path:out1:RootOutput                          6.592e-06    1.12237e-05   3.8302e-05     9.618e-06    6.65371e-06      19     
end_path:out1:RootOutput(write)                    5.16503       7.67993       9.5941        7.4505        1.34075        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5563.16 MB
  Peak resident set size usage (VmHWM): 3571.07 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0385_df-s02-d0_dw_0_20260203T210827_reco_stage0_20260204T011440_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0385_df-s02-d0_dw_0_20260203T210827_reco_stage0_20260204T011440_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0385_df-s02-d0_dw_0_20260203T210827_reco_stage0_20260204T011440_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163340.0,
    "core.end_time": 1770167682.0,
    "core.events": [
      43896,
      43902,
      43908,
      43914,
      43920,
      43926,
      43932,
      43938,
      43944,
      43950,
      43956,
      43962,
      43968,
      43974,
      43980,
      43986,
      43992,
      43998,
      44004
    ],
    "core.event_count": 19,
    "core.first_event_number": 43896,
    "core.last_event_number": 44004,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0385_df-s02-d0_dw_0_20260203T210827.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:37:44 UTC       justIN version: 01.06.00