justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.68@dunegpschedd02.fnal.gov

Jobsub ID297923.68@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2026-02-04 01:02:31
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:58:53
Input filesvd-protodune:np02vd_raw_run042433_0617_df-s03-d1_dw_0_20260203T223639.hdf5
JobscriptExit code0
Real time1h (3667s)
CPU time55m (3328s = 90%)
Max RSS bytes3356741632 (3201 MiB)
Outputting started2026-02-04 01:00:01
Output files
Finished2026-02-04 01:02:31
Saved logsjustin-logs:297923.68-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ptr... 0x118f8480, 0x59be8cc0
Caught exception in looking up space point ptr... 0x5dcf03e0, 0x5dcf1400
Caught exception in looking up space point ptr... 0x5dcf1400, 0x525ceeb0
Caught exception in looking up space point ptr... 0x3b317140, 0x7c756df0
Caught exception in looking up space point ptr... 0x7c756df0, 0x3b317140
Caught exception in looking up space point ptr... 0x39097230, 0x390975a0
Caught exception in looking up space point ptr... 0x390975a0, 0x39097730
Caught exception in looking up space point ptr... 0x8a24d9d0, 0x8a24d710
Caught exception in looking up space point ptr... 0x8a24d710, 0x8a24d9d0
Caught exception in looking up space point ptr... 0x5dcfb1a0, 0x5dcfc830
Caught exception in looking up space point ptr... 0x5dcfc830, 0x5dcfc960
Caught exception in looking up space point ptr... 0x50d3dcd0, 0x3160de50
Caught exception in looking up space point ptr... 0x3160de50, 0x2adeb200
Caught exception in looking up space point ptr... 0x59589340, 0x5956d5d0
Caught exception in looking up space point ptr... 0x5956d5d0, 0x59589340
Caught exception in looking up space point ptr... 0xace80a0, 0xace6c80
Caught exception in looking up space point ptr... 0xace6c80, 0xace80a0
Caught exception in looking up space point ptr... 0x86bd02e0, 0x86bd4c10
Caught exception in looking up space point ptr... 0x86bd4c10, 0x86bd49b0
Caught exception in looking up space point ptr... 0x9ef2100, 0x823938b0
Caught exception in looking up space point ptr... 0x823938b0, 0x9ef2100
Caught exception in looking up space point ptr... 0x6a199c10, 0x6a199a30
Caught exception in looking up space point ptr... 0x5276d670, 0x6a199c10
Caught exception in looking up space point ptr... 0x49030c30, 0x50b72f20
Caught exception in looking up space point ptr... 0x50b72f20, 0x50b71fa0
Caught exception in looking up space point ptr... 0x808de940, 0x808d3130
Caught exception in looking up space point ptr... 0x808d3130, 0x808de940
Caught exception in looking up space point ptr... 0x1a483b20, 0x3160c720
Caught exception in looking up space point ptr... 0x3160c720, 0x822cf30
Caught exception in looking up space point ptr... 0x4113d770, 0x4113dc20
Caught exception in looking up space point ptr... 0x4113dc20, 0x4113ddb0
Caught exception in looking up space point ptr... 0x2af60940, 0x647bc3b0
Caught exception in looking up space point ptr... 0x647bc3b0, 0x2af60940
Caught exception in looking up space point ptr... 0x5278a210, 0x5278a370
Caught exception in looking up space point ptr... 0x5278a370, 0x5278a210
Caught exception in looking up space point ptr... 0x547c0650, 0x547c4df0
Caught exception in looking up space point ptr... 0x547c4df0, 0x547b5890
Caught exception in looking up space point ptr... 0x66ad6a80, 0x66ad6f20
Caught exception in looking up space point ptr... 0x66ad6f20, 0x66ad6a80
Caught exception in looking up space point ptr... 0x547ab120, 0x547ab020
Caught exception in looking up space point ptr... 0x547ab020, 0x547ab120
Caught exception in looking up space point ptr... 0x5dd08470, 0x5dcf2a30
Caught exception in looking up space point ptr... 0x5dcf2a30, 0x5dcf28e0
Caught exception in looking up space point ptr... 0x390f0610, 0x390f04c0
Caught exception in looking up space point ptr... 0x390f04c0, 0x390f0610
Caught exception in looking up space point ptr... 0x592b0740, 0x592b0b00
Caught exception in looking up space point ptr... 0x592b0b00, 0x592b0920
Caught exception in looking up space point ptr... 0x4c8d9630, 0x418befa0
Caught exception in looking up space point ptr... 0x418befa0, 0x4c8d9630
Caught exception in looking up space point ptr... 0x8e0dbc10, 0x8e0dbab0
Caught exception in looking up space point ptr... 0x8e0dbab0, 0x8e0dbc10
Caught exception in looking up space point ptr... 0x8e0dbc10, 0x8e0db6d0
Caught exception in looking up space point ptr... 0x8e0db6d0, 0x8e0dbc10
Caught exception in looking up space point ptr... 0x4af8de10, 0x390e8ab0
Caught exception in looking up space point ptr... 0x390e8ab0, 0x4af8de10
Caught exception in looking up space point ptr... 0x80342b10, 0x80342980
Caught exception in looking up space point ptr... 0x80342980, 0x80342ca0
Caught exception in looking up space point ptr... 0x390789b0, 0x3906f460
Caught exception in looking up space point ptr... 0x3906f460, 0x390789b0
Caught exception in looking up space point ptr... 0x592e1a70, 0x383d8690
Caught exception in looking up space point ptr... 0x383d8690, 0x592e1a70
Caught exception in looking up space point ptr... 0x592e1a70, 0x592e1f50
Caught exception in looking up space point ptr... 0x592e1f50, 0x592e1a70
Caught exception in looking up space point ptr... 0x592e1a70, 0x592d52b0
Caught exception in looking up space point ptr... 0x592d52b0, 0x592e1a70
Caught exception in looking up space point ptr... 0x4795ee50, 0x3bc1da20
Caught exception in looking up space point ptr... 0x3bc1da20, 0x4795ee50
++++>>>> total num hits: 122306, num free: 117289
04-Feb-2026 00:59:53 GMT  Closed output file "np02vd_raw_run042433_0617_df-s03-d1_dw_0_20260203T223639_reco_stage0_20260204T005953_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         167.713       188.125       215.857       189.574       14.2899        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.4617e-05    0.000159942   0.000233277   0.00015503    4.44353e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.7298       15.6733       16.4193       15.6602      0.409804        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.527408      0.543908      0.632586      0.531447      0.0253721       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0176553     0.0180407     0.018667      0.0179855    0.000260518      19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0175376     0.0179779     0.0185012     0.0179412    0.000246475      19     
produce:beamevent:BeamEvent                      0.000147847   0.000226549   0.000986808   0.000183544   0.000181433      19     
produce:pdvddaphne:DAPHNEReaderPDVD                3.12271       3.82422       4.30672       3.85646      0.242652        19     
produce:ophit:OpHitFinder                         0.0378366     0.0477782     0.0586881     0.0493754    0.00614496       19     
produce:wclsdatavd:WireCellToolkit                 112.992       135.864       155.401       139.014       11.2607        19     
produce:gaushit:GausHitFinder                      1.09186       1.73061       2.51435       1.74914       0.42843        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000314159   0.000464215   0.000963434   0.000457186   0.000137245      19     
produce:reco3d:SpacePointSolver                    7.45315       19.4547       28.2226       20.3563       5.08283        19     
produce:hitpdune:DisambigFromSpacePoints          0.0833855     0.264331       0.40628      0.262431      0.0891712       19     
produce:cluster3d:Cluster3D                        1.62364       3.68504       6.04163       3.6704        1.22325        19     
[art]:TriggerResults:TriggerResultInserter       2.6099e-05    4.45806e-05   0.000172002   3.5868e-05    3.11137e-05      19     
end_path:out1:RootOutput                          4.067e-06    1.18475e-05   5.4482e-05     9.308e-06    1.10276e-05      19     
end_path:out1:RootOutput(write)                    4.96924       6.94016       7.93791       7.11363      0.906096        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5374.85 MB
  Peak resident set size usage (VmHWM): 3356.74 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0617_df-s03-d1_dw_0_20260203T223639_reco_stage0_20260204T005953_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0617_df-s03-d1_dw_0_20260203T223639_reco_stage0_20260204T005953_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0617_df-s03-d1_dw_0_20260203T223639_reco_stage0_20260204T005953_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163147.0,
    "core.end_time": 1770166795.0,
    "core.events": [
      70347,
      70353,
      70359,
      70365,
      70371,
      70377,
      70383,
      70389,
      70395,
      70401,
      70407,
      70413,
      70419,
      70425,
      70431,
      70437,
      70443,
      70449,
      70455
    ],
    "core.event_count": 19,
    "core.first_event_number": 70347,
    "core.last_event_number": 70455,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0617_df-s03-d1_dw_0_20260203T223639.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:12:46 UTC       justIN version: 01.06.00