justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 40584.74@dunegpschedd02.fnal.gov

Jobsub ID40584.74@dunegpschedd02.fnal.gov
Workflow ID2657
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-16 19:48:18
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2025-09-16 21:51:39
From worker nodeHostnamecomp05-26
cpuinfoIntel(R) Xeon(R) Platinum 8352Y CPU @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-16 19:49:06
Input filesvd-protodune:np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444.hdf5
JobscriptExit code0
Real time1h (6998s)
CPU time1h (3877s = 55%)
Max RSS bytes2992836608 (2854 MiB)
Outputting started2025-09-16 21:45:45
Output filesvd-protodune-det-reco:np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214537_keepup.root
Finished2025-09-16 21:51:39
Saved logsjustin-logs:40584.74-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

desired window for channel 11203
wclsFrameSaver: no samples within desired window for channel 11203
wclsFrameSaver: no samples within desired window for channel 11203
wclsFrameSaver: no samples within desired window for channel 11204
wclsFrameSaver: no samples within desired window for channel 11204
wclsFrameSaver: no samples within desired window for channel 11205
wclsFrameSaver: no samples within desired window for channel 11205
wclsFrameSaver: no samples within desired window for channel 11205
wclsFrameSaver: no samples within desired window for channel 11348
wclsFrameSaver: no samples within desired window for channel 11348
wclsFrameSaver: no samples within desired window for channel 11348
wclsFrameSaver: no samples within desired window for channel 11348
wclsFrameSaver: no samples within desired window for channel 11403
wclsFrameSaver: no samples within desired window for channel 11404
wclsFrameSaver: no samples within desired window for channel 11512
wclsFrameSaver: no samples within desired window for channel 11512
wclsFrameSaver: no samples within desired window for channel 11512
wclsFrameSaver: no samples within desired window for channel 11610
wclsFrameSaver: no samples within desired window for channel 11610
wclsFrameSaver: no samples within desired window for channel 11610
wclsFrameSaver: no samples within desired window for channel 12152
wclsFrameSaver: no samples within desired window for channel 12152
wclsFrameSaver: no samples within desired window for channel 12153
wclsFrameSaver: no samples within desired window for channel 12153
wclsFrameSaver: no samples within desired window for channel 12154
wclsFrameSaver: no samples within desired window for channel 12155
wclsFrameSaver: no samples within desired window for channel 12155
wclsFrameSaver: no samples within desired window for channel 12156
wclsFrameSaver: no samples within desired window for channel 12156
wclsFrameSaver: no samples within desired window for channel 12157
wclsFrameSaver: no samples within desired window for channel 12157
wclsFrameSaver: no samples within desired window for channel 12158
wclsFrameSaver: no samples within desired window for channel 12158
wclsFrameSaver: no samples within desired window for channel 12159
wclsFrameSaver: no samples within desired window for channel 12163
FrameSaver: q=8.58004e+06 n=1013929 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 2036 XUs and 1943 XVs -> 88 XUVs
C:0 T:1 379 XUs and 360 XVs -> 17 XUVs
C:0 T:2 1195 XUs and 1023 XVs -> 69 XUVs
C:0 T:3 668 XUs and 764 XVs -> 37 XUVs
C:0 T:4 381 XUs and 417 XVs -> 20 XUVs
C:0 T:5 3180 XUs and 4529 XVs -> 270 XUVs
C:0 T:6 1500 XUs and 1420 XVs -> 78 XUVs
C:0 T:7 848 XUs and 761 XVs -> 41 XUVs
C:0 T:8 24524 XUs and 26375 XVs -> 1553 XUVs
C:0 T:9 388 XUs and 196 XVs -> 28 XUVs
C:0 T:10 1032 XUs and 1424 XVs -> 70 XUVs
C:0 T:11 1454 XUs and 2026 XVs -> 111 XUVs
C:0 T:12 1352 XUs and 1871 XVs -> 90 XUVs
C:0 T:13 2857 XUs and 3951 XVs -> 214 XUVs
C:0 T:14 668 XUs and 775 XVs -> 49 XUVs
C:0 T:15 1233 XUs and 1278 XVs -> 89 XUVs
2824 XUVs total
1900 collection wire objects
2824 potential space points
Neighbour search...
44830 tests to find 13142 neighbours
Iterating with no regularization...
Begin: 3.04719e+09
0 2.98596e+09
1 2.9824e+09
2 2.98235e+09
Now with regularization...
Begin: 2.9625e+09
0 2.96248e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
16-Sep-2025 22:45:37 BST  Closed output file "np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214537_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        257.661       365.138       522.456       358.039       75.8449        19     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       5.797e-05    7.57715e-05   0.000173179   6.4352e-05    2.56669e-05      19     
produce:tpcrawdecoder:PDVDTPCReader               94.3019       167.528       241.01        157.634       50.774         19     
produce:triggerrawdecoder:PDVDTriggerReader4     0.421005      0.532837       1.01019      0.469491      0.134702        19     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000323707   0.000384581   0.00068629    0.000369452   7.46042e-05      19     
produce:ophit:OpHitFinder                       0.000104783   0.000177993   0.000730573   0.000138559   0.000135031      19     
produce:opflash:OpFlashFinderVerticalDrift      3.8992e-05    6.01999e-05   0.000280695   4.7774e-05    5.21403e-05      19     
produce:wclsdatavd:WireCellToolkit                79.468        89.0032       99.5702       88.3569       5.89454        19     
produce:gaushit:GausHitFinder                     1.37353       1.92561       2.83457       1.78722      0.408589        19     
produce:nhitsfilter:NumberOfHitsFilter          0.000381127   0.000512448   0.000726078   0.000493033   9.1278e-05       19     
produce:reco3d:SpacePointSolver                   12.2927       20.8312       35.3473       19.2837       6.13357        19     
produce:hitpdune:DisambigFromSpacePoints         0.150636      0.282463      0.542127      0.254905      0.100723        19     
produce:pandora:StandardPandora                   28.0207       77.6721       209.856       61.9424       45.139         19     
produce:pandoraTrack:LArPandoraTrackCreation     0.841704       2.00086       6.04226       1.81632       1.14111        19     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0228718     0.0343419     0.0529672     0.0321957    0.00739516       19     
[art]:TriggerResults:TriggerResultInserter      2.6459e-05    3.84726e-05   7.8705e-05    3.8394e-05    1.06533e-05      19     
end_path:out1:RootOutput                         7.38e-06     1.03854e-05   2.5422e-05     9.308e-06    3.8654e-06       19     
end_path:out1:RootOutput(write)                   4.60579       5.28184       5.64892       5.38678      0.298688        19     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4892.05 MB
  Peak resident set size usage (VmHWM): 2992.84 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214537_keepup.root
\tHists: np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214538_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214537_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214537_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758059138.0,
    "core.end_time": 1758059138.0,
    "core.events": [
      20355,
      20359,
      20363,
      20367,
      20371,
      20375,
      20379,
      20383,
      20387,
      20391,
      20395,
      20399,
      20403,
      20407,
      20411,
      20415,
      20419,
      20423,
      20427
    ],
    "core.event_count": 19,
    "core.first_event_number": 20355,
    "core.last_event_number": 20427,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39388
    ],
    "core.runs_subruns": [
      3938800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444_reco_stage1_20250916T214538_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758059138.0,
    "core.end_time": 1758059138.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39388
    ],
    "core.runs_subruns": [
      3938800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039388_0264_df-s05-d3_dw_0_20250916T184444.hdf5"
    }
  ]
}
justIN time: 2025-09-19 04:24:36 UTC       justIN version: 01.05.00