justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41655.93@dunegpschedd02.fnal.gov

Jobsub ID41655.93@dunegpschedd02.fnal.gov
Workflow ID2867
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 05:44:05
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2025-09-18 07:20:51
From worker nodeHostnameheplnc172.pp.rl.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 06:21:12
Input filesvd-protodune:np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143.hdf5
JobscriptExit code0
Real time57m (3453s)
CPU time51m (3109s = 90%)
Max RSS bytes2520477696 (2403 MiB)
Outputting started2025-09-18 07:18:46
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup.root
Finished2025-09-18 07:20:51
Saved logsjustin-logs:41655.93-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lete 
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.35 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.62 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.09 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.88 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.36 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.36 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[08:18:11.775] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[08:18:11.775] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[08:18:11.775] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[08:18:11.775] I [ timer  ] Timer: Total node execution : 23.340000092983246 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 19306 traces tagged "gauss"
FrameSaver: q=3.33411e+06 n=452275 tag=gauss
wclsFrameSaver: saving 23865 traces tagged "wiener"
FrameSaver: q=3.54795e+06 n=439011 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 265 XUs and 218 XVs -> 11 XUVs
C:0 T:1 162 XUs and 283 XVs -> 3 XUVs
C:0 T:2 504 XUs and 491 XVs -> 13 XUVs
C:0 T:3 177 XUs and 180 XVs -> 4 XUVs
C:0 T:4 8734 XUs and 11604 XVs -> 795 XUVs
C:0 T:5 1723 XUs and 2186 XVs -> 59 XUVs
C:0 T:6 497 XUs and 469 XVs -> 22 XUVs
C:0 T:7 936 XUs and 1195 XVs -> 74 XUVs
981 XUVs total
598 collection wire objects
981 potential space points
Neighbour search...
27677 tests to find 7020 neighbours
Iterating with no regularization...
Begin: 6.14293e+09
0 6.12105e+09
1 6.1207e+09
Now with regularization...
Begin: 5.98171e+09
0 5.98169e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
18-Sep-2025 08:18:32 BST  Closed output file "np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        35.1547       70.4812       348.683       56.9181       51.9772        48     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      5.9921e-05    0.000103135   0.000257043   9.6326e-05    3.38974e-05      48     
produce:tpcrawdecoder:PDVDTPCReader               5.49934       5.96324       9.74549       5.81633      0.634782        48     
produce:triggerrawdecoder:PDVDTriggerReader4     0.299613      0.304846      0.341896      0.302753     0.00751265       48     
produce:pdvddaphne:DAPHNEReaderPDVD               2.34739       2.85451       3.17863       2.85993      0.210076        48     
produce:ophit:OpHitFinder                        0.0342486     0.045738      0.059424      0.0461945    0.00486585       48     
produce:opflash:OpFlashFinderVerticalDrift      0.00413739    0.00899651     0.0152601     0.0085366    0.00280759       48     
produce:wclsdatavd:WireCellToolkit                19.6686       28.7196       50.546        24.5517       7.68605        48     
produce:gaushit:GausHitFinder                    0.233264       0.58559       1.03333      0.564295      0.188287        48     
produce:nhitsfilter:NumberOfHitsFilter          6.5241e-05    0.000152718   0.000405434   0.000144491   5.20044e-05      48     
produce:reco3d:SpacePointSolver                   1.45209       4.99988       12.7641       4.27042       2.71063        48     
produce:hitpdune:DisambigFromSpacePoints         0.0196847     0.068655      0.216439      0.055326      0.0447606       48     
produce:pandora:StandardPandora                   2.55703       24.3249       272.31        10.9805       44.3304        48     
produce:pandoraTrack:LArPandoraTrackCreation     0.0676386      0.24859      0.765946      0.190098      0.152497        48     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00499685     0.0142245     0.0291402     0.0128922    0.00512412       48     
[art]:TriggerResults:TriggerResultInserter       1.697e-05    3.14023e-05   9.5991e-05     2.855e-05    1.28779e-05      48     
end_path:out1:RootOutput                         3.57e-06     7.2051e-06     3.698e-05     5.715e-06    5.86094e-06      48     
end_path:out1:RootOutput(write)                   1.75421       2.32282       3.11771       2.28252      0.320824        48     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4497.04 MB
  Peak resident set size usage (VmHWM): 2520.48 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup.root
\tHists: np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758179913.0,
    "core.end_time": 1758179913.0,
    "core.events": [
      98935,
      98941,
      98947,
      98953,
      98959,
      98965,
      98971,
      98977,
      98983,
      98989,
      98995,
      99001,
      99007,
      99013,
      99019,
      99025,
      99031,
      99037,
      99043,
      99049,
      99055,
      99061,
      99067,
      99073,
      99079,
      99085,
      99091,
      99097,
      99103,
      99109,
      99115,
      99121,
      99127,
      99133,
      99139,
      99145,
      99151,
      99157,
      99163,
      99169,
      99175,
      99181,
      99187,
      99193,
      99199,
      99205,
      99211,
      99217
    ],
    "core.event_count": 48,
    "core.first_event_number": 98935,
    "core.last_event_number": 99217,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143_reco_stage1_20250918T071832_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758179913.0,
    "core.end_time": 1758179913.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0345_df-s05-d1_dw_0_20250918T053143.hdf5"
    }
  ]
}
justIN time: 2025-09-19 01:41:00 UTC       justIN version: 01.05.00