justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 42635.5@dunegpschedd02.fnal.gov

Jobsub ID42635.5@dunegpschedd02.fnal.gov
Workflow ID2920
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 20:10:49
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-09-18 23:34:11
From worker nodeHostnameheplnc162.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 22:27:24
Input filesvd-protodune:np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351.hdf5
JobscriptExit code0
Real time1h (3859s)
CPU time58m (3489s = 90%)
Max RSS bytes2837884928 (2706 MiB)
Outputting started2025-09-18 23:31:43
Output filesvd-protodune-det-reco:np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup.root
Finished2025-09-18 23:34:11
Saved logsjustin-logs:42635.5-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e=1 
[00:31:09.581] D [  glue  ] <ChannelSelector:chsel0> see EOS at call=39 
[00:31:09.581] D [  aux   ] <Resampler:resmp0> EOS at call=79 
[00:31:09.581] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=79 anode=0 
[00:31:09.581] D [  glue  ] <FrameFanin:nfsp> EOS at call=79 with 8 
[00:31:09.581] D [  glue  ] frame sink sees EOS
[00:31:09.581] D [ pgraph ] <Pgrapher:> graph execution complete 
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.1 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.73 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.59 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.3 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.4 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.25 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.25 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.22 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0.01 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[00:31:09.581] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[00:31:09.581] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[00:31:09.581] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[00:31:09.581] I [ timer  ] Timer: Total node execution : 39.860000271350145 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 16676 traces tagged "gauss"
FrameSaver: q=1.95848e+06 n=341177 tag=gauss
wclsFrameSaver: saving 21670 traces tagged "wiener"
FrameSaver: q=2.07617e+06 n=322468 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 2702 XUs and 4165 XVs -> 260 XUVs
C:0 T:1 536 XUs and 676 XVs -> 40 XUVs
C:0 T:2 1062 XUs and 1587 XVs -> 100 XUVs
C:0 T:3 437 XUs and 948 XVs -> 34 XUVs
C:0 T:4 738 XUs and 696 XVs -> 30 XUVs
C:0 T:5 549 XUs and 861 XVs -> 55 XUVs
C:0 T:6 769 XUs and 1508 XVs -> 55 XUVs
C:0 T:7 528 XUs and 356 XVs -> 30 XUVs
604 XUVs total
444 collection wire objects
604 potential space points
Neighbour search...
13570 tests to find 6324 neighbours
Iterating with no regularization...
Begin: 9.0978e+07
0 8.70946e+07
1 8.68499e+07
2 8.68415e+07
Now with regularization...
Begin: 8.5161e+07
0 8.51582e+07
19-Sep-2025 00:31:30 BST  Closed output file "np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        49.144        94.6612       268.049       78.2809       46.5539        40     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.2026e-05    0.000117731   0.000520506   9.9186e-05    7.18276e-05      40     
produce:tpcrawdecoder:PDVDTPCReader               5.93918       6.74816       7.89264       6.61786      0.503466        40     
produce:triggerrawdecoder:PDVDTriggerReader4     0.300092      0.305615      0.322848      0.304027     0.00516703       40     
produce:pdvddaphne:DAPHNEReaderPDVD               2.53515       3.18459       3.66095       3.18607      0.232149        40     
produce:ophit:OpHitFinder                        0.0354995     0.0535644     0.0732228     0.0545078     0.0063641       40     
produce:opflash:OpFlashFinderVerticalDrift      0.00430883     0.0120639     0.0196146     0.0122816    0.00364778       40     
produce:wclsdatavd:WireCellToolkit                26.5256       39.8532       74.0592       34.1271       11.7845        40     
produce:gaushit:GausHitFinder                    0.458659      0.802764       1.40816      0.769625       0.21087        40     
produce:nhitsfilter:NumberOfHitsFilter          0.000118402   0.000179057   0.00050124    0.000164018   6.31525e-05      40     
produce:reco3d:SpacePointSolver                   2.18556       6.94492       18.3206       6.12677       3.31605        40     
produce:hitpdune:DisambigFromSpacePoints         0.0228104      0.10165      0.306201      0.0854265     0.0676437       40     
produce:pandora:StandardPandora                   6.02497       33.1136       180.745       18.2892       36.9588        40     
produce:pandoraTrack:LArPandoraTrackCreation     0.110249      0.361494       1.1566       0.292641      0.217888        40     
produce:pandoraGnocalo:GnocchiCalorimetry        0.008979      0.0204129     0.0369447     0.0208436    0.00636112       40     
[art]:TriggerResults:TriggerResultInserter      1.4998e-05    3.27129e-05   0.00010146    3.0863e-05    1.26361e-05      40     
end_path:out1:RootOutput                         3.957e-06    7.1642e-06    2.9406e-05     5.556e-06    4.1954e-06       40     
end_path:out1:RootOutput(write)                   2.65029       3.14186       3.70828       3.10176      0.257407        40     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4779.63 MB
  Peak resident set size usage (VmHWM): 2818.58 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup.root
\tHists: np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758238291.0,
    "core.end_time": 1758238291.0,
    "core.events": [
      20738,
      20744,
      20750,
      20756,
      20762,
      20768,
      20774,
      20780,
      20786,
      20792,
      20798,
      20804,
      20810,
      20816,
      20822,
      20828,
      20834,
      20840,
      20846,
      20852,
      20858,
      20864,
      20870,
      20876,
      20882,
      20888,
      20894,
      20900,
      20906,
      20912,
      20918,
      20924,
      20930,
      20936,
      20942,
      20948,
      20954,
      20960,
      20966,
      20972
    ],
    "core.event_count": 40,
    "core.first_event_number": 20738,
    "core.last_event_number": 20972,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351_reco_stage1_20250918T233130_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758238291.0,
    "core.end_time": 1758238291.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0085_df-s04-d3_dw_0_20250918T203351.hdf5"
    }
  ]
}
justIN time: 2025-09-19 08:53:24 UTC       justIN version: 01.05.00