justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 42777.143@dunegpschedd02.fnal.gov

Jobsub ID42777.143@dunegpschedd02.fnal.gov
Workflow ID2973
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 23:45:01
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2025-09-19 06:13:03
From worker nodeHostnamedune001-7807705.0-lcg2697.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-19 05:19:50
Input filesvd-protodune:np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853.hdf5
JobscriptExit code0
Real time51m (3108s)
CPU time45m (2710s = 87%)
Max RSS bytes3010285568 (2870 MiB)
Outputting started2025-09-19 06:11:38
Output filesvd-protodune-det-reco:np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup.root
Finished2025-09-19 06:13:03
Saved logsjustin-logs:42777.143-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

roc ] <OmnibusSigProc:anode0sigproc0> EOS at call=81 anode=0 
[06:11:05.873] D [  glue  ] <FrameFanin:nfsp> EOS at call=81 with 8 
[06:11:05.873] D [  glue  ] frame sink sees EOS
[06:11:05.873] D [ pgraph ] <Pgrapher:> graph execution complete 
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.33 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.94 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.43 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.06 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.19 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.18 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.17 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.15 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[06:11:05.873] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:11:05.874] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[06:11:05.874] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[06:11:05.874] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[06:11:05.874] I [ timer  ] Timer: Total node execution : 23.4799997638911 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 20451 traces tagged "gauss"
FrameSaver: q=3.17766e+06 n=487234 tag=gauss
wclsFrameSaver: saving 26154 traces tagged "wiener"
FrameSaver: q=3.34487e+06 n=466093 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1313 XUs and 1167 XVs -> 47 XUVs
C:0 T:1 1378 XUs and 1571 XVs -> 83 XUVs
C:0 T:2 10869 XUs and 12390 XVs -> 673 XUVs
C:0 T:3 626 XUs and 1109 XVs -> 66 XUVs
C:0 T:4 280 XUs and 400 XVs -> 23 XUVs
C:0 T:5 91 XUs and 106 XVs -> 4 XUVs
C:0 T:6 266 XUs and 289 XVs -> 9 XUVs
C:0 T:7 674 XUs and 787 XVs -> 32 XUVs
937 XUVs total
715 collection wire objects
937 potential space points
Neighbour search...
15913 tests to find 4548 neighbours
Iterating with no regularization...
Begin: 2.60257e+09
0 2.57676e+09
1 2.57658e+09
Now with regularization...
Begin: 2.51592e+09
0 2.51587e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
19-Sep-2025 06:11:28 UTC  Closed output file "np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        40.9855       73.6407       281.061       55.8902       50.0536        41     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.4397e-05    0.000131489   0.000437068   0.000107081   7.20017e-05      41     
produce:tpcrawdecoder:PDVDTPCReader               6.95445       7.52683       9.88324       7.38355       0.51836        41     
produce:triggerrawdecoder:PDVDTriggerReader4     0.391986      0.402168       0.45271      0.397006      0.0127944       41     
produce:pdvddaphne:DAPHNEReaderPDVD               1.94747       2.54393       3.19929       2.50153      0.246754        41     
produce:ophit:OpHitFinder                        0.0260235     0.0342198     0.0506019     0.0328676    0.00651811       41     
produce:opflash:OpFlashFinderVerticalDrift      0.00432733     0.0106455     0.0289914    0.00952166    0.00492933       41     
produce:wclsdatavd:WireCellToolkit                21.3418       28.9811       64.7767       25.9092       8.64883        41     
produce:gaushit:GausHitFinder                    0.217897      0.393435      0.747014      0.374819      0.113547        41     
produce:nhitsfilter:NumberOfHitsFilter          0.000152069   0.000485006    0.0012577    0.000435154   0.000256542      41     
produce:reco3d:SpacePointSolver                   2.11744       4.87629       11.477        4.65271       1.89299        41     
produce:hitpdune:DisambigFromSpacePoints         0.0153782     0.0716524      0.33971      0.0550577     0.0581849       41     
produce:pandora:StandardPandora                   2.22987       26.4624       223.853       11.3675       46.3009        41     
produce:pandoraTrack:LArPandoraTrackCreation     0.0488298     0.628012       4.67111      0.536466      0.717743        41     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00527114     0.0141785     0.0304744     0.0129939    0.00560248       41     
[art]:TriggerResults:TriggerResultInserter      1.8818e-05    3.83317e-05   0.000115334   3.5093e-05    1.82842e-05      41     
end_path:out1:RootOutput                         3.726e-06    1.05788e-05   3.3631e-05     8.593e-06    6.57135e-06      41     
end_path:out1:RootOutput(write)                   1.40661       1.66057       1.90421       1.65888      0.124238        41     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5356.04 MB
  Peak resident set size usage (VmHWM): 3010.29 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup.root
\tHists: np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758262289.0,
    "core.end_time": 1758262289.0,
    "core.events": [
      53025,
      53031,
      53037,
      53043,
      53049,
      53055,
      53061,
      53067,
      53073,
      53079,
      53085,
      53091,
      53097,
      53103,
      53109,
      53115,
      53121,
      53127,
      53133,
      53139,
      53145,
      53151,
      53157,
      53163,
      53169,
      53175,
      53181,
      53187,
      53193,
      53199,
      53205,
      53211,
      53217,
      53223,
      53229,
      53235,
      53241,
      53247,
      53253,
      53259,
      53265
    ],
    "core.event_count": 41,
    "core.first_event_number": 53025,
    "core.last_event_number": 53265,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853_reco_stage1_20250919T061128_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758262289.0,
    "core.end_time": 1758262289.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0217_df-s05-d3_dw_0_20250919T005853.hdf5"
    }
  ]
}
justIN time: 2025-09-19 08:47:24 UTC       justIN version: 01.05.00