justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 42777.46@dunegpschedd02.fnal.gov

Jobsub ID42777.46@dunegpschedd02.fnal.gov
Workflow ID2973
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 23:45:01
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-09-19 05:39:15
From worker nodeHostnamewn1208251.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-19 03:20:33
Input filesvd-protodune:np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341.hdf5
JobscriptExit code0
Real time2h (8057s)
CPU time1h (5793s = 71%)
Max RSS bytes2710953984 (2585 MiB)
Outputting started2025-09-19 05:34:51
Output filesvd-protodune-det-reco:np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053429_keepup.root
Finished2025-09-19 05:39:15
Saved logsjustin-logs:42777.46-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

] <ChannelSelector:chsel0> see EOS at call=40 
[06:33:56.715] D [  aux   ] <Resampler:resmp0> EOS at call=81 
[06:33:56.715] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=81 anode=0 
[06:33:56.715] D [  glue  ] <FrameFanin:nfsp> EOS at call=81 with 8 
[06:33:56.715] D [  glue  ] frame sink sees EOS
[06:33:56.715] D [ pgraph ] <Pgrapher:> graph execution complete 
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.94 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 13.3 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.53 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.07 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.45 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.41 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.41 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.39 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[06:33:56.716] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[06:33:56.716] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:33:56.716] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[06:33:56.716] I [ timer  ] Timer: Total node execution : 52.52999916486442 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 18819 traces tagged "gauss"
FrameSaver: q=3.35675e+06 n=441016 tag=gauss
wclsFrameSaver: saving 26019 traces tagged "wiener"
FrameSaver: q=3.69317e+06 n=418217 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 23918 XUs and 27890 XVs -> 1606 XUVs
C:0 T:1 190 XUs and 397 XVs -> 15 XUVs
C:0 T:2 4126 XUs and 3012 XVs -> 287 XUVs
C:0 T:3 2040 XUs and 1736 XVs -> 279 XUVs
C:0 T:4 77 XUs and 80 XVs -> 2 XUVs
C:0 T:5 1934 XUs and 1542 XVs -> 540 XUVs
C:0 T:6 4142 XUs and 3600 XVs -> 729 XUVs
C:0 T:7 630 XUs and 458 XVs -> 128 XUVs
3586 XUVs total
1419 collection wire objects
3586 potential space points
Neighbour search...
155206 tests to find 68664 neighbours
Iterating with no regularization...
Begin: 3.31907e+09
0 2.9338e+09
1 2.90424e+09
2 2.904e+09
Now with regularization...
Begin: 2.88683e+09
0 2.88676e+09
19-Sep-2025 06:34:29 BST  Closed output file "np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053429_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        112.331       193.432       788.017       152.277       122.627        41     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      8.4179e-05    0.000107094   0.000325434   0.000100249   3.66765e-05      41     
produce:tpcrawdecoder:PDVDTPCReader               24.4249       41.5517       57.0874       41.4643       7.82612        41     
produce:triggerrawdecoder:PDVDTriggerReader4     0.378838      0.421088      0.603725       0.40382      0.0499319       41     
produce:pdvddaphne:DAPHNEReaderPDVD               5.99201       10.3902       14.5834       10.3084       1.97656        41     
produce:ophit:OpHitFinder                        0.0564776     0.0731738     0.117988      0.0711933     0.0101279       41     
produce:opflash:OpFlashFinderVerticalDrift       0.008857      0.0205753     0.0416417     0.0199259    0.00663685       41     
produce:wclsdatavd:WireCellToolkit                44.9781       64.2517       121.783       55.955        18.3279        41     
produce:gaushit:GausHitFinder                    0.644607       1.17803       2.63177       1.10521      0.401203        41     
produce:nhitsfilter:NumberOfHitsFilter          0.000113594   0.00026279    0.000674403   0.000242648   0.000114778      41     
produce:reco3d:SpacePointSolver                   2.74073       10.1958       34.2133       8.58045       6.46099        41     
produce:hitpdune:DisambigFromSpacePoints         0.0558546     0.168386      0.897829      0.115907      0.187721        41     
produce:pandora:StandardPandora                   10.1749       60.0552       561.557       28.4229       100.091        41     
produce:pandoraTrack:LArPandoraTrackCreation     0.183365      0.687815       3.59754      0.509227       0.66288        41     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0153476     0.0321262     0.0632857     0.0302007     0.0102856       41     
[art]:TriggerResults:TriggerResultInserter      2.2752e-05    3.3978e-05     7.941e-05    2.9355e-05    1.27018e-05      41     
end_path:out1:RootOutput                         4.359e-06    5.92563e-06   3.1119e-05     5.23e-06     4.04211e-06      41     
end_path:out1:RootOutput(write)                   3.86459       4.37839       5.21827       4.27948      0.346399        41     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4667.88 MB
  Peak resident set size usage (VmHWM): 2708.86 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053429_keepup.root
\tHists: np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053430_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053429_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053429_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758260071.0,
    "core.end_time": 1758260071.0,
    "core.events": [
      43916,
      43922,
      43928,
      43934,
      43940,
      43946,
      43952,
      43958,
      43964,
      43970,
      43976,
      43982,
      43988,
      43994,
      44000,
      44006,
      44012,
      44018,
      44024,
      44030,
      44036,
      44042,
      44048,
      44054,
      44060,
      44066,
      44072,
      44078,
      44084,
      44090,
      44096,
      44102,
      44108,
      44114,
      44120,
      44126,
      44132,
      44138,
      44144,
      44150,
      44156
    ],
    "core.event_count": 41,
    "core.first_event_number": 43916,
    "core.last_event_number": 44156,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341_reco_stage1_20250919T053430_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758260071.0,
    "core.end_time": 1758260071.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0180_df-s04-d3_dw_0_20250918T234341.hdf5"
    }
  ]
}
justIN time: 2025-09-19 08:02:06 UTC       justIN version: 01.05.00