justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 49243.143@dunegpschedd01.fnal.gov

Jobsub ID49243.143@dunegpschedd01.fnal.gov
Workflow ID2973
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-19 00:47:04
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-09-19 05:54:03
From worker nodeHostnamewn1208241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-19 03:58:09
Input filesvd-protodune:np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958.hdf5
JobscriptExit code0
Real time1h (6698s)
CPU time1h (4742s = 70%)
Max RSS bytes2670972928 (2547 MiB)
Outputting started2025-09-19 05:49:47
Output filesvd-protodune-det-reco:np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054927_keepup.root
Finished2025-09-19 05:54:03
Saved logsjustin-logs:49243.143-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

81 anode=1 
[06:48:56.651] D [  glue  ] <ChannelSelector:chsel0> see EOS at call=40 
[06:48:56.651] D [  aux   ] <Resampler:resmp0> EOS at call=81 
[06:48:56.651] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=81 anode=0 
[06:48:56.651] D [  glue  ] <FrameFanin:nfsp> EOS at call=81 with 8 
[06:48:56.651] D [  glue  ] frame sink sees EOS
[06:48:56.651] D [ pgraph ] <Pgrapher:> graph execution complete 
[06:48:56.651] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.4 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.65 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 10.3 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.74 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.4 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.4 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.38 sec
[06:48:56.651] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.38 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[06:48:56.652] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[06:48:56.652] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:48:56.652] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[06:48:56.652] I [ timer  ] Timer: Total node execution : 45.669999200850725 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 19014 traces tagged "gauss"
FrameSaver: q=4.13751e+06 n=467066 tag=gauss
wclsFrameSaver: saving 23134 traces tagged "wiener"
FrameSaver: q=4.35863e+06 n=454582 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 447 XUs and 633 XVs -> 23 XUVs
C:0 T:1 326 XUs and 398 XVs -> 9 XUVs
C:0 T:2 207 XUs and 247 XVs -> 5 XUVs
C:0 T:3 238 XUs and 162 XVs -> 3 XUVs
C:0 T:4 609 XUs and 652 XVs -> 30 XUVs
C:0 T:5 771 XUs and 840 XVs -> 23 XUVs
C:0 T:6 201 XUs and 185 XVs -> 9 XUVs
C:0 T:7 7313 XUs and 9034 XVs -> 472 XUVs
574 XUVs total
439 collection wire objects
574 potential space points
Neighbour search...
8886 tests to find 2112 neighbours
Iterating with no regularization...
Begin: 2.33949e+10
0 2.3107e+10
1 2.31045e+10
Now with regularization...
Begin: 2.30566e+10
0 2.30565e+10
19-Sep-2025 06:49:27 BST  Closed output file "np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054927_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        106.969       160.253       639.678       138.975       84.6908        41     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      7.7957e-05    9.84778e-05   0.000269828   8.9589e-05    3.23547e-05      41     
produce:tpcrawdecoder:PDVDTPCReader               26.5004       37.1748       51.9832       36.5866       6.69827        41     
produce:triggerrawdecoder:PDVDTriggerReader4     0.375831      0.420368      0.747333      0.399592      0.0629837       41     
produce:pdvddaphne:DAPHNEReaderPDVD               7.13699       9.26283       12.7487       9.03662       1.33056        41     
produce:ophit:OpHitFinder                        0.056979      0.0695738     0.121708      0.0677727     0.0102666       41     
produce:opflash:OpFlashFinderVerticalDrift       0.0110861     0.0196754     0.0452578     0.0186758    0.00598003       41     
produce:wclsdatavd:WireCellToolkit                40.2302       56.1309       94.9472       49.9621       13.9787        41     
produce:gaushit:GausHitFinder                    0.542859       1.04532       2.1078        1.0217       0.305188        41     
produce:nhitsfilter:NumberOfHitsFilter          0.000114295   0.00020444    0.000651126   0.00018563    8.46278e-05      41     
produce:reco3d:SpacePointSolver                   3.36968       9.26312       30.3122       8.35958       4.50874        41     
produce:hitpdune:DisambigFromSpacePoints         0.0361062     0.131763      0.833176      0.108827      0.122115        41     
produce:pandora:StandardPandora                   6.06828       41.8946       489.784       25.1827       74.8668        41     
produce:pandoraTrack:LArPandoraTrackCreation     0.147597       0.54444       2.59954      0.492713       0.38567        41     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0161705     0.0297158     0.0637061     0.0283059    0.00874755       41     
[art]:TriggerResults:TriggerResultInserter      2.0559e-05    2.86324e-05   8.9488e-05    2.5878e-05    1.10876e-05      41     
end_path:out1:RootOutput                         4.098e-06    5.4701e-06    2.9475e-05     4.819e-06    3.83976e-06      41     
end_path:out1:RootOutput(write)                   3.65722       4.24646       5.67698       4.15286      0.414627        41     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4603.78 MB
  Peak resident set size usage (VmHWM): 2670.97 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054927_keepup.root
\tHists: np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054928_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054927_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054927_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758260969.0,
    "core.end_time": 1758260969.0,
    "core.events": [
      47081,
      47087,
      47093,
      47099,
      47105,
      47111,
      47117,
      47123,
      47129,
      47135,
      47141,
      47147,
      47153,
      47159,
      47165,
      47171,
      47177,
      47183,
      47189,
      47195,
      47201,
      47207,
      47213,
      47219,
      47225,
      47231,
      47237,
      47243,
      47249,
      47255,
      47261,
      47267,
      47273,
      47279,
      47285,
      47291,
      47297,
      47303,
      47309,
      47315,
      47321
    ],
    "core.event_count": 41,
    "core.first_event_number": 47081,
    "core.last_event_number": 47321,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958_reco_stage1_20250919T054928_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758260969.0,
    "core.end_time": 1758260969.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0193_df-s05-d5_dw_0_20250919T000958.hdf5"
    }
  ]
}
justIN time: 2025-09-19 08:45:29 UTC       justIN version: 01.05.00