justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 49074.0@dunegpschedd01.fnal.gov

Jobsub ID49074.0@dunegpschedd01.fnal.gov
Workflow ID2867
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 20:42:50
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce03
Last heartbeat2025-09-19 03:37:39
From worker nodeHostnamedune001-8710001.0-lcg2677.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 22:40:38
Input filesvd-protodune:np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332.hdf5
JobscriptExit code0
Real time4h (17727s)
CPU time4h (17077s = 96%)
Max RSS bytes4334448640 (4133 MiB)
Outputting started2025-09-19 03:36:06
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup.root
Finished2025-09-19 03:37:39
Saved logsjustin-logs:49074.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

call=95 
[03:34:50.071] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=95 anode=0 
[03:34:50.071] D [  glue  ] <FrameFanin:nfsp> EOS at call=95 with 8 
[03:34:50.071] D [  glue  ] frame sink sees EOS
[03:34:50.071] D [ pgraph ] <Pgrapher:> graph execution complete 
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 10.33 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.97 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.55 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.31 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.49 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.28 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[03:34:50.071] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[03:34:50.072] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[03:34:50.072] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:34:50.072] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[03:34:50.072] I [ timer  ] Timer: Total node execution : 30.659999864175916 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 26219 traces tagged "gauss"
FrameSaver: q=5.13364e+06 n=632981 tag=gauss
wclsFrameSaver: saving 34156 traces tagged "wiener"
FrameSaver: q=5.60743e+06 n=606488 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1543 XUs and 1811 XVs -> 122 XUVs
C:0 T:1 704 XUs and 1047 XVs -> 41 XUVs
C:0 T:2 5024 XUs and 5449 XVs -> 295 XUVs
C:0 T:3 1840 XUs and 1564 XVs -> 176 XUVs
C:0 T:4 36733 XUs and 35077 XVs -> 3450 XUVs
C:0 T:5 8845 XUs and 6922 XVs -> 515 XUVs
C:0 T:6 585 XUs and 882 XVs -> 38 XUVs
C:0 T:7 304 XUs and 540 XVs -> 10 XUVs
4647 XUVs total
1644 collection wire objects
4647 potential space points
Neighbour search...
238729 tests to find 55794 neighbours
Iterating with no regularization...
Begin: 5.6164e+09
0 4.78466e+09
1 4.70595e+09
2 4.7028e+09
Now with regularization...
Begin: 4.63039e+09
0 4.62985e+09
19-Sep-2025 03:35:53 UTC  Closed output file "np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        40.5181       367.123       12059.5       63.1437       1721.77        48     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      7.9539e-05    0.000231137   0.00442592    0.000127795   0.000614391      48     
produce:tpcrawdecoder:PDVDTPCReader               7.08386       8.24678       30.5669       7.44965       3.36501        48     
produce:triggerrawdecoder:PDVDTriggerReader4     0.390323      0.407934       0.74305      0.394825      0.053195        48     
produce:pdvddaphne:DAPHNEReaderPDVD               2.79137       3.40746       4.78433       3.37523      0.399601        48     
produce:ophit:OpHitFinder                        0.0301232     0.0495062     0.0732616     0.0489748    0.00840034       48     
produce:opflash:OpFlashFinderVerticalDrift      0.00545894     0.0114276     0.0200905     0.0108611    0.00362946       48     
produce:wclsdatavd:WireCellToolkit                22.5517       34.909        88.9664       29.7079       12.8432        48     
produce:gaushit:GausHitFinder                    0.248026      0.643355       1.88903      0.583182      0.356103        48     
produce:nhitsfilter:NumberOfHitsFilter          9.4657e-05    0.000271528   0.000955261   0.00018977    0.000196958      48     
produce:reco3d:SpacePointSolver                   1.24097       6.59837       45.5119       4.77896       7.67322        48     
produce:hitpdune:DisambigFromSpacePoints         0.011855      0.106208       1.13148      0.0616391     0.170455        48     
produce:pandora:StandardPandora                   2.41877       309.918       11904.6       11.4474       1705.96        48     
produce:pandoraTrack:LArPandoraTrackCreation     0.0693605     0.438412       3.46893      0.217851      0.700489        48     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00473219     0.0175387     0.103628       0.01278      0.015353        48     
[art]:TriggerResults:TriggerResultInserter      2.1891e-05    4.05244e-05   9.2002e-05    3.7896e-05    1.36188e-05      48     
end_path:out1:RootOutput                         3.436e-06    1.01565e-05   3.0197e-05    9.5825e-06    4.11454e-06      48     
end_path:out1:RootOutput(write)                   1.77874       2.3396         3.303        2.23645      0.348503        48     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6535.87 MB
  Peak resident set size usage (VmHWM): 4334.45 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup.root
\tHists: np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758252954.0,
    "core.end_time": 1758252954.0,
    "core.events": [
      95625,
      95631,
      95637,
      95643,
      95649,
      95655,
      95661,
      95667,
      95673,
      95679,
      95685,
      95691,
      95697,
      95703,
      95709,
      95715,
      95721,
      95727,
      95733,
      95739,
      95745,
      95751,
      95757,
      95763,
      95769,
      95775,
      95781,
      95787,
      95793,
      95799,
      95805,
      95811,
      95817,
      95823,
      95829,
      95835,
      95841,
      95847,
      95853,
      95859,
      95865,
      95871,
      95877,
      95883,
      95889,
      95895,
      95901,
      95907
    ],
    "core.event_count": 48,
    "core.first_event_number": 95625,
    "core.last_event_number": 95907,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332_reco_stage1_20250919T033553_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758252954.0,
    "core.end_time": 1758252954.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0333_df-s05-d3_dw_0_20250918T050332.hdf5"
    }
  ]
}
justIN time: 2025-09-19 08:42:39 UTC       justIN version: 01.05.00