justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 49050.0@dunegpschedd01.fnal.gov

Jobsub ID49050.0@dunegpschedd01.fnal.gov
Workflow ID2920
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 20:08:50
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-09-18 22:04:16
From worker nodeHostnamewn1209290.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 20:12:27
Input filesvd-protodune:np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620.hdf5
JobscriptExit code0
Real time1h (6543s)
CPU time1h (5918s = 90%)
Max RSS bytes2791153664 (2661 MiB)
Outputting started2025-09-18 22:01:32
Output filesvd-protodune-det-reco:np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220116_keepup.root
Finished2025-09-18 22:04:16
Saved logsjustin-logs:49050.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

proc1> EOS at call=81 anode=1 
[23:00:18.432] D [  glue  ] <ChannelSelector:chsel0> see EOS at call=40 
[23:00:18.432] D [  aux   ] <Resampler:resmp0> EOS at call=81 
[23:00:18.432] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=81 anode=0 
[23:00:18.432] D [  glue  ] <FrameFanin:nfsp> EOS at call=81 with 8 
[23:00:18.432] D [  glue  ] frame sink sees EOS
[23:00:18.432] D [ pgraph ] <Pgrapher:> graph execution complete 
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 15.36 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.18 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.13 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.51 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.58 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.44 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.44 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.44 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[23:00:18.432] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[23:00:18.432] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[23:00:18.432] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[23:00:18.432] I [ timer  ] Timer: Total node execution : 58.130000280216336 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 19896 traces tagged "gauss"
FrameSaver: q=4.05312e+06 n=518575 tag=gauss
wclsFrameSaver: saving 24802 traces tagged "wiener"
FrameSaver: q=4.3739e+06 n=506936 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1505 XUs and 1704 XVs -> 75 XUVs
C:0 T:1 12398 XUs and 15528 XVs -> 1069 XUVs
C:0 T:2 345 XUs and 318 XVs -> 7 XUVs
C:0 T:3 160 XUs and 186 XVs -> 14 XUVs
C:0 T:4 6393 XUs and 9086 XVs -> 637 XUVs
C:0 T:5 292 XUs and 304 XVs -> 11 XUVs
C:0 T:6 342 XUs and 447 XVs -> 12 XUVs
C:0 T:7 359 XUs and 509 XVs -> 20 XUVs
1845 XUVs total
1001 collection wire objects
1845 potential space points
Neighbour search...
50011 tests to find 12608 neighbours
Iterating with no regularization...
Begin: 5.13041e+09
0 5.02656e+09
1 5.02244e+09
Now with regularization...
Begin: 4.92076e+09
0 4.92045e+09
18-Sep-2025 23:01:16 BST  Closed output file "np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220116_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        79.461        157.366       513.068       129.982       83.6785        41     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.319e-05    0.00015455    0.000305019   0.000142125   4.86972e-05      41     
produce:tpcrawdecoder:PDVDTPCReader               7.50879       8.69228       10.4453       8.54846      0.670271        41     
produce:triggerrawdecoder:PDVDTriggerReader4     0.375412      0.385852      0.423378      0.381398      0.0107285       41     
produce:pdvddaphne:DAPHNEReaderPDVD               4.74949       6.28863       6.93994       6.2641       0.400719        41     
produce:ophit:OpHitFinder                        0.0519981     0.0808633     0.120496      0.0778119     0.0143169       41     
produce:opflash:OpFlashFinderVerticalDrift      0.00829923     0.0195109     0.0345268     0.0184224    0.00576941       41     
produce:wclsdatavd:WireCellToolkit                45.7764       69.8939       139.709       60.5125       22.8518        41     
produce:gaushit:GausHitFinder                     0.46946       1.18961       1.81175       1.1853       0.325079        41     
produce:nhitsfilter:NumberOfHitsFilter          0.000155909   0.000285531   0.000539089   0.000277395   7.40214e-05      41     
produce:reco3d:SpacePointSolver                   2.9595        8.82264       19.9686       7.93377       3.94651        41     
produce:hitpdune:DisambigFromSpacePoints         0.0294744     0.199602      0.624239      0.170475      0.138753        41     
produce:pandora:StandardPandora                   5.59356       56.7264       395.867       32.3928       70.705         41     
produce:pandoraTrack:LArPandoraTrackCreation     0.128564      0.835029       3.14723      0.666084       0.58352        41     
produce:pandoraGnocalo:GnocchiCalorimetry        0.011748      0.028672      0.0468898     0.0290463    0.00912896       41     
[art]:TriggerResults:TriggerResultInserter      2.1732e-05    3.38064e-05   7.4072e-05    3.3166e-05    8.19691e-06      41     
end_path:out1:RootOutput                         3.357e-06    8.63449e-06    4.079e-05     8.45e-06     5.42552e-06      41     
end_path:out1:RootOutput(write)                   2.62817       4.17221       5.25486       4.15667      0.549045        41     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4697.35 MB
  Peak resident set size usage (VmHWM): 2791.15 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220116_keepup.root
\tHists: np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220117_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220116_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220116_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758232878.0,
    "core.end_time": 1758232878.0,
    "core.events": [
      7801,
      7807,
      7813,
      7819,
      7825,
      7831,
      7837,
      7843,
      7849,
      7855,
      7861,
      7867,
      7873,
      7879,
      7885,
      7891,
      7897,
      7903,
      7909,
      7915,
      7921,
      7927,
      7933,
      7939,
      7945,
      7951,
      7957,
      7963,
      7969,
      7975,
      7981,
      7987,
      7993,
      7999,
      8005,
      8011,
      8017,
      8023,
      8029,
      8035,
      8041
    ],
    "core.event_count": 41,
    "core.first_event_number": 7801,
    "core.last_event_number": 8041,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620_reco_stage1_20250918T220117_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758232878.0,
    "core.end_time": 1758232878.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39499
    ],
    "core.runs_subruns": [
      3949900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039499_0032_df-s04-d2_dw_0_20250918T184620.hdf5"
    }
  ]
}
justIN time: 2025-09-18 23:36:56 UTC       justIN version: 01.05.00