justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41739.134@dunegpschedd02.fnal.gov

Jobsub ID41739.134@dunegpschedd02.fnal.gov
Workflow ID2867
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 07:48:12
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-09-18 10:31:30
From worker nodeHostnamewn1209300.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 08:22:26
Input filesvd-protodune:np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529.hdf5
JobscriptExit code0
Real time2h (7622s)
CPU time1h (4880s = 64%)
Max RSS bytes2890735616 (2756 MiB)
Outputting started2025-09-18 10:29:29
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102912_keepup.root
Finished2025-09-18 10:31:30
Saved logsjustin-logs:41739.134-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

call=93 
[11:28:44.439] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=93 anode=0 
[11:28:44.439] D [  glue  ] <FrameFanin:nfsp> EOS at call=93 with 8 
[11:28:44.439] D [  glue  ] frame sink sees EOS
[11:28:44.440] D [ pgraph ] <Pgrapher:> graph execution complete 
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 17.24 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 15.82 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 15.21 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.93 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.93 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.92 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.74 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.06 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[11:28:44.440] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[11:28:44.440] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[11:28:44.440] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[11:28:44.440] I [ timer  ] Timer: Total node execution : 69.88999966531992 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 16357 traces tagged "gauss"
FrameSaver: q=1.89498e+06 n=327989 tag=gauss
wclsFrameSaver: saving 20340 traces tagged "wiener"
FrameSaver: q=2.00052e+06 n=313565 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 682 XUs and 557 XVs -> 39 XUVs
C:0 T:1 179 XUs and 153 XVs -> 8 XUVs
C:0 T:2 995 XUs and 977 XVs -> 50 XUVs
C:0 T:3 162 XUs and 188 XVs -> 7 XUVs
C:0 T:4 648 XUs and 716 XVs -> 67 XUVs
C:0 T:5 2179 XUs and 1655 XVs -> 62 XUVs
C:0 T:6 433 XUs and 540 XVs -> 27 XUVs
C:0 T:7 1286 XUs and 1421 XVs -> 86 XUVs
346 XUVs total
294 collection wire objects
346 potential space points
Neighbour search...
2282 tests to find 1188 neighbours
Iterating with no regularization...
Begin: 5.31232e+07
0 5.16018e+07
1 5.15895e+07
Now with regularization...
Begin: 5.08512e+07
0 5.08509e+07
18-Sep-2025 11:29:12 BST  Closed output file "np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102912_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        109.344       159.996       239.418       154.996       34.0753        47     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      9.6213e-05    0.000331931   0.00711975    0.000166911   0.00100344       47     
produce:tpcrawdecoder:PDVDTPCReader               29.7387       50.4881       87.5691       49.5814       10.4463        47     
produce:triggerrawdecoder:PDVDTriggerReader4      0.37895      0.445251      0.690447      0.410812      0.0800859       47     
produce:pdvddaphne:DAPHNEReaderPDVD               9.74714       14.1435       19.7195       14.2381       2.25986        47     
produce:ophit:OpHitFinder                        0.0440046     0.0746051     0.118526      0.0751381     0.0147057       47     
produce:opflash:OpFlashFinderVerticalDrift      0.00327746     0.0176598     0.0385671     0.0176927    0.00827953       47     
produce:wclsdatavd:WireCellToolkit                41.0728       58.6003       88.1877       54.707        13.2261        47     
produce:gaushit:GausHitFinder                    0.433324       1.01174       2.45999      0.946524      0.371951        47     
produce:nhitsfilter:NumberOfHitsFilter          0.000135044   0.000268782   0.000504075   0.000256236   7.31935e-05      47     
produce:reco3d:SpacePointSolver                   2.29803       6.50692       14.1243       6.45063       2.68224        47     
produce:hitpdune:DisambigFromSpacePoints         0.0389305     0.120762      0.347311      0.100701      0.0626127       47     
produce:pandora:StandardPandora                   7.22038       24.1556       68.7913       18.743        16.0169        47     
produce:pandoraTrack:LArPandoraTrackCreation     0.138527      0.521413       1.28059      0.426331       0.2726         47     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00760904     0.0238249     0.0466064     0.024027     0.00896269       47     
[art]:TriggerResults:TriggerResultInserter      2.9982e-05    4.54964e-05   0.00011017     4.145e-05    1.47367e-05      47     
end_path:out1:RootOutput                         5.202e-06    1.06493e-05   4.8155e-05     9.854e-06    6.62472e-06      47     
end_path:out1:RootOutput(write)                   2.39991       3.8454        5.58548       3.74898      0.643589        47     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4801.52 MB
  Peak resident set size usage (VmHWM): 2890.74 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102912_keepup.root
\tHists: np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102914_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102912_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102912_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758191355.0,
    "core.end_time": 1758191355.0,
    "core.events": [
      113388,
      113394,
      113400,
      113406,
      113412,
      113418,
      113424,
      113430,
      113436,
      113442,
      113448,
      113454,
      113460,
      113466,
      113472,
      113478,
      113484,
      113490,
      113496,
      113502,
      113508,
      113514,
      113520,
      113526,
      113532,
      113538,
      113544,
      113550,
      113556,
      113562,
      113568,
      113574,
      113580,
      113586,
      113592,
      113598,
      113604,
      113610,
      113616,
      113622,
      113628,
      113634,
      113640,
      113646,
      113652,
      113658,
      113664
    ],
    "core.event_count": 47,
    "core.first_event_number": 113388,
    "core.last_event_number": 113664,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529_reco_stage1_20250918T102914_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758191355.0,
    "core.end_time": 1758191355.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0395_df-s05-d0_dw_0_20250918T073529.hdf5"
    }
  ]
}
justIN time: 2025-09-18 15:58:50 UTC       justIN version: 01.05.00