justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 36945.48@dunegpschedd02.fnal.gov

Jobsub ID36945.48@dunegpschedd02.fnal.gov
Workflow ID2365
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-09 12:43:05
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat2025-09-09 14:06:48
From worker nodeHostnamewn-b8-26-01.brunel.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-09 12:48:30
Input filesvd-protodune:np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539.hdf5
JobscriptExit code0
Real time1h (4618s)
CPU time59m (3586s = 77%)
Max RSS bytes2390024192 (2279 MiB)
Outputting started2025-09-09 14:05:29
Output filesvd-protodune-det-reco:np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140519_keepup.root
Finished2025-09-09 14:06:48
Saved logsjustin-logs:36945.48-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

all=55 with 8 
[15:04:08.520] D [  glue  ] frame sink sees EOS
[15:04:08.520] D [ pgraph ] <Pgrapher:> graph execution complete 
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.01 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.94 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.33 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.81 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.72 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.46 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.38 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.99 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.4 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.39 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.39 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.37 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:04:08.520] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[15:04:08.520] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[15:04:08.520] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[15:04:08.520] I [ timer  ] Timer: Total node execution : 65.23999937064946 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 39296 traces tagged "gauss"
FrameSaver: q=6.53243e+06 n=798210 tag=gauss
wclsFrameSaver: saving 48451 traces tagged "wiener"
FrameSaver: q=6.86875e+06 n=761654 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 234 XUs and 447 XVs -> 24 XUVs
C:0 T:1 1200 XUs and 1724 XVs -> 40 XUVs
C:0 T:2 242 XUs and 237 XVs -> 11 XUVs
C:0 T:3 1166 XUs and 1559 XVs -> 81 XUVs
C:0 T:4 961 XUs and 1308 XVs -> 57 XUVs
C:0 T:5 404 XUs and 535 XVs -> 18 XUVs
C:0 T:6 381 XUs and 693 XVs -> 10 XUVs
C:0 T:7 113 XUs and 211 XVs -> 3 XUVs
C:0 T:8 6952 XUs and 9401 XVs -> 1180 XUVs
C:0 T:9 2218 XUs and 3174 XVs -> 247 XUVs
C:0 T:10 953 XUs and 1759 XVs -> 270 XUVs
C:0 T:11 4320 XUs and 4766 XVs -> 205 XUVs
C:0 T:12 370 XUs and 506 XVs -> 30 XUVs
C:0 T:13 811 XUs and 643 XVs -> 53 XUVs
C:0 T:14 1858 XUs and 2384 XVs -> 135 XUVs
C:0 T:15 3415 XUs and 4542 XVs -> 409 XUVs
2773 XUVs total
1205 collection wire objects
2773 potential space points
Neighbour search...
149437 tests to find 77034 neighbours
Iterating with no regularization...
Begin: 5.65366e+08
0 5.34753e+08
1 5.32733e+08
2 5.32223e+08
Now with regularization...
Begin: 5.26142e+08
0 5.26016e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
09-Sep-2025 15:05:19 BST  Closed output file "np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140519_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        131.503       162.819       217.526       160.441       23.6175        28     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      4.8928e-05    7.86881e-05   0.000220982   6.14075e-05   3.53737e-05      28     
produce:tpcrawdecoder:PDVDTPCReader               30.5371       35.1477       42.0189       33.6587       3.66315        28     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0349986     0.047281      0.0991671     0.0468413     0.0151225       28     
produce:pdvddaphne:DAPHNEReaderPDVD               5.26283       6.34996       8.75894       6.10426      0.836986        28     
produce:ophit:OpHitFinder                        0.0363587     0.0430639     0.0486291     0.0436504    0.00279787       28     
produce:opflash:OpFlashFinderVerticalDrift      0.00729212     0.0132152     0.0202819     0.0126461     0.0030774       28     
produce:wclsdatavd:WireCellToolkit                55.1617       64.8289       70.9525       64.5532       4.38266        28     
produce:gaushit:GausHitFinder                    0.655365       1.21421       1.86632       1.23078      0.290179        28     
produce:nhitsfilter:NumberOfHitsFilter          0.000122825   0.000236829   0.00052107    0.000207325   8.9003e-05       28     
produce:reco3d:SpacePointSolver                   3.7673        9.07138       16.3879       9.4484        2.91575        28     
produce:hitpdune:DisambigFromSpacePoints         0.0625106     0.137367      0.332049      0.130927      0.0522538       28     
produce:pandora:StandardPandora                   13.7364       40.3702       79.5219       40.5335       16.3885        28     
produce:pandoraTrack:LArPandoraTrackCreation     0.273469      0.627419       1.35075      0.579802       0.26347        28     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0145854     0.0228111     0.0355844     0.0237219     0.0055078       28     
[art]:TriggerResults:TriggerResultInserter      2.0043e-05    2.60651e-05   6.1614e-05    2.48595e-05   7.38424e-06      28     
end_path:out1:RootOutput                         3.644e-06    6.84204e-06   2.5754e-05     6.941e-06    3.9623e-06       28     
end_path:out1:RootOutput(write)                   4.5428        4.91618       5.33586       4.94486      0.214183        28     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4308.15 MB
  Peak resident set size usage (VmHWM): 2390.02 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140519_keepup.root
\tHists: np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140520_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140519_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140519_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757426721.0,
    "core.end_time": 1757426721.0,
    "core.events": [
      10008,
      10028,
      10048,
      10068,
      10088,
      10108,
      10128,
      10148,
      10168,
      10188,
      10208,
      10228,
      10248,
      10268,
      10288,
      10308,
      10328,
      10348,
      10368,
      10388,
      10408,
      10428,
      10448,
      10468,
      10488,
      10508,
      10528,
      10548
    ],
    "core.event_count": 28,
    "core.first_event_number": 10008,
    "core.last_event_number": 10548,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39349
    ],
    "core.runs_subruns": [
      3934900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539_reco_stage1_20250909T140520_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757426721.0,
    "core.end_time": 1757426721.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39349
    ],
    "core.runs_subruns": [
      3934900001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039349_0018_df-s03-d2_dw_0_20250909T121539.hdf5"
    }
  ]
}
justIN time: 2025-09-18 19:19:06 UTC       justIN version: 01.05.00