justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242471.112@dunegpschedd01.fnal.gov

Jobsub ID242471.112@dunegpschedd01.fnal.gov
Workflow ID9538
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors2
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-03 17:53:07
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-03 20:23:47
From worker nodeHostnamewn2207251.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors2
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 17:54:55
Input filesvd-protodune:np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906.hdf5
JobscriptExit code0
Real time2h (8762s)
CPU time1h (5379s = 61%)
Max RSS bytes3387691008 (3230 MiB)
Outputting started2025-11-03 20:20:58
Output filesvd-protodune-det-reco:np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202041_keepup.root
Finished2025-11-03 20:23:47
Saved logsjustin-logs:242471.112-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gproc0> EOS at call=47 anode=0 
[20:19:28.883] D [  glue  ] <FrameFanin:nfsp> EOS at call=47 with 8 
[20:19:28.883] D [  glue  ] frame sink sees EOS
[20:19:28.883] D [ pgraph ] <Pgrapher:> graph execution complete 
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.85 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.8 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 13.74 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.54 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.45 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.19 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.98 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.17 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.46 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.44 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.42 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.42 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.04 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.02 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[20:19:28.883] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[20:19:28.883] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[20:19:28.883] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[20:19:28.883] I [ timer  ] Timer: Total node execution : 105.5799992904067 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 47231 traces tagged "gauss"
FrameSaver: q=1.11311e+07 n=950206 tag=gauss
wclsFrameSaver: saving 58858 traces tagged "wiener"
FrameSaver: q=1.21441e+07 n=910036 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 471 XUs and 613 XVs -> 27 XUVs
C:0 T:1 1210 XUs and 1106 XVs -> 57 XUVs
C:0 T:2 247 XUs and 208 XVs -> 16 XUVs
C:0 T:3 476 XUs and 532 XVs -> 18 XUVs
C:0 T:4 2291 XUs and 2006 XVs -> 102 XUVs
C:0 T:5 6707 XUs and 11307 XVs -> 505 XUVs
C:0 T:6 456 XUs and 557 XVs -> 26 XUVs
C:0 T:7 9460 XUs and 12832 XVs -> 1070 XUVs
C:0 T:8 346 XUs and 503 XVs -> 39 XUVs
C:0 T:9 1392 XUs and 1526 XVs -> 99 XUVs
C:0 T:10 527 XUs and 736 XVs -> 40 XUVs
C:0 T:11 510 XUs and 805 XVs -> 29 XUVs
C:0 T:12 606 XUs and 1276 XVs -> 39 XUVs
C:0 T:13 1393 XUs and 1142 XVs -> 66 XUVs
C:0 T:14 856 XUs and 784 XVs -> 23 XUVs
C:0 T:15 1878 XUs and 1661 XVs -> 51 XUVs
2207 XUVs total
1220 collection wire objects
2207 potential space points
Neighbour search...
109455 tests to find 27772 neighbours
Iterating with no regularization...
Begin: 4.90873e+09
0 4.58495e+09
1 4.56644e+09
2 4.56578e+09
Now with regularization...
Begin: 4.51263e+09
0 4.51239e+09
03-Nov-2025 20:20:41 GMT  Closed output file "np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202041_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        283.321       360.554       513.796       326.865       80.9762        24     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.6543e-05    0.000148168   0.000370259   0.000127063   6.30422e-05      24     
produce:tpcrawdecoder:PDVDTPCReader               103.771       141.468       230.321       139.608       27.7586        24     
produce:triggerrawdecoder:PDVDTriggerReader4     0.801728       1.03368       1.45342      0.985717       0.18105        24     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000581644   0.000746223   0.00133855    0.000674291   0.00018479       24     
produce:ophit:OpHitFinder                       0.000118908   0.000226315   0.00099144    0.000196239   0.000165133      24     
produce:opflash:OpFlashFinderVerticalDrift       7.968e-05    0.000164277   0.000560613   0.000129955   0.000114497      24     
produce:wclsdatavd:WireCellToolkit                89.8045       128.44        220.602       108.942       37.2625        24     
produce:gaushit:GausHitFinder                     1.25695       2.13049       2.89199       2.13156      0.400755        24     
produce:nhitsfilter:NumberOfHitsFilter          0.000393318   0.000553994   0.000951141   0.000544721   0.000111017      24     
produce:reco3d:SpacePointSolver                   10.0905       15.8587       21.8305       16.2277       3.36468        24     
produce:hitpdune:DisambigFromSpacePoints         0.173536      0.385456      0.945874      0.326599      0.170512        24     
produce:pandora:StandardPandora                   23.0468       60.168        186.334       41.6312       41.8057        24     
produce:pandoraTrack:LArPandoraTrackCreation      1.47692       3.76829       7.03955       3.29166       1.52121        24     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0457619     0.0640507     0.0840512     0.0640747     0.0102457       24     
[art]:TriggerResults:TriggerResultInserter      6.4448e-05    7.77017e-05   0.000151878   6.99715e-05   1.93136e-05      24     
end_path:out1:RootOutput                        1.0573e-05    2.06771e-05   5.1552e-05    1.88665e-05   8.29584e-06      24     
end_path:out1:RootOutput(write)                   6.32195       7.14636       8.99059       7.13336      0.577505        24     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5436.72 MB
  Peak resident set size usage (VmHWM): 3387.69 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202041_keepup.root
\tHists: np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202042_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202041_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202041_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762201243.0,
    "core.end_time": 1762201243.0,
    "core.events": [
      347364,
      347368,
      347372,
      347376,
      347380,
      347384,
      347388,
      347392,
      347396,
      347400,
      347404,
      347408,
      347412,
      347416,
      347420,
      347424,
      347428,
      347432,
      347436,
      347440,
      347444,
      347448,
      347452,
      347456
    ],
    "core.event_count": 24,
    "core.first_event_number": 347364,
    "core.last_event_number": 347456,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906_reco_stage1_20251103T202042_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762201243.0,
    "core.end_time": 1762201243.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3763_df-s04-d0_dw_0_20251103T170906.hdf5"
    }
  ]
}
justIN time: 2025-11-05 05:01:45 UTC       justIN version: 01.05.01