justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242278.18@dunegpschedd01.fnal.gov

Jobsub ID242278.18@dunegpschedd01.fnal.gov
Workflow ID9538
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors2
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-03 10:40:44
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-03 11:32:42
From worker nodeHostnamewn-s04-05-34-03.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors2
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 10:42:59
Input filesvd-protodune:np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247.hdf5
JobscriptExit code0
Real time43m (2618s)
CPU time35m (2140s = 81%)
Max RSS bytes3788247040 (3612 MiB)
Outputting started2025-11-03 11:26:38
Output filesvd-protodune-det-reco:np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup.root
Finished2025-11-03 11:32:42
Saved logsjustin-logs:242278.18-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

[12:25:54.149] D [  glue  ] frame sink sees EOS
[12:25:54.150] D [ pgraph ] <Pgrapher:> graph execution complete 
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.7 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.31 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.26 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.23 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.93 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.66 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.04 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.56 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.95 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.9 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.55 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.51 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[12:25:54.150] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[12:25:54.150] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[12:25:54.150] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[12:25:54.150] I [ timer  ] Timer: Total node execution : 53.63999954611063 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 51758 traces tagged "gauss"
FrameSaver: q=1.28484e+07 n=1100237 tag=gauss
wclsFrameSaver: saving 64056 traces tagged "wiener"
FrameSaver: q=1.35693e+07 n=1049436 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 391 XUs and 373 XVs -> 11 XUVs
C:0 T:1 2313 XUs and 2645 XVs -> 133 XUVs
C:0 T:2 966 XUs and 855 XVs -> 23 XUVs
C:0 T:3 1198 XUs and 1354 XVs -> 35 XUVs
C:0 T:4 5930 XUs and 6004 XVs -> 328 XUVs
C:0 T:5 766 XUs and 715 XVs -> 30 XUVs
C:0 T:6 264 XUs and 326 XVs -> 14 XUVs
C:0 T:7 155 XUs and 180 XVs -> 4 XUVs
C:0 T:8 566 XUs and 893 XVs -> 31 XUVs
C:0 T:9 625 XUs and 967 XVs -> 42 XUVs
C:0 T:10 418 XUs and 756 XVs -> 24 XUVs
C:0 T:11 1149 XUs and 2044 XVs -> 78 XUVs
C:0 T:12 387 XUs and 642 XVs -> 38 XUVs
C:0 T:13 1820 XUs and 2339 XVs -> 100 XUVs
C:0 T:14 1570 XUs and 2235 XVs -> 76 XUVs
C:0 T:15 1546 XUs and 1528 XVs -> 90 XUVs
1057 XUVs total
864 collection wire objects
1057 potential space points
Neighbour search...
10081 tests to find 4350 neighbours
Iterating with no regularization...
Begin: 3.19065e+09
0 3.17631e+09
1 3.1757e+09
Now with regularization...
Begin: 3.15653e+09
0 3.15652e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
03-Nov-2025 12:26:29 CET  Closed output file "np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        59.7323       100.911       388.508       87.8007       64.2433        23     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      3.6359e-05    5.42429e-05   9.1733e-05    4.9323e-05    1.55689e-05      23     
produce:tpcrawdecoder:PDVDTPCReader               7.79297       10.0233       11.4433       9.99876      0.815314        23     
produce:triggerrawdecoder:PDVDTriggerReader4     0.299213      0.314253      0.377117      0.309111      0.0180217       23     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000171372   0.000237777   0.000486465   0.000200988   7.94081e-05      23     
produce:ophit:OpHitFinder                       4.0366e-05    7.67056e-05   0.000423416   5.7719e-05    7.65677e-05      23     
produce:opflash:OpFlashFinderVerticalDrift      2.8113e-05    4.91677e-05   0.000242145   3.6078e-05    4.23764e-05      23     
produce:wclsdatavd:WireCellToolkit                33.0643       46.9749       72.2656       42.797        12.1504        23     
produce:gaushit:GausHitFinder                    0.482381      0.822162       1.31717      0.722349      0.227027        23     
produce:nhitsfilter:NumberOfHitsFilter          0.000107032   0.000213458   0.000845049   0.000174528   0.000142283      23     
produce:reco3d:SpacePointSolver                   4.2041        8.55158       19.0973       7.58694       3.01517        23     
produce:hitpdune:DisambigFromSpacePoints         0.0671594     0.134624       0.38077      0.115929      0.0650612       23     
produce:pandora:StandardPandora                   9.03434       29.8928       275.839       17.3619       53.0621        23     
produce:pandoraTrack:LArPandoraTrackCreation     0.384697       0.79834       4.41237      0.566265      0.803972        23     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0121613     0.0225401     0.0455744     0.0187504    0.00958472       23     
[art]:TriggerResults:TriggerResultInserter       1.066e-05    1.61843e-05   3.8031e-05    1.4627e-05    5.44154e-06      23     
end_path:out1:RootOutput                         2.875e-06    7.55161e-06   1.7553e-05     6.332e-06    3.36552e-06      23     
end_path:out1:RootOutput(write)                   2.67432       3.35498       4.7259        3.01227      0.654481        23     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5823.7 MB
  Peak resident set size usage (VmHWM): 3788.25 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup.root
\tHists: np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762169190.0,
    "core.end_time": 1762169190.0,
    "core.events": [
      302012,
      302016,
      302020,
      302024,
      302028,
      302032,
      302036,
      302040,
      302044,
      302048,
      302052,
      302056,
      302060,
      302064,
      302068,
      302072,
      302076,
      302080,
      302084,
      302088,
      302092,
      302096,
      302100
    ],
    "core.event_count": 23,
    "core.first_event_number": 302012,
    "core.last_event_number": 302100,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247_reco_stage1_20251103T112629_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762169190.0,
    "core.end_time": 1762169190.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3272_df-s04-d0_dw_0_20251103T095247.hdf5"
    }
  ]
}
justIN time: 2025-11-05 04:48:29 UTC       justIN version: 01.05.01