justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41352.20@dunegpschedd02.fnal.gov

Jobsub ID41352.20@dunegpschedd02.fnal.gov
Workflow ID2862
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-17 23:43:47
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_juk_multicore
Last heartbeat2025-09-18 01:11:07
From worker nodeHostnamewn-sate-028.farm.nikhef.nl
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-17 23:44:44
Input filesvd-protodune:np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825.hdf5
JobscriptExit code0
Real time1h (5110s)
CPU time49m (2952s = 57%)
Max RSS bytes2640330752 (2518 MiB)
Outputting started2025-09-18 01:09:54
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup.root
Finished2025-09-18 01:11:07
Saved logsjustin-logs:41352.20-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

EOS at call=93 with 8 
[03:07:30.579] D [  glue  ] frame sink sees EOS
[03:07:30.579] D [ pgraph ] <Pgrapher:> graph execution complete 
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.75 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.88 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.54 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.06 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.35 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[03:07:30.579] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[03:07:30.579] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:07:30.579] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[03:07:30.579] I [ timer  ] Timer: Total node execution : 32.630000023171306 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 31340 traces tagged "gauss"
FrameSaver: q=9.81117e+06 n=834081 tag=gauss
wclsFrameSaver: saving 41726 traces tagged "wiener"
FrameSaver: q=1.05941e+07 n=798159 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 7502 XUs and 8272 XVs -> 558 XUVs
C:0 T:1 1061 XUs and 1011 XVs -> 54 XUVs
C:0 T:2 8 XUs and 18 XVs -> 0 XUVs
C:0 T:3 908 XUs and 1214 XVs -> 53 XUVs
C:0 T:4 544 XUs and 651 XVs -> 22 XUVs
C:0 T:5 69960 XUs and 84715 XVs -> 5368 XUVs
C:0 T:6 1882 XUs and 2317 XVs -> 97 XUVs
C:0 T:7 9459 XUs and 11098 XVs -> 550 XUVs
6702 XUVs total
2940 collection wire objects
6702 potential space points
Neighbour search...
302640 tests to find 65184 neighbours
Iterating with no regularization...
Begin: 3.76921e+10
0 3.47508e+10
1 3.45676e+10
2 3.45639e+10
Now with regularization...
Begin: 3.42871e+10
0 3.42865e+10
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
18-Sep-2025 03:09:45 CEST  Closed output file "np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        78.009        107.388       195.98        101.241       21.0042        47     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      5.1337e-05    6.46417e-05   0.00020226    6.0103e-05    2.15834e-05      47     
produce:tpcrawdecoder:PDVDTPCReader               24.1289       39.7113       67.3567       37.2137       9.92718        47     
produce:triggerrawdecoder:PDVDTriggerReader4     0.290864      0.367408      0.645194      0.341724      0.0791952       47     
produce:pdvddaphne:DAPHNEReaderPDVD               4.46695       9.4002        15.7998       9.19981       1.95511        47     
produce:ophit:OpHitFinder                        0.0277482     0.040306      0.0499876     0.0407212    0.00469831       47     
produce:opflash:OpFlashFinderVerticalDrift      0.00332312    0.00944145     0.0147957    0.00949935    0.00244707       47     
produce:wclsdatavd:WireCellToolkit                22.938        32.0085       57.8791       29.792        7.40483        47     
produce:gaushit:GausHitFinder                    0.371117      0.578808      0.867225      0.541378      0.117686        47     
produce:nhitsfilter:NumberOfHitsFilter          8.6583e-05    0.000139311   0.000319951   0.000130856   4.39225e-05      47     
produce:reco3d:SpacePointSolver                   2.4513        5.40519       14.4081       4.71623       2.25581        47     
produce:hitpdune:DisambigFromSpacePoints         0.0193622     0.0632393     0.185142      0.0552785     0.0327983       47     
produce:pandora:StandardPandora                   5.05069       17.1573       115.12        11.6095       16.6574        47     
produce:pandoraTrack:LArPandoraTrackCreation     0.0975591     0.283742      0.985755      0.252969      0.165315        47     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00945233     0.0144438     0.0287927     0.0137269    0.00371159       47     
[art]:TriggerResults:TriggerResultInserter      1.5319e-05    1.98596e-05   6.1325e-05    1.8505e-05    6.54314e-06      47     
end_path:out1:RootOutput                         3.898e-06    6.19528e-06   2.2432e-05     4.999e-06    3.06149e-06      47     
end_path:out1:RootOutput(write)                   2.04144       2.3309        2.87935        2.262       0.228126        47     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4525.52 MB
  Peak resident set size usage (VmHWM): 2640.33 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup.root
\tHists: np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758157785.0,
    "core.end_time": 1758157785.0,
    "core.events": [
      53439,
      53445,
      53451,
      53457,
      53463,
      53469,
      53475,
      53481,
      53487,
      53493,
      53499,
      53505,
      53511,
      53517,
      53523,
      53529,
      53535,
      53541,
      53547,
      53553,
      53559,
      53565,
      53571,
      53577,
      53583,
      53589,
      53595,
      53601,
      53607,
      53613,
      53619,
      53625,
      53631,
      53637,
      53643,
      53649,
      53655,
      53661,
      53667,
      53673,
      53679,
      53685,
      53691,
      53697,
      53703,
      53709,
      53715
    ],
    "core.event_count": 47,
    "core.first_event_number": 53439,
    "core.last_event_number": 53715,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825_reco_stage1_20250918T010945_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758157785.0,
    "core.end_time": 1758157785.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0186_df-s05-d3_dw_0_20250917T230825.hdf5"
    }
  ]
}
justIN time: 2025-09-18 20:59:15 UTC       justIN version: 01.05.00