justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41655.77@dunegpschedd02.fnal.gov

Jobsub ID41655.77@dunegpschedd02.fnal.gov
Workflow ID2867
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 05:44:05
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-09-18 11:09:03
From worker nodeHostnamewn-la-04.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 06:17:12
Input filesvd-protodune:np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557.hdf5
JobscriptExit code0
Real time4h (17341s)
CPU time3h (14079s = 81%)
Max RSS bytes4834127872 (4610 MiB)
Outputting started2025-09-18 11:06:14
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110557_keepup.root
Finished2025-09-18 11:09:03
Saved logsjustin-logs:41655.77-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

D [ pgraph ] <Pgrapher:> graph execution complete 
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.89 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.44 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.36 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.32 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.27 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.26 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.26 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.25 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[13:05:46.458] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[13:05:46.458] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[13:05:46.458] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[13:05:46.458] I [ timer  ] Timer: Total node execution : 31.07000022009015 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 14744 traces tagged "gauss"
FrameSaver: q=2.84174e+06 n=345038 tag=gauss
wclsFrameSaver: saving 18571 traces tagged "wiener"
FrameSaver: q=3.07173e+06 n=332823 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 441 XUs and 690 XVs -> 24 XUVs
C:0 T:1 332 XUs and 368 XVs -> 8 XUVs
C:0 T:2 1005 XUs and 795 XVs -> 78 XUVs
C:0 T:3 109 XUs and 126 XVs -> 7 XUVs
C:0 T:4 513 XUs and 670 XVs -> 18 XUVs
C:0 T:5 35 XUs and 36 XVs -> 2 XUVs
C:0 T:6 2332 XUs and 2991 XVs -> 192 XUVs
C:0 T:7 9436 XUs and 11018 XVs -> 924 XUVs
1253 XUVs total
738 collection wire objects
1253 potential space points
Neighbour search...
45941 tests to find 10868 neighbours
Iterating with no regularization...
Begin: 2.96218e+09
0 2.81698e+09
1 2.8097e+09
2 2.80963e+09
Now with regularization...
Begin: 2.73425e+09
0 2.73416e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
18-Sep-2025 13:05:57 CEST  Closed output file "np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110557_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        80.8957       350.89        11445.5       115.668       1601.57        49     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.1372e-05    8.48296e-05   0.000229122   7.4311e-05    3.6697e-05       49     
produce:tpcrawdecoder:PDVDTPCReader               30.3175       55.0608       102.235       53.2393       15.063         49     
produce:triggerrawdecoder:PDVDTriggerReader4     0.324091      0.488931       1.1481       0.441005      0.164461        49     
produce:pdvddaphne:DAPHNEReaderPDVD               6.51961       12.3964       17.3878       12.0948       2.77451        49     
produce:ophit:OpHitFinder                        0.0274662     0.0384564     0.0509484     0.0380748    0.00541716       49     
produce:opflash:OpFlashFinderVerticalDrift      0.00288054    0.00693418     0.014688      0.0064446    0.00215903       49     
produce:wclsdatavd:WireCellToolkit                21.0232       33.0277       103.988       29.0748       13.1391        49     
produce:gaushit:GausHitFinder                    0.200114      0.446378        1.674       0.381728      0.227485        49     
produce:nhitsfilter:NumberOfHitsFilter          7.5722e-05    0.000159343   0.000736802   0.000114591   0.000115853      49     
produce:reco3d:SpacePointSolver                  0.969173       5.08192       50.1254       3.89171       6.82202        49     
produce:hitpdune:DisambigFromSpacePoints         0.0171408     0.0692432     0.955896      0.0411136     0.131535        49     
produce:pandora:StandardPandora                   1.85327       241.925       11202.4       7.99736       1582.07        49     
produce:pandoraTrack:LArPandoraTrackCreation     0.0629857     0.429866       10.6632      0.182032       1.48602        49     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00475833     0.0111867     0.0822113    0.00965991     0.0106399       49     
[art]:TriggerResults:TriggerResultInserter      1.6654e-05    2.64237e-05   6.9264e-05    2.2844e-05    1.10985e-05      49     
end_path:out1:RootOutput                         4.337e-06    6.19914e-06   2.3626e-05     5.378e-06    3.53097e-06      49     
end_path:out1:RootOutput(write)                   1.6001        1.87383       2.98071       1.80566      0.232406        49     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6720.7 MB
  Peak resident set size usage (VmHWM): 4834.13 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110557_keepup.root
\tHists: np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110558_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110557_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110557_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758193558.0,
    "core.end_time": 1758193558.0,
    "core.events": [
      98211,
      98217,
      98223,
      98229,
      98235,
      98241,
      98247,
      98253,
      98259,
      98265,
      98271,
      98277,
      98283,
      98289,
      98295,
      98301,
      98307,
      98313,
      98319,
      98325,
      98331,
      98337,
      98343,
      98349,
      98355,
      98361,
      98367,
      98373,
      98379,
      98385,
      98391,
      98397,
      98403,
      98409,
      98415,
      98421,
      98427,
      98433,
      98439,
      98445,
      98451,
      98457,
      98463,
      98469,
      98475,
      98481,
      98487,
      98493,
      98499
    ],
    "core.event_count": 49,
    "core.first_event_number": 98211,
    "core.last_event_number": 98499,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557_reco_stage1_20250918T110558_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758193558.0,
    "core.end_time": 1758193558.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0342_df-s05-d3_dw_0_20250918T052557.hdf5"
    }
  ]
}
justIN time: 2025-09-18 23:37:43 UTC       justIN version: 01.05.00