Jobsub ID 40956.0@dunegpschedd02.fnal.gov

Jobsub ID	40956.0@dunegpschedd02.fnal.gov
Workflow ID	2744
Stage ID	1
User name	ykermaid@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-17 12:03:10
Site	NL_NIKHEF
Entry	VIRGO_NL_NIKHEF_brug
Last heartbeat	2025-09-17 14:22:01
From worker node	Hostname	wn-sate-048.farm.nikhef.nl
	cpuinfo	AMD EPYC 7551P 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-17 12:03:55
Input files	vd-protodune:np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530.hdf5
Jobscript	Exit code	0
	Real time	2h (8219s)
	CPU time	1h (4887s = 59%)
	Max RSS bytes	3490947072 (3329 MiB)
Outputting started	2025-09-17 14:20:54
Output files	vd-protodune-det-reco:np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup.root
Finished	2025-09-17 14:22:01
Saved logs	justin-logs:40956.0-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ndow for channel 11687
wclsFrameSaver: no samples within desired window for channel 11688
wclsFrameSaver: no samples within desired window for channel 11688
wclsFrameSaver: no samples within desired window for channel 11689
wclsFrameSaver: no samples within desired window for channel 11690
wclsFrameSaver: no samples within desired window for channel 11691
wclsFrameSaver: no samples within desired window for channel 11691
wclsFrameSaver: no samples within desired window for channel 11692
wclsFrameSaver: no samples within desired window for channel 11692
wclsFrameSaver: no samples within desired window for channel 11692
wclsFrameSaver: no samples within desired window for channel 11693
wclsFrameSaver: no samples within desired window for channel 11694
wclsFrameSaver: no samples within desired window for channel 11694
wclsFrameSaver: no samples within desired window for channel 11695
wclsFrameSaver: no samples within desired window for channel 11696
wclsFrameSaver: no samples within desired window for channel 11696
wclsFrameSaver: no samples within desired window for channel 11704
wclsFrameSaver: no samples within desired window for channel 11705
wclsFrameSaver: no samples within desired window for channel 11706
wclsFrameSaver: no samples within desired window for channel 11706
wclsFrameSaver: no samples within desired window for channel 11707
wclsFrameSaver: no samples within desired window for channel 11708
wclsFrameSaver: no samples within desired window for channel 11708
wclsFrameSaver: no samples within desired window for channel 11709
wclsFrameSaver: no samples within desired window for channel 11709
wclsFrameSaver: no samples within desired window for channel 11710
wclsFrameSaver: no samples within desired window for channel 11710
wclsFrameSaver: no samples within desired window for channel 11711
wclsFrameSaver: no samples within desired window for channel 11711
wclsFrameSaver: no samples within desired window for channel 11712
wclsFrameSaver: no samples within desired window for channel 11712
wclsFrameSaver: no samples within desired window for channel 11785
wclsFrameSaver: no samples within desired window for channel 11785
wclsFrameSaver: no samples within desired window for channel 12287
wclsFrameSaver: no samples within desired window for channel 12287
FrameSaver: q=1.26886e+07 n=1288749 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1702 XUs and 2219 XVs -> 77 XUVs
C:0 T:1 319 XUs and 334 XVs -> 32 XUVs
C:0 T:2 573 XUs and 558 XVs -> 17 XUVs
C:0 T:3 140 XUs and 175 XVs -> 6 XUVs
C:0 T:4 964 XUs and 1328 XVs -> 50 XUVs
C:0 T:5 732 XUs and 954 XVs -> 33 XUVs
C:0 T:6 978 XUs and 1663 XVs -> 53 XUVs
C:0 T:7 845 XUs and 1769 XVs -> 52 XUVs
C:0 T:8 10856 XUs and 40216 XVs -> 2934 XUVs
C:0 T:9 8292 XUs and 11519 XVs -> 1825 XUVs
C:0 T:10 11782 XUs and 31530 XVs -> 4124 XUVs
C:0 T:11 13871 XUs and 12780 XVs -> 1710 XUVs
C:0 T:12 15863 XUs and 23701 XVs -> 4327 XUVs
C:0 T:13 9121 XUs and 14490 XVs -> 1618 XUVs
C:0 T:14 2546 XUs and 4226 XVs -> 443 XUVs
C:0 T:15 5222 XUs and 6435 XVs -> 443 XUVs
17744 XUVs total
3349 collection wire objects
17744 potential space points
Neighbour search...
1619892 tests to find 761378 neighbours
Iterating with no regularization...
Begin: 6.22576e+08
0 5.24243e+08
1 5.17813e+08
2 5.16609e+08
3 5.16164e+08
Now with regularization...
Begin: 4.9588e+08
0 4.95468e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
17-Sep-2025 16:20:45 CEST  Closed output file "np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        346.541       453.538       818.48        423.334       109.808        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.0063e-05    7.70642e-05   0.000224481   6.8519e-05    3.5891e-05       18     
produce:tpcrawdecoder:PDVDTPCReader               139.368       173.574       211.513       176.06        22.5237        18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0342677     0.0382767     0.0548555     0.0346099    0.00628949       18     
produce:pdvddaphne:DAPHNEReaderPDVD               15.1586       19.9423       25.4978       20.4164       2.64368        18     
produce:ophit:OpHitFinder                        0.0603287     0.0706457     0.0785825     0.0722484    0.00600979       18     
produce:opflash:OpFlashFinderVerticalDrift      0.00817693     0.0168734     0.0261917     0.0168135    0.00475316       18     
produce:wclsdatavd:WireCellToolkit                79.6137       93.3312       103.757       94.2144       6.72122        18     
produce:gaushit:GausHitFinder                     1.16779       1.83311       2.67269       1.8651       0.378682        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000277991   0.000402817   0.000599777   0.000404334   7.19833e-05      18     
produce:reco3d:SpacePointSolver                   11.8691       23.5015       36.7266       23.0878       6.64284        18     
produce:hitpdune:DisambigFromSpacePoints          0.16221      0.379352      0.669764      0.371762       0.14586        18     
produce:pandora:StandardPandora                   36.0904       131.832       441.372       110.267       93.0706        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.807417       2.41629       4.21424       2.36471      0.955106        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0301112     0.0462255     0.0633586     0.044542     0.00887211       18     
[art]:TriggerResults:TriggerResultInserter      2.5398e-05    3.85952e-05   0.000104777   3.37985e-05   1.70074e-05      18     
end_path:out1:RootOutput                         4.168e-06    8.6485e-06    2.9556e-05    7.7595e-06    5.39854e-06      18     
end_path:out1:RootOutput(write)                   6.04774       6.50009       7.04425       6.48584      0.263758        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5370.91 MB
  Peak resident set size usage (VmHWM): 3490.95 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup.root
\tHists: np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758118846.0,
    "core.end_time": 1758118846.0,
    "core.events": [
      186489,
      186509,
      186529,
      186549,
      186569,
      186589,
      186609,
      186629,
      186649,
      186669,
      186689,
      186709,
      186729,
      186749,
      186769,
      186789,
      186809,
      186829
    ],
    "core.event_count": 18,
    "core.first_event_number": 186489,
    "core.last_event_number": 186829,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39275
    ],
    "core.runs_subruns": [
      3927500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530_reco_stage1_20250917T142045_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758118846.0,
    "core.end_time": 1758118846.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39275
    ],
    "core.runs_subruns": [
      3927500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039275_0518_df-s03-d3_dw_0_20250902T081530.hdf5"
    }
  ]
}