Jobsub ID 39336.103@dunegpschedd02.fnal.gov

Jobsub ID	39336.103@dunegpschedd02.fnal.gov
Workflow ID	2576
Stage ID	1
User name	ykermaid@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-13 15:36:17
Site	NL_NIKHEF
Entry	VIRGO_NL_NIKHEF_brug
Last heartbeat	2025-09-13 18:12:58
From worker node	Hostname	wn-snel-028.farm.nikhef.nl
	cpuinfo	AMD EPYC 7H12 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-09-13 16:54:28
Input files	vd-protodune:np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148.hdf5
Outputting started	2025-09-13 18:06:25
Output files	vd-protodune-det-reco:np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup.root
Finished	2025-09-13 18:12:58
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

ver: no samples within desired window for channel 8782
wclsFrameSaver: no samples within desired window for channel 8986
wclsFrameSaver: no samples within desired window for channel 8986
wclsFrameSaver: no samples within desired window for channel 8986
wclsFrameSaver: no samples within desired window for channel 8988
wclsFrameSaver: no samples within desired window for channel 8988
wclsFrameSaver: no samples within desired window for channel 9216
wclsFrameSaver: no samples within desired window for channel 9328
wclsFrameSaver: no samples within desired window for channel 9706
wclsFrameSaver: no samples within desired window for channel 9809
wclsFrameSaver: no samples within desired window for channel 9809
wclsFrameSaver: no samples within desired window for channel 9810
wclsFrameSaver: no samples within desired window for channel 9811
wclsFrameSaver: no samples within desired window for channel 9811
wclsFrameSaver: no samples within desired window for channel 9833
wclsFrameSaver: no samples within desired window for channel 9884
wclsFrameSaver: no samples within desired window for channel 9885
wclsFrameSaver: no samples within desired window for channel 9885
wclsFrameSaver: no samples within desired window for channel 10251
wclsFrameSaver: no samples within desired window for channel 10395
wclsFrameSaver: no samples within desired window for channel 10721
wclsFrameSaver: no samples within desired window for channel 10722
wclsFrameSaver: no samples within desired window for channel 10722
wclsFrameSaver: no samples within desired window for channel 10851
wclsFrameSaver: no samples within desired window for channel 10852
wclsFrameSaver: no samples within desired window for channel 10854
wclsFrameSaver: no samples within desired window for channel 11827
wclsFrameSaver: no samples within desired window for channel 11827
wclsFrameSaver: no samples within desired window for channel 11827
wclsFrameSaver: no samples within desired window for channel 11828
wclsFrameSaver: no samples within desired window for channel 11828
wclsFrameSaver: no samples within desired window for channel 11909
wclsFrameSaver: no samples within desired window for channel 11909
wclsFrameSaver: no samples within desired window for channel 11909
wclsFrameSaver: no samples within desired window for channel 11909
wclsFrameSaver: no samples within desired window for channel 11910
wclsFrameSaver: no samples within desired window for channel 11910
wclsFrameSaver: no samples within desired window for channel 11912
wclsFrameSaver: no samples within desired window for channel 11912
FrameSaver: q=7.52427e+06 n=899640 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 297 XUs and 452 XVs -> 20 XUVs
C:0 T:1 72 XUs and 93 XVs -> 0 XUVs
C:0 T:2 246 XUs and 218 XVs -> 18 XUVs
C:0 T:3 411 XUs and 221 XVs -> 14 XUVs
C:0 T:4 662 XUs and 961 XVs -> 27 XUVs
C:0 T:5 984 XUs and 1309 XVs -> 83 XUVs
C:0 T:6 749 XUs and 843 XVs -> 35 XUVs
C:0 T:7 2399 XUs and 2603 XVs -> 142 XUVs
C:0 T:8 2135 XUs and 1963 XVs -> 124 XUVs
C:0 T:9 1957 XUs and 2526 XVs -> 104 XUVs
C:0 T:10 1154 XUs and 926 XVs -> 66 XUVs
C:0 T:11 1140 XUs and 1549 XVs -> 74 XUVs
C:0 T:12 2891 XUs and 4139 XVs -> 132 XUVs
C:0 T:13 3512 XUs and 4047 XVs -> 277 XUVs
C:0 T:14 1166 XUs and 1469 XVs -> 90 XUVs
C:0 T:15 641 XUs and 886 XVs -> 58 XUVs
1264 XUVs total
1056 collection wire objects
1264 potential space points
Neighbour search...
10814 tests to find 5270 neighbours
Iterating with no regularization...
Begin: 6.99414e+08
0 6.89955e+08
1 6.89738e+08
Now with regularization...
Begin: 6.83618e+08
0 6.83612e+08
13-Sep-2025 20:06:17 CEST  Closed output file "np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        180.124       237.502       336.21        230.327       41.5687        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      5.4222e-05    7.50909e-05   0.000159099   7.0342e-05    2.17012e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               44.6439       91.202        114.668       91.5867       17.535         18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0344018     0.0346296     0.0356223     0.0345482    0.000263864      18     
produce:pdvddaphne:DAPHNEReaderPDVD               9.72191       13.0908       16.1821       13.2269       1.73679        18     
produce:ophit:OpHitFinder                        0.0578508     0.0760768     0.0882413     0.0770022    0.00859935       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0107231     0.0168856     0.0244062     0.0171156    0.00368668       18     
produce:wclsdatavd:WireCellToolkit                57.2412       65.9761       71.3813       66.2458       3.30477        18     
produce:gaushit:GausHitFinder                     0.6399        1.18067       1.84649       1.12781      0.324555        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000274746   0.000386223   0.000723448   0.000350383   0.000121185      18     
produce:reco3d:SpacePointSolver                   6.79519       13.8129       23.1956       12.8149       4.35596        18     
produce:hitpdune:DisambigFromSpacePoints         0.0790412     0.179564      0.330631      0.164494      0.0620413       18     
produce:pandora:StandardPandora                   16.8682       46.3729       120.466       38.0289       25.2824        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.315239      0.837895       1.41183      0.768838       0.29596        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0190756     0.028466      0.0392995     0.0270931    0.00551855       18     
[art]:TriggerResults:TriggerResultInserter      1.4387e-05    1.79631e-05   4.4644e-05    1.6361e-05    6.60177e-06      18     
end_path:out1:RootOutput                         3.186e-06    4.32983e-06   1.6751e-05     3.486e-06    3.03131e-06      18     
end_path:out1:RootOutput(write)                   4.39514       4.65696       4.93893       4.64739      0.152467        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4818.78 MB
  Peak resident set size usage (VmHWM): 2934.28 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup.root
\tHists: np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757786778.0,
    "core.end_time": 1757786778.0,
    "core.events": [
      53295,
      53315,
      53335,
      53355,
      53375,
      53395,
      53415,
      53435,
      53455,
      53475,
      53495,
      53515,
      53535,
      53555,
      53575,
      53595,
      53615,
      53635
    ],
    "core.event_count": 18,
    "core.first_event_number": 53295,
    "core.last_event_number": 53635,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39275
    ],
    "core.runs_subruns": [
      3927500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148_reco_stage1_20250913T180617_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757786778.0,
    "core.end_time": 1757786778.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39275
    ],
    "core.runs_subruns": [
      3927500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039275_0148_df-s05-d1_dw_0_20250901T193148.hdf5"
    }
  ]
}