Jobsub ID 45988.80@dunegpschedd01.fnal.gov

Jobsub ID	45988.80@dunegpschedd01.fnal.gov
Workflow ID	2575
Stage ID	1
User name	ykermaid@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	18000 (5 hours)
Submitted time	2025-09-13 15:34:15
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2025-09-13 18:15:39
From worker node	Hostname	comp17-00
	cpuinfo	Intel(R) Xeon(R) 6767P
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-09-13 16:08:48
Input files	vd-protodune:np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501.hdf5
Outputting started	2025-09-13 17:29:17
Output files
Finished	2025-09-13 18:15:39
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

no samples within desired window for channel 8684
wclsFrameSaver: no samples within desired window for channel 9821
wclsFrameSaver: no samples within desired window for channel 9832
wclsFrameSaver: no samples within desired window for channel 9834
wclsFrameSaver: no samples within desired window for channel 9835
wclsFrameSaver: no samples within desired window for channel 9835
wclsFrameSaver: no samples within desired window for channel 9882
wclsFrameSaver: no samples within desired window for channel 9914
wclsFrameSaver: no samples within desired window for channel 9916
wclsFrameSaver: no samples within desired window for channel 10071
wclsFrameSaver: no samples within desired window for channel 10194
wclsFrameSaver: no samples within desired window for channel 10204
wclsFrameSaver: no samples within desired window for channel 10224
wclsFrameSaver: no samples within desired window for channel 10261
wclsFrameSaver: no samples within desired window for channel 10347
wclsFrameSaver: no samples within desired window for channel 10352
wclsFrameSaver: no samples within desired window for channel 10440
wclsFrameSaver: no samples within desired window for channel 10737
wclsFrameSaver: no samples within desired window for channel 10762
wclsFrameSaver: no samples within desired window for channel 10763
wclsFrameSaver: no samples within desired window for channel 10764
wclsFrameSaver: no samples within desired window for channel 10833
wclsFrameSaver: no samples within desired window for channel 10835
wclsFrameSaver: no samples within desired window for channel 11774
wclsFrameSaver: no samples within desired window for channel 11775
wclsFrameSaver: no samples within desired window for channel 11775
wclsFrameSaver: no samples within desired window for channel 11775
wclsFrameSaver: no samples within desired window for channel 11776
wclsFrameSaver: no samples within desired window for channel 11776
wclsFrameSaver: no samples within desired window for channel 11777
wclsFrameSaver: no samples within desired window for channel 11778
wclsFrameSaver: no samples within desired window for channel 11779
wclsFrameSaver: no samples within desired window for channel 11866
wclsFrameSaver: no samples within desired window for channel 11866
wclsFrameSaver: no samples within desired window for channel 11866
wclsFrameSaver: no samples within desired window for channel 11867
wclsFrameSaver: no samples within desired window for channel 12047
wclsFrameSaver: no samples within desired window for channel 12047
wclsFrameSaver: no samples within desired window for channel 12048
FrameSaver: q=4.91301e+06 n=602778 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 250 XUs and 316 XVs -> 8 XUVs
C:0 T:1 317 XUs and 501 XVs -> 20 XUVs
C:0 T:2 500 XUs and 964 XVs -> 24 XUVs
C:0 T:3 28 XUs and 41 XVs -> 0 XUVs
C:0 T:4 9 XUs and 14 XVs -> 0 XUVs
C:0 T:5 166 XUs and 295 XVs -> 6 XUVs
C:0 T:6 802 XUs and 959 XVs -> 52 XUVs
C:0 T:7 192 XUs and 516 XVs -> 21 XUVs
C:0 T:8 2475 XUs and 2849 XVs -> 225 XUVs
C:0 T:9 839 XUs and 929 XVs -> 70 XUVs
C:0 T:10 1250 XUs and 1507 XVs -> 167 XUVs
C:0 T:11 844 XUs and 806 XVs -> 59 XUVs
C:0 T:12 779 XUs and 639 XVs -> 85 XUVs
C:0 T:13 1241 XUs and 1065 XVs -> 55 XUVs
C:0 T:14 930 XUs and 986 XVs -> 30 XUVs
C:0 T:15 543 XUs and 696 XVs -> 42 XUVs
864 XUVs total
736 collection wire objects
864 potential space points
Neighbour search...
8094 tests to find 4538 neighbours
Iterating with no regularization...
Begin: 1.83869e+08
0 1.78116e+08
1 1.7804e+08
Now with regularization...
Begin: 1.74291e+08
0 1.74272e+08
13-Sep-2025 18:29:08 BST  Closed output file "np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501_reco_stage1_20250913T172908_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        194.211       265.862       518.749       246.664       70.0505        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      5.5089e-05    6.65528e-05   0.000149671   6.0164e-05    2.08223e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               77.3002       117.412       159.395       117.234       20.9824        18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0501269     0.0777606     0.351127      0.0597187     0.0674985       18     
produce:pdvddaphne:DAPHNEReaderPDVD               9.0331        13.5697       19.1432       13.9297       2.65015        18     
produce:ophit:OpHitFinder                        0.0326982     0.0366232     0.0395189     0.0369664     0.0018067       18     
produce:opflash:OpFlashFinderVerticalDrift      0.00650003     0.0090924     0.0123442     0.0089428    0.00125823       18     
produce:wclsdatavd:WireCellToolkit                50.8865       55.4817       64.6353       55.2143       3.64742        18     
produce:gaushit:GausHitFinder                    0.461389      0.906906       1.30083      0.869334      0.184087        18     
produce:nhitsfilter:NumberOfHitsFilter          0.00019146    0.000394751   0.000606912   0.00039727    0.000108607      18     
produce:reco3d:SpacePointSolver                   5.26069       13.5709       22.3788       13.4707       3.93468        18     
produce:hitpdune:DisambigFromSpacePoints         0.0628397     0.183395      0.342969      0.171544      0.0684435       18     
produce:pandora:StandardPandora                   14.4245       59.5941       282.645       45.4196       56.9052        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.497194       1.62761       3.44262       1.44323      0.688396        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.012753      0.0220642     0.0311377     0.0217649    0.00432167       18     
[art]:TriggerResults:TriggerResultInserter      1.7566e-05    2.31529e-05   5.0165e-05    2.08935e-05   7.33377e-06      18     
end_path:out1:RootOutput                         6.801e-06    8.55822e-06   1.8871e-05     7.642e-06    2.76826e-06      18     
end_path:out1:RootOutput(write)                   2.99461       3.3226        3.6152        3.29391      0.156929        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5287.51 MB
  Peak resident set size usage (VmHWM): 3371.51 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501_reco_stage1_20250913T172908_keepup.root
\tHists: np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501_reco_stage1_20250913T172908_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501_reco_stage1_20250913T172908_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501_reco_stage1_20250913T172908_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757784548.0,
    "core.end_time": 1757784548.0,
    "core.events": [
      619215,
      619235,
      619255,
      619275,
      619295,
      619315,
      619335,
      619355,
      619375,
      619395,
      619415,
      619435,
      619455,
      619475,
      619495,
      619515,
      619535,
      619555
    ],
    "core.event_count": 18,
    "core.first_event_number": 619215,
    "core.last_event_number": 619555,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39273
    ],
    "core.runs_subruns": [
      3927300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501_reco_stage1_20250913T172908_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757784548.0,
    "core.end_time": 1757784548.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39273
    ],
    "core.runs_subruns": [
      3927300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039273_1720_df-s05-d1_dw_0_20250901T125501.hdf5"
    }
  ]
}