Jobsub ID 43070.89@dunegpschedd01.fnal.gov
| Jobsub ID | 43070.89@dunegpschedd01.fnal.gov |
| Workflow ID | 2323 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 11:37:51 |
| Site | UK_Lancaster |
| Entry | UBoone_UK_Lancaster_HEC_grendel_ce02 |
| Last heartbeat | 2025-09-08 15:52:27 |
| From worker node | Hostname | comp05-25 |
| cpuinfo | Intel(R) Xeon(R) Platinum 8352Y CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:28:22 |
| Input files | vd-protodune:np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (4935s) |
| CPU time | 1h (4162s = 84%) |
| Max RSS bytes | 2964414464 (2827 MiB) |
| Outputting started | 2025-09-08 15:50:37 |
| Output files | |
| Finished | 2025-09-08 15:52:27 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.95 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.39 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.25 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.11 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.97 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.56 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.51 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.27 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Aux::Resampler : 0.43 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Aux::Resampler : 0.39 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Aux::Resampler : 0.38 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Aux::Resampler : 0.35 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[16:44:40.713] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:44:40.713] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[16:44:40.713] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[16:44:40.713] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[16:44:40.713] I [ timer ] Timer: Total node execution : 59.66000020876527 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 49239 traces tagged "gauss"
FrameSaver: q=8.43509e+06 n=1063763 tag=gauss
wclsFrameSaver: saving 62645 traces tagged "wiener"
FrameSaver: q=9.04134e+06 n=1016603 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1410 XUs and 1527 XVs -> 57 XUVs
C:0 T:1 252 XUs and 145 XVs -> 5 XUVs
C:0 T:2 4425 XUs and 5444 XVs -> 237 XUVs
C:0 T:3 263 XUs and 238 XVs -> 17 XUVs
C:0 T:4 970 XUs and 812 XVs -> 56 XUVs
C:0 T:5 504 XUs and 448 XVs -> 23 XUVs
C:0 T:6 2355 XUs and 2656 XVs -> 72 XUVs
C:0 T:7 991 XUs and 800 XVs -> 27 XUVs
C:0 T:8 8232 XUs and 18948 XVs -> 3998 XUVs
C:0 T:9 6553 XUs and 14194 XVs -> 1616 XUVs
C:0 T:10 5804 XUs and 8553 XVs -> 1881 XUVs
C:0 T:11 2949 XUs and 4803 XVs -> 820 XUVs
C:0 T:12 13438 XUs and 18303 XVs -> 1764 XUVs
C:0 T:13 2776 XUs and 3199 XVs -> 327 XUVs
C:0 T:14 2152 XUs and 2229 XVs -> 165 XUVs
C:0 T:15 1306 XUs and 1687 XVs -> 97 XUVs
11162 XUVs total
2236 collection wire objects
11162 potential space points
Neighbour search...
1466606 tests to find 612496 neighbours
Iterating with no regularization...
Begin: 4.53237e+08
0 4.04086e+08
1 4.01312e+08
2 4.00868e+08
3 4.00714e+08
Now with regularization...
Begin: 3.90367e+08
0 3.90216e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 16:46:25 BST Closed output file "np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156_reco_stage1_20250908T154625_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 101.867 177.91 308.24 167.473 51.6485 26
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.1129e-05 9.56772e-05 0.000227911 8.1619e-05 3.44165e-05 26
produce:tpcrawdecoder:PDVDTPCReader 15.718 22.106 37.0649 20.3773 6.04186 26
produce:triggerrawdecoder:PDVDTriggerReader4 0.0493185 0.0548829 0.0749173 0.050839 0.00735725 26
produce:pdvddaphne:DAPHNEReaderPDVD 4.59793 5.50899 7.8101 5.36236 0.70247 26
produce:ophit:OpHitFinder 0.0436987 0.0627579 0.0876041 0.0627959 0.0088 26
produce:opflash:OpFlashFinderVerticalDrift 0.0087654 0.0154056 0.0215104 0.01529 0.00319406 26
produce:wclsdatavd:WireCellToolkit 46.4497 68.6691 113.07 65.1952 16.6413 26
produce:gaushit:GausHitFinder 0.89846 1.52541 2.35537 1.53025 0.393413 26
produce:nhitsfilter:NumberOfHitsFilter 0.000290449 0.000458598 0.000710023 0.000449127 0.000100235 26
produce:reco3d:SpacePointSolver 7.57001 15.4753 25.9059 15.6248 5.34747 26
produce:hitpdune:DisambigFromSpacePoints 0.0978333 0.224949 0.397335 0.216931 0.0814968 26
produce:pandora:StandardPandora 17.6198 57.9554 167.028 52.7549 38.0503 26
produce:pandoraTrack:LArPandoraTrackCreation 0.33543 1.43667 3.77501 1.13782 1.03009 26
produce:pandoraGnocalo:GnocchiCalorimetry 0.0151648 0.0304497 0.0480496 0.0292165 0.00854735 26
[art]:TriggerResults:TriggerResultInserter 3.2075e-05 4.71289e-05 0.000100125 4.31045e-05 1.31051e-05 26
end_path:out1:RootOutput 1.0465e-05 1.39568e-05 2.993e-05 1.31435e-05 3.82522e-06 26
end_path:out1:RootOutput(write) 3.85744 4.78168 6.8102 4.55311 0.780776 26
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4869.71 MB
Peak resident set size usage (VmHWM): 2964.41 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156_reco_stage1_20250908T154625_keepup.root
\tHists: np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156_reco_stage1_20250908T154625_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156_reco_stage1_20250908T154625_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156_reco_stage1_20250908T154625_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346386.0,
"core.end_time": 1757346386.0,
"core.events": [
893598,
893618,
893638,
893658,
893678,
893698,
893718,
893738,
893758,
893778,
893798,
893818,
893838,
893858,
893878,
893898,
893918,
893938,
893958,
893978,
893998,
894018,
894038,
894058,
894078,
894098
],
"core.event_count": 26,
"core.first_event_number": 893598,
"core.last_event_number": 894098,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156_reco_stage1_20250908T154625_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346386.0,
"core.end_time": 1757346386.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_1707_df-s05-d4_dw_0_20250907T041156.hdf5"
}
]
} 