Jobsub ID 36343.15@dunegpschedd02.fnal.gov
| Jobsub ID | 36343.15@dunegpschedd02.fnal.gov |
| Workflow ID | 2323 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 12:01:52 |
| Site | UK_Lancaster |
| Entry | UBoone_UK_Lancaster_HEC_grendel_ce02 |
| Last heartbeat | 2025-09-08 15:52:43 |
| From worker node | Hostname | comp23-26 |
| cpuinfo | Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:19:56 |
| Input files | vd-protodune:np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (5418s) |
| CPU time | 1h (4030s = 74%) |
| Max RSS bytes | 3001098240 (2862 MiB) |
| Outputting started | 2025-09-08 15:50:15 |
| Output files | |
| Finished | 2025-09-08 15:52:43 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
mnibusSigProc : 8.27 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.9 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.41 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.16 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.99 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.61 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.54 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Aux::Resampler : 0.32 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Aux::Resampler : 0.3 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Aux::Resampler : 0.29 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Aux::Resampler : 0.29 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.04 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[16:44:01.311] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:44:01.311] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[16:44:01.311] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[16:44:01.311] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[16:44:01.311] I [ timer ] Timer: Total node execution : 61.45000055804849 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 46338 traces tagged "gauss"
FrameSaver: q=8.18195e+06 n=1003858 tag=gauss
wclsFrameSaver: saving 57965 traces tagged "wiener"
FrameSaver: q=8.7797e+06 n=960549 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 675 XUs and 838 XVs -> 20 XUVs
C:0 T:1 1225 XUs and 1357 XVs -> 42 XUVs
C:0 T:2 1948 XUs and 2339 XVs -> 52 XUVs
C:0 T:3 54 XUs and 65 XVs -> 0 XUVs
C:0 T:4 806 XUs and 964 XVs -> 57 XUVs
C:0 T:5 320 XUs and 353 XVs -> 7 XUVs
C:0 T:6 343 XUs and 490 XVs -> 12 XUVs
C:0 T:7 106 XUs and 136 XVs -> 6 XUVs
C:0 T:8 2239 XUs and 3541 XVs -> 173 XUVs
C:0 T:9 7364 XUs and 12573 XVs -> 1724 XUVs
C:0 T:10 8271 XUs and 15074 XVs -> 2091 XUVs
C:0 T:11 5576 XUs and 10739 XVs -> 2039 XUVs
C:0 T:12 4877 XUs and 8648 XVs -> 1301 XUVs
C:0 T:13 2997 XUs and 4539 XVs -> 585 XUVs
C:0 T:14 3644 XUs and 8465 XVs -> 841 XUVs
C:0 T:15 15619 XUs and 26200 XVs -> 3373 XUVs
12323 XUVs total
1965 collection wire objects
12323 potential space points
Neighbour search...
1575337 tests to find 670246 neighbours
Iterating with no regularization...
Begin: 1.57548e+09
0 1.51764e+09
1 1.51434e+09
2 1.51369e+09
Now with regularization...
Begin: 1.47424e+09
0 1.47384e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 16:45:42 BST Closed output file "np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058_reco_stage1_20250908T154542_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 129.433 195.725 419.011 175.618 56.8472 26
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.0843e-05 0.000153222 0.000246027 0.000139935 4.31742e-05 26
produce:tpcrawdecoder:PDVDTPCReader 19.0238 43.9178 75.5271 42.7082 13.6903 26
produce:triggerrawdecoder:PDVDTriggerReader4 0.0498411 0.0550981 0.0685517 0.0504992 0.0066652 26
produce:pdvddaphne:DAPHNEReaderPDVD 5.35681 7.56963 10.3696 7.6712 1.33465 26
produce:ophit:OpHitFinder 0.0344765 0.0418239 0.0499464 0.0416033 0.00347168 26
produce:opflash:OpFlashFinderVerticalDrift 0.00739505 0.0121365 0.0178558 0.0117271 0.00270488 26
produce:wclsdatavd:WireCellToolkit 51.372 63.0817 87.7783 60.9931 9.22046 26
produce:gaushit:GausHitFinder 0.796457 1.28546 2.00787 1.28884 0.313999 26
produce:nhitsfilter:NumberOfHitsFilter 0.000309497 0.000468196 0.000772334 0.000468404 0.000109549 26
produce:reco3d:SpacePointSolver 5.46103 11.9039 20.1979 11.1998 3.90266 26
produce:hitpdune:DisambigFromSpacePoints 0.104565 0.232154 0.416478 0.213172 0.0841136 26
produce:pandora:StandardPandora 22.175 62.285 269.921 48.822 48.9383 26
produce:pandoraTrack:LArPandoraTrackCreation 0.376197 1.52028 3.75488 1.31431 0.896838 26
produce:pandoraGnocalo:GnocchiCalorimetry 0.0157807 0.0284161 0.0428197 0.0288178 0.00732476 26
[art]:TriggerResults:TriggerResultInserter 2.8324e-05 4.77883e-05 8.8516e-05 4.52935e-05 1.33053e-05 26
end_path:out1:RootOutput 9.195e-06 1.2129e-05 3.5951e-05 1.0703e-05 5.02412e-06 26
end_path:out1:RootOutput(write) 3.21912 3.72555 5.27835 3.52365 0.545164 26
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4905.97 MB
Peak resident set size usage (VmHWM): 3001.1 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058_reco_stage1_20250908T154542_keepup.root
\tHists: np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058_reco_stage1_20250908T154543_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058_reco_stage1_20250908T154542_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058_reco_stage1_20250908T154542_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346344.0,
"core.end_time": 1757346344.0,
"core.events": [
888331,
888351,
888371,
888391,
888411,
888431,
888451,
888471,
888491,
888511,
888531,
888551,
888571,
888591,
888611,
888631,
888651,
888671,
888691,
888711,
888731,
888751,
888771,
888791,
888811,
888831
],
"core.event_count": 26,
"core.first_event_number": 888331,
"core.last_event_number": 888831,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058_reco_stage1_20250908T154543_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346344.0,
"core.end_time": 1757346344.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_1696_df-s04-d1_dw_0_20250907T040058.hdf5"
}
]
} 