Jobsub ID 36361.36@dunegpschedd02.fnal.gov
| Jobsub ID | 36361.36@dunegpschedd02.fnal.gov |
| Workflow ID | 2329 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 12:39:56 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep206 |
| Last heartbeat | 2025-09-08 15:52:37 |
| From worker node | Hostname | heplnc159.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:42:20 |
| Input files | vd-protodune:np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (3908s) |
| CPU time | 55m (3312s = 84%) |
| Max RSS bytes | 3207659520 (3059 MiB) |
| Outputting started | 2025-09-08 15:47:29 |
| Output files | |
| Finished | 2025-09-08 15:52:37 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
gProc : 7.67 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.51 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.49 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.08 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.81 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.77 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.21 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.14 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Aux::Resampler : 0.4 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Aux::Resampler : 0.38 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Aux::Resampler : 0.33 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Aux::Resampler : 0.32 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:34.761] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:34.761] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:34.761] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[16:41:34.761] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[16:41:34.761] I [ timer ] Timer: Total node execution : 50.18999983370304 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 45038 traces tagged "gauss"
FrameSaver: q=7.8856e+06 n=967462 tag=gauss
wclsFrameSaver: saving 56723 traces tagged "wiener"
FrameSaver: q=8.4069e+06 n=925013 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 204 XUs and 205 XVs -> 13 XUVs
C:0 T:1 341 XUs and 363 XVs -> 5 XUVs
C:0 T:2 1466 XUs and 1341 XVs -> 38 XUVs
C:0 T:3 1151 XUs and 939 XVs -> 69 XUVs
C:0 T:4 1195 XUs and 1290 XVs -> 41 XUVs
C:0 T:5 1380 XUs and 1377 XVs -> 55 XUVs
C:0 T:6 512 XUs and 486 XVs -> 36 XUVs
C:0 T:7 45 XUs and 58 XVs -> 1 XUVs
C:0 T:8 8426 XUs and 10749 XVs -> 1922 XUVs
C:0 T:9 12250 XUs and 13912 XVs -> 1958 XUVs
C:0 T:10 9841 XUs and 19171 XVs -> 2979 XUVs
C:0 T:11 1674 XUs and 1165 XVs -> 56 XUVs
C:0 T:12 2558 XUs and 3465 XVs -> 487 XUVs
C:0 T:13 9382 XUs and 11436 XVs -> 1113 XUVs
C:0 T:14 1075 XUs and 998 XVs -> 72 XUVs
C:0 T:15 2884 XUs and 4486 XVs -> 608 XUVs
9453 XUVs total
1899 collection wire objects
9453 potential space points
Neighbour search...
1042363 tests to find 459292 neighbours
Iterating with no regularization...
Begin: 4.91308e+08
0 4.44138e+08
1 4.39999e+08
2 4.39295e+08
3 4.39049e+08
Now with regularization...
Begin: 4.23253e+08
0 4.23074e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 16:43:06 BST Closed output file "np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027_reco_stage1_20250908T154306_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 97.3611 149.796 281.76 135.131 47.379 24
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.0963e-05 0.000134739 0.000367892 0.000114325 5.7986e-05 24
produce:tpcrawdecoder:PDVDTPCReader 11.7524 12.9962 15.0136 12.3862 1.18758 24
produce:triggerrawdecoder:PDVDTriggerReader4 0.0353343 0.0358957 0.0382637 0.0357098 0.000605942 24
produce:pdvddaphne:DAPHNEReaderPDVD 2.89947 3.25139 3.64351 3.23326 0.198088 24
produce:ophit:OpHitFinder 0.0422131 0.0536082 0.0627042 0.0538158 0.00579176 24
produce:opflash:OpFlashFinderVerticalDrift 0.00562548 0.0134375 0.0233489 0.0126313 0.00421869 24
produce:wclsdatavd:WireCellToolkit 46.3358 57.6644 83.6642 50.6872 12.22 24
produce:gaushit:GausHitFinder 0.737198 1.42466 2.55485 1.37404 0.42424 24
produce:nhitsfilter:NumberOfHitsFilter 0.000186601 0.000364789 0.000826336 0.000327928 0.00013465 24
produce:reco3d:SpacePointSolver 6.09668 13.6915 25.3689 12.287 4.9782 24
produce:hitpdune:DisambigFromSpacePoints 0.0898159 0.214618 0.556925 0.184495 0.106331 24
produce:pandora:StandardPandora 20.7607 54.28 169.287 46.6413 33.163 24
produce:pandoraTrack:LArPandoraTrackCreation 0.215531 0.928297 2.54565 0.744341 0.605452 24
produce:pandoraGnocalo:GnocchiCalorimetry 0.0185862 0.0344221 0.0591024 0.0328659 0.0116358 24
[art]:TriggerResults:TriggerResultInserter 2.3725e-05 4.14302e-05 0.00010093 3.36285e-05 1.85836e-05 24
end_path:out1:RootOutput 3.798e-06 1.03332e-05 3.3182e-05 9.4575e-06 5.04417e-06 24
end_path:out1:RootOutput(write) 3.80465 5.14488 7.31554 4.92979 1.15795 24
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5174.52 MB
Peak resident set size usage (VmHWM): 3207.66 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027_reco_stage1_20250908T154306_keepup.root
\tHists: np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027_reco_stage1_20250908T154307_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027_reco_stage1_20250908T154306_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027_reco_stage1_20250908T154306_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346187.0,
"core.end_time": 1757346187.0,
"core.events": [
1499041,
1499061,
1499081,
1499101,
1499121,
1499141,
1499161,
1499181,
1499201,
1499221,
1499241,
1499261,
1499281,
1499301,
1499321,
1499341,
1499361,
1499381,
1499401,
1499421,
1499441,
1499461,
1499481,
1499501
],
"core.event_count": 24,
"core.first_event_number": 1499041,
"core.last_event_number": 1499501,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027_reco_stage1_20250908T154307_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346187.0,
"core.end_time": 1757346187.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2857_df-s01-d3_dw_0_20250908T033027.hdf5"
}
]
} 