Jobsub ID 36325.119@dunegpschedd02.fnal.gov
| Jobsub ID | 36325.119@dunegpschedd02.fnal.gov |
| Workflow ID | 2329 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 10:55:54 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2025-09-08 15:52:26 |
| From worker node | Hostname | n157.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 13:47:50 |
| Input files | vd-protodune:np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (7189s) |
| CPU time | 1h (4860s = 67%) |
| Max RSS bytes | 3014025216 (2874 MiB) |
| Outputting started | 2025-09-08 15:47:40 |
| Output files | |
| Finished | 2025-09-08 15:52:26 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.29 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.17 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.8 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.16 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.99 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.64 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.07 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Aux::Resampler : 0.36 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Aux::Resampler : 0.36 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Aux::Resampler : 0.36 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Aux::Resampler : 0.36 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.06 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Gen::Retagger : 0.02 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:41:19.343] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[16:41:19.343] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[16:41:19.343] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[16:41:19.343] I [ timer ] Timer: Total node execution : 72.39000020734966 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 40329 traces tagged "gauss"
FrameSaver: q=7.89039e+06 n=936477 tag=gauss
wclsFrameSaver: saving 50054 traces tagged "wiener"
FrameSaver: q=8.3599e+06 n=903168 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1974 XUs and 1701 XVs -> 47 XUVs
C:0 T:1 542 XUs and 525 XVs -> 10 XUVs
C:0 T:2 1600 XUs and 1297 XVs -> 42 XUVs
C:0 T:3 435 XUs and 408 XVs -> 28 XUVs
C:0 T:4 173 XUs and 123 XVs -> 9 XUVs
C:0 T:5 123 XUs and 123 XVs -> 6 XUVs
C:0 T:6 8 XUs and 15 XVs -> 0 XUVs
C:0 T:7 44 XUs and 57 XVs -> 0 XUVs
C:0 T:8 5473 XUs and 22334 XVs -> 1156 XUVs
C:0 T:9 8986 XUs and 29021 XVs -> 2237 XUVs
C:0 T:10 18130 XUs and 35126 XVs -> 7092 XUVs
C:0 T:11 1508 XUs and 5441 XVs -> 378 XUVs
C:0 T:12 10616 XUs and 16038 XVs -> 1899 XUVs
C:0 T:13 633 XUs and 575 XVs -> 45 XUVs
C:0 T:14 2250 XUs and 1500 XVs -> 95 XUVs
C:0 T:15 358 XUs and 255 XVs -> 19 XUVs
13063 XUVs total
1601 collection wire objects
13063 potential space points
Neighbour search...
2068731 tests to find 870922 neighbours
Iterating with no regularization...
Begin: 4.427e+08
0 3.67947e+08
1 3.64525e+08
2 3.64041e+08
3 3.63819e+08
Now with regularization...
Begin: 3.55911e+08
0 3.55693e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 16:43:22 BST Closed output file "np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031_reco_stage1_20250908T154322_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 169.599 253.466 335.128 252.405 44.118 27
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.4896e-05 8.5022e-05 0.000213165 7.7309e-05 2.76626e-05 27
produce:tpcrawdecoder:PDVDTPCReader 60.1165 72.1322 141.871 69.5083 16.0627 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.0594327 0.0644165 0.101148 0.0596745 0.010425 27
produce:pdvddaphne:DAPHNEReaderPDVD 8.46766 10.8425 15.3704 10.4415 1.68316 27
produce:ophit:OpHitFinder 0.0418189 0.0591595 0.0690911 0.0594082 0.00549102 27
produce:opflash:OpFlashFinderVerticalDrift 0.00656409 0.01373 0.0203543 0.0136852 0.00295189 27
produce:wclsdatavd:WireCellToolkit 55.7123 70.9466 105.893 69.7921 8.41037 27
produce:gaushit:GausHitFinder 0.732128 1.61919 2.30546 1.67006 0.36801 27
produce:nhitsfilter:NumberOfHitsFilter 0.000337826 0.000548933 0.000880883 0.000527268 0.000151655 27
produce:reco3d:SpacePointSolver 6.25863 14.6759 24.0395 15.1657 4.35947 27
produce:hitpdune:DisambigFromSpacePoints 0.115945 0.295545 0.483045 0.311179 0.100701 27
produce:pandora:StandardPandora 21.7009 75.8892 131.232 74.8703 30.4631 27
produce:pandoraTrack:LArPandoraTrackCreation 0.324559 2.51592 5.93754 2.45974 1.12965 27
produce:pandoraGnocalo:GnocchiCalorimetry 0.0153072 0.0373883 0.0563893 0.0388189 0.00978904 27
[art]:TriggerResults:TriggerResultInserter 2.9611e-05 4.51936e-05 9.6531e-05 4.2061e-05 1.33982e-05 27
end_path:out1:RootOutput 6.757e-06 1.04586e-05 2.5123e-05 8.917e-06 4.20888e-06 27
end_path:out1:RootOutput(write) 3.80169 4.30149 6.22081 4.24903 0.424662 27
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4918.97 MB
Peak resident set size usage (VmHWM): 3014.03 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031_reco_stage1_20250908T154322_keepup.root
\tHists: np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031_reco_stage1_20250908T154322_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031_reco_stage1_20250908T154322_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031_reco_stage1_20250908T154322_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346202.0,
"core.end_time": 1757346202.0,
"core.events": [
1457897,
1457917,
1457937,
1457957,
1457977,
1457997,
1458017,
1458037,
1458057,
1458077,
1458097,
1458117,
1458137,
1458157,
1458177,
1458197,
1458217,
1458237,
1458257,
1458277,
1458297,
1458317,
1458337,
1458357,
1458377,
1458397,
1458417
],
"core.event_count": 27,
"core.first_event_number": 1457897,
"core.last_event_number": 1458417,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031_reco_stage1_20250908T154322_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346202.0,
"core.end_time": 1757346202.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2780_df-s05-d3_dw_0_20250908T021031.hdf5"
}
]
} 