Jobsub ID 43065.97@dunegpschedd01.fnal.gov
| Jobsub ID | 43065.97@dunegpschedd01.fnal.gov |
| Workflow ID | 2327 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 11:23:53 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2025-09-08 15:52:35 |
| From worker node | Hostname | n202.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 13:49:50 |
| Input files | vd-protodune:np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (6773s) |
| CPU time | 1h (4348s = 64%) |
| Max RSS bytes | 2946887680 (2810 MiB) |
| Outputting started | 2025-09-08 15:42:45 |
| Output files | |
| Finished | 2025-09-08 15:52:35 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
[16:38:12.778] D [ pgraph ] <Pgrapher:> graph execution complete
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.48 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.89 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.84 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.74 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.71 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.56 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.51 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.99 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Aux::Resampler : 0.37 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Aux::Resampler : 0.35 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Aux::Resampler : 0.35 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Aux::Resampler : 0.34 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.04 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Gen::Retagger : 0.02 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:38:12.778] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[16:38:12.778] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[16:38:12.778] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[16:38:12.778] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[16:38:12.778] I [ timer ] Timer: Total node execution : 70.2299997843802 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 46336 traces tagged "gauss"
FrameSaver: q=9.35577e+06 n=1042466 tag=gauss
wclsFrameSaver: saving 59768 traces tagged "wiener"
FrameSaver: q=9.95757e+06 n=984296 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1378 XUs and 1154 XVs -> 71 XUVs
C:0 T:1 301 XUs and 326 XVs -> 9 XUVs
C:0 T:2 2253 XUs and 1749 XVs -> 90 XUVs
C:0 T:3 73 XUs and 99 XVs -> 5 XUVs
C:0 T:4 263 XUs and 418 XVs -> 12 XUVs
C:0 T:5 59 XUs and 46 XVs -> 0 XUVs
C:0 T:6 978 XUs and 1515 XVs -> 51 XUVs
C:0 T:7 141 XUs and 258 XVs -> 4 XUVs
C:0 T:8 13279 XUs and 17572 XVs -> 2708 XUVs
C:0 T:9 18622 XUs and 29099 XVs -> 3443 XUVs
C:0 T:10 19604 XUs and 39944 XVs -> 6568 XUVs
C:0 T:11 8078 XUs and 7699 XVs -> 823 XUVs
C:0 T:12 5418 XUs and 8071 XVs -> 1120 XUVs
C:0 T:13 1433 XUs and 1638 XVs -> 77 XUVs
C:0 T:14 1075 XUs and 740 XVs -> 35 XUVs
C:0 T:15 2273 XUs and 3622 XVs -> 309 XUVs
15325 XUVs total
2534 collection wire objects
15325 potential space points
Neighbour search...
2003959 tests to find 865456 neighbours
Iterating with no regularization...
Begin: 1.6682e+09
0 1.55023e+09
1 1.5448e+09
2 1.54385e+09
Now with regularization...
Begin: 1.52573e+09
0 1.52523e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 16:41:34 BST Closed output file "np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249_reco_stage1_20250908T154134_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 153.395 245.256 378.16 242.085 55.8436 27
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.4282e-05 9.5461e-05 0.000258387 7.7679e-05 3.99216e-05 27
produce:tpcrawdecoder:PDVDTPCReader 53.8208 73.7717 104.226 71.4068 14.3869 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.0591659 0.0678267 0.137598 0.0596229 0.016047 27
produce:pdvddaphne:DAPHNEReaderPDVD 8.20677 11.2893 15.4388 10.6892 2.28846 27
produce:ophit:OpHitFinder 0.0400172 0.0589091 0.211291 0.0532566 0.0306198 27
produce:opflash:OpFlashFinderVerticalDrift 0.00871892 0.0138667 0.0200968 0.0131596 0.00296509 27
produce:wclsdatavd:WireCellToolkit 49.7842 72.1431 162.886 66.7524 21.8226 27
produce:gaushit:GausHitFinder 0.851591 1.42308 2.66382 1.34629 0.373754 27
produce:nhitsfilter:NumberOfHitsFilter 0.000289821 0.000441059 0.00083737 0.000407466 0.000119818 27
produce:reco3d:SpacePointSolver 7.47964 12.5946 24.5112 11.6175 3.92274 27
produce:hitpdune:DisambigFromSpacePoints 0.136377 0.272372 0.766202 0.239428 0.148122 27
produce:pandora:StandardPandora 24.2952 67.3075 170.947 51.6685 37.0722 27
produce:pandoraTrack:LArPandoraTrackCreation 0.474066 1.69444 7.57627 1.21341 1.53578 27
produce:pandoraGnocalo:GnocchiCalorimetry 0.0198305 0.0384279 0.225925 0.0279557 0.0394337 27
[art]:TriggerResults:TriggerResultInserter 2.8565e-05 4.2822e-05 9.8359e-05 4.0084e-05 1.35655e-05 27
end_path:out1:RootOutput 4.243e-06 9.07333e-06 3.6294e-05 7.765e-06 5.68299e-06 27
end_path:out1:RootOutput(write) 3.88547 4.52379 9.30319 4.09921 1.29839 27
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4839 MB
Peak resident set size usage (VmHWM): 2946.89 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249_reco_stage1_20250908T154134_keepup.root
\tHists: np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249_reco_stage1_20250908T154134_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249_reco_stage1_20250908T154134_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249_reco_stage1_20250908T154134_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346095.0,
"core.end_time": 1757346095.0,
"core.events": [
1301260,
1301280,
1301300,
1301320,
1301340,
1301360,
1301380,
1301400,
1301420,
1301440,
1301460,
1301480,
1301500,
1301520,
1301540,
1301560,
1301580,
1301600,
1301620,
1301640,
1301660,
1301680,
1301700,
1301720,
1301740,
1301760,
1301780
],
"core.event_count": 27,
"core.first_event_number": 1301260,
"core.last_event_number": 1301780,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249_reco_stage1_20250908T154134_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346095.0,
"core.end_time": 1757346095.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2486_df-s01-d2_dw_0_20250907T203249.hdf5"
}
]
} 