Jobsub ID 44661.174@dunegpschedd01.fnal.gov
| Jobsub ID | 44661.174@dunegpschedd01.fnal.gov |
| Workflow ID | 2483 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-10 23:22:58 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce515 |
| Last heartbeat | 2025-09-11 03:04:13 |
| From worker node | Hostname | b9p24p7422.cern.ch |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | aborted |
| Started | 2025-09-11 01:46:45 |
| Input files | vd-protodune:np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012.hdf5
|
| Outputting started | 2025-09-11 02:49:20 |
| Output files | |
| Finished | 2025-09-11 03:04:13 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
6:59.193] D [ glue ] frame sink sees EOS
[04:46:59.193] D [ pgraph ] <Pgrapher:> graph execution complete
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.16 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.62 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.43 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.35 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.21 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.18 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.89 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.66 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Aux::Resampler : 0.76 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Aux::Resampler : 0.76 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Aux::Resampler : 0.75 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Aux::Resampler : 0.73 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.01 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[04:46:59.193] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[04:46:59.193] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[04:46:59.193] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[04:46:59.193] I [ timer ] Timer: Total node execution : 71.56000047549605 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 70716 traces tagged "gauss"
FrameSaver: q=1.21045e+07 n=1527445 tag=gauss
wclsFrameSaver: saving 88872 traces tagged "wiener"
FrameSaver: q=1.29597e+07 n=1466744 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 713 XUs and 807 XVs -> 34 XUVs
C:0 T:1 1706 XUs and 1803 XVs -> 60 XUVs
C:0 T:2 1769 XUs and 2101 XVs -> 37 XUVs
C:0 T:3 1103 XUs and 1470 XVs -> 47 XUVs
C:0 T:4 548 XUs and 663 XVs -> 21 XUVs
C:0 T:5 969 XUs and 872 XVs -> 59 XUVs
C:0 T:6 1766 XUs and 2076 XVs -> 51 XUVs
C:0 T:7 1723 XUs and 1781 XVs -> 55 XUVs
C:0 T:8 17231 XUs and 29871 XVs -> 6092 XUVs
C:0 T:9 14344 XUs and 13034 XVs -> 1797 XUVs
C:0 T:10 4734 XUs and 6772 XVs -> 1187 XUVs
C:0 T:11 8324 XUs and 12429 XVs -> 2112 XUVs
C:0 T:12 6946 XUs and 10204 XVs -> 1322 XUVs
C:0 T:13 12635 XUs and 17112 XVs -> 1933 XUVs
C:0 T:14 8434 XUs and 18067 XVs -> 1940 XUVs
C:0 T:15 698 XUs and 772 XVs -> 54 XUVs
16801 XUVs total
3239 collection wire objects
16801 potential space points
Neighbour search...
2015559 tests to find 881350 neighbours
Iterating with no regularization...
Begin: 5.85287e+08
0 5.19535e+08
1 5.14537e+08
2 5.13576e+08
3 5.13228e+08
Now with regularization...
Begin: 4.95486e+08
0 4.95187e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
11-Sep-2025 04:49:02 CEST Closed output file "np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012_reco_stage1_20250911T024902_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 141.056 205.827 302.342 203.528 41.5993 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.14e-05 8.65886e-05 0.000181042 7.1346e-05 3.81642e-05 18
produce:tpcrawdecoder:PDVDTPCReader 6.51647 7.93814 16.2811 7.50518 2.0512 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.00112594 0.00537321 0.0436408 0.00142736 0.0101067 18
produce:pdvddaphne:DAPHNEReaderPDVD 3.74946 4.74157 5.69037 4.69707 0.518339 18
produce:ophit:OpHitFinder 0.0353736 0.0486737 0.0610988 0.0491775 0.00736667 18
produce:opflash:OpFlashFinderVerticalDrift 0.00846837 0.0140624 0.019504 0.013178 0.00313068 18
produce:wclsdatavd:WireCellToolkit 67.1851 77.595 98.5751 75.5383 9.07622 18
produce:gaushit:GausHitFinder 0.834624 1.35225 2.4407 1.31233 0.345146 18
produce:nhitsfilter:NumberOfHitsFilter 0.000290853 0.000430792 0.000801909 0.000394564 0.000126278 18
produce:reco3d:SpacePointSolver 11.395 20.2683 28.2304 20.6411 5.25838 18
produce:hitpdune:DisambigFromSpacePoints 0.186712 0.330886 0.436569 0.342513 0.0695669 18
produce:pandora:StandardPandora 40.9455 86.5372 154.176 83.9955 33.2295 18
produce:pandoraTrack:LArPandoraTrackCreation 0.843185 1.77732 3.87382 1.65734 0.833494 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0223443 0.0375276 0.0774434 0.0341032 0.0134496 18
[art]:TriggerResults:TriggerResultInserter 1.61e-05 2.34818e-05 6.74e-05 1.82405e-05 1.29453e-05 18
end_path:out1:RootOutput 3.84e-06 8.21667e-06 2.264e-05 8.175e-06 4.18454e-06 18
end_path:out1:RootOutput(write) 4.57144 5.13935 7.04551 4.83264 0.677719 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5414.49 MB
Peak resident set size usage (VmHWM): 3481.15 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012_reco_stage1_20250911T024902_keepup.root
\tHists: np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012_reco_stage1_20250911T024902_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012_reco_stage1_20250911T024902_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012_reco_stage1_20250911T024902_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757558943.0,
"core.end_time": 1757558943.0,
"core.events": [
493285,
493305,
493325,
493345,
493365,
493385,
493405,
493425,
493445,
493465,
493485,
493505,
493525,
493545,
493565,
493585,
493605,
493625
],
"core.event_count": 18,
"core.first_event_number": 493285,
"core.last_event_number": 493625,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39255
],
"core.runs_subruns": [
3925500001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012_reco_stage1_20250911T024902_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757558943.0,
"core.end_time": 1757558943.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39255
],
"core.runs_subruns": [
3925500001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039255_1369_df-s04-d2_dw_0_20250831T012012.hdf5"
}
]
} 