Jobsub ID 240018.51@dunegpschedd01.fnal.gov
| Jobsub ID | 240018.51@dunegpschedd01.fnal.gov |
| Workflow ID | 9375 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-10-29 08:22:14 |
| Site | NL_NIKHEF |
| Entry | VIRGO_NL_NIKHEF_juk |
| Last heartbeat | 2025-10-29 12:15:53 |
| From worker node | Hostname | wn-lot-032.farm.nikhef.nl |
| cpuinfo | AMD EPYC 7702P 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | aborted |
| Started | 2025-10-29 10:43:24 |
| Input files | vd-protodune:np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448.hdf5
|
| Outputting started | 2025-10-29 11:58:03 |
| Output files | |
| Finished | 2025-10-29 12:15:53 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
e=0
[12:56:08.443] D [ glue ] <FrameFanin:nfsp> EOS at call=45 with 8
[12:56:08.443] D [ glue ] frame sink sees EOS
[12:56:08.443] D [ pgraph ] <Pgrapher:> graph execution complete
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 15.04 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 14.05 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 11.06 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.14 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.12 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.65 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.4 sec
[12:56:08.443] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.21 sec
[12:56:08.443] I [ timer ] Timer: WireCell::Aux::Resampler : 0.25 sec
[12:56:08.443] I [ timer ] Timer: WireCell::Aux::Resampler : 0.25 sec
[12:56:08.443] I [ timer ] Timer: WireCell::Aux::Resampler : 0.24 sec
[12:56:08.443] I [ timer ] Timer: WireCell::Aux::Resampler : 0.24 sec
[12:56:08.444] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[12:56:08.444] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:56:08.444] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[12:56:08.444] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[12:56:08.444] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[12:56:08.444] I [ timer ] Timer: Total node execution : 89.68000006489456 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 58599 traces tagged "gauss"
FrameSaver: q=2.40989e+07 n=1404167 tag=gauss
wclsFrameSaver: saving 73930 traces tagged "wiener"
FrameSaver: q=2.56772e+07 n=1347143 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1931 XUs and 1554 XVs -> 72 XUVs
C:0 T:1 38925 XUs and 50752 XVs -> 3659 XUVs
C:0 T:2 978 XUs and 1528 XVs -> 41 XUVs
C:0 T:3 14553 XUs and 20480 XVs -> 860 XUVs
C:0 T:4 695 XUs and 672 XVs -> 25 XUVs
C:0 T:5 113 XUs and 101 XVs -> 2 XUVs
C:0 T:6 838 XUs and 914 XVs -> 58 XUVs
C:0 T:7 174 XUs and 151 XVs -> 6 XUVs
C:0 T:8 407 XUs and 459 XVs -> 34 XUVs
C:0 T:9 1829 XUs and 1794 XVs -> 102 XUVs
C:0 T:10 1467 XUs and 1804 XVs -> 82 XUVs
C:0 T:11 165 XUs and 171 XVs -> 29 XUVs
C:0 T:12 1164 XUs and 1118 XVs -> 59 XUVs
C:0 T:13 3141 XUs and 3278 XVs -> 171 XUVs
C:0 T:14 932 XUs and 734 XVs -> 79 XUVs
C:0 T:15 2588 XUs and 2931 XVs -> 182 XUVs
5461 XUVs total
2596 collection wire objects
5461 potential space points
Neighbour search...
257035 tests to find 54362 neighbours
Iterating with no regularization...
Begin: 1.21964e+11
0 1.12621e+11
1 1.12239e+11
2 1.12233e+11
Now with regularization...
Begin: 1.11351e+11
0 1.11349e+11
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
29-Oct-2025 12:57:54 CET Closed output file "np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448_reco_stage1_20251029T115754_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 109.331 192.319 505.681 137.956 120.168 23
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.4691e-05 8.52753e-05 0.000247634 7.7725e-05 3.52196e-05 23
produce:tpcrawdecoder:PDVDTPCReader 14.9543 23.0752 29.679 23.5426 2.93882 23
produce:triggerrawdecoder:PDVDTriggerReader4 0.29432 0.397261 2.24476 0.297151 0.39473 23
produce:pdvddaphne:DAPHNEReaderPDVD 0.000470492 0.000511397 0.000920765 0.00049104 8.8692e-05 23
produce:ophit:OpHitFinder 7.4008e-05 0.000102364 0.000652342 7.6804e-05 0.000117286 23
produce:opflash:OpFlashFinderVerticalDrift 5.7728e-05 7.55207e-05 0.000403136 5.9962e-05 6.99355e-05 23
produce:wclsdatavd:WireCellToolkit 51.8488 77.3637 169.373 71.7534 24.8686 23
produce:gaushit:GausHitFinder 0.814885 1.4591 5.31261 1.22251 0.906129 23
produce:nhitsfilter:NumberOfHitsFilter 0.000233798 0.00055794 0.0010301 0.000522919 0.000216454 23
produce:reco3d:SpacePointSolver 6.82625 15.6982 39.7248 13.1083 8.10038 23
produce:hitpdune:DisambigFromSpacePoints 0.0707197 0.249408 0.885294 0.196411 0.185356 23
produce:pandora:StandardPandora 11.7363 68.0429 332.031 26.9161 96.1742 23
produce:pandoraTrack:LArPandoraTrackCreation 0.29561 1.53085 7.37374 0.931107 1.60954 23
produce:pandoraGnocalo:GnocchiCalorimetry 0.0205863 0.0354473 0.0558667 0.033643 0.00964813 23
[art]:TriggerResults:TriggerResultInserter 1.7453e-05 2.30563e-05 5.9772e-05 2.145e-05 8.19184e-06 23
end_path:out1:RootOutput 4.318e-06 5.55039e-06 2.4486e-05 4.749e-06 4.04277e-06 23
end_path:out1:RootOutput(write) 3.81465 4.40084 5.12187 4.38993 0.402499 23
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 7940.4 MB
Peak resident set size usage (VmHWM): 6057.04 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448_reco_stage1_20251029T115754_keepup.root
\tHists: np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448_reco_stage1_20251029T115754_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448_reco_stage1_20251029T115754_keepup.root
Ran successfully
{
"name": "np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448_reco_stage1_20251029T115754_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1761739075.0,
"core.end_time": 1761739075.0,
"core.events": [
52009,
52017,
52025,
52033,
52041,
52049,
52057,
52065,
52073,
52081,
52089,
52097,
52105,
52113,
52121,
52129,
52137,
52145,
52153,
52161,
52169,
52177,
52185
],
"core.event_count": 23,
"core.first_event_number": 52009,
"core.last_event_number": 52185,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40267
],
"core.runs_subruns": [
4026700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448_reco_stage1_20251029T115754_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1761739075.0,
"core.end_time": 1761739075.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40267
],
"core.runs_subruns": [
4026700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040267_0281_df-s03-d1_dw_0_20251026T022448.hdf5"
}
]
} 