Jobsub ID 47593.10@dunegpschedd01.fnal.gov
| Jobsub ID | 47593.10@dunegpschedd01.fnal.gov |
| Workflow ID | 2815 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-17 14:47:21 |
| Site | CZ_FZU |
| Entry | Nova_CZ_FZU_osgce1 |
| Last heartbeat | 2025-09-17 17:12:19 |
| From worker node | Hostname | mahagon01.farm.particle.cz |
| cpuinfo | AMD EPYC 7301 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2025-09-17 14:52:08 |
| Input files | vd-protodune:np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428.hdf5
|
| Outputting started | 2025-09-17 16:28:45 |
| Output files | |
| Finished | 2025-09-17 18:10:49 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
: WireCell::SigProc::OmnibusSigProc : 18.92 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 17.35 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 16.88 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 14.93 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 13.96 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.83 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.21 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.05 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Aux::Resampler : 1.41 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Aux::Resampler : 1.3 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Aux::Resampler : 1.29 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Aux::Resampler : 1.28 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.06 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.03 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Gen::Retagger : 0.02 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[18:22:42.070] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[18:22:42.070] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[18:22:42.070] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[18:22:42.070] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[18:22:42.070] I [ timer ] Timer: Total node execution : 122.54999996349216 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 81649 traces tagged "gauss"
FrameSaver: q=4.35094e+07 n=2166376 tag=gauss
wclsFrameSaver: saving 99761 traces tagged "wiener"
FrameSaver: q=4.45759e+07 n=2105834 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1755 XUs and 1914 XVs -> 58 XUVs
C:0 T:1 842 XUs and 1043 XVs -> 39 XUVs
C:0 T:2 495 XUs and 579 XVs -> 16 XUVs
C:0 T:3 3782 XUs and 3262 XVs -> 161 XUVs
C:0 T:4 25229 XUs and 31883 XVs -> 2188 XUVs
C:0 T:5 3511 XUs and 3283 XVs -> 135 XUVs
C:0 T:6 99 XUs and 123 XVs -> 6 XUVs
C:0 T:7 1498 XUs and 1627 XVs -> 45 XUVs
C:0 T:8 3623 XUs and 3178 XVs -> 168 XUVs
C:0 T:9 5609 XUs and 7340 XVs -> 576 XUVs
C:0 T:10 3376 XUs and 2918 XVs -> 223 XUVs
C:0 T:11 2206 XUs and 2410 XVs -> 94 XUVs
C:0 T:12 9697 XUs and 16473 XVs -> 893 XUVs
C:0 T:13 1971 XUs and 2673 XVs -> 156 XUVs
C:0 T:14 2352 XUs and 3405 XVs -> 176 XUVs
C:0 T:15 2316 XUs and 3735 XVs -> 156 XUVs
5090 XUVs total
2991 collection wire objects
5090 potential space points
Neighbour search...
102966 tests to find 31228 neighbours
Iterating with no regularization...
Begin: 9.22715e+10
0 9.16662e+10
1 9.16396e+10
Now with regularization...
Begin: 9.02388e+10
0 9.02378e+10
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
17-Sep-2025 18:28:31 CEST Closed output file "np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428_reco_stage1_20250917T162831_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 197.658 300.273 492.838 263.991 88.653 19
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 9.9638e-05 0.000204429 0.000522208 0.00014833 0.000127169 19
produce:tpcrawdecoder:PDVDTPCReader 15.7421 18.1864 38.0582 16.3634 5.24477 19
produce:triggerrawdecoder:PDVDTriggerReader4 0.272812 0.27833 0.306632 0.276571 0.00719851 19
produce:pdvddaphne:DAPHNEReaderPDVD 0.000446103 0.000583424 0.00105355 0.000503111 0.000194895 19
produce:ophit:OpHitFinder 0.000209386 0.000465921 0.00138725 0.00036496 0.000265936 19
produce:opflash:OpFlashFinderVerticalDrift 7.7236e-05 0.000144492 0.000462224 0.000106562 0.000101973 19
produce:wclsdatavd:WireCellToolkit 109.038 127.624 166.969 125.235 14.843 19
produce:gaushit:GausHitFinder 1.19642 2.29858 4.72539 1.93043 0.86314 19
produce:nhitsfilter:NumberOfHitsFilter 0.000338841 0.000642482 0.00167892 0.000542346 0.000302971 19
produce:reco3d:SpacePointSolver 12.8925 22.2939 40.8316 20.6017 8.05824 19
produce:hitpdune:DisambigFromSpacePoints 0.209576 0.379322 0.799187 0.345746 0.162619 19
produce:pandora:StandardPandora 34.3902 118.415 284.465 90.1536 71.5077 19
produce:pandoraTrack:LArPandoraTrackCreation 1.05412 2.7705 8.12699 2.32808 1.58985 19
produce:pandoraGnocalo:GnocchiCalorimetry 0.0355232 0.0603997 0.0962266 0.0589033 0.0163475 19
[art]:TriggerResults:TriggerResultInserter 2.5078e-05 6.32486e-05 0.00016907 5.5565e-05 3.13023e-05 19
end_path:out1:RootOutput 7.865e-06 1.46778e-05 4.4354e-05 1.1441e-05 8.23646e-06 19
end_path:out1:RootOutput(write) 6.4127 7.88897 9.29467 8.04805 0.770361 19
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5301.05 MB
Peak resident set size usage (VmHWM): 3316.99 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428_reco_stage1_20250917T162831_keepup.root
\tHists: np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428_reco_stage1_20250917T162831_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428_reco_stage1_20250917T162831_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428_reco_stage1_20250917T162831_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1758126512.0,
"core.end_time": 1758126512.0,
"core.events": [
26380,
26384,
26388,
26392,
26396,
26400,
26404,
26408,
26412,
26416,
26420,
26424,
26428,
26432,
26436,
26440,
26444,
26448,
26452
],
"core.event_count": 19,
"core.first_event_number": 26380,
"core.last_event_number": 26452,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39388
],
"core.runs_subruns": [
3938800001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428_reco_stage1_20250917T162831_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1758126512.0,
"core.end_time": 1758126512.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39388
],
"core.runs_subruns": [
3938800001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039388_0343_df-s05-d0_dw_0_20250916T202428.hdf5"
}
]
} 