Minimum Mean Square Error Estimator Research Articles

On training targets for deep learning approaches to clean speech magnitude spectrum estimation.

Estimation of the clean speech short-time magnitude spectrum (MS) is key for speech enhancement and separation. Moreover, an automatic speech recognition (ASR) system that employs a front-end relies on clean speech MS estimation to remain robust. Training targets for deep learning approaches to clean speech MS estimation fall into three categories: computational auditory scene analysis (CASA), MS, and minimum mean square error (MMSE) estimator training targets. The choice of the training target can have a significant impact on speech enhancement/separation and robust ASR performance. Motivated by this, the training target that produces enhanced/separated speech at the highest quality and intelligibility and that which is best for an ASR front-end is found. Three different deep neural network (DNN) types and two datasets, which include real-world nonstationary and coloured noise sources at multiple signal-to-noise ratio (SNR) levels, were used for evaluation. Ten objective measures were employed, including the word error rate of the Deep Speech ASR system. It is found that training targets that estimate the a priori SNR for MMSE estimators produce the highest objective quality scores. Moreover, it is established that the gain of MMSE estimators and the ideal amplitude mask produce the highest objective intelligibility scores and are most suitable for an ASR front-end.

An improved MMSE estimator based modified group delay spectrum for Forensic Automatic Speaker Recognition

This paper presents an improved speech enhancement algorithm called Minimum Mean Square Error (MMSE), based on the MODified Group Delay spectrum (MODGD), for Forensic Automatic Speaker Recognition (FASR) under noisy environments. This algorithm uses the MODGD instead of the amplitude spectrum, to compute the power spectrum of the noise-corrupt signal. In the proposed estimator, the MODGD retains most of the formants information. Therefore, it enhances the noisy speech signal with high quality even at extremely low Signal-to-Noise Ratio (SNR) levels. The evaluation of the improved algorithm in simulated FASR scenarios was performed by adding different noise levels, extracted from the NOISEX-92 database to the clean NIST2000-traces. The results obtained show that the proposed MMSE–MODGD estimator provides greater suppression of noise components in regions of low SNR than the MMSE estimator. In addition, there is a drastic reduction in Equal Proportion Probability (EPP) (the improvements are 1.84% for babble noise and 1.25% for factory and white noises), combining FASR techniques with the proposed MMSE–MODGD estimator than with the conventional estimator.

An enhanced iterative receiver based on vector approximate message passing for deep-sea vertical underwater acoustic communications.

Vertical underwater acoustic (UWA) communications play a crucial role in deep-sea applications. A vertical UWA channel generally features a moderate multipath but with time-varying Doppler shifts as well as loud impulsive noise. To achieve a robust vertical single-carrier UWA communication, this paper proposes an enhanced iterative receiver. First, a spline interpolation-based timing estimation approach is proposed to compensate for the time-varying Doppler effects efficiently. Then, the residual timing errors and the multipath interference are tackled by a fractionally spaced self-iterative soft equalizer (SISE) based on the vector approximate message passing (VAMP) algorithm. The VAMP-SISE consists of four parts: an inner soft slicer and an inner soft equalizer for symbol detection as well as a denoiser and a minimum mean-squared error estimator for impulsive noise suppression. Different parts iteratively exchange extrinsic information to improve the equalization performance. Last, a channel-fitting irregular convolutional code and a unity-rate code are employed at the transmitter to lower the signal-to-noise ratio threshold for reliable communications. Deep-sea experiments verify the performance superiority of the proposed receiver over existing schemes.

An Estimator for Weather Radar Doppler Power Spectrum via Minimum Mean Square Error

This article proposes a method for estimating the Doppler power spectrum (DPS) of a weather radar via minimum mean square error (MMSE). In order to detect severe weather phenomena that mostly occur within the lowest few kilometers of the atmosphere, weather radars have to direct their beams at low elevation angles, and the received signals from such observations usually contain reflections from the ground and buildings, so-called ground clutter. The MMSE estimator, which is an adaptive spectral estimator, allows weather radar DPSs to be obtained with excellently reduced ground clutter contaminations. The MMSE estimator was examined by numerical simulations, which supposed various precipitation and ground clutter scenarios and DPS estimation parameter values. The MMSE estimator provided DPSs almost as accurate as those from the traditional Fourier and windowed Fourier estimators in simulations with no ground clutter and much better DPSs than those estimators in the presence of ground clutter. Furthermore, the MMSE estimator gave better suppression of ground clutter contamination than the Capon estimator, which is another adaptive spectral estimator. As a result of statistical evaluations, ground clutter signals with a strong clutter-to-noise ratio of 70 dB appeared only in a narrow velocity range of the MMSE DPS, from −2 to 2 m/s, and caused degradation of the mean and standard errors outside this velocity range by just 1 dB. The MMSE estimator was also applied to signals received by actual weather radar, and DPS estimation of precipitation signals with similarly low ground clutter contamination was demonstrated.

Fast iterative adaptive approach for 3D seismic data reconstruction

ABSTRACT In this paper, we introduce a fast iterative adaptive approach for the reconstruction of 3D seismic data with randomly missing traces. Our method starts by transforming 3D seismic volume to 2D harmonic signal for each frequency slice, and then the power spectrum of the frequency slice is iteratively estimated by a weighted least square fitting criterion. The missing data can be recovered with the obtained spectral estimate using a linear minimum mean-squared error estimator. However, estimation of the power spectrum depends on matrix-vector multiplications for each iteration that leads to high computation complexity when the data are large scale and high dimension. To solve the associated spectrum estimation problem above, a fast iterative adaptive scheme is adopted by utilising 2D fast discrete Fourier transform, which makes use of the block-Vandermonde structure and the 2D Fourier property of the steering matrix. Simulation experiments on synthetic and field data verify the effectiveness of the proposed algorithm. Compared with other reconstruction methods based on frequency slice, such as minimum-weighted norm interpolation approach, multi-channel singular spectrum analysis, and Curvelet transform method, the proposed method achieves better reconstruction performance and low computational complexity.

A Tensor Framework for Multi-Linear Complex MMSE Estimation

Tensors are higher order generalization of vectors and matrices which can be used to describe signals indexed by more than two indices. This paper introduces a tensor framework for minimum mean square error (MMSE) estimation for multi-domain signals and data using the Einstein Product. The framework addresses both proper and improper complex tensors. The multi-domain nature of tensors has been harnessed to provide an augmented representation of improper complex tensors to account for covariance and pseudo-covariance. The classical notions of linear and widely linear MMSE estimators are extended to tensor case leading to the notion of multi-linear and widely multi-linear MMSE estimation. The Tucker product based n-mode Wiener filtering approach more commonly used in tensor estimation has been shown to be a special case of the proposed multi-linear MMSE estimation. An application of the tensor based estimation in a multiple antenna Orthogonal Frequency Division Multiplexing (MIMO OFDM) system is presented where the tensor formulation allows a convenient treatment of inter-carrier interference. A comparison between the tensor estimation and per sub-carrier estimation used for MIMO OFDM is presented which shows a significant performance advantage of using the tensor framework.

Multi-Agent Estimation and Filtering for Minimizing Team Mean-Squared Error

Motivated by estimation problems arising in autonomous vehicles and decentralized control of unmanned aerial vehicles, we consider multi-agent estimation and filtering problems in which multiple agents generate state estimates based on decentralized information and the objective is to minimize a coupled mean-squared error which we call <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">team mean-square error</i> . We call the resulting estimates as minimum team mean-squared error (MTMSE) estimates. We show that MTMSE estimates are different from minimum mean-squared error (MMSE) estimates. We derive closed-form expressions for MTMSE estimates, which are linear function of the observations where the corresponding gain depends on the weight matrix that couples the estimation error. We then consider a filtering problem where a linear stochastic process is monitored by multiple agents which can share their observations (with delay) over a communication graph. We derive expressions to recursively compute the MTMSE estimates. To illustrate the effectiveness of the proposed scheme we consider an example of estimating the distances between vehicles in a platoon and show that MTMSE estimates significantly outperform MMSE estimates and consensus Kalman filtering estimates.

Wire-Mesh Sensor Super-Resolution Based on Statistical Reconstruction

A wire-mesh sensor (WMS) is a widely used instrument to visualize and estimate derived parameters of multiphase flows, e.g., gas void fraction or liquid hold-up. The spatial resolution of obtained flow images is associated with the number of crossing points formed by the transmitter and receiver wires of a given sensor. This may be a limitation for applications that require high spatial resolution since WMS is an intrusive device and the increase of electrodes may increase pressure drop and deform/fragment bubbles. In order to minimize such undesirable effects and maximize the sensor resolution, we employed a reconstruction algorithm based on the minimum mean-square error (MMSE) estimator to increase image resolution of WMS with fewer wires than commonly reported in the literature, i.e., here, we apply <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8\times 8$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6\times 6$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4\times 4$ </tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\times 2$ </tex-math></inline-formula> sensors for 1-in pipe. Since standard regularization approaches may provide incorrect solutions for such configurations, a new methodology to obtain the prior model is presented. In our approach, the prior is assumed as a multivariate Gaussian model, which is extracted from experimental flow data of a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16\times 16$ </tex-math></inline-formula> WMS (the most common resolution for 1-in pipe). Finally, the sensitivity matrix obtained by electric field simulation and the experimental prior model is incorporated into the MMSE algorithm to restore experimental flow data of the low-resolution sensors. The experiments were performed in a flow loop operating at slug flow. The experimental results suggest that the MMSE estimator combined with the experimental prior model has a high potential not only to improve image resolution but also to correct the average void fraction estimation.

Channel Estimation with an Interpolation Trained Deep Neural Network

This paper proposes a deep learning-based channel estimation method for orthogonal frequency-division multiplexing (OFDM) systems. The existing OFDM receiver has low estimation accuracy when estimating channel state information (CSI) with fewer pilots. To tackle the problem, in this paper, a deep learning model is first trained by the interpolated channel frequency responses (CFRs) and then used to denoise the CFR estimated by least square (LS) estimation. The proposed deep neural network (DNN) can also be trained in a short time because it only learns the CFR and the network structure is simple. According to the simulation results, the performance of the DNN estimator can be compared with the minimum mean-square error (MMSE) estimator. Furthermore, the DNN approach is more robust than conventional methods when fewer pilots are used. In summary, deep learning is a promising tool for channel estimation in wireless communications.

Robust Importance Sampling for Error Estimation in the Context of Optimal Bayesian Transfer Learning

Classification has been a major task for building intelligent systems as it enables decision-making under uncertainty. Classifier design aims at building models from training data for representing feature-label distributions--either explicitly or implicitly. In many scientific or clinical settings, training data are typically limited, which makes designing accurate classifiers and evaluating their classification error extremely challenging. While transfer learning (TL) can alleviate this issue by incorporating data from relevant source domains to improve learning in a different target domain, it has received little attention for performance assessment, notably in error estimation. In this paper, we fill this gap by investigating knowledge transferability in the context of classification error estimation within a Bayesian paradigm. We introduce a novel class of Bayesian minimum mean-square error (MMSE) estimators for optimal Bayesian transfer learning (OBTL), which enables rigorous evaluation of classification error under uncertainty in a small-sample setting. Using Monte Carlo importance sampling, we employ the proposed estimator to evaluate the classification accuracy of a broad family of classifiers that span diverse learning capabilities. Experimental results based on both synthetic data as well as real-world RNA sequencing (RNA-seq) data show that our proposed OBTL error estimation scheme clearly outperforms standard error estimators, especially in a small-sample setting, by tapping into the data from other relevant domains.

Resource Allocation and Dithering of Bayesian Parameter Estimation Using Mixed-Resolution Data

Quantization of signals is an integral part of modern signal processing applications, such as sensing, communication, and inference. While signal quantization provides many physical advantages, it usually degrades the subsequent estimation performance that is based on quantized data. In order to maintain physical constraints and simultaneously bring substantial performance gain, in this work we consider systems with mixed-resolution, 1-bit quantized and continuous-valued, data. First, we describe the linear minimum mean-squared error (LMMSE) estimator and its associated mean-squared error (MSE) for the general mixed-resolution model. However, the MSE of the LMMSE requires matrix inversion in which the number of measurements defines the matrix dimensions and thus, is not a tractable tool for optimization and system design. Therefore, we present the linear Gaussian orthonormal (LGO) measurement model and derive a closed-form analytic expression for the MSE of the LMMSE estimator under this model. In addition, we present two common special cases of the LGO model: 1) scalar parameter estimation and 2) channel estimation in mixed-ADC multiple-input multiple-output (MIMO) communication systems. We then solve the resource allocation optimization problem of the LGO model with the proposed tractable form of the MSE as an objective function and under a power constraint using a one-dimensional search. Moreover, we present the concept of dithering for mixed-resolution models and optimize the dithering noise as part of the resource allocation optimization problem for two dithering schemes: 1) adding noise only to the quantized measurements and 2) adding noise to both measurement types. Finally, we present simulations that demonstrate the advantages of using mixed-resolution measurements and the possible improvement introduced with dithering and resource allocation.

The Art of Piecewise Linear Approximation in MMSE Estimator for Most Accurate and Fast Frequency Extraction in DIFM Receivers

Detection of wideband passive radar signals, accomplished by instantaneous frequency measurement (IFM) systems, is crucial for gaining a profound knowledge of environmental circumstances. The need for fast pulse detection has hindered the investigation of frequency extraction with higher accuracy in IFM receivers. A noteworthy research in recent years shows that fast and reliable frequency extraction in high bit-count (HBC) IFMs while attaining a better frequency accuracy [called intelligent, reliable, and design-independent (InReDI)] is feasible. In addition, some investigated process-intensive algorithms like the minimum mean squared error (MMSE) estimator promise excellent frequency accuracy. In this work, we present a new method of MMSE algorithm modified by a piecewise linear approximation [named linear approximation of MMSE (LAMMSE)]. By utilizing the InReDI methodology and the novel LAMMSE algorithm, we propose a new architecture for a fast, accurate, and reliable frequency extraction (FAREx) in HBC digital IFMs (DIFM) receivers. The proposed FAREx architecture resolves the compromise challenge between frequency accuracy and extraction speed that has been long lingered in DIFM receivers. We have implemented the new architecture using Xilinx field-programmable gate array (FPGA) on a fabricated 15 bit, 2–4 GHz, and 70 dB dynamic range IFM receiver. The measurement results show an average frequency rms error of 1.43 MHz for a CW signal, 2.49 MHz average rms error for a 50 ns pulsed input signal, and 2.7 MP/s detection speed, representing a state-of-the-art receiver and a breakthrough compared to the latest works.

Optimal Fractional Fourier Filtering for Graph Signals

Graph signal processing has recently received considerable attention. Several concepts, tools, and applications in signal processing such as filtering, transforming, and sampling have been extended to graph signal processing. One such extension is the optimal filtering problem. The minimum mean-squared error estimate of an original graph signal can be obtained from its distorted and noisy version. However, the best separation of signal and noise, and thus the least error, is not always achieved in the ordinary Fourier domain, but rather a fractional Fourier domain. In this work, the optimal filtering problem for graph signals is extended to fractional Fourier domains, and theoretical analysis and solution of the proposed problem are provided along with computational cost considerations. Numerical results are presented to illustrate the benefits of filtering in fractional Fourier domains.

Deep Learning for Channel Estimation: Interpretation, Performance, and Comparison

Deep learning (DL) has emerged as an effective tool for channel estimation in wireless communication systems, especially under some imperfect environments. However, even with such unprecedented success, DL methods are often regarded as black boxes and are lack of explanations on their internal mechanisms, which severely limits their further improvement and extension. In this paper, we present preliminary theoretical analysis on DL based channel estimation for single-input multiple-output (SIMO) systems to understand and interpret its internal mechanisms. As deep neural network (DNN) with rectified linear unit (ReLU) activation function is mathematically equivalent to a piecewise linear function, the corresponding DL estimator can achieve universal approximation to a large family of functions by making efficient use of piecewise linearity. We demonstrate that DL based channel estimation does not restrict to any specific signal model and asymptotically approaches to the minimum mean-squared error (MMSE) estimation in various scenarios without requiring any prior knowledge of channel statistics. Therefore, DL based channel estimation outperforms or is at least comparable with traditional channel estimation, depending on the types of channels. Simulation results confirm the accuracy of the proposed interpretation and demonstrate the effectiveness of DL based channel estimation under both linear and nonlinear signal models.

State estimation with multi-level vector quantisation and communication uncertainty

In this paper, we design a state estimation algorithm for vector state-vector measurement systems over wireless sensor networks subject to bandwidth limitation and communication uncertainty. With the aid of Mahalanobis transformation, vector measurement innovations are decorrelated into the normalised ones to facilitate parallel quantisation. Then, taking account of Gaussian channel noises, a generalised multi-level quantisation mechanism and the minimum mean square error (MMSE) estimator are jointly designed, where optimal quantisation parameters can be solved by minimising the estimation error covariance with given quantisation level. The proposed MMSE estimator not only has a similar recursive structure as the classical Kalman filter, but also dramatically reduces the sensor-to-estimator communication requirement with only a slight deterioration of estimation performance. The combined effect of quantisation mechanism and communication uncertainty on estimation performance is also discussed. Finally, Monte Carlo simulation results illustrate the effectiveness and efficiency of the proposed quantised estimator.

Improved Performance in the Detection of ACO-OFDM Modulated Signals Using Deep Learning Modules

Free space optical communication (FSO) is widely deployed to transmit high data rates for rapid communication traffic increase. Asymmetrically clipped optical orthogonal frequency division multiplexing (ACO-OFDM) modulation is a very efficient FSO communication technique in terms of transmitted optical power. However, its performance is limited by atmospheric turbulence. When the channel includes strong turbulence or is non-deterministic, the bit error rate (BER) increases. To reach optimal performance, the ACO-OFDM decoder needs to know accurate channel state information (CSI). We propose novel detection using different deep learning (DL) algorithms. Our DL models are compared with minimum mean square error (MMSE) detection methods in different turbulent channels and improve performance especially for non-stationary and non-deterministic channels. Our models yield performance very close to that of the MMSE estimator when the channel is characterized by weak or strong turbulence and is stationary. However, when the channel is non-stationary and variable, our DL model succeeds in improving the performance of the system and decreasing the signal to noise ratio (SNR) by more than 8 dB compared to that of the MMSE estimator, and it succeeds in recovering the received data without needing to know accurate CSI. Our DL decoders also show notable speed and energy efficiency improvement.

Joint Power Control and LSFD for Wireless-Powered Cell-Free Massive MIMO

This paper considers wireless uplink information and downlink power transfer in cell-free massive multiple-input multiple-output systems. The single-antenna user equipments (UEs) utilize the energy harvested in the downlink to transmit uplink pilot and information signals to the multiple-antenna access points (APs). We consider Rician fading and maximum ratio processing based on either linear minimum mean-squared error (LMMSE) or least-squares (LS) channel estimation. We derive the average harvested energy by using a practical non-linear energy harvesting circuit model for both coherent and non-coherent transmission schemes. Furthermore, the uplink spectral efficiency (SE) is derived for all the considered methods and the max-min fairness problem is cast where the optimization variables are the AP and UE power control coefficients together with the large-scale fading decoding vectors. The objective is to maximize the minimum SE of the UEs’ under APs’ and UEs’ transmission power constraints. A novel alternating optimization algorithm with guaranteed convergence and improvement at each step is proposed to solve the highly-coupled non-convex problem.

On the MMSE estimation of norm of a Gaussian vector under additive white Gaussian noise with randomly missing input entries

This paper considers the task of estimating the l2 norm of a n-dimensional random Gaussian vector from noisy measurements taken after many of the entries of the vector are missed and only K (0 ≤ K ≤ n) entries are retained while the rest of the entries are erased and set to 0. Specifically, we evaluate the minimum mean square error (MMSE) estimator of the l2 norm of the unknown Gaussian vector performing measurements under additive white Gaussian noise (AWGN) on the vector after the data missing and derive expressions for the corresponding mean square error (MSE). We find that the corresponding MSE normalized by n tends to 0 as n → ∞ for any 1 ≤ k ≤ n. Furthermore, expressions for the MSE is derived when the variance of the AWGN noise tends to either 0 or ∞. These results generalize the results of Dytso et al. [1] where the case K=n is considered, i.e. the MMSE estimator of norm of random Gaussian vector is derived from measurements under AWGN noise without considering the data missing phenomenon.

Ergodic spectral efficiency of massive MIMO with correlated Rician channel and MRC detection based on LS and MMSE channel estimation

The authors study the spectral efficiency (SE) of a multi-cell massive multiple-input multiple-output (MIMO) system with a spatially correlated Rician channel. The correlation between least squares (LS) estimator and its error complicates SE analysis, since signal and interference components become cross-correlated, too. Minimum mean square error (MMSE) estimators do not suffer from this burden. In some previous works, a proper part of the signal is referred to interference, which makes them cross-uncorrelated, and leads to a SE lower bound. In their modified approach, the authors extract and refer the cross-correlated part of interference to the signal to attain this objective. Here, they use this approach for calculating the instantaneous SE of maximum ratio combining (MRC) detector under LS and MMSE estimation methods. They further derive closed-form approximations of their ergodic SE. This approach is also applicable to other linear channel estimators or data detectors. Numerical results show that achievable SE surpasses that of the previous approach. Moreover, they show that their approximation is close enough to Monte Carlo simulation results, especially at the high number of the base station antennas.

Distributed consensus-based estimation with unknown inputs and random link failures

This paper focuses on the distributed state estimation problem for linear time-varying systems with unknown exogenous inputs and random link failures. The unknown inputs appear both in the system equation and the measurement equation, and no information of them is available. The communication links between sensor nodes are unreliable and suffer from random link failures governed by a set of independent Bernoulli processes. By modeling the unknown inputs as processes with non-informative priors, a novel minimum mean square error (MMSE) estimator is derived. Then, a distributed consensus-based estimation algorithm is developed by repeatedly fusing local information from the neighbors possessing the successful link communications, in the sense that each sensor exchanges the local information obtained by performing the local MMSE estimation with its neighbors. Further, sufficient conditions are given to guarantee the stability of the proposed distributed estimator, in which the estimation error in each sensor is uniformly bounded in mean square. Finally, numerical examples are provided to show the effectiveness of the proposed technique.