Minimum Mean Square Error Estimator Research Articles

On the Equivalence of Maximum SNR and MMSE Estimation: Applications to Additive Non-Gaussian Channels and Quantized Observations

The minimum mean-squared error (MMSE) is one of the most popular criteria for Bayesian estimation. Conversely, the signal-to-noise ratio (SNR) is a typical performance criterion in communications, radar, and generally detection theory. In this paper, we first formalize an SNR criterion to design an estimator, and then we prove that there exists an equivalence between MMSE and maximum-SNR estimators, for any statistics. We also extend this equivalence to specific classes of suboptimal estimators, which are expressed by a basis expansion model (BEM). Then, by exploiting an orthogonal BEM for the estimator, we derive the MMSE estimator constrained to a given quantization resolution of the noisy observations, and we prove that this suboptimal MMSE estimator tends to the optimal MMSE estimator that uses an infinite resolution of the observation. Besides, we derive closed-form expressions for the mean-squared error (MSE) and for the SNR of the proposed suboptimal estimators, and we show that these expressions constitute tight, asymptotically exact, bounds for the optimal MMSE, and maximum SNR.

A novel super-resolution image and video reconstruction approach based on Newton-Thiele’s rational kernel in sparse principal component analysis

In this paper, we propose a new image and video sequences reconstruction approach, where the Newton-Thiele's vector valued rational interpolation is combined with the sparse principal component analysis. Through observation of the degraded model, the reconstruction scheme is performed by two steps. Firstly, the sparse principal component analysis and the linear minimum mean square-error estimation method are used to remove the noise from the degraded image. And then, the Newton-Thiele's vector valued rational interpolation is used to magnify the denoising result, by which the details and texture regions of image can be well preserved. By using this novel reconstruction model by Newton-Thiele's rational kernel in sparse principal component analysis, the final reconstructed results not only have good visual effect, but also have rich texture details. In order to show the effectiveness and robustness of the proposed method, we have done plenty of experiments on images and video sequences, and the experimental results show that the proposed method can produce better high-quality resolution results, as compared with the state-of-the-art methods.

Diffusion scheme using mean filter and wavelet coefficient magnitude for image denoising

A novel wavelet-domain diffusion scheme is proposed for image denoising. In the proposed scheme, the shrinkage function plays an important role for denoising performance. By researching the signal information extraction feature map of locally adaptive linear minimum mean square-error estimation (LALMMSE) method, the proposed shrinkage function is produced. In the design of the function, local mean filter is used to effectively extract noise-free wavelet coefficient information while original noisy wavelet coefficient magnitude is used to offset the negative effect produced by information extraction. Tests show that the proposed new method is always on par with or better than the state-of-the-art image denoising techniques in the wavelet domain. Furthermore, the proposed method is also very efficient compared to other methods.

Estimating the Variance of an Exponential Distribution in the Presence of Large True Observations

The present paper discusses some classes of shrinkage estimators for the variance of the exponential distribution in the presence of large true observations when some a priori or guessed interval containing the variance parameter is available from some past experiences. Empirical study shows the high efficiency of the developed classes of shrinkage estimators when compared with Pandey and Singh’s estimator, minimum MSE estimator and special classes of shrinkage estimators.

Higher‐order MRFs based image super resolution: why not MAP?

A trainable filter-based higher-order Markov random fields model – the so called fields of experts (FoE), has proved a highly effective image prior model for many classic image restoration problems. Generally, two options are available to incorporate the learned FoE prior in the inference procedure: (i) sampling-based minimum mean square error (MMSE) estimate, and (ii) energy minimisation-based maximum a posteriori (MAP) estimate. This study is devoted to the FoE prior based single image super resolution (SR) problem, and the author suggest to make use of the MAP estimate for inference based on two facts: (i) It is well-known that the MAP inference has a remarkable advantage of high computational efficiency, while the sampling-based MMSE estimate is very time consuming. (ii) Practical SR experiment results demonstrate that the MAP estimate works equally well compared with the MMSE estimate with exactly the same FoE prior model. Moreover, it can lead to even further improvements by incorporating the discriminatively trained FoE prior model. In summary, the author hold that for higher-order natural image prior based SR problem, it is better to employ the MAP estimate for inference.

Joint beamforming and spectral enhancement for robust automatic speech recognition in reverberant environments

This work evaluates multi-microphone beamforming techniques and single-microphone spectral enhancement strategies to alleviate the reverberation effect for robust automatic speech recognition (ASR) systems in different reverberant environments characterized by different reverberation times T60 and direct-to- reverberation ratios (DRRs). The systems under test consist of minimum variance distortionless response (MVDR) beamformers in combination with minimum mean square error (MMSE) estimators. For the later, reliable late reverberation spectral variance (LRSV) estimation employing a generalized model of the room impulse response (RIR) is crucial. Based on the generalized RIR model which separates the direct path from the remaining RIR, two different frequency resolutions in the short time Fourier transform (STFT) domain are evaluated, referred to as short- and long-term, to effectively estimate the direct signal. Regarding to the fusion between the MVDR beamformer and the MMSE estimator, the LRSV estimator can operate either on the multi-channel observed speech signals or on the single-channel beamformer output. By this, in this contribution, four different combination system architectures are evaluated and analyzed with a focus on optimal ASR performance w.r.t. word error rate (WER).

Event-based state estimation of linear dynamic systems with unknown exogenous inputs

In this work, an event-based optimal state estimation problem for linear-time varying systems with unknown inputs is investigated. By treating the unknown input as a process with a non-informative prior, the event-based minimum mean square error (MMSE) estimator is obtained in a recursive form. It is shown that for the general time-varying case, the closed-loop matrix of the optimal event-based estimator is exponentially stable and the estimation error covariance matrix is asymptotically bounded for each sample path of the event-triggering process. The results are also extended to the multiple sensor scenario, where each sensor is allowed to have its own event-triggering condition. The efficiency of the proposed results is illustrated by a numerical example and comparative simulation with the MMSE estimators obtained based on time-triggered measurements. The results are potentially applicable to event-based secure state estimation of cyber-physical systems.

Image enhancement via MMSE estimation of Gaussian scale mixture with Maxwell density in AWGN

In optical techniques, noise signal is a classical problem in medical image processing. Recently, there has been considerable interest in using the wavelet transform with Bayesian estimation as a powerful tool for recovering image from noisy data. In wavelet domain, if Bayesian estimator is used for denoising problem, the solution requires a prior knowledge about the distribution of wavelet coefficients. Indeed, wavelet coefficients might be better modeled by super Gaussian density. The super Gaussian density can be generated by Gaussian scale mixture (GSM). So, we present new minimum mean square error (MMSE) estimator for spherically-contoured GSM with Maxwell distribution in additive white Gaussian noise (AWGN). We compare our proposed method to current state-of-the-art method applied on standard test image and we quantify achieved performance improvement.

Bayesian Hypothesis Testing for Block Sparse Signal Recovery

A novel block Bayesian hypothesis testing algorithm (BBHTA) is presented for reconstructing block-sparse signals with unknown block structures. The BBHTA detects and recovers the supports and then estimates the amplitudes of block sparse signal. The support detection and recovery are performed by a Bayesian hypothesis testing. Using the detected and reconstructed supports, the nonzero amplitudes are then estimated by linear minimum mean-square error estimation. Numerical experiments demonstrate the effectiveness of BBHTA.

Linear estimation with transformed measurement for nonlinear estimation

The linear minimum mean-square error (LMMSE) estimation plays an important role in nonlinear estimation. It is the best of all estimators that are linear in the measurement. However, it may not perform well for a highly nonlinear problem. A generalized linear estimation framework, namely, linear in transform (LIT) estimation, is proposed in this work. It employs a measurement transform function (MTF) and finds the best estimator among all estimators that are linear in MTF, rather than in the measurement itself. The performance of LIT estimation with a proper MTF can be superior to LMMSE estimation due to the benefit of a more appropriate or larger candidate set of estimators determined by the MTF (compared to the set of all linear estimators in LMMSE estimation). Since systematic procedures for constructing an MTF that guarantees enhanced performance for the general case are difficult to produce, we provide several design guidelines, which are illustrated by a numerical example. Further, similar to LMMSE estimation, moments involved in LIT estimation are difficult to compute analytically in general. Fortunately, many numerical approximations for LMMSE estimation are also applicable to LIT estimation. Approximation of LIT estimation based on the Gauss-Hermite quadrature is presented. Our LIT estimation is demonstrated by applications to target tracking. Its performance is compared with LMMSE estimation by Monte Carlo simulations.

Medical image denoising using simple form of MMSE estimation in Poisson–Gaussian noise model

Poisson–Gaussian noise is the basis of image formation for a great number of imaging systems used in variety of applications, including medical and astronomical imaging. In wavelet domain, the application of Bayesian estimation method with generalized Anscombe transform in Poisson–Gaussian noise reduction algorithm has shown remarkable success over the last decade. The generalized Anscombe transform is exerted to convert the Poisson–Gaussian noise into an additive white Gaussian noise (AWGN). So, the resulting data can be denoised with any algorithm designed for the removal of AWGN. Here, we present simple form of minimum mean square error (MMSE) estimator for logistic distribution in Poisson–Gaussian noise. The experimental results show that the proposed method yields good denoising results.

An Opportunistic Sensor Scheduling Solution to Remote State Estimation Over Multiple Channels

We consider a sensor scheduling problem where the sensors have multiple choices of communication channel to send their local measurements to a remote state estimator for state estimation. Specifically, the sensors can transmit high-precision data packets over an expensive channel or low-precision data packets, which are quantized in several bits, over some cheap channels. The expensive channel, though being able to deliver more accurate data which leads to good estimation quality at the remote estimator, can only be used scarcely due to its high cost (e.g., high energy consumption). On the other hand, the cheap channel, though having a small cost, delivers less accurate data which inevitably deteriorates the remote estimation quality. In this work we propose a new framework in which the sensors switch between the two channels to achieve a better tradeoff among the communication cost, the estimation performance and the computational complexity, where the two-channel case can be easily extended to a multiple-channel case. We propose an opportunistic sensor schedule which reduces the communication cost by randomly switching among the expensive and cheap channels, and in the meantime maintains low computational complexity while introducing data quantization into the estimation problem. We present a minimum mean square error (MMSE) estimator in a closed-form under the proposed opportunistic sensor schedule. We also formulate an optimization problem to search the best opportunistic schedule with a linear quantizer. Furthermore, we show that the MMSE estimator in the limiting case becomes the standard Kalman filter.

Performance evaluation of DFT-based channel estimation in MIMO-OFDM system

Multiple input multiple output-orthogonal frequency division multiplexing (MIMO-OFDM) is an emerging multiplexing technique used in modern mobile communication systems. A main challenge to MIMO-OFDM is retrieval of the channel state information (CSI) accurately and synchronisation between the transmitter and receiver. The CSI is retrieved with the help of channel estimation algorithms such as training-based, blind and semi-blind channel estimation by various researchers. The mean square error (MSE) of existing training-based least square (LS) and minimum mean square error (MMSE) estimation are usually high. In order to minimise MSE, discrete Fourier transform (DFT)-based channel estimation is proposed. This paper deals with MIMO-OFDM system and simulation of training-based LS and MMSE channel estimation, performance evaluation of DFT-based channel estimation. The simulation results clearly indicate the drastic minimisation of MSE by the implementation of DFT channel estimation with LS and MMSE estimation.

On the Performance Enhancement When Combining STBC and Channel Estimation Techniques in Underground MIMO Channels

One way to enhance the channel data rate in enclosed environments is by incorporating multiple-input-multiple-output (MIMO) systems. Very often, this comes at the cost of data reliability. Motivated by the proven efficiency of space-time block coding (STBC) technique in conventional MIMO systems in improving the communication quality, we propose to incorporate it in wireless underground MIMO communication system and investigate data reliability at the reception in terms of bit error rate (BER). For this purpose, an intensive channel measurement campaign was performed in a mine-laboratory in northern Canada to obtain the perfect channel state. As a second step, the underground MIMO channel is estimated by sending a pilot sequence and using minimum mean square error (MMSE) principle on the received pilots. The resulting MIMO channel estimate is used to evaluate the data part in the STBC block and perform a decision using the maximum likelihood (ML) detector. Moreover, both the channel and transmitted data estimates are given analytically, and the degradation of the BER performance of the system due to MMSE estimation errors is studied using Monte Carlo simulations.

Speech enhancement by Bayesian estimation of clean speech modeled as super Gaussian given a priori knowledge of phase

In this paper, STFT based speech enhancement algorithms based on estimation of short time spectral amplitudes are proposed. These algorithms use Maximum Likelihood (ML), Maximum a posterior (MAP) and Minimum mean square error (MMSE) estimators which respectively uses Laplace, Gaussian probability density functions (pdf) as noise spectral amplitude priors and Nakagami, Gamma distributions as speech spectral amplitude priors. The method uses a joint MMSE estimate of the clean speech amplitude and clean speech phase for a given uncertainty phase information for improved single channel speech enhancement. In the most of the speech enhancement algorithms, we only concentrate on the frequency domain amplitude of speech, but not on the phase of noisy speech since it may cause undesired artifacts. In this paper, a recent phase reconstruction algorithm is used to estimate the phase of clean speech. The reconstructed phase is treated as an uncertain prior knowledge when deriving a joint MMSE estimate of the Complex speech coefficients given Uncertain Phase (CUP) information. The proposed MMSE optimal CUP estimator reduces undesired artifacts and also gives satisfactory values between the phase of noisy signal and the estimate of prior phase. We evaluate all the above estimators using speech signals uttered by 10 male speakers and 10 female speakers are taken from TIMIT database. The proposed method outperforms other benchmark algorithms in terms of segmental signal to noise ratio (SSNR), short-time objective intelligibility (STOI) and perceptual evaluation of speech quality (PESQ).

Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part I: Performance Bounds

How accurately can one estimate a random parameter subject to unknown deterministic nuisance parameters? The hybrid Cramer-Rao bound (HCRB) provides an answer to this question for a restricted class of estimators. The HCRB is the most popular performance bound on the mean-square-error (MSE) for random parameter estimation problems which involve deterministic parameters. The HCRB is useful when one is interested in both the random and the deterministic parameters and in the coupling between their estimation errors. This bound refers to locally weak-sense unbiased estimators with respect to (w.r.t.) the deterministic parameters. However, if these parameters are nuisance, it is unnecessary to restrict their estimation as unbiased. This paper is the first of a two-part study of Bayesian parameter estimation in the presence of deterministic nuisance parameters. It begins with a study on order relations between existing Cramer-Rao (CR)-type bounds of mean-unbiased Bayesian estimators. Then, a new CR-type bound is developed with no assumption of unbiasedness on the nuisance parameters. Alternatively, Lehmann’s concept of unbiasedness is employed rather than conventional mean-unbiasedness. It is imposed on a risk that measures the distance between the estimator and the minimum MSE (MMSE) estimator which assumes perfect knowledge of the nuisance parameters. In the succeeding paper, asymptotic performances of some Bayesian estimators with maximum likelihood based estimates for the nuisance parameters are investigated. The proposed risk-unbiased bound (RUB) is proved to be asymptotically achieved by the MMSE estimator with maximum likelihood estimates for the nuisance parameters, while the existing CR-type bounds are not necessarily achievable.

Fundamental Limits on RSS Based Range Estimation in Visible Light Positioning Systems

In this study, theoretical limits are obtained for the accuracy of range (distance) estimation in visible light positioning (VLP) systems. In particular, the Ziv-Zakai bound (ZZB) and the weighted Cramer-Rao bound (WCRB) are derived for range estimation based on received signal strength (RSS) measurements. Also, the maximum a posteriori probability (MAP) and the minimum mean-squared error (MMSE) estimators are obtained for RSS based range estimation, and compared against the theoretical limits.

Low-Complexity Bayesian Estimation of Cluster-Sparse Channels

This paper addresses the problem of channel impulse response estimation for cluster-sparse channels under the Bayesian estimation framework. We develop a novel low-complexity minimum mean squared error (MMSE) estimator by exploiting the sparsity of the received signal profile and the structure of the measurement matrix. It is shown that, due to the banded Toeplitz/circulant structure of the measurement matrix, a channel impulse response, such as underwater acoustic channel impulse responses, can be partitioned into a number of orthogonal or approximately orthogonal clusters. The orthogonal clusters, the sparsity of the channel impulse response, and the structure of the measurement matrix, all combined, result in a computationally superior realization of the MMSE channel estimator. The MMSE estimator calculations boil down to simpler in-cluster calculations that can be reused in different clusters. The reduction in computational complexity allows for a more accurate implementation of the MMSE estimator. The proposed approach is tested using synthetic Gaussian channels, as well as simulated underwater acoustic channels. Symbol-error-rate performance and computation time confirm the superiority of the proposed method compared to selected benchmark methods in systems with preamble-based training signals transmitted over cluster-sparse channels.

A Joint MMSE Channel and Noise Variance Estimation for OFDM/OQAM Modulation

This paper deals with the minimum mean square error (MMSE)-based multipath channel and noise variance estimation in the case of a pilot-aided orthogonal frequency division multiplexing/offset quadrature amplitude modulation (OFDM/OQAM) system. The theoretical expression of the LMMSE channel estimation is formulated and a simpler closed-form is derived. The MSE analysis shows that LMMSE inevitably reaches an error floor due to the interference inherent to OFDM/OQAM modulation. Furthermore, an algorithm for the joint MMSE estimation of both the channel and the noise variance (J-MCNE) is proposed and features a reduced complexity thanks to the low-rank approximation method. The noise level estimation represents a challenge as it is concealed in the intrinsic interferences generated by the OFDM/OQAM. Simulations reveal the capability of the proposed J-MCNE method to estimate the noise variance and show that the achieved bit error rate (BER) is close to that of the perfect estimation.

State estimation with quantised innovations and communication channels

This study considers the minimum mean-squared error (MMSE) state estimation for the discrete linear stochastic systems subject to both the quantisation and the communication unreliability. Based on the binary quantised innovation and a binary symmetric channel, the recursive MMSE estimator is established, and its performance is analysed. It is also discussed how the quantisation scheme and the communication uncertainty jointly affect the estimation quality. Consequently, the related results are extended to the case of the time-invariant discrete memoryless channel. Numerical simulations are included to illustrate the main results obtained.