MMSE Estimator Research Articles

Uncertainty decoding for distributed speech recognition over error-prone networks

In this paper, we propose an enhanced error concealment strategy at the server side of a distributed speech recognition (DSR) system, which is fully compatible with the existing DSR standard. It is based on a Bayesian approach, where the a posteriori probability density of the error-free feature vector is computed, given all received feature vectors which are possibly corrupted by transmission errors. Rather than computing a point estimate, such as the MMSE estimate, and plugging it into the Bayesian decision rule, we employ uncertainty decoding, which results in an integration over the uncertainty in the feature domain. In a typical scenario the communication between the thin client, often a mobile device, and the recognition server spreads across heterogeneous networks. Both bit errors on circuit-switched links and lost data packets on IP connections are mitigated by our approach in a unified manner. The experiments reveal improved robustness both for small- and large-vocabulary recognition tasks.

Integrity-directed sequential state estimation: Assessing high reliability requirements via safe confidence intervals

This study deals with the problem of dynamic state estimation of continuous-time systems from discrete-time measurements in the context of high-integrity applications. The objective of integrity-directed estimation is to provide confidence intervals for the state with an extremely low risk of error. We suppose that the process noise can be modelled by Gaussian sequences, and that the measurement noise is Gaussian in the normal operating mode of the sensors. The evolution of the posterior probability distribution of the system’s state is deduced from recursive linear MMSE estimation. The estimation scheme presented here is equivalent to the Kalman filter, with the difference that the data is not processed directly, but collected in sets in preparation for an ulterior, slightly delayed, grouped processing. This strategy is particularly suitable for fault detection, because the estimator naturally takes into account the cross-correlations of close-in-time measurements and the decisions can be based on more data. Next, we introduce dynamic tools for detecting faults and sensor failures. A full Bayesian modelling of the sensors leads to the derivation of a dynamic multiple-model estimator performing the linear MMSE state estimation under various fault hypotheses. This estimator provides estimates of the posterior density function of the state, on which safe confidence intervals certifying very high-integrity levels can be fixed. In practice, the exponential growth of the complexity of the multiple-model estimator requires the simplification of the posterior mixture distributions. A new method for limiting the complexity of the posterior distributions in an integrity-oriented context is presented. Unlike the known mixture simplification strategies (GPB, IMM), the present method has the property to quantify and minimize the loss in integrity of the process of simplification of the distributions. Finally, the estimator is tested on a typical rail navigation problem.

Filtering of Interferometric SAR Phase Images as a Fuzzy Matching-Pursuit Blind Estimation

We present an original application of fuzzy logic to restoration of phase images from interferometric synthetic aperture radar (InSAR), which are affected by zero-mean uncorrelated noise, whose variance depends on the underlying coherence, thereby yielding a nonstationary random noise process. Spatial filtering of the phase noise is recommended, either before phase unwrapping is accomplished, or simultaneously with it. In fact, phase unwrapping basically relies on a smoothness constraint of the phase field, which is severely hampered by the noise. Space-varying linear MMSE estimation is stated as a problem of matching pursuit, in which the estimator is obtained as an expansion in series of a finite number of prototype estimators, fitting the spatial features of the different statistical classes encountered, for example, fringes and steep slope areas. Such estimators are calculated in a fuzzy fashion through an automatic training procedure. The space-varying coefficients of the expansion are stated as degrees of fuzzy membership of a pixel to each of the estimators. Neither a priori knowledge on the noise variance is required nor particular signal and noise models are assumed. Filtering performances on simulated phase images show a steady SNR improvement over conventional box filtering. Applications of the proposed filter to interferometric phase images demonstrate a superior ability of restoring fringes yet preserving their discontinuities, together with an effective noise smoothing performance, irrespective of locally varying coherence characteristics.

A Network of Kalman Filters for MAI and ISI Compensation in a Non-Gaussian Environment

This paper develops a new multiuser detector based on a network of kalman filters (NKF) dealing with multiple access-interference (MAI), intersymbol Interference (ISI), and an impulsive observation noise. The two proposed schemes are based on the modeling of the DS-CDMA system by a discrete-time linear system that has non-Gaussian state and measurement noises. By approximating the non-Gaussian densities of the noises by a weighted sum of Gaussian terms and under the common MMSE estimation criterion, we first derive an NKF detector. This version is further optimized by introducing a feedback exploiting the ISI interference structure. The resulting scheme is an NKF detector based on a likelihood ratio test (LRT). Monte-Carlo simulations have shown that the NKF and the NKF based on LRT detectors significantly improve the efficiency and the performance of the classical Kalman algorithm.

Estimating Fisher information in normal population with prior information

This paper is contemplated to propose a class of shrunken estimators which is further used in constructing a class of preliminary test estimators for amount of information in complete samples from normal population when some ‘apriori’ or guessed value of amount of information is available and analyses their characteristics. The proposed classes of shrunken estimators and preliminary test estimators are compared with the usual unbiased estimator and MMSE estimator. Eventually, empirical study is carried out to demonstrate the performance of some shrunken estimators and preliminary test estimators of the proposed classes over the MMSE estimator. The suggested class of estimators is found to give gratifying results. Subsequently, the usage of proposed classes of estimators in estimating the precision of sample mean has been exclusively discussed followed by an example.

Suppression of Additive Noise Using a Power Spectral Density MMSE Estimator

In this letter, we propose a novel speech enhancement approach, called power spectral density minimum mean-square error (PSD-MMSE) estimation-based speech enhancement, which is implemented in the power spectral domain where stationary stochastic noise can be modeled as the exponential distribution. Speech magnitude-squared spectra are modeled as the mixed exponential distribution. And an MMSE estimator is constructed based on the parametric distributions. Besides, a fast algorithm is presented to implement the approach in real time. Experimental results of Itakura-Saito distortion measures show that the proposed approach is superior to alternative speech enhancement algorithms.

Improving parallel EKF-based nonlinear channel equalization using unscented transformation

The paper presents a new review of parallel Kalman filtering for nonlinear channel equalization. A Network of Extended Kalman Filters (nekf) has already been suggested for this purpose. This equalizer gives recursively a minimum mean squared error (mmse) estimation of a sequence of transmitted symbols according to a state formulation of a digital communication scheme. It is essentially based on two mechanisms: the approximation of the non Gaussiana posteriori probability density function (pdf) of the symbol sequence by a Weighted Gaussian Sum (wgs); and the local linearization of the nonlinear channel function for each branch of the network. Since the linearization, bearing on scattered symbol states, is one of the major limitations of thenekf, a new Kalman filtering approach, the Unscented Kalman Filter (ukf) suggested by Julier and Uhlman is considered in this paper for an interesting adaptation to the equalization context. Theukf algorithm is based on the equations of a Kalman filter, as the optimal linear minimum variance estimator, and on determining conditional expectations based on a kind of deterministic Monte-Carlo simulations. The new equalizer referred to as the Network ofukf (nukf), thus combines density approximation by awgs and the Unscented Transformation (ut) principle to circumvent the linearization brought within eachekf and is shown to perform better than thenekf based equalizer for severe nonlinear channels. Also, an adaptive version of thenukf is developed using the k-means clustering algorithm for noise-free channel output identification, since thenukf-based algorithm does not require the knowledge of the channel nonlinearity model.

Multichannel Direction-Independent Speech Enhancement Using Spectral Amplitude Estimation

This paper introduces two short-time spectral amplitude estimators for speech enhancement with multiple microphones. Based on joint Gaussian models of speech and noise Fourier coefficients, the clean speech amplitudes are estimated with respect to the MMSE or the MAP criterion. The estimators outperform single microphone minimum mean square amplitude estimators when the speech components are highly correlated and the noise components are sufficiently uncorrelated. Whereas the first MMSE estimator also requires knowledge of the direction of arrival, the second MAP estimator performs a direction-independent noise reduction. The estimators are generalizations of the well-known single channel MMSE estimator derived by Ephraim and Malah (1984) and the MAP estimator derived by Wolfe and Godsill (2001), respectively.

Pilot‐aided acquisition algorithms for asynchronous

AbstractThe problem of estimating propagation delays of orthogonally modulated signals in asynchronous DS‐CDMA systems over time‐varing Rayleigh‐fading channels is treated in this paper. The maximum likelihood (ML) estimator and its unaffordable complexity for implementation are discussed. Some suboptimal solutions, e.g., whitened sliding correlator, MMSE estimator, subspace‐based estimator, approximate ML estimator, are proposed to combat the multiple access interference in the fading channels. The performance of these estimators are evaluated with the computer simulations and shown to have better acquisition performance than the standard sliding correlator. They also achieve reduced computational complexity compared with the ML estimator, while maintaining an acceptable performance degradation. Copyright © 2003 AEI

Pilot-aided acquisition algorithms for asynchronous DS-CDMA systems

The problem of estimating propagation delays of orthogonally modulated signals in asynchronous DS-CDMA systems over time-varing Rayleigh-fading channels is treated in this paper. The maximum likelihood (ML) estimator and its unaffordable complexity for implementation are discussed. Some suboptimal solutions, e.g., whitened sliding correlator, MMSE estimator, subspace-based estimator, approximate ML estimator, are proposed to combat the multiple access interference in the fading channels. The performance of these estimators are evaluated with the computer simulations and shown to have better acquisition performance than the standard sliding correlator. They also achieve reduced computational complexity compared with the ML estimator, while maintaining an acceptable performance degradation.

A new family of soft-output adaptive receivers exploiting nonlinear mmse estimates for TDMA-based wireless links

Decision-directed adaptive receivers suffer performance degradation on time varying and intersymbol interference-impaired links because of two major problems: the use of predicted channel estimates due to the unavoidable decision delay of any detector, and the unreliability of hard decisions used for channel estimation and tracking. It is shown here that combining a recursive nonlinear symbol estimator with a channel estimator with a low prediction order may alleviate this performance degradation. In particular, it is here proposed to employ the nonlinear minimum mean-square error (NL-MMSE) filtered and fixed-lag smoothed estimates of the transmitted symbols in place of the usual hard decisions for channel estimation and tracking. It is also shown that these NL-MMSE estimates can be recursively computed on the basis of a linear transformation of the vector of the a posteriori probabilities (APPs) of the states of the channel. This approach allows the prediction order of the channel estimates to be limited and, at the same time, limits the performance degradation due to erroneous hard decisions. Another result presented here is that the use of NL-MMSE estimates in place of hard decisions is not based on mere intuition only, but is a straightforward consequence of the statement of the problem of MMSE channel estimation when the overly optimistic assumption of correct decisions is dropped. On the basis of this novel approach, a new family of soft-output adaptive receivers is presented for time-division multiple-access-based radio communications. The proposed family of adaptive receivers is based on an APP-computer and exploits the APPs for both channel estimation and detection. The versatility of the APPs ensures that the architecture of the proposed receiver is flexible, so that several estimators and detectors can be embedded in it.

Alignment-based codeword-dependent cepstral normalization

This paper proposes the alignment-based codeword dependent cepstral normalization algorithm (ACDC/sub N/) which aims to alleviate the acoustical mismatch that occurs when the speech recognizer faces environmental conditions not observed in the training data. ACDC/sub N/ is based on the linear channel model of the environment originally proposed by Acero (1990) and on the CDCN solution to this model. ACDC/sub N/ substitutes the codebook (Gaussian mixture model) employed by CDCN for the state distributions employed by the recognizer's HMMs under the assumption that these HMM distributions will model the associated speech segments better than the general GMM distribution. The feature-frame to HMM-state association is obtained through an alignment of a first decoding-pass hypothesis. From this alignment, ACDC/sub N/ obtains an estimate of the environmental parameters (noise and channel vectors) which are then employed to obtain an MMSE estimate of the clean speech vectors, in a way similar to Aero's method. ACDC/sub N/ produces an overall reduction of the error rate of over 30 % in the noise range of 0 to 20 dB on experiments conducted on the Aurora-2 noisy digits database.

Pilot-based estimation of time-varying multipath channels for coherent CDMA receivers

Reliable coherent wireless communication requires accurate estimation of the time-varying multipath channel. This paper addresses two issues in the context of direct-sequence code-division multiple access (CDMA) systems: (i) linear minimum-mean-squared-error (MMSE) channel estimation based on a pilot transmission and (ii) impact of channel estimation errors on coherent receiver performance. A simple characterization of the MMSE estimator in terms of a bank of filters is derived. A key channel characteristic controlling system performance is the normalized coherence time, which is approximately the number of symbols over which the channel remains strongly correlated. It is shown that the estimator performance is characterized by an effective signal-to-noise ratio (SNR)-the product of the pilot SNR and the normalized coherence time. A simple uniform averaging estimator is also proposed that is easy to implement and delivers near-optimal performance if properly designed. The receivers analyzed in this paper are based on a time-frequency RAKE structure that exploits joint multipath-Doppler diversity. It is shown that the overall receiver performance is controlled by two competing effects: shorter coherence times lead to degraded channel estimation but improved inherent receiver performance due to Doppler diversity, with opposite effects for longer coherence times. Our results demonstrate that exploiting Doppler diversity can significantly mitigate the error probability floors that plague conventional CDMA receivers under fast fading due to errors in channel estimation.

Modified stochastic Luenberger observers

A modified stochastic Luenberger observer (MSLO) structure is proposed to recover the optimal performance of the coventional SLO for obtaining full-state estimates in linear discrete-time stochastic systems . The optimal MSLO (OMSLO) which gives the MMSE estimates is derived by using the general two-stage Kalman filter . A reduced-order form of the OMSLO is also proposed for systems having singular measurement noises. The connection between the OMSLO and the optimal minimal-order observer of Leondes and Novak is also shown.

Modified stochastic Luenberger observers

A modified stochastic Luenberger observer (MSLO) structure is proposed to recover the optimal performance of the coventional SLO for obtaining full-state estimates in linear discrete-time stochastic systems. The optimal MSLO (OMSLO) which gives the MMSE estimates is derived by using the general two-stage Kalman filter. A reduced-order form of the OMSLO is also proposed for systems having singular measurement noises. The connection between the OMSLO and the optimal minimal-order observer of Leondes and Novak is also shown.

A Bayesian approach to geometric subspace estimation

This paper presents a geometric approach to estimating subspaces as elements of the complex Grassmann-manifold, with each subspace represented by its unique, complex projection matrix. Variation between the subspaces is modeled by rotating their projection matrices via the action of unitary matrices [elements of the unitary group U(n)]. Subspace estimation or tracking then corresponds to inferences on U(n). Taking a Bayesian approach, a posterior density is derived on U(n), and certain expectations under this posterior are empirically generated. For the choice of the Hilbert-Schmidt norm on U(n), to define estimation errors, an optimal MMSE estimator is derived. It is shown that this estimator achieves a lower bound on the expected squared errors associated with all possible estimators. The estimator and the bound are computed using (Metropolis-adjusted) Langevin's-diffusion algorithm for sampling from the posterior. For use in subspace tracking, a prior model on subspace rotation, that utilizes Newtonian dynamics, is suggested.

Optimal modal wave-front compensation for anisoplanatism in adaptive optics

We examine how modal aberration measurements degraded by turbulence-induced anisoplanatism may be used to optimally conjugate atmospheric phase aberrations. By examining the form of the aperture-averaged mean square residual phase error, we show that atmospheric compensation is suboptimal when the measured coefficients from off-axis or finite-altitude guide stars are applied directly. The optimal compensation is obtained only when conjugate phase coefficients are estimated, given the guide-star measurements and knowledge of the spatial correlation of the on-axis and measured phase coefficients, by use of a minimum-mean-square-error (MMSE) estimator. The form of this estimator is outlined, thus motivating the need to quantify the spatial cross correlation of the Zernike coefficients of the phase aberrations. With a knowledge of the modal cross correlation, we show that wave-front compensation performance can be enhanced by use of the MMSE estimator over use of the beacon measurements directly for all orders of correction. For high-order off-axis natural-guide-star correction, equivalent imaging performance is obtained at a beacon offset 10% larger than when beacon measurements are used directly. For high-order laser-guide-star correction, equivalent imaging performance is obtained at laser-guide-star altitudes 20% lower when the MMSE estimator is employed.

MMSE estimation of nonlinear parameters of multiple linear/quadratic chirps

An iterative algorithm similar to our MMSE estimation procedure proposed in another paper by the authors is applied to parameter estimation of multiple superimposed chirp signals in white Gaussian noise. The necessary parameter initializations of the procedure are accomplished by peak detections in the Choi-Williams (1981) time-frequency distribution of the data followed by application of the least-squares principle. A comparison between our scheme and the alternating projection (AP) method is demonstrated by computer simulation.

Lossy compression of noisy images

Noise degrades the performance of any image compression algorithm. This paper studies the effect of noise on lossy image compression. The effect of Gaussian, Poisson, and film-grain noise on compression is studied. To reduce the effect of the noise on compression, the distortion is measured with respect to the original image not to the input of the coder. Results of noisy source coding are then used to design the optimal coder. In the minimum-mean-square-error (MMSE) sense, this is equivalent to an MMSE estimator followed by an MMSE coder. The coders for the Poisson noise and the film-grain noise cases are derived and their performance is studied. The effect of this preprocessing step is studied using standard coders, e.g., JPEG, also. As is demonstrated, higher quality is achieved at lower bit rates.

Channel estimation for OFDM systems based on comb-type pilot arrangement in frequency selective fading channels

The channel estimation methods for OFDM systems based on a comb-type pilot sub-carrier arrangement are investigated. The channel estimation algorithm based on comb-type pilots is divided into pilot signal estimation and channel interpolation. The pilot signal estimation based on LS or MMSE criteria, together with channel interpolation based on piecewise-linear interpolation or piecewise second-order polynomial interpolation is studied. Owing to the MMSE estimate of the pilot signals, the inter-carrier interference and additive white Gaussian noise are reduced considerably. The computational complexity of pilot signal estimation based on MMSE criterion can be reduced by using a simplified LMMSE estimator with low-rank approximation using singular value decomposition. Phase compensators before and after interpolation are also presented to combat the phase changes of subchannel symbols arising from the frame synchronization errors. Compared to the transform-domain processing based channel estimation algorithm the MMSE estimate of pilot signals together with phase compensated linear interpolation algorithm provides a better performance and requires less computations.