Recursive Expectation-maximization Research Articles

HMM-Based Gain Modeling for Enhancement of Speech in Noise

Accurate modeling and estimation of speech and noise gains facilitate good performance of speech enhancement methods using data-driven prior models. In this paper, we propose a hidden Markov model (HMM)-based speech enhancement method using explicit gain modeling. Through the introduction of stochastic gain variables, energy variation in both speech and noise is explicitly modeled in a unified framework. The speech gain models the energy variations of the speech phones, typically due to differences in pronunciation and/or different vocalizations of individual speakers. The noise gain helps to improve the tracking of the time-varying energy of nonstationary noise. The expectation-maximization (EM) algorithm is used to perform offline estimation of the time-invariant model parameters. The time-varying model parameters are estimated online using the recursive EM algorithm. The proposed gain modeling techniques are applied to a novel Bayesian speech estimator, and the performance of the proposed enhancement method is evaluated through objective and subjective tests. The experimental results confirm the advantage of explicit gain modeling, particularly for nonstationary noise sources

Recursive K-distribution parameter estimation

Recursive estimation of the parameter of the K-distribution is studied and tested. The probability density function (pdf) of the K-distribution is seen as a mixture pdf allowing the application of Titterington's recursive expectation-maximization (EM) technique. Under mild conditions, the technique produces estimates that are strongly consistent and asymptotically normal. For the K-distribution, the complete data information matrix required by the recursive EM has an explicit form making the algorithm easy to implement. The algorithm is tested using K-distributed data with both constant and time-varying parameter. For the constant parameter case, recursive EM estimates are compared to numerical maximization of the likelihood. For the time-varying parameter case, recursive EM estimates are compared to estimates obtained using a fast routine from the literature and implemented by sliding a rectangular window over the observations.

Tracking of Multiple Moving Sources Using Recursive EM Algorithm

We deal with recursive direction-of-arrival (DOA) estimation of multiple moving sources. Based on the recursive EM algorithm, we develop two recursive procedures to estimate the time-varying DOA parameter for narrowband signals. The first procedure requires no prior knowledge about the source movement. The second procedure assumes that the motion of moving sources is described by a linear polynomial model. The proposed recursion updates the polynomial coefficients when a new data arrives. The suggested approaches have two major advantages: simple implementation and easy extension to wideband signals. Numerical experiments show that both procedures provide excellent results in a slowly changing environment. When the DOA parameter changes fast or two source directions cross with each other, the procedure designed for a linear polynomial model has a better performance than the general procedure. Compared to the beamforming technique based on the same parameterization, our approach is computationally favorable and has a wider range of applications.

Channel estimation for multicarrier multiple input single output systems using the em algorithm

This paper investigates the problem of blindly and semi-blindly acquiring the channel gains for an underdetermined synchronous multiuser multicarrier system. The special case of a multiple-input single-output (MISO) channel is considered where the different users transmit at the same time and in the same bandwidth. In order to separate the different users blindly, techniques exploiting the finite alphabet are used. For such techniques, and for a general underdetermined MIMO system, we study conditions under which the channel and the data for each user are blindly and semi-blindly identifiable. We consider the stochastic maximum likelihood (SML) criterion in which the unknown input symbols are modeled as discrete random variables. We apply the expectation-maximization (EM) algorithm in the frequency domain to get blind and semi-blind channel estimates for each user in the MISO case. We also present a recursive EM solution that updates the channel and noise estimates at each time instant. Simulations show that users can be separated, even at low SNR. Furthermore, semi-blind estimation allows for a more robust estimation solution since a possible singularity problem is avoided.

Adaptive nonlinear filters for narrow-band interference suppression in spread-spectrum CDMA systems

This paper presents a novel nonlinear filter and parameter estimator for narrow band interference suppression in code division multiple access spread-spectrum systems. As in the article by Rusch and Poor (1994), the received sampled signal is modeled as the sum of the spread-spectrum signal (modeled as a finite state independently identically distributed (i.i.d.) process-here we generalize to a finite state Markov chain), narrow-band interference (modeled as a Gaussian autoregressive process), and observation noise (modeled as a zero-mean white Gaussian process). The proposed algorithm combines a recursive hidden Markov model (HMM) estimator, Kalman filter (KF), and the recursive expectation maximization algorithm. The nonlinear filtering techniques for narrow-band interference suppression presented in Rusch and Poor and our proposed HMM-KF algorithm have the same computational cost. Detailed simulation studies show that the HMM-KF algorithm outperforms the filtering techniques in Rusch and Poor. In particular, significant improvements in the bit error rate and signal-to-noise ratio (SNR) enhancement are obtained in low to medium SNR. Furthermore, in simulation studies we investigate the effect on the performance of the HMM-KF and the approximate conditional mean (ACM) filter in the paper by Rusch and Poor, when the observation noise variance is increased. As expected, the performance of the HMM-KF and ACM algorithms worsen with increasing observation noise and number of users. However, HMM-KF significantly outperforms ACM in medium to high observation noise.

Recursive expectation-maximization (EM) algorithms for time-varying parameters with applications to multiple target tracking

We investigate the application of expectation maximization (EM) algorithms to the classical problem of multiple target tracking (MTT) for a known number of targets. Conventional algorithms, which deal with this problem, have a computational complexity that depends exponentially on the number of targets, and usually divide the problem into a localization stage and a tracking stage. The new algorithms achieve a linear dependency and integrate these two stages. Three optimization criteria are proposed, using deterministic and stochastic dynamic models for the targets.

EM-based recursive estimation of channel parameters

Recursive (online) expectation-maximization (EM) algorithm along with stochastic approximation is employed in this paper to estimate unknown time-invariant/variant parameters. The impulse response of a linear system (channel) is modeled as an unknown deterministic vector/process and as a Gaussian vector/process with unknown stochastic characteristics. Using these models which are embedded in white or colored Gaussian noise, different types of recursive least squares (RLS), Kalman filtering and smoothing and combined RLS and Kalman-type algorithms are derived directly from the recursive EM algorithm. The estimation of unknown parameters also generates new recursive algorithms for situations, such as additive colored noise modeled by an autoregressive process. The recursive EM algorithm is shown as a powerful tool which unifies the derivations of many adaptive estimation methods.

A reduced-complexity online state sequence and parameter estimator for superimposed convolutional coded signals

This paper develops a reduced-complexity online state sequence and parameter estimator for superimposed convolutional coded signals. Joint state sequence and parameter estimation is achieved by iteratively estimating the state sequence via a variable reduced-complexity Viterbi algorithm (VRCVA) and the model parameters via a recursive expectation maximization (EM) approach. The VRCVA is developed from a fixed reduced-complexity Viterbi algorithm (FRCVA). The FRCVA is a special case of the delayed decision-feedback sequence estimation (DDFSE) algorithm. The performance of online versions of the FRCVA, VRCVA, and the standard Viterbi algorithm (VA) are compared when they are used to estimate the state sequence as part of the reduced-complexity online state sequence and parameter estimator.

Applying Kalman Filter on Solving Simultaneous Equations with Overidentifying Rank Restrictions: The Analysis of the Demand and Supply Model of Medium-size Scooter Market in Taiwan

Once the structure form of demand and supply is translated into areduced form, one can solve the reduced form with a state space modelof the Kalman filter method. This paper discusses an innovationrepresentation that links the structure form with the state space model.For the state space model, the recursive Expectation Maximization(EM) algorithm is used to estimate the parameters of a structure form.This research successfully applied the Kalman filter method to theestimation of the coefficients of simultaneous equations withoveridentifying rank restrictions. The empirical monthly data set camefrom the medium-size scooter market in Taiwan during 1987 to 1992period.

Estimation of Markov-modulated time-series via EM algorithm

We consider the estimation of various Markov-modulated time series. We obtain maximum likelihood estimates of the time-series parameters including the Markov chain transition probabilities and the time-series coefficients using the expectation maximization (EM) algorithm. In addition, the recursive EM algorithm is used to obtain on-line parameter estimates. Simulation studies show that both algorithms yield satisfactory results. >

RECURSIVE ESTIMATION IN SWITCHING AUTOREGRESSIONS WITH A MARKOV REGIME

Abstract. A hidden Markov regime is a Markov process that governs the time or space dependent distributions of an observed stochastic process. We propose a recursive algorithm for parameter estimation in a switching autoregressive process governed by a hidden Markov chain. A common approach to the recursive estimation problem is to base the estimation on suboptimal modifications of Kalman filtering techniques. The main idea in this paper is to use the maximum likelihood method and from this develop a recursive EM algorithm.

A Recursive EM Algorithm for Identification Subject to Missing Data

In this paper we derive a recursive EM algorithm for identification of ARX-models in the case of missing data. Convergence analysis using the ODE approach is indicated. The algorithm is tested via Monte-Carlo simulation and compared to other approaches.

Multiple Wideband Signal Detection and Tracking from Towed Array Data

In this paper towed array data are investigated. The estimation of both the number of signals and the model parameters are considered. Recursive estimation of the parameters is performed by stochastic approximation and a recursive EM algorithm. The proposed algorithms are applied to towed array data measured in the Baltic Sea.

Adaptive estimation of hidden nearly completely decomposable Markov chains with applications in blind equalization

AbstractThis paper proposes maximum likelihood (ML) estimation schemes for nearly completely decomposable Markov chains (NCDMCs) in white Gaussian Noise. Aggregation techniques based on stochastic complementation are applied to significantly reduce the dimension of the resulting hidden Markov model (HMM) and hence substantially reduce the computational requirements of the estimation algorithms. Stochastic complementation results in exact aggregation in that no approximations are involved in the steady state probability distribution of the Markov chain.We then present an off‐line estimation algorithm for the parameters and states of the HMM based on the estimation of the aggregated HMM. This off‐line algorithm is an ML estimation scheme and is based on the expectation maximization (EM) algorithm. It has a significantly reduced computational complexity compared with the standard (full‐order) EM‐based HMM estimation scheme.Finally we present an application of our techniques. We show that hidden NCDMCs can be used to formulate the blind equalization problem for noisy FIR channels with Markov inputs, e.g. phase‐shiftkeyed (PSK) signals. We then propose recursive EM and gradient estimation techniques for the aggregated HMM resulting in on‐line estimates of the channel coefficients and signal estimate.For an Na‐state Markov chain our aggregate‐based estimation scheme has a computational complexity O(N−2a), whereas standard algorithms have a complexity O(Na−L + 1) at each time instant, where L is the length of the FIR channel.