Identification Of Hidden Markov Models Research Articles

Bayesian Identification of Hidden Markov Models and Their Use for Condition-Based Monitoring

In this paper, we propose a Bayesian estimation scheme for hidden Markov model (HMM) parameters, as well as a method for monitoring systems whose degradation processes are modeled using HMMs identified using this novel estimation approach. The Bayesian estimation naturally yields information about model parametric uncertainties via posterior distributions of HMM parameters emanating from the estimation procedure. Numerous simulations implementing the newly proposed method for estimation of HMM parameters were conducted demonstrating its capability to identify several types of HMMs, including the commonly encountered ergodic and homogeneous HMM, as well as less commonly studied nonergodic or/and nonhomogeneous HMM. In addition, a novel condition monitoring scheme based on uncertain HMMs of the degradation process is proposed and demonstrated on a large dataset obtained from a semiconductor-manufacturing facility. A small portion of the data was used to build operating mode specific HMMs of machine degradation via the newly proposed Bayesian estimation, while the remainder of the data was used for condition monitoring based on the uncertain degradation HMMs yielded by the novel Bayesian estimation method. Comparison with a traditional sensory signature-based statistical monitoring method showed that the newly proposed approach significantly outperforms the traditional method in terms of its detection capabilities and false alarm ratios.

Evaluation of Spectral Learning for the Identification of Hidden Markov Models

Hidden Markov models have successfully been applied as models of discrete time series in many fields. Often, when applied in practice, the parameters of these models have to be estimated. The currently predominating identification methods, such as maximum- likelihood estimation and especially expectation-maximization, are iterative and prone to have problems with local minima. A non-iterative method employing a spectral subspace-like approach has recently been proposed in the machine learning literature. This paper evaluates the performance of this algorithm, and compares it to the performance of the expectation- maximization algorithm, on a number of numerical examples. We find that the performance is mixed; it successfully identifies some systems with relatively few available observations, but fails completely for some systems even when a large amount of observations is available. An open question is how this discrepancy can be explained. We provide some indications that it could be related to how well-conditioned some system parameters are.

Identification of hidden Markov models for ion channel currents .III. Bandlimited, sampled data

For pt.II. see ibid., vol.46, p.1916-29 (1998). Hidden Markov models (HMMs) have been used to model single channel currents as recorded with the patch clamp technique from living cells. Continuous time patch-clamp recordings are typically passed through an antialiasing filter and sampled before analysis. In this paper, an adaptation of the Baum-Welch weighted least squares (BW-WLS) algorithm called the H-noise algorithm is presented to estimate the HMM and noise model parameters from bandlimited, sampled data. The effects of the antialiasing filter and the correlated background noise are considered in a metastate or vector HMM framework. The "correlated emission probability", which plays a central role in the algorithm, is redefined to consider the noise correlation in successive filtered, sampled data points. The performance of the H-noise algorithm is demonstrated with simulated data.

Identification of hidden Markov models for ion channel currents. I. Colored background noise

Signal processing based on hidden Markov models (HMM's) has been applied recently to the characterization of single ion channel currents as recorded with the patch clamp technique from living cells. The estimation of HMM parameters using the traditional forward-backward and Baum-Welch algorithms can be performed at signal-to-noise ratios (SNR's) that are too low for conventional analysis; however, the application of these algorithms relies on the assumption that the background noise is white. In this paper, the observed single channel current is modeled as a vector hidden Markov process. An extension of the forward-backward and Baum-Welch algorithms is described to model ion channel kinetics under conditions of colored noise like that seen in patch clamp recordings. Using simulated data, we demonstrate that the traditional algorithms result in biased estimates and that the vector HMM approach provides unbiased estimates of the parameters of the underlying hidden Markov scheme.

Identification of hidden Markov models for ion channel currents. II. State-dependent excess noise

For pt.I see ibid., vol.46, no.7, p.1901 (1998). Hidden Markov modeling (HMM) techniques have been applied in the past few years to characterize single ion channel current events at low signal-to-noise ratios (SNRs). In this paper, an adaptation of the forward-backward procedure and Baum-Welch algorithm is presented to model ion channel kinetics under conditions of correlated and state-dependent excess noise like that observed in patch-clamp recordings. An autoregressive with additive nonstationary (ARANS) noise model is introduced to model the experimentally observed noise, and an algorithm called the Baum-Welch weighted least squares (BW-WLS) procedure is presented to re-estimate the noise model parameters along with the parameters of the underlying HMM. The performance of the algorithm is demonstrated with simulated data.

On-line estimation and identification of HMMs with grouped state values

This paper presents a signal-processing scheme for the class of lumpable or weakly lumpable hidden Markov models (HMMs) which have state values clustered into groups. Attention is focused not only on state estimation for known models but also on on-line model identification. The approach taken employs a new technique whereby separate state estimators are used for each group of state values. The state estimator for each group estimates the discrete states in that group together with an associated flag state which represents all the other groups. The result is that the computational complexity is greatly reduced. Hidden Markov model parameters associated with lumpable or weakly lumpable Markov chains can be identified on-line using available techniques such as the recursive prediction error (RPE) approach taken in this paper. These techniques estimate the transition probabilities and discrete state values of the Markov chain on-line. Other parameters, such as the noise density associated with the observations, can also be identified.

Identification of Hidden Markov Models with Grouped State Values

This paper presents a recursive prediction error algorithm which addresses the problem of computational complexity for on-line identification of hidden Markov models (HMMs). The particular class of HMMs considered has state-values which are clustered into groups. This allows a reformulation of the Markov model and results in a sub-optimal reduced order identification scheme. Actually, an exact definition of clustering is not discussed, rather, a general identification technique is presented for which the computational requirements are greatly reduced when the state-values are divided in some way into groups. The applicability to certain types of cluster patterns is tested via simulation studies.

Tracking the output of a poorly known markov process

Tracking the output of an unknown Markov process with unknown generator and unknown output function is considered. It is assumed the unknown quantities have a known prior probability distribution. It is shown that the optimal control is a linear feedback in the tracking error plus the conditional expectation of a quantity involving the unknown generator and output function of the Markov process. The results also have application to Bayesian identification of hidden Markov models

On-line identification of hidden Markov models via recursive prediction error techniques

An on-line state and parameter identification scheme for hidden Markov models (HMMs) with states in a finite-discrete set is developed using recursive prediction error (RPE) techniques. The parameters of interest are the transition probabilities and discrete state values of a Markov chain. The noise density associated with the observations can also be estimated. Implementation aspects of the proposed algorithms are discussed, and simulation studies are presented to show that the algorithms converge for a wide variety of initializations. In addition, an improved version of an earlier proposed scheme (the Recursive Kullback-Leibler (RKL) algorithm) is presented with a parameterization that ensures positivity of transition probability estimates.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>