Adaptive hidden Markov model estimation and applications

Abstract

Initially introduced in the late 1960's and early 1970's, hidden Markov models (HMMs) have become increasingly popular in the last decade. The major reason for the increasing popularity of HMMs has been the richness of the model class and the power of the signal processing tools. In this thesis we propose several algorithms for estimation of HMM parameters. Initially, we propose recursive prediction error algorithms for separately estimating the state values and the state transition probability matrix. Local convergence results and corresponding convergence rates are obtained via an ordinary differential equation (ODE) approach. Suboptimal extended least squares algorithms are also presented and convergence results are established in idealized situations. These algorithms exploit the discrete-valued nature of HMMs. Following this, globally convergent parameter estimators for HMMs are presented. These estimators have parallels to the well known Baum-Welch EM algorithm for off-line estimation of HMM parameters. Almost sure convergence results and convergence rates results are established using martingale convergence results, the Kronecker lemma and an ODE approach. This inspires the proposal of globally convergent parameter estimators for partially observed linear systems and hybrid linear systems. Almost sure convergence results are established using martingale convergence results, the Kronecker lemma and an ODE approach. Finally, as a contribution towards applications, optimal HMM filters are developed for demodulation of differentially encoded transmission systems and a decision feedback equalizer is proposed.