Hidden Markov Model Classification based on Empirical Frequencies of Observed Symbols

Abstract

Given a sequence of observations, classification among two known hidden Markov models (HMMs) can be accomplished with a classifier that minimizes the probability of error (i.e., the probability of misclassification) by enforcing the maximum a posteriori probability (MAP) rule. For this MAP classifier, the a priori probability of error (before any observations are made) can be obtained, as a function of the length of the sequence of observations, by summing up the probability of error over all possible observation sequences of the given length, which is a computationally expensive task. In this paper, we obtain an upper bound on the probability of error of the MAP classifier. Our results are based on a suboptimal decision rule that ignores the order with which observations occur and relies solely on the empirical frequencies with which different symbols appear. We describe necessary and sufficient conditions under which this bound on the probability of error decreases exponentially with the length of the observation sequence. Apart from the usefulness of the suboptimal rule in bounding the probability of misclassification, its numerous advantages (such as low computational complexity, reduced storage requirements, and potential applicability to distributed or decentralized decision schemes) could prove a useful alternative to the MAP rule for HMM classification in many applications.