Unsupervised Learning Minimum Risk Pattern Classification for Dependent Hypotheses and Dependent Measurements

Abstract

A recursive Bayes optimal solution is found for the problem of sequential multicategory pattern recognition when unsupervised learning is required. An unknown parameter model is developed which, for the pattern classification problem, allows for 1) both constant and time-varying unknown parameters, 2) partially unknown probability laws of the hypotheses and time-varying parameter sequences, 3) dependence of the observations on past as well as present hypotheses and parameters, and most significantly, 4) sequential dependencies in the observations arising from either (or both) dependency in the pattern or information source (context dependence) or in the observation medium (sequential measurement correlation), these dependencies being up to any finite Markov orders. For finite parameter spaces, the solution which is Bayes optimal (minimum risk) at each step is found and shown to be realizable in recursive form with fixed memory requirements. The asymptotic properties of the optimal solution are studied and conditions established for the solution (in addition to making best use of available data at each step) to converge in performance to operation with knowledge of the (unobservable) constant unknown parameters.