Variational Bayes method for Mixture of Principal Component Analyzers

Abstract

AbstractWhen dealing with high‐dimensional data, it is proper to construct an appropriate feature space consisting of relevant features extracted from given data. Mixture of Principal Component Analyzers (MPCA) is a feature extracting tool in which clustering in the data space and principal component analysis in each cluster are simultaneously done. A probabilistic model of MPCA and its maximum likelihood (ML) inference were presented by Tipping and Bishop (1999). Although the probabilistic formulation can be satisfactorily advantageous within Bayesian inference instead of ML inference, Bayesian inference often requires difficult integrations. Such difficulty comes to be overcome by the variational Bayes (VB) approximation that has recently been developed. In this paper, we present a VB inference algorithm for MPCA, and its application to a recognition problem of handwritten digit images. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(11): 55–66, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10394