Infinite max-margin factor analysis via data augmentation

Abstract

This paper addresses the Bayesian estimation of the discriminative probabilistic latent models, especially the mixture models. We develop the max-margin factor analysis (MMFA) model, which utilizes the latent variable support vector machine (LVSVM) as the classification criterion in the latent space to learn a discriminative subspace with max-margin constraint. Furthermore, to deal with multimodally distributed data, we further extend MMFA to infinite Gaussian mixture model and develop the infinite max-margin factor analysis (iMMFA) model, via the consideration of Dirichlet process mixtures (DPM). It jointly learns clustering, max-margin classifiers and the discriminative latent space in a united framework to improve the prediction performance. Moreover, both of MMFA and iMMFA are natural to handle outlier rejection problem, since the observations are described by a single or a mixture of Gaussian distributions. Additionally, thanks to the conjugate property, the parameters in the two models can be inferred efficiently via the simple Gibbs sampler. Finally, we implement our models on synthesized and real-world data, including multimodally distributed datasets and measured radar echo data, to validate the classification and rejection performance of the proposed models.