Semi-supervised Feature Extraction Using Independent Factor Analysis

Abstract

Efficient dimensionality reduction can involve generative latent variable models such as probabilistic principal component analysis (PPCA) or independent component analysis (ICA). Such models aim to extract a reduced set of variables (latent variables) from the original ones. In most cases, the learning of these models occur within an unsupervised framework where only unlabeled samples are used. In this paper, we investigate the possibility of estimating an independent factor analysis model (IFA), and thus projecting original data onto a lower dimensional space, when prior knowledge on the cluster membership of some training samples is incorporated. We propose to allow this model to learn within a semi-supervised framework in which likelihood of both labeled and unlabeled samples is maximized by a generalized expectation-maximization (GEM) algorithm. Experimental results with real data sets are provided to demonstrate the ability of our approach to find a low dimensional manifold with good explanatory power.