Chapter 19 - Dimensionality Reduction and Latent Variable Modeling

Abstract

This chapter deals with latent variable modeling and dimensionality reduction techniques. It starts with the more classical principal component analysis (PCA) method. Its various properties are analyzed and its interpretation as a low rank matrix factorization is emphasized. Then, the canonical correlation analysis (CCA) and its relatives, such as partial least-squares (PLS), are introduced. Independent component analysis (ICA) is reviewed and the cocktail party problem is presented. Dictionary learning, as a matrix factorization approach, is defined and the k-SVD algorithm is considered. The probabilistic approach to latent variable modeling is reviewed, starting with the factor analysis method, and moving on to discuss probabilistic PCA and the method of mixture of factor analyzers, as a Bayesian alternative to compressed sensing. Nonlinear dimensionality reduction techniques, including kernel PCA, Laplacian eigenmaps, local linear embedding (LLE), and isometric mapping (ISOPAM), are considered. Matrix completion and robust PCA are introduced and related applications are discussed. Finally, the chapter concludes with a case study concerning fMRI data analysis.