Chapter 19 - Dimensionality Reduction and Latent Variables Modeling

Abstract

This chapter deals with latent variables modeling and dimensionality reduction techniques. It starts with the more classical principle components analysis (PCA) method. Its various properties are analyzed and its interpretation as a low-rank matrix factorization is emphasized. Then, the canonical correlation analysis (CCA) and its relatives, such as partial least-squares (PLS) are introduced. Independent component analysis (ICA) is reviewed and the cocktail party problem is presented. Dictionary learning, as a matrix factorization approach, is defined and the k-SVD algorithm is considered. The probabilistic approach to latent variables modeling is reviewed, starting with the factor analysis method and moves on to discuss probabilistic PCA and the method of mixture of factor analyzers, as a Bayesian alternative to compressed sensing. Nonlinear dimensionality reduction techniques, including kernel PCA, Laplacian eigenmaps, local linear embedding (LLE), and isometric mapping (ISOPAM) are considered. Matrix completion and robust PCA are introduced and related applications are discussed. Finally, the chapter concludes with a case study concerning fMRI data-analysis.