Bayesian Factor Regression Models in the “Large p, Small n” Paradigm

Abstract

Abstract I discuss Bayesian factor regression models with many explanatory variables. These models are of particular interest and applicability in problems of prediction, but also for elucidating underlying structure in predictor variables. One key motivating application here is in studies of gene expression in functional genomics. I first discuss empirical factor (principal components) regression, and the use of general classes of shrinkage priors, with an example. These models raise foundational questions for Bayesians, and related practical issues, due to the use of design-dependent priors and the need to recover inferences on the effects of the original, high-dimensional predictors. I then discuss latent factor models for high-dimensional variables, and regression approaches in which low-dimensional latent factors are the predictor variables. These models generalize empirical factor regression, provide for more incisive evaluation of factor structure underlying high-dimensional predictors, and resolve the modelling and practical issues in empirical factor models by casting the latter as limiting special cases. Finally, I turn to questions of prior specification in these models, and introduce sparse latent factor models to induce sparsity in factor loadings matrices. Embedding such sparse latent factor models in factor regressions provides a novel approach to variable selection with very many predictors. The paper concludes with an example of sparse factor analysis of gene expression data and comments about further research.