Pairwise sparse + low-rank models for variables of mixed type

Abstract

Factor models have been proposed for a broad range of observed variables such as binary, Gaussian, and variables of mixed types. They typically model a pairwise interaction parameter matrix with a low-rank and a diagonal component. The low-rank component can be interpreted as the effect of a few latent quantitative variables which are common in social science applications. Another line of research has investigated graphical models for the same types of observed variables, where the pairwise interactions are usually assumed to be sparse. Sparse network structures are often found in the natural sciences. Still overall, while factor and sparse models are suitable for many applications, they sometimes might not be expressive enough to fit certain data well. This has motivated the confluence of factor and graphical models, yielding models where the interaction parameters are decomposed into respectively sparse and low-rank components. Up to now, this has only been done separately for observed Gaussian and for observed binary variables, but never jointly. The present work accomplishes a unified treatment of pairwise sparse and low-rank models. It does so by simultaneously allowing observed binary and quantitative (conditional Gaussian) variables. The model parameters of the joint model can be estimated using a convex regularized likelihood optimization problem. We show that the resulting estimator has consistency properties in the high-dimensional setting.