High-dimensional graph estimation and density estimation

Abstract

Graphical models have become a common tool in many fields and a useful way of modeling probability distributions. In this thesis, we investigate approaches for graph estimation and density estimation problems in high dimensions. The estimation and model selection problems in Gaussian graphical models are equivalent to the estimation of precision matrix and identification of its zero-pattern. In Chapter 2, we propose a new estimation method by considering a convex combination of a set of individual estimators with various sparsity patterns. We analyze the risk of this aggregation estimator and show by an oracle that it is comparable to the risk of the best estimator based on a single graph. In Chapter 3, we investigate robust methods for Gaussian graphical models in the presence of possible outliers and corrupted data. We consider the neighborhood selection and graphical lasso algorithms and show that the robust counterparts obtained using the trimmed inner product or the nonparanormal give stronger performance guarantees. Gaussian graphical models maintain tractable inference, but they are limited in their ability to flexibly model the bivariate and higher order marginals. In Chapter 4, we study tree-based graphical models and develop new density estimation approaches under structural constraints---scale-free network and shared edges among multiple graphs. Our methods arise from a Bayesian formulation as the MAP estimates and solve the optimization problems via a minorize-maximization procedure with Kruskal's algorithm. The ability of tree-based graphical models to model complex independence graphs is compromised. In Chapter 5, we combine the ideas behind Gaussian graphical models and tree-based graphical models to form a new nonparametric family of graphical models, which relax the normality assumption and increase statistical efficiency by modeling the forest with kernel density estimators and modeling each blossom with the nonparanormal. Our analysis and experimental results indicate that the newly proposed methods in this thesis can be powerful alternatives to standard approaches for graph estimation and density estimation problems.