Abstract
Graphical models are useful tools for describing structured high-dimensional probability distributions. Development of efficient algorithms for learning graphical models with least amount of data remains an active research topic. Reconstruction of graphical models that describe the statistics of discrete variables is a particularly challenging problem, for which the maximum likelihood approach is intractable. In this work, we provide the first sample-efficient method based on the interaction screening framework that allows one to provably learn fully general discrete factor models with node-specific discrete alphabets and multi-body interactions, specified in an arbitrary basis. We identify a single condition related to model parametrization that leads to rigorous guarantees on the recovery of model structure and parameters in any error norm, and is readily verifiable for a large class of models. Importantly, our bounds make explicit distinction between parameters that are proper to the model and priors used as an input to the algorithm. Finally, we show that the interaction screening framework includes all models previously considered in the literature as special cases, and for which our analysis shows a systematic improvement in sample complexity.
Highlights
Representing and understanding the structure of direct correlations between distinct random variables with graphical models is a fundamental task that is essential to scientific and engineering endeavors
Even though it has been later shown in [13] that regularized pseudo-likelihood supplemented with a crucial post-processing step leads to a structure estimator for pairwise binary models, strong numerical and theoretical evidence provided in that work demonstrated that RISE is superior in terms of worst-case sample complexity
We propose a generalization of the estimator RISE, first introduced in [18] for pairwise binary graphical models, in order to reconstruct general discrete graphical models defined in (1)
Summary
Representing and understanding the structure of direct correlations between distinct random variables with graphical models is a fundamental task that is essential to scientific and engineering endeavors. The algorithm RISE suggested in this work is based on the minimization of a novel local convex loss function, called the Interaction Screening objective, supplemented with an l1 penalty to promote sparsity Even though it has been later shown in [13] that regularized pseudo-likelihood supplemented with a crucial post-processing step leads to a structure estimator for pairwise binary models, strong numerical and theoretical evidence provided in that work demonstrated that RISE is superior in terms of worst-case sample complexity. The so-called SPARSITRON algorithm in [12] has the flavor of a stochastic first order method with multiplicative updates It has a low computational complexity and is sample-efficient for structure recovery of two subclasses of discrete graphical models: multiwise graphical models over binary variables or pairwise models with general alphabets. We provide a fully parallelizable algorithmic formulation for the GRISE estimator and SUPRISE algorithm, and show that they have efficient run times of O(pL) for a model of size p with L-order interactions, that includes the best-known O(p2) scaling for pairwise models
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Journal of Statistical Mechanics: Theory and Experiment
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
