Instability, Sensitivity, and Degeneracy of Discrete Exponential Families

Abstract

A number of discrete exponential family models for dependent data, first and foremost relational data, have turned out to be near-degenerate and problematic in terms of Markov chain Monte Carlo (MCMC) simulation and statistical inference. I introduce the notion of instability with an eye to characterize, detect, and penalize discrete exponential family models that are near-degenerate and problematic in terms of MCMC simulation and statistical inference. I show that unstable discrete exponential family models are characterized by excessive sensitivity and near-degeneracy. In special cases, the subset of the natural parameter space corresponding to nondegenerate distributions and mean-value parameters far from the boundary of the mean-value parameter space turns out to be a lower-dimensional subspace of the natural parameter space. These characteristics of unstable discrete exponential family models tend to obstruct MCMC simulation and statistical inference. In applications to relational data, I show that discrete exponential family models with Markov dependence tend to be unstable, and that the parameter space of some curved exponential families contains unstable subsets.