Geometry, moments and conditional independence trees with hidden variables

Abstract

We study the geometry of the parameter space for Bayesian directed graphical models with hidden variables that have a tree structure and where all the nodes are binary. We show that the conditional independence statements implicit in such models can be expressed in terms of polynomial relationships among the central moments. This algebraic structure will enable us to identify the inequality constraints on the space of the manifest variables that are induced by the conditional independence assumptions as well as determine the degree of unidentifiability of the parameters associated with the hidden variables. By understanding the geometry of the sample space under this class of models we shall propose and discuss simple diagnostic methods. 1. Introduction. Graphical models have proved to be a powerful tool for building Bayesian models to analyze multivariate problems where all variables are observed [e.g., Spiegelhalter, Dawid, Lauritzen and Cowell (1993)]. In particular it is possible to estimate all the conditional probabilities that parameterize such models by using a conjugate analysis. However, when all the data on certain variables in an explanatory model are missing, conjugacy usually disappears, estimates of these conditional probabilities become highly dependent on one another and they often cannot be determined from data no matter how extensive that data is [see, e.g., Settimi and Smith (1998)]. In this paper we propose a geometrical approach to analyze such difficulties. We first observe that conditional independence assumptions implicit in directed graphical models induce some constraints on the model space, that can be expressed as polynomial equations among the central moments. We then exploit such an algebraic structure to explore the geometry and the irregularities of the parameter space for Bayesian directed graphical models with hidden variables, defined over a set of binary variables, and such that the conditional independence assumptions are represented via a directed tree. Understanding the geometry and the singularities of the parameter spaces will enable us to investigate practical statistical issues, such as parameter identifiability, model dimension and diagnostic methods. In the statistical analyses of problems with missing data it has been common practice either to use various methods of approximation to calculate the posterior probabilities of the model parameters [see, e.g., Spiegelhalter and