Independence Statements Research Articles

Foundations of compositional models: structural properties

The paper is a follow-up of [R.J.: Foundations of compositional model theory. IJGS, 40(2011): 623–678], where basic properties of compositional models, as one of the approaches to multidimensional probability distributions representation and processing, were introduced. In fact, it is an algebraic alternative to graphical models, which does not use graphs to represent conditional independence statements. Here, these statements are encoded in a sequence of distributions to which an operator of composition – the key element of this theory – is applied in order to assemble a multidimensional model from its low-dimensional parts. In this paper, we show a way to read conditional independence relations, and to solve related topics, above all the so-called equivalence problem, i.e. the problem of recognizing whether two different structures induce the same system of conditional independence relations.

Positive margins and primary decomposition

We study random walks on contingency tables with fixed marginals, corresponding to a (log-linear) hierarchical model. If the set of allowed moves is not a Markov basis, then there exist tables with the same marginals that are not connected. We study linear conditions on the values of the marginals that ensure that all tables in a given fiber are connected. We show that many graphical models have the positive margins property, which says that all fibers with strictly positive marginals are connected by the quadratic moves that correspond to conditional independence statements. The property persists under natural operations such as gluing along cliques, but we also construct examples of graphical models not enjoying this property. We also provide a negative answer to a question of Engstr\"om, Kahle, and Sullivant by demonstrating that the global Markov ideal of the complete bipartite graph K_(3,3) is not radical. Our analysis of the positive margins property depends on computing the primary decomposition of the associated conditional independence ideal. The main technical results of the paper are primary decompositions of the conditional independence ideals of graphical models of the $N$-cycle and the complete bipartite graph $K_(2,N-2)$, with various restrictions on the size of the nodes.

On the completeness of the semigraphoid axioms for deriving arbitrary from saturated conditional independence statements

Conditional independence (CI) statements occur in several areas of computer science and artificial intelligence, e.g., as embedded multivalued dependencies in database theory, disjunctive association rules in data mining, and probabilistic CI statements in probability theory. Although, syntactically, such constraints can always be represented in the form I(A,B|C), with A, B, and C subsets of some universe S, their semantics is very dependent on their interpretation, and, therefore, inference rules valid under one interpretation need not be valid under another. However, all aforementioned interpretations obey the so-called semigraphoid axioms. In this paper, we consider the restricted case of deriving arbitrary CI statements from so-called saturated ones, i.e., which involve all elements of S. Our main result is a necessary and sufficient condition under which the semigraphoid axioms are also complete for such derivations. Finally, we apply these results to the examples mentioned above to show that, for these semantics, the semigraphoid axioms are both sound and complete for the derivation of arbitrary CI statements from saturated ones.

A class of smooth models satisfying marginal and context specific conditional independencies

We study a class of conditional independence models for discrete data with the property that one or more log-linear interactions are defined within two different marginal distributions and then constrained to 0; all the conditional independence models which are known to be non-smooth belong to this class. We introduce a new marginal log-linear parameterization and show that smoothness may be restored by restricting one or more independence statements to hold conditionally to a restricted subset of the configurations of the conditioning variables. Our results are based on a specific reconstruction algorithm from log-linear parameters to probabilities and fixed point theory. Several examples are examined and a general rule for determining the implied conditional independence restrictions is outlined.

Practical Guidelines for Learning Bayesian Networks from Small Data Sets

Model learning is the process of extracting, analysing and synthesising information from data sets. Graphical models are a suitable framework for probabilistic modelling. A Bayesian Network (BN) is a probabilistic graphical model, which represents joint distributions in an intuitive and efficient way. It encodes the probability density (or mass) function of a set of variables by specifying a number of conditional independence statements in the form of a directed acyclic graph. Specifying the structure of the model is one of the most important design choices in graphical modelling. Notwithstanding their potential, there are several limitations to learning BNs from small data sets. In this paper, we introduce a set of practical guidelines a modeller can use to deal with these limitations. The main goal of the guidelines is to increase awareness of the underlying assumptions and the tacit implications of several learning techniques. Unsurprisingly, one of the authors’ findings is that learning BNs from small data sets is a complex and challenging task, yet potentially very rewarding. The paper also draws attention to the amount of subjective input needed from the modeller and the necessity to tailor solutions on the particularity of the application.

Minimal primes of ideals arising from conditional independence statements

We study ideals whose primary decomposition specifies the relevant structural zeros of certain conditional independence models. The ideals we study generalize the class of ideals considered by Fink (2011) [5] in a way distinct from the generalizations of Herzog, Hibi, Hreinsdottir, Kahle, and Rauh (2010) [10] and Ay and Rauh (2011) [1]. We introduce switchable sets to give a combinatorial description of the minimal prime ideals, and for some classes we describe the minimal components. We discuss possible interpretations of the ideals we study, including as 2×2 minors of generic hypermatrices. We also introduce a definition of diagonal monomial orders on generic hypermatrices to compute some Gröbner bases.

Sound approximate reasoning about saturated conditional probabilistic independence under controlled uncertainty

Knowledge about complex events is usually incomplete in practice. We distinguish between random variables that can be assigned a designated marker to model missing data values, and certain random variables to which the designated marker cannot be assigned. The ability to specify an arbitrary set of certain random variables provides an effective mechanism to control the uncertainty in form of missing data values. A finite axiomatization for the implication problem of saturated conditional independence statements is established under controlled uncertainty, relative to discrete probability measures. The completeness proof utilizes special probability models where two assignments have probability one half. The special probability models enable us to establish an equivalence between the implication problem and that of a propositional fragment in Cadoli and Schaerfʼs S-3 logic. Here, the propositional variables in S correspond to the random variables specified to be certain. The duality leads to an almost linear time algorithm to decide implication. It is shown that this duality cannot be extended to cover general conditional independence statements. All results subsume classical reasoning about saturated conditional independence statements as the idealized special case where every random variable is certain. Under controlled uncertainty, certain random variables allow us to soundly approximate classical reasoning about saturated conditional independence statements.

Probability distributions with summary graph structure

A set of independence statements may define the independence structure of interest in a family of joint probability distributions. This structure is often captured by a graph that consists of nodes representing the random variables and of edges that couple node pairs. One important class contains regression graphs. Regression graphs are a type of so-called chain graph and describe stepwise processes, in which at each step single or joint responses are generated given the relevant explanatory variables in their past. For joint densities that result after possible marginalising or conditioning, we introduce summary graphs. These graphs reflect the independence structure implied by the generating process for the reduced set of variables and they preserve the implied independences after additional marginalising and conditioning. They can identify generating dependences that remain unchanged and alert to possibly severe distortions due to direct and indirect confounding. Operators for matrix representations of graphs are used to derive these properties of summary graphs and to translate them into special types of paths in graphs.

Properties of semi-elementary imsets as sums of elemen- tary imsets

We study properties of semi-elementary imsets and elementary imsets introduced byStuden´y [10]. The rules of the semi-graphoid axiom (decomposition, weak union and contraction)for conditional independence statements can be translated into a simple identity among threesemi-elementary imsets. By recursively applying the identity, any semi-elementary imset can bewritten as a sum of elementary imsets, which we call a representation of the semi-elementary imset.A semi-elementary imset has many representations. We study properties of the set of possiblerepresentations of a semi-elementary imset and prove that all representations are connected byrelations among four elementary imsets.

Characteristic properties of equivalent structures in compositional models

Compositional model theory serves as an alternative approach to multidimensional probability distribution representation and processing. Every compositional model over a finite non-empty set of variables N is uniquely defined by its generating sequence – an ordered set of low-dimensional probability distributions. A generating sequence structure induces a system of conditional independence statements over N valid for every multidimensional distribution represented by a compositional model with this structure. The equivalence problem is how to characterise whether all independence statements induced by structure P are induced by a second structure P ′ and vice versa. This problem can be solved in several ways. A partial solution of the so-called direct characterisation of an equivalence problem is represented here. We deduce and describe three properties of equivalent structures necessary for equivalence of the respective structures. We call them characteristic properties of classes of equivalent structures.

Gaussian Covariance Faithful Markov Trees

Graphical models are useful for characterizing conditional and marginal independence structures in high‐dimensional distributions. An important class of graphical models is covariance graph models, where the nodes of a graph represent different components of a random vector, and the absence of an edge between any pair of variables implies marginal independence. Covariance graph models also represent more complex conditional independence relationships between subsets of variables. When the covariance graph captures or reflects all the conditional independence statements present in the probability distribution, the latter is said to be faithful to its covariance graph—though in general this is not guaranteed. Faithfulness however is crucial, for instance, in model selection procedures that proceed by testing conditional independences. Hence, an analysis of the faithfulness assumption is important in understanding the ability of the graph, a discrete object, to fully capture the salient features of the probability distribution it aims to describe. In this paper, we demonstrate that multivariate Gaussian distributions that have trees as covariance graphs are necessarily faithful.

Exploiting independencies to compute semigraphoid and graphoid structures

We deal with conditional independencies, which have a fundamental role in probability and multivariate statistics. The structure of probabilistic independencies is described by semigraphoids or, for strictly positive probabilities, by graphoids. In this paper, given a set of independencies compatible with a probability, the attention is focused toward the problem of computing efficiently the closure with respect to the semigraphoid and graphoid structures. We introduce a suitable notion of projection in order to provide a new method which properly uses conditional independence statements.

Marginal parameterizations of discrete models defined by a set of conditional independencies

It is well-known that a conditional independence statement for discrete variables is equivalent to constraining to zero a suitable set of log–linear interactions. In this paper we show that this is also equivalent to zero constraints on suitable sets of marginal log–linear interactions, that can be formulated within a class of smooth marginal log–linear models. This result allows much more flexibility than known until now in combining several conditional independencies into a smooth marginal model. This result is the basis for a procedure that can search for such a marginal parameterization, so that, if one exists, the model is smooth.

Causal analysis with Chain Event Graphs

As the Chain Event Graph (CEG) has a topology which represents sets of conditional independence statements, it becomes especially useful when problems lie naturally in a discrete asymmetric non-product space domain, or when much context-specific information is present. In this paper we show that it can also be a powerful representational tool for a wide variety of causal hypotheses in such domains. Furthermore, we demonstrate that, as with Causal Bayesian Networks (CBNs), the identifiability of the effects of causal manipulations when observations of the system are incomplete can be verified simply by reference to the topology of the CEG. We close the paper with a proof of a Back Door Theorem for CEGs, analogous to Pearl's Back Door Theorem for CBNs.

Binomial edge ideals and conditional independence statements

We introduce binomial edge ideals attached to a simple graph G and study their algebraic properties. We characterize those graphs for which the quadratic generators form a Gröbner basis in a lexicographic order induced by a vertex labeling. Such graphs are chordal and claw-free. We give a reduced squarefree Gröbner basis for general G. It follows that all binomial edge ideals are radical ideals. Their minimal primes can be characterized by particular subsets of the vertices of G. We provide sufficient conditions for Cohen–Macaulayness for closed and nonclosed graphs. Binomial edge ideals arise naturally in the study of conditional independence ideals. Our results apply for the class of conditional independence ideals where a fixed binary variable is independent of a collection of other variables, given the remaining ones. In this case the primary decomposition has a natural statistical interpretation.

Conditional independence structure and its closure: Inferential rules and algorithms

In this paper, we deal with conditional independence models closed with respect to graphoid properties. Such models come from different uncertainty measures, in particular in a probabilistic setting. We study some inferential rules and describe methods and algorithms to compute efficiently the closure of a set of conditional independence statements.

Matrix representations and independencies in directed acyclic graphs

For a directed acyclic graph, there are two known criteria to decide whether any specific conditional independence statement is implied for all distributions factorized according to the given graph. Both criteria are based on special types of path in graphs. They are called separation criteria because independence holds whenever the conditioning set is a separating set in a graph theoretical sense. We introduce and discuss an alternative approach using binary matrix representations of graphs in which zeros indicate independence statements. A matrix condition is shown to give a new path criterion for separation and to be equivalent to each of the previous two path criteria.

Tropical fans and the moduli spaces of tropical curves

AbstractWe give a rigorous definition of tropical fans (the ‘local building blocks for tropical varieties’) and their morphisms. For a morphism of tropical fans of the same dimension we show that the number of inverse images (counted with suitable tropical multiplicities) of a point in the target does not depend on the chosen point; a statement that can be viewed as one of the important first steps of tropical intersection theory. As an application we consider the moduli spaces of rational tropical curves (both abstract and in some ℝr) together with the evaluation and forgetful morphisms. Using our results this gives new, easy and unified proofs of various tropical independence statements, e.g. of the fact that the numbers of rational tropical curves (in any ℝr) through given points are independent of the points.

Gaussian conditional independence relations have no finite complete characterization

We show that there can be no finite list of conditional independence relations which can be used to deduce all conditional independence implications among Gaussian random variables. To do this, we construct, for each n > 3 a family of n conditional independence statements on n random variables which together imply that X 1 ⫫ X 2 , and such that no subset have this same implication. The proof relies on binomial primary decomposition.

Mining and visualising ordinal data with non-parametric continuous BBNs

Data mining is the process of extracting and analysing information from large databases. Graphical models are a suitable framework for probabilistic modelling. A Bayesian Belief Net (BBN) is a probabilistic graphical model, which represents joint distributions in an intuitive and efficient way. It encodes the probability density (or mass) function of a set of variables by specifying a number of conditional independence statements in the form of a directed acyclic graph. Specifying the structure of the model is one of the most important design choices in graphical modelling. Notwithstanding their potential, there are only a limited number of applications of graphical models on very complex and large databases. A method for mining ordinal multivariate data using non-parametric BBNs is presented. The main advantage of this method is that it can handle a large number of continuous variables, without making any assumptions about their marginal distributions, in a very fast manner. Once the BBN is learned from data, it can be further used for prediction. This approach allows for rapid conditionalisation, which is a very important feature of a BBN from a user’s standpoint.