Conditional Independence Statements Research Articles

Describing Conditional Independence Statements Using Undirected Graphs

This paper investigates the capability of undirected graphs (UGs) to represent a set of Conditional Independence (CI) statements derived from a given probability distribution of a random vector. While it is established that certain axioms can govern this set, providing sufficient conditions for UGs to capture specific CI statements, our focus is on covariance and concentration graphs. These remain the only known families of UGs capable of describing CI statements. We explore the issue of complete representation of CI statements through their corresponding covariance and concentration graphs. Two parameters are defined, one each from the covariance and concentration graphs, to determine the limitations concerning the cardinality of the conditioning subset that the graph can represent. We establish a relationship between these parameters and the cardinality of the separators in each graph, providing a straightforward computational method to evaluate them. In conclusion, we enhance the aforementioned procedure and introduce criteria to ascertain, without additional computations, whether the graphs can fully represent a given set of CI statements. We demonstrate that either the concentration or the covariance graph forms a cycle, and when considered in conjunction, they can represent the entire relation. These criteria also enable us, in specific cases, to deduce the covariance graph from the concentration graph and vice versa.

Nature of decision valuations in elimination of redundant attributes

Information systems are the basic building blocks of the theory of rough sets which, based on information signatures of objects, develops strategies for classifying concepts and/or deriving significant information about decision attributes. During such processes of aggregating decision information, one needs to focus on the aspects of designing its representation and considering various ways for reducing attributes. We introduce a general mathematical apparatus of decision valuations that are aimed at representing information derivable from data. We establish a strong connection between the notion of attribute reduction considered in the context of decision valuations and the analogous notions developed in relational databases, semigraphoid models of conditional independence statements, etc. Based on different decision making strategies, we discuss different properties of decision valuations and explore different examples in the light of those properties. We also investigate interrelationships among those properties. Going back to attribute reduction, i.e. elimination of redundant attributes, we consider the prevalent property of discernibility and compare it with the properties of semigraphoid models, such as weak union and decomposition.

Inversion of Bayesian networks

Variational autoencoders and Helmholtz machines use a recognition network (encoder) to approximate the posterior distribution of a generative model (decoder). In this paper we establish some necessary and some sufficient properties of a recognition network so that it can model the true posterior distribution exactly. These results are derived in the general context of probabilistic graphical modelling / Bayesian networks, for which the network represents a set of conditional independence statements. We derive both global conditions, in terms of d-separation, and local conditions for the recognition network to have the desired qualities. It turns out that for the local conditions the perfectness property (for every node, all parents are joined) plays an important role.

Greedy Causal Discovery Is Geometric

Finding a directed acyclic graph (DAG) that best encodes the conditional independence statements observable from data is a central question within causality. Algorithms that greedily transform one candidate DAG into another given a fixed set of moves have been particularly successful, for example, the greedy equivalence search, greedy interventional equivalence search, and max-min hill climbing algorithms. In 2010, Studený, Hemmecke, and Lindner introduced the characteristic imset (CIM) polytope, , whose vertices correspond to Markov equivalence classes, as a way of transforming causal discovery into a linear optimization problem. We show that the moves of the aforementioned algorithms are included within classes of edges of and that restrictions placed on the skeleton of the candidate DAGs correspond to faces of . Thus, we observe that greedy equivalence search, greedy interventional equivalence search, and max-min hill climbing all have geometric realizations as greedy edge-walks along . Furthermore, the identified edges of strictly generalize the moves of these algorithms. Exploiting this generalization, we introduce a greedy simplex-type algorithm called greedy CIM, and a hybrid variant, skeletal greedy CIM, that outperforms current competitors among hybrid and constraint-based algorithms.

Development and application of an evidence-based directed acyclic graph to evaluate the associations between metal mixtures and cardiometabolic outcomes.

Specifying causal models to assess relationships among metal mixtures and cardiometabolic outcomes requires evidence-based models of the causal structures; however, such models have not been previously published. The objective of this study was to develop and evaluate a directed acyclic graph (DAG) diagraming metal mixture exposure and cardiometabolic outcomes. We conducted a literature search to develop the DAG of metal mixtures and cardiometabolic outcomes. To evaluate consistency of the DAG, we tested the suggested conditional independence statements using linear and logistic regression analyses with data from the San Luis Valley Diabetes Study (SLVDS; n=1795). We calculated the proportion of statements supported by the data and compared this to the proportion of conditional independence statements supported by 1,000 DAGs with the same structure but randomly permuted nodes. Next, we used our DAG to identify minimally sufficient adjustment sets needed to estimate the association between metal mixtures and cardiometabolic outcomes (i.e., cardiovascular disease, fasting glucose, and systolic blood pressure). We applied them to the SLVDS using Bayesian kernel machine regression, linear mixed effects, and Cox proportional hazards models. From the 42 articles included in the review, we developed an evidence-based DAG with 74 testable conditional independence statements (43 % supported by SLVDS data). We observed evidence for an association between As and Mn and fasting glucose. We developed, tested, and applied an evidence-based approach to analyze associations between metal mixtures and cardiometabolic health.

Lattice conditional independence models and Hibi ideals

Lattice Conditional Independence models are a class of models developed first for the Gaussian case in which a distributive lattice classifies all the conditional independence statements. The main result is that these models can equivalently be described via a transitive directed acyclic graph (TDAG) in which, as is normal for causal models, the conditional independence is in terms of conditioning on ancestors in the graph. We demonstrate that a parallel stream of research in algebra, the theory of Hibi ideals, not only maps directly to the LCI models but gives a vehicle to generalise the theory from the linear Gaussian case. Given a distributive lattice (i) each conditional independence statement is associated with a Hibi relation defined on the lattice, (ii) the directed graph is given by chains in the lattice which correspond to chains of conditional independence, (iii) the elimination ideal of product terms in the chains gives the Hibi ideal and (iv) the TDAG can be recovered from a special bipartite graph constructed via the Alexander dual of the Hibi ideal. It is briefly demonstrated that there are natural applications to statistical log-linear models, time series, and Shannon information flow.

Conditional independence in max-linear Bayesian networks

Motivated by extreme value theory, max-linear Bayesian networks have been recently introduced and studied as an alternative to linear structural equation models. However, for max-linear systems the classical independence results for Bayesian networks are far from exhausting valid conditional independence statements. We use tropical linear algebra to derive a compact representation of the conditional distribution given a partial observation, and exploit this to obtain a complete description of all conditional independence relations. In the context-specific case, where conditional independence is queried relative to a specific value of the conditioning variables, we introduce the notion of a source DAG to disclose the valid conditional independence relations. In the context-free case we characterize conditional independence through a modified separation concept, $\ast$-separation, combined with a tropical eigenvalue condition. We also introduce the notion of an impact graph which describes how extreme events spread deterministically through the network and we give a complete characterization of such impact graphs. Our analysis opens up several interesting questions concerning conditional independence and tropical geometry.

Complexity as Causal Information Integration.

Complexity measures in the context of the Integrated Information Theory of consciousness try to quantify the strength of the causal connections between different neurons. This is done by minimizing the KL-divergence between a full system and one without causal cross-connections. Various measures have been proposed and compared in this setting. We will discuss a class of information geometric measures that aim at assessing the intrinsic causal cross-influences in a system. One promising candidate of these measures, denoted by , is based on conditional independence statements and does satisfy all of the properties that have been postulated as desirable. Unfortunately it does not have a graphical representation, which makes it less intuitive and difficult to analyze. We propose an alternative approach using a latent variable, which models a common exterior influence. This leads to a measure , Causal Information Integration, that satisfies all of the required conditions. Our measure can be calculated using an iterative information geometric algorithm, the em-algorithm. Therefore we are able to compare its behavior to existing integrated information measures.

Modeling “Equitable and Sustainable Well-being” (BES) Using Bayesian Networks: A Case Study of the Italian Regions

Measurement of well-being has been a highly debated topic since the end of the last century. While some specific aspects are still open issues, a multidimensional approach as well as the construction of shared and well-rooted systems of indicators are now accepted as the main route to measure this complex phenomenon. A meaningful effort, in this direction, is that of the Italian "Equitable and Sustainable Well-being" (BES) system of indicators, developed by the Italian National Institute of Statistics (ISTAT) and the National Council for Economics and Labour (CNEL). The BES framework comprises a number of atomic indicators measured yearly at the regional level and reflecting the different domains of well-being (e.g. Health, Education, Work \& Life Balance, Environment,...). In this work we aim at dealing with the multidimensionality of the BES system of indicators and try to answer three main research questions: I) What is the structure of the relationships among the BES atomic indicators; II) What is the structure of the relationships among the BES domains; III) To what extent the structure of the relationships reflects the current BES theoretical framework. We address these questions by implementing Bayesian Networks (BNs), a widely accepted class of multivariate statistical models, particularly suitable for handling reasoning with uncertainty. Implementation of a BN results in a set of nodes and a set of conditional independence statements that provide an effective tool to explore associations in a system of variables. In this work, we also suggest two strategies for encoding prior knowledge in the BN estimating algorithm so that the BES theoretical framework can be represented into the network.

Conditional independence ideals with hidden variables

We study a class of determinantal ideals that are related to conditional independence (CI) statements with hidden variables. Such CI statements correspond to determinantal conditions on a matrix whose entries are probabilities of events involving the observed random variables. We focus on an example that generalizes the CI ideals of the intersection axiom. In this example, the minimal primes are again determinantal ideals, which is not true in general.

On the primary decomposition of some determinantal hyperedge ideal

In this paper we describe the method which we applied to successfully compute the primary decomposition of a certain ideal coming from applications in combinatorial algebra and algebraic statistics regarding conditional independence statements with hidden variables. While our method is based on the algorithm for primary decomposition by Gianni, Trager and Zacharias, we were not able to decompose the ideal using the standard form of that algorithm, nor by any other method known to us.

Iterative Approximate Nonlinear Inference via Gaussian Message Passing on Factor Graphs

Factor graphs are graphical models able to represent the factorization of probability density functions. By visualizing conditional independence statements, they provide an intuitive and versatile interface to sparsity exploiting message passing algorithms as a unified framework for constructing algorithms in signal processing, estimation, and control in a mix-and-match style. Especially, when assuming Gaussian distributed variables, tabulated message passing rules allow for easy automated derivations of algorithms. This letter’s contribution consists in the combination of statistical or Jacobian-based linearization approaches to handling nonlinear factors with efficient message parametrizations in a Gaussian message passing setting. Tabulated message passing rules for a multivariate nonlinear factor node are presented that implement a re-linearization about the most current belief (marginal) of each adjacent variable. When utilized in a nonlinear Kalman smoothing setting, the iterated nonlinear modified Bryson–Frazier smoother is recovered, while retaining the flexibility of the factor graph framework. This application is illustrated by deriving an input estimation algorithm for a nonlinear system.

A linear-algebraic tool for conditional independence inference

In this note, we propose a new linear-algebraic method for the implication problem among conditional independence statements, which is inspired by the factorization characterization of conditional independence. First, we give a criterion in the case of a discrete strictly positive density and relate it to an earlier linear-algebraic approach. Then, we extend the method to the case of a discrete density that need not be strictly positive. Finally, we provide a computational result in the case of six variables.

The three faces of faithfulness

In the causal inference framework of Spirtes, Glymour, and Scheines (SGS), inferences about causal relationships are made from samples from probability distributions and a number of assumptions relating causal relations to probability distributions. The most controversial of these assumptions is the Causal Faithfulness Assumption, which roughly states that if a conditional independence statement is true of a probability distribution generated by a causal structure, it is entailed by the causal structure and not just for particular parameter values. In this paper we show that the addition of the Causal Faithfulness Assumption plays three quite different roles in the SGS framework: (i) it reduces the degree of underdetermination of causal structure by probability distribution; (ii) computationally, it justifies reliable (constraint-based) causal inference algorithms that would otherwise have to be slower in order to be reliable; and (iii) statistically, it implies that those algorithms reliably obtain the correct answer at smaller sample sizes than would otherwise be the case. We also consider a number of variations on the Causal Faithfulness Assumption, and show how they affect each of these three roles.

The dynamic chain event graph

In this paper we develop a formal dynamic version of Chain Event Graphs (CEGs), a particularly expressive family of discrete graphical models. We demonstrate how this class links to semi-Markov models and provides a convenient generalization of the Dynamic Bayesian Network (DBN). In particular we develop a repeating time-slice Dynamic CEG providing a useful and simpler model in this family. We demonstrate how the Dynamic CEG’s graphical formulation exhibits asymmetric conditional independence statements and also how each model can be estimated in a closed form enabling fast model search over the class. The expressive power of this model class together with its estimation is illustrated throughout by a variety of examples that include the risk of childhood hospitalization and the efficacy of a flu vaccine.

Foundations of compositional models: structural properties

The paper is a follow-up of [R.J.: Foundations of compositional model theory. IJGS, 40(2011): 623–678], where basic properties of compositional models, as one of the approaches to multidimensional probability distributions representation and processing, were introduced. In fact, it is an algebraic alternative to graphical models, which does not use graphs to represent conditional independence statements. Here, these statements are encoded in a sequence of distributions to which an operator of composition – the key element of this theory – is applied in order to assemble a multidimensional model from its low-dimensional parts. In this paper, we show a way to read conditional independence relations, and to solve related topics, above all the so-called equivalence problem, i.e. the problem of recognizing whether two different structures induce the same system of conditional independence relations.

Positive margins and primary decomposition

We study random walks on contingency tables with fixed marginals, corresponding to a (log-linear) hierarchical model. If the set of allowed moves is not a Markov basis, then there exist tables with the same marginals that are not connected. We study linear conditions on the values of the marginals that ensure that all tables in a given fiber are connected. We show that many graphical models have the positive margins property, which says that all fibers with strictly positive marginals are connected by the quadratic moves that correspond to conditional independence statements. The property persists under natural operations such as gluing along cliques, but we also construct examples of graphical models not enjoying this property. We also provide a negative answer to a question of Engstr\"om, Kahle, and Sullivant by demonstrating that the global Markov ideal of the complete bipartite graph K_(3,3) is not radical. Our analysis of the positive margins property depends on computing the primary decomposition of the associated conditional independence ideal. The main technical results of the paper are primary decompositions of the conditional independence ideals of graphical models of the $N$-cycle and the complete bipartite graph $K_(2,N-2)$, with various restrictions on the size of the nodes.

On the completeness of the semigraphoid axioms for deriving arbitrary from saturated conditional independence statements

Conditional independence (CI) statements occur in several areas of computer science and artificial intelligence, e.g., as embedded multivalued dependencies in database theory, disjunctive association rules in data mining, and probabilistic CI statements in probability theory. Although, syntactically, such constraints can always be represented in the form I(A,B|C), with A, B, and C subsets of some universe S, their semantics is very dependent on their interpretation, and, therefore, inference rules valid under one interpretation need not be valid under another. However, all aforementioned interpretations obey the so-called semigraphoid axioms. In this paper, we consider the restricted case of deriving arbitrary CI statements from so-called saturated ones, i.e., which involve all elements of S. Our main result is a necessary and sufficient condition under which the semigraphoid axioms are also complete for such derivations. Finally, we apply these results to the examples mentioned above to show that, for these semantics, the semigraphoid axioms are both sound and complete for the derivation of arbitrary CI statements from saturated ones.

Practical Guidelines for Learning Bayesian Networks from Small Data Sets

Model learning is the process of extracting, analysing and synthesising information from data sets. Graphical models are a suitable framework for probabilistic modelling. A Bayesian Network (BN) is a probabilistic graphical model, which represents joint distributions in an intuitive and efficient way. It encodes the probability density (or mass) function of a set of variables by specifying a number of conditional independence statements in the form of a directed acyclic graph. Specifying the structure of the model is one of the most important design choices in graphical modelling. Notwithstanding their potential, there are several limitations to learning BNs from small data sets. In this paper, we introduce a set of practical guidelines a modeller can use to deal with these limitations. The main goal of the guidelines is to increase awareness of the underlying assumptions and the tacit implications of several learning techniques. Unsurprisingly, one of the authors’ findings is that learning BNs from small data sets is a complex and challenging task, yet potentially very rewarding. The paper also draws attention to the amount of subjective input needed from the modeller and the necessity to tailor solutions on the particularity of the application.

Minimal primes of ideals arising from conditional independence statements

We study ideals whose primary decomposition specifies the relevant structural zeros of certain conditional independence models. The ideals we study generalize the class of ideals considered by Fink (2011) [5] in a way distinct from the generalizations of Herzog, Hibi, Hreinsdottir, Kahle, and Rauh (2010) [10] and Ay and Rauh (2011) [1]. We introduce switchable sets to give a combinatorial description of the minimal prime ideals, and for some classes we describe the minimal components. We discuss possible interpretations of the ideals we study, including as 2×2 minors of generic hypermatrices. We also introduce a definition of diagonal monomial orders on generic hypermatrices to compute some Gröbner bases.