Conditional Independence Relations Research Articles

Qualitative inequalities for squared partial correlations of a Gaussian random vector

We describe various sets of conditional independence relationships, sufficient for qualitatively comparing non-vanishing squared partial correlations of a Gaussian random vector. These sufficient conditions are satisfied by several graphical Markov models. Rules for comparing degree of association among the vertices of such Gaussian graphical models are also developed. We apply these rules to compare conditional dependencies on Gaussian trees. In particular for trees, we show that such dependence can be completely characterized by the length of the paths joining the dependent vertices to each other and to the vertices conditioned on. We also apply our results to postulate rules for model selection for polytree models. Our rules apply to mutual information of Gaussian random vectors as well.

Conditional independence graph for nonlinear time series and its application to international financial markets

Conditional independence graphs are proposed for describing the dependence structure of multivariate nonlinear time series, which extend the graphical modeling approach based on partial correlation. The vertexes represent the components of a multivariate time series and edges denote direct dependence between corresponding series. The conditional independence relations between component series are tested efficiently and consistently using conditional mutual information statistics and a bootstrap procedure. Furthermore, a method combining information theory with surrogate data is applied to test the linearity of the conditional dependence. The efficiency of the methods is approved through simulation time series with different linear and nonlinear dependence relations. Finally, we show how the method can be applied to international financial markets to investigate the nonlinear independence structure.

Measuring monetary policy with empirically grounded identifying restrictions

This article reevaluates the impulse response functions (IRFs) to a monetary policy shock of the structural vector autoregression (SVAR). Identifying restrictions are specified and justified based on empirical evidence,i.e., conditional independence relations of variables, which is an important dimension that a good model must be able to mimic. The empirical-based approach is able to significant narrow down the set of admissible causal orders to identify the IRFs to a monetary policy shock (from 2,482 to 8). I find that most of the qualitative “stylized” features reported in the literature remain intact. However, the quantitative predictions are much less certain than what is commonly perceived.

Labeling of Human Motion Based on CBGA and Probabilistic Model

Abstract In this paper, we present a novel method for the labeling of human motion which uses Constraint-Based Genetic Algorithm (CBGA) to optimize the probabilistic model of body features and construct the set of conditional independence relations among the body features by a fitness function. The approach also allows the user to add custom rules to produce valid candidate solutions to achieve more accurate results with constraint-based genetic operators. Specifically, we design the fitness function using a probability model based on the decomposable triangle model(DTM), which is learned through the EM algorithm with the minimum description length (MDL) principle and CBGA algorithm to characterize the stochastic and dynamic relations of articulated human motions. We also extend these results to learning the probabilistic structure of human body to improve the labeling results, the handling of missing body parts, the integration of multi-frame information and the accuracy rates. Finally, we analyze the performance of our proposed approach and show that it outperforms most of the current state of the art methods on a set of motion captured walking, running and dancing sequences in terms of quality and robustness.

LOGARITHMIC MARKETS CORING RULES FOR MODULAR COMBINATORIAL INFORMATION AGGREGATION

In practice, scoring rules elicit good probability estimates from individuals, while betting markets elicit good consensus estimates from groups. Market scoring rules combine these features, eliciting estimates from individuals or groups, with groups costing no more than individuals. Regarding a bet on one event given another event, only logarithmic versions preserve the probability of the given event. Logarithmic versions also preserve the conditional probabilities of other events, and so preserve conditional independence relations. Given logarithmic rules that elicit relative probabilities of base event pairs, it costs no more to elicit estimates on all combinations of these base events.

Covariate-Adjusted Precision Matrix Estimation with an Application in Genetical Genomics.

Motivated by analysis of genetical genomics data, we introduce a sparse high dimensional multivariate regression model for studying conditional independence relationships among a set of genes adjusting for possible genetic effects. The precision matrix in the model specifies a covariate-adjusted Gaussian graph, which presents the conditional dependence structure of gene expression after the confounding genetic effects on gene expression are taken into account. We present a covariate-adjusted precision matrix estimation method using a constrained ℓ1 minimization, which can be easily implemented by linear programming. Asymptotic convergence rates in various matrix norms and sign consistency are established for the estimators of the regression coefficients and the precision matrix, allowing both the number of genes and the number of the genetic variants to diverge. Simulation shows that the proposed method results in significant improvements in both precision matrix estimation and graphical structure selection when compared to the standard Gaussian graphical model assuming constant means. The proposed method is also applied to analyze a yeast genetical genomics data for the identification of the gene network among a set of genes in the mitogen-activated protein kinase pathway.

Characterization and greedy learning of interventional Markov equivalence classes of directed acyclic graphs

The investigation of directed acyclic graphs (DAGs) encoding the same Markov property, that is the same conditional independence relations of multivariate observational distributions, has a long tr...

Local computations in Dempster–Shafer theory of evidence

When applying any technique of multidimensional models to problems of practice, one always has to cope with two problems: the necessity to represent the models with a ”reasonable” number of parameters and to have sufficiently efficient computational procedures at one’s disposal. When considering graphical Markov models in probability theory, both of these conditions are fulfilled; various computational procedures for decomposable models are based on the ideas of local computations, whose theoretical foundations were laid by Lauritzen and Spiegelhalter.The presented contribution studies a possibility of transferring these ideas from probability theory into Dempster-Shafer theory of evidence. The paper recalls decomposable models, discusses connection of the model structure with the corresponding system of conditional independence relations, and shows that under special additional conditions, one can locally compute specific basic assignments which can be considered to be conditional.

Likelihood analysis of the binary instrumental variable model

Instrumental variables are widely used for the identification of the causal effect of one random variable on another under unobserved confounding. The distribution of the observable variables for a discrete instrumental variable model satisfies certain inequalities but no conditional independence relations. Such models are usually tested by checking whether the relative frequency estimators of the parameters satisfy the constraints. This ignores sampling uncertainty in the data. Using the observable constraints for the instrumental variable model, a likelihood analysis is conducted. A significance test for its validity is developed, and a bootstrap algorithm for computing confidence intervals for the causal effect is proposed. Applications are given to illustrate the advantage of the suggested approach. Copyright 2011, Oxford University Press.

Foundations of compositional model theory

Graphical Markov models, most of all Bayesian networks, have become a very popular way for multidimensional probability distribution representation and processing. What makes representation of a very-high-dimensional probability distribution possible is its independence structure, i.e. a system of conditional independence relations valid for the distribution in question. The fact that some of independence systems can be successfully represented with the help of graphs is reflected in the general title: graphical modelling. However, graphical representation of independence structures is also associated with some disadvantages: only a small part of different independence structures can be faithfully represented by graphs; and still one structure is usually equally well represented by several graphs. These reasons, among others, initiated development of an alternative approach, called here theory of compositional models, which enables us to represent exactly the same class of distributions as Bayesian networks. This paper is a survey of the most important basic concepts and results concerning compositional models necessary for reading advanced papers on computational procedures and other aspects connected with this (relatively new) approach for multidimensional distribution representation.

Gaussian Covariance Faithful Markov Trees

Graphical models are useful for characterizing conditional and marginal independence structures in high‐dimensional distributions. An important class of graphical models is covariance graph models, where the nodes of a graph represent different components of a random vector, and the absence of an edge between any pair of variables implies marginal independence. Covariance graph models also represent more complex conditional independence relationships between subsets of variables. When the covariance graph captures or reflects all the conditional independence statements present in the probability distribution, the latter is said to be faithful to its covariance graph—though in general this is not guaranteed. Faithfulness however is crucial, for instance, in model selection procedures that proceed by testing conditional independences. Hence, an analysis of the faithfulness assumption is important in understanding the ability of the graph, a discrete object, to fully capture the salient features of the probability distribution it aims to describe. In this paper, we demonstrate that multivariate Gaussian distributions that have trees as covariance graphs are necessarily faithful.

Identification of nonlinear VAR models using general conditional independence graphs

The general conditional independence graphs are proposed for identifying nonlinear vector autoregressive model, which extend the graphical modeling approach for linear structure VAR. The conditional independence relations between time series variables and their lags can be tested efficiently and consistently using conditional mutual information statistics and a permutation procedure. Furthermore, a statistic resulted from the difference between general conditional mutual information and linear conditional mutual information is proposed to test the nonlinearity of the dependence. A bootstrap method based on surrogate data is used to determine the significance of the nonlinear test statistic. The finite sample behavior of the procedure through simulation time series with different linear and nonlinear dependence relations is investigated.

BASSUM: A Bayesian semi-supervised method for classification feature selection

Feature selection is an important preprocessing step for building efficient, generalizable and interpretable classifiers on high dimensional data sets. Given the assumption on the sufficient labelled samples, the Markov Blanket provides a complete and sound solution to the selection of optimal features, by exploring the conditional independence relationships among the features. In real-world applications, unfortunately, it is usually easy to get unlabelled samples, but expensive to obtain the corresponding accurate labels on the samples. This leads to the potential waste of valuable classification information buried in unlabelled samples. In this paper, we propose a new BAyesian Semi-SUpervised Method, or BASSUM in short, to exploit the values of unlabelled samples on classification feature selection problem. Generally speaking, the inclusion of unlabelled samples helps the feature selection algorithm on (1) pinpointing more specific conditional independence tests involving fewer variable features and (2) improving the robustness of individual conditional independence tests with additional statistical information. Our experimental results show that BASSUM enhances the efficiency of traditional feature selection methods and overcomes the difficulties on redundant features in existing semi-supervised solutions.

Identifying product failure rate based on a conditional Bayesian network classifier

To identify the product failure rate grade under diverse configuration and operation conditions, a new conditional Bayesian networks (CBN) model is brought forward. By indicating the conditional independence relationship between attribute variables given the target variable, this model could provide an effective approach to classify the grade of failure rate. Furthermore, on the basis of the CBN model, the procedure of building product failure rate grade classifier is elaborated with modeling and application. At last, a case study is carried out and the results show that, with comparison to other Bayesian networks classifiers and traditional decision tree C4.5, the CBN model not only increases the total classification accuracy, but also reduces the complexity of network structure.

On meta-analyses of imaging data and the mixture of records

Neumann et al. (2010) aim to find directed graphical representations of the independence and dependence relations among activities in brain regions by applying a search procedure to merged fMRI activity records from a large number of contrasts obtained under a variety of conditions. To that end, Neumann et al., obtain three graphical models, justifying their search procedure with simulations that find that merging the data sampled from probability distributions characterized by two distinct Bayes net graphs results in a graphical object that combines the edges in the individual graphs. We argue that the graphical objects they obtain cannot be interpreted as representations of conditional independence and dependence relations among localized neural activities; specifically, directed edges and directed pathways in their graphical results may be artifacts of the manner in which separate studies are combined in the meta-analytic procedure. With a larger simulation study, we argue that their simulation results with combined data sets are an artifact of their choice of examples. We provide sufficient conditions and necessary conditions for the merger of two or more probability distributions, each characterized by the Markov equivalence class of a directed acyclic graph, to be describable by a Markov equivalence class whose edges are a union of those for the individual distributions. Contrary to Neumann et al., we argue that the scientific value of searches for network representations from imaging data lies in attempting to characterize large scaled neural mechanisms, and we suggest several alternative strategies for combining data from multiple experiments.

A universal algebraic approach for conditional independence

In this paper we show that elementary properties of joint probability density functions naturally induce a universal algebraic structure suitable for studying probabilistic conditional independence (PCI) relations. We call this algebraic system the cain. In the cain algebra, PCI relations are represented in equational forms. In particular, we show that the cain satisfies the axioms of the graphoid of Pearl and Paz (Advances in artificial intelligence. North-Holland, Amsterdam, 1987) and the separoid of Dawid (Ann. Math. Artif. Intell. 32:335–372, 2001), these axiomatic systems being useful for general probabilistic reasoning.

Conditional independence relations among biological markers may improve clinical decision as in the case of triple negative breast cancers

The associations existing among different biomarkers are important in clinical settings because they contribute to the characterisation of specific pathways related to the natural history of the disease, genetic and environmental determinants. Despite the availability of binary/linear (or at least monotonic) correlation indices, the full exploitation of molecular information depends on the knowledge of direct/indirect conditional independence (and eventually causal) relationships among biomarkers, and with target variables in the population of interest. In other words, that depends on inferences which are performed on the joint multivariate distribution of markers and target variables. Graphical models, such as Bayesian Networks, are well suited to this purpose. Therefore, we reconsidered a previously published case study on classical biomarkers in breast cancer, namely estrogen receptor (ER), progesterone receptor (PR), a proliferative index (Ki67/MIB-1) and to protein HER2/neu (NEU) and p53, to infer conditional independence relations existing in the joint distribution by inferring (learning) the structure of graphs entailing those relations of independence. We also examined the conditional distribution of a special molecular phenotype, called triple-negative, in which ER, PR and NEU were absent. We confirmed that ER is a key marker and we found that it was able to define subpopulations of patients characterized by different conditional independence relations among biomarkers. We also found a preliminary evidence that, given a triple-negative profile, the distribution of p53 protein is mostly supported in 'zero' and 'high' states providing useful information in selecting patients that could benefit from an adjuvant anthracyclines/alkylating agent-based chemotherapy.

Markov equivalence for ancestral graphs

Ancestral graphs can encode conditional independence relations that arise in directed acyclic graph (DAG) models with latent and selection variables. However, for any ancestral graph, there may be several other graphs to which it is Markov equivalent. We state and prove conditions under which two maximal ancestral graphs are Markov equivalent to each other, thereby extending analogous results for DAGs given by other authors. These conditions lead to an algorithm for determining Markov equivalence that runs in time that is polynomial in the number of vertices in the graph.

Efficient Markov Network Structure Discovery Using Independence Tests

We present two algorithms for learning the structure of a Markov network from data: GSMN* and GSIMN. Both algorithms use statistical independence tests to infer the structure by successively constraining the set of structures consistent with the results of these tests. Until very recently, algorithms for structure learning were based on maximum likelihood estimation, which has been proved to be NP-hard for Markov networks due to the difficulty of estimating the parameters of the network, needed for the computation of the data likelihood. The independence-based approach does not require the computation of the likelihood, and thus both GSMN* and GSIMN can compute the structure efficiently (as shown in our experiments). GSMN* is an adaptation of the Grow-Shrink algorithm of Margaritis and Thrun for learning the structure of Bayesian networks. GSIMN extends GSMN* by additionally exploiting Pearl's well-known properties of the conditional independence relation to infer novel independences from known ones, thus avoiding the performance of statistical tests to estimate them. To accomplish this efficiently GSIMN uses the Triangle theorem, also introduced in this work, which is a simplified version of the set of Markov axioms. Experimental comparisons on artificial and real-world data sets show GSIMN can yield significant savings with respect to GSMN*, while generating a Markov network with comparable or in some cases improved quality. We also compare GSIMN to a forward-chaining implementation, called GSIMN-FCH, that produces all possible conditional independences resulting from repeatedly applying Pearl's theorems on the known conditional independence tests. The results of this comparison show that GSIMN, by the sole use of the Triangle theorem, is nearly optimal in terms of the set of independences tests that it infers.

Markov Properties for Linear Causal Models with Correlated Errors

A linear causal model with correlated errors, represented by a DAG with bi-directed edges, can be tested by the set of conditional independence relations implied by the model. A global Markov prope...