Conditional Independence Properties Research Articles

Syntactic complexity of spoken language in the diagnosis of schizophrenia: A probabilistic Bayes network model.

In the clinical linguistics of schizophrenia, syntactic complexity has received much attention. In this study, we address whether syntactic complexity deteriorates within the six months following the first episode of psychosis in those who develop a diagnosis of schizophrenia. We collected data from a cohort of twenty-six first-episode psychosis and 12 healthy control subjects using the Thought and Language Index interview in response to three pictures from the Thematic Apperception Test at first assessment and after six months (the time of consensus diagnosis). An automated labeling (part-of-speech tagging) for specific syntactic elements calculated large and granular syntactic complexity indices with a focus on clause complexity as a particular case from this spoken language data. Probabilistic reasoning leveraging the conditional independence properties of Bayes networks revealed that consensus diagnosis of schizophrenia predicted a decrease in nominal subjects per clause among individuals with first episode psychosis. From the entire sample, we estimate a 95.4% probability that a 50% decrease in mean nominal subjects per clause after six months is explained by the presence of first episode psychosis. Among those with psychosis, a 30% decrease in this clause-complexity index after six months of experiencing the first episode predicted with 95% probability a consensus diagnosis of schizophrenia, representing a conditional relationship between a longitudinal decrease in syntactic complexity and a diagnosis of schizophrenia. We conclude that an early drift towards linguistic disorganization/impoverishment of clause complexity-at the granular level of nominal subject per clause-is a distinctive feature of schizophrenia that decreases longitudinally, thus differentiating schizophrenia from other psychotic illnesses with shared phenomenology.

Undirected Structural Markov Property for Bayesian Model Determination

This paper generalizes the structural Markov properties for undirected decomposable graphs to arbitrary ones. This helps us to exploit the conditional independence properties of joint prior laws to analyze and compare multiple graphical structures, while being able to take advantage of the common conditional independence constraints. This work provides a theoretical support for full Bayesian posterior updating about the structure of a graph using data from a certain distribution. We further investigate the ratio of graph law so as to simplify the acceptance probability of the Metropolis–Hastings sampling algorithms.

Causal inference for the covariance between breeding values under identity disequilibrium

BackgroundThe covariance matrix of breeding values is at the heart of prediction methods. Prediction of breeding values can be formulated using either an “observed” or a theoretical covariance matrix, and a major argument for choosing one or the other is the reduction of the computational burden for inverting such a matrix. In this regard, covariance matrices that are derived from Markov causal models possess properties that deliver sparse inverses.ResultsBy using causal Markov models, we express the breeding value of an individual as a linear regression on ancestral breeding values, plus a residual term, which we call residual breeding value (RBV). The latter is a noise term that accounts for the uncertainty in prediction due to lack of fit of the linear regression. A notable property of these models is the parental Markov condition, through which the multivariate distribution of breeding values is uniquely determined by the distribution of the mutually independent RBV. Animal breeders have long been relying on a causal Markov model, while using the additive relationship matrix as the covariance matrix structure of breeding values, which is calculated assuming gametic equilibrium. However, additional covariances among breeding values arise due to identity disequilibrium, which is defined as the difference between the covariance matrix under the multi-loci probability of identity-by-descent (varvec{Sigma}) and its expectation under gametic phase equilibrium, i.e., A. The disequilibrium term varvec{Sigma}−A is considered in the model for predicting breeding values called the “ancestral regression” (AR), a causal Markov model. Here, we introduce the “ancestral regression to parents” (PAR) causal Markov model, which reduces the computational burden of the AR approach. By taking advantage of the conditional independence property of the PAR Markov model, we derive covariances between the breeding values of grandparents and grand-offspring and between parents and offspring. In addition, we obtain analytical expressions for the covariance between collateral relatives under the PAR model, as well as for the inbreeding coefficient.ConclusionsWe introduced the causal PAR Markov model that captures identity disequilibrium in the covariances among breeding values and produces a sparse inverse covariance matrix to build and solve a set of mixed model equations.

Diagonal nonlinear transformations preserve structure in covariance and precision matrices

For a multivariate normal distribution, the sparsity of the covariance and precision matrices encodes complete information about independence and conditional independence properties. For general distributions, the covariance and precision matrices reveal correlations and so-called partial correlations between variables, but these do not, in general, have any correspondence with respect to independence properties. In this paper, we prove that, for a certain class of non-Gaussian distributions, these correspondences still hold, exactly for the covariance and approximately for the precision. The distributions—sometimes referred to as “nonparanormal”—are given by diagonal transformations of multivariate normal random variables. We provide several analytic and numerical examples illustrating these results.

Splitting models for multivariate count data

We investigate the class of splitting distributions as the composition of a singular multivariate distribution and a univariate distribution. It will be shown that most common parametric count distributions (multinomial, negative multinomial, multivariate hypergeometric, multivariate negative hypergeometric, …) can be written as splitting distributions with separate parameters for both components, thus facilitating their interpretation, inference, the study of their probabilistic characteristics and their extensions to regression models. We highlight many probabilistic properties deriving from the compound aspect of splitting distributions and their underlying algebraic properties. Parameter inference and model selection are thus reduced to two separate problems, preserving time and space complexity of the base models. Based on this principle, we introduce several new distributions. In the case of multinomial splitting distributions, conditional independence and asymptotic normality properties for estimators are obtained. Mixtures of splitting regression models are used on a mango tree dataset in order to analyze the patchiness.

Mobile Robotic Sensors for Environmental Monitoring using Gaussian Markov Random Field

SUMMARYThis paper addresses the issue of monitoring spatial environmental phenomena of interest utilizing information collected by a network of mobile, wireless, and noisy sensors that can take discrete measurements as they navigate through the environment. It is proposed to employ Gaussian Markov random field (GMRF) represented on an irregular discrete lattice by using the stochastic partial differential equations method to model the physical spatial field. It then derives a GMRF-based approach to effectively predict the field at unmeasured locations, given available observations, in both centralized and distributed manners. Furthermore, a novel but efficient optimality criterion is then proposed to design centralized and distributed adaptive sampling strategies for the mobile robotic sensors to find the most informative sampling paths in taking future measurements. By taking advantage of conditional independence property in the GMRF, the adaptive sampling optimization problem is proven to be resolved in a deterministic time. The effectiveness of the proposed approach is compared and demonstrated using pre-published data sets with appealing results.

Mutual conditional independence and its applications to model selection in Markov networks

The fundamental concepts underlying Markov networks are the conditional independence and the set of rules called Markov properties that translate conditional independence constraints into graphs. We introduce the concept of mutual conditional independence in an independent set of a Markov network, and we prove its equivalence to the Markov properties under certain regularity conditions. This extends the notion of similarity between separation in graph and conditional independence in probability to similarity between the mutual separation in graph and the mutual conditional independence in probability. Model selection in graphical models remains a challenging task due to the large search space. We show that mutual conditional independence property can be exploited to reduce the search space. We present a new forward model selection algorithm for graphical log-linear models using mutual conditional independence. We illustrate our algorithm with a real data set example. We show that for sparse models the size of the search space can be reduced from mathcal {O} (n^{3}) to mathcal {O}(n^{2}) using our proposed forward selection method rather than the classical forward selection method. We also envision that this property can be leveraged for model selection and inference in different types of graphical models.

On the Strong Law of Large Numbers for Linear Combinations of Concomitants

A theorem on the strong law of large numbers for linear functions of concomitants (induced order statistics) for sequences of independent identically distributed two-dimensional random vectors is proved in this paper. The result complements previous work by S.S. Yang (1981) and N. Gribkova and R. Zitikis (2017, 2019). The proof is based on the conditional independence property of the concomitants established by P. K. Bhattacharya (1974); the van Zwet strong law of large numbers for linear functions of order statistics (1980) is used and classical inequalities apply, including the Rosenthal inequality (1970).

A logical approach to context-specific independence

Directed acyclic graphs (DAGs) constitute a qualitative representation for conditional independence (CI) properties of a probability distribution. It is known that every CI statement implied by the topology of a DAG is witnessed over it under a graph-theoretic criterion of d-separation. Alternatively, all such implied CI statements are derivable from the local independencies encoded by a DAG using the so-called semi-graphoid axioms. We consider Labeled Directed Acyclic Graphs (LDAGs) modeling graphically scenarios exhibiting context-specific independence (CSI). Such CSI statements are modeled by labeled edges, where labels encode contexts in which the edge vanishes. We study the problem of identifying all independence statements implied by the structure and the labels of an LDAG. We show that this problem is coNP-hard for LDAGs and formulate a sound extension of the semi-graphoid axioms for the derivation of such implied independencies. Finally we connect our study to certain qualitative versions of independence ubiquitous in database theory and teams semantics.

Addressing the Practical Limitations of Noisy-OR Using Conditional Inter-Causal Anti-Correlation with Ranked Nodes

Numerous methods have been proposed to simplify the problem of eliciting complex conditional probability tables in Bayesian networks. One of the most popular methods -“Noisy-OR”- approximates the required relationship in many real-world situations between a set of variables that are potential causes of an effect variable. However, the Noisy-OR function has the conditional inter-causal independence (CII) property which means that ‘explaining away’ behavior—one of the most powerful benefits of BN inference—is not present when the effect variable is observed as false. Hence, for many real-world problems where the Noisy-OR has been used or proposed, it may be deficient as an approximation of the required relationship. However, there is a very simple alternative solution, namely to define the variables as ranked nodes and to use the ranked node weighted average function. This does not have the CII property—instead, we prove it has the conditional anti-correlation property required to ensure that explaining away works in all cases. Moreover, ranked node variables are not restricted to binary states, and hence provide a more comprehensive and general solution to Noisy-OR in all cases.

Characterizing the Manifest Probability Distributions of Three Latent Trait Models for Accuracy and Response Time

In this paper we study the statistical relations between three latent trait models for accuracies and response times: the hierarchical model (HM) of van der Linden (Psychometrika 72(3):287–308, 2007), the signed residual time model (SM) proposed by Maris and van der Maas (Psychometrika 77(4):615–633, 2012), and the drift diffusion model (DM) as proposed by Tuerlinckx and De Boeck (Psychometrika 70(4):629–650, 2005). One important distinction between these models is that the HM and the DM either assume or imply that accuracies and response times are independent given the latent trait variables, while the SM does not. In this paper we investigate the impact of this conditional independence property—or a lack thereof—on the manifest probability distribution for accuracies and response times. We will find that the manifest distributions of the latent trait models share several important features, such as the dependency between accuracy and response time, but we also find important differences, such as in what function of response time is being modeled. Our method for characterizing the manifest probability distributions is related to the Dutch identity (Holland in Psychometrika 55(6):5–18, 1990).

Modified Weights-of-Evidence Modeling with Example of Missing Geochemical Data

Weights of evidence (WofE) and logistic regression (LR) are two loglinear methods for mineral potential mapping. Both models are limited by their respective basic assumptions in application. Ideally, WofE indicator patterns have the property of conditional independence (CI) with respect to the point pattern of mineral deposits to be predicted; in LR, there supposedly are no interactions between the point pattern and two or more of the indicator patterns. If the CI assumption is satisfied, estimated LR coefficients become approximately equal to WofE contrasts and the two methods produce similar results; additionally, bias then is avoided in that the sum of all estimated posterior probabilities becomes approximately equal to the number of observed discrete events. WofE allows construction of input layers that have missing data as a separate category in addition to known presence-absence type input, while logistic regression as such is not capable of handling missing data. As an improved WofE model based on LR, modified weights of evidence (MWofE) inherit the advantages of both LR and WofE, i.e., eliminates bias due to lack of CI and can handle missing data as well. Pixel or unit area input for MWofE consists of positive and negative weights for presence and absence of a pattern plus zeros for missing data. MWofE first is illustrated by application to simple examples. Next, it is applied to a study area with 20 known gold occurrences in southwestern Nova Scotia in relation to four input layers based on geological and lake geochemical data. Assuming that geochemical data were missing for the northern part of the study area, MWofE, like WofE but unlike LR, provides posterior probabilities for the entire area.

Large-Scale Bayesian Probabilistic Matrix Factorization with Memo-Free Distributed Variational Inference

Bayesian Probabilistic Matrix Factorization (BPMF) is a powerful model in many dyadic data prediction problems, especially the applications of Recommender system. However, its poor scalability has limited its wide applications on massive data. Based on the conditional independence property of observed entries in BPMF model, we propose a novel distributed memo-free variational inference method for large-scale matrix factorization problems. Compared with the state-of-the-art methods, the proposed method is favored for several attractive properties. Specifically, it does not require tuning of learning rate carefully, shuffling the training set at each iteration, or storing massive redundant variables, and can introduce new agents into the computations on the fly. We conduct extensive experiments on both synthetic and real-world datasets. The experimental results show that our method can converge significantly faster with better prediction performance than alternative algorithms.

Singular Gaussian graphical models: Structure learning

ABSTRACTThe goal of this article is to introduce singular Gaussian graphical models and their conditional independence properties. In fact, we extend the concept of Gaussian Markov Random Field to the case of a multivariate normally distributed vector with a singular covariance matrix. We construct, then, the associated graph’s structure from the covariance matrix’s pseudo-inverse on the basis of a characterization of the pairwise conditional independence. The proposed approach can also be used when the covariance matrix is ill-conditioned, through projecting data on a smaller subspace. In this case, our method ensures numerical stability and consistency of the constructed graph and significantly reduces the inference problem’s complexity. These aspects are illustrated using numerical experiments.

Cooperative Simultaneous Localization and Synchronization in Mobile Agent Networks

Cooperative localization in agent networks based on interagent time-of-flight measurements is closely related to synchronization. To leverage this relation, we propose a Bayesian factor graph framework for cooperative simultaneous localization and synchronization (CoSLAS). This framework is suited to mobile agents and time-varying local clock parameters. Building on the CoSLAS factor graph, we develop a distributed (decentralized) belief propagation algorithm for CoSLAS in the practically important case of an affine clock model and asymmetric time stamping. Our algorithm allows for real-time operation and is suitable for a time-varying network connectivity. To achieve high accuracy at reduced complexity and communication cost, the algorithm combines particle implementations with parametric message representations and takes advantage of a conditional independence property. Simulation results demonstrate the good performance of the proposed algorithm in a challenging scenario with time-varying network connectivity.

Analytical SLAM without linearization

This paper solves the classical problem of simultaneous localization and mapping (SLAM) in a fashion that avoids linearized approximations altogether. Based on the creation of virtual synthetic measurements, the algorithm uses a linear time-varying Kalman observer, bypassing errors and approximations brought by the linearization process in traditional extended Kalman filtering SLAM. Convergence rates of the algorithm are established using contraction analysis. Different combinations of sensor information can be exploited, such as bearing measurements, range measurements, optical flow, or time-to-contact. SLAM-DUNK, a more advanced version of the algorithm in global coordinates, exploits the conditional independence property of the SLAM problem, decoupling the covariance matrices between different landmarks and reducing computational complexity to O(n). As illustrated in simulations, the proposed algorithm can solve SLAM problems in both 2D and 3D scenarios with guaranteed convergence rates in a full nonlinear context.

Complexity Reduction and Sensitivity Analysis in Road Probabilistic Safety Assessment Bayesian Network Models

AbstractThis article is concerned with improving some existing methods for probabilistic safety analysis of roads and highways. After a quick review of a Bayesian network model, in which special attention is devoted to human error and all safety related items or elements existing along the road are considered, important problems are dealt with and some solutions provided. This includes: (1) a new and general method for a detailed description of the conditional probabilities of variables given their parents leading to closed‐form formulas, (2) a partitioning technique that allows us to reduce drastically the CPU time required for the calculations, based on dividing the Bayesian network into very small subnetworks using the conditional independence property and leading to a reduced complexity, which is linear in the number of variables or road length instead of the nonlinear character of alternative methods, and (3) a range sensitivity analysis method, which takes advantage of the partitioning technique and is superior to a local sensitivity analysis. Finally, some real examples are provided to show the usefulness of the proposed methodologies to assess the safety of highways or conventional roads.

Two-stage filtration algorithm with interframe causal processing for multichannel image with presence of uncorrelated noise

Introduction. When solving a number of practical problems the usage of multichannel images is common practice. Multichannel of this data permits or increases the efficiency of solving the problem, or allows to obtain useful information, which in principle cannot be extracted from the single-channel images. One of the main types of noise occurring in a multichannel image is uncorrelated noise. The optimal image filtering algorithms require enormous computational cost. Therefore, the important practical value is the synthesis of multi-channel image filtering algorithms, providing the required performance indicators at moderate computational cost. Theoretical results. Using conditional independence properties, the expression for the a posteriori probability density of pixels at the two-stage multi-channel image filtration with causal frame processing in the presence of uncorrelated noise is obtained. Gaussian algorithm for determining the estimates of image pixels and error variance estimation with causal intra and inter-frame processing is obtained in the case of multichannel image. Experimental results. The developed algorithm for considered example allows increasing the filtration accuracy of the sequence of homogeneous Gaussian images on a 20% - 45% compared to inter-frame averaging algorithm. Conclusion. Optimal and quasi-two-stage multi-channel image filtration algorithms were synthesized. In algorithms the first stage is one-dimensional causal filtration along each of the coordinates, and the second is the union of the results. These algorithms allow reducing the computational cost in comparison with the optimal algorithm and thus ensuring acceptable accuracy characteristics.

On the Intersection Property of Conditional Independence and its Application to Causal Discovery

AbstractThis work investigates the intersection property of conditional independence. It states that for random variables $$A,B,C$$ and X we have that $$X \bot \bot A{\kern 1pt} {\kern 1pt} |{\kern 1pt} {\kern 1pt} B,C$$ and $$X\, \bot \bot\, B{\kern 1pt} {\kern 1pt} |{\kern 1pt} {\kern 1pt} A,C$$ implies $$X\, \bot \bot\, (A,B){\kern 1pt} {\kern 1pt} |{\kern 1pt} {\kern 1pt} C$$. Here, “$$ \bot \bot $$” stands for statistical independence. Under the assumption that the joint distribution has a density that is continuous in $$A,B$$ and C, we provide necessary and sufficient conditions under which the intersection property holds. The result has direct applications to causal inference: it leads to strictly weaker conditions under which the graphical structure becomes identifiable from the joint distribution of an additive noise model.

Local percolative properties of the vacant set of random interlacements with small intensity

Un entrelac aleatoire au niveau $u$ est une famille a un parametre de sous-ensembles connexes aleatoires de $\mathbb{Z}^{d}$, $d\geq3$, introduit dans (Ann. Math. 171 (2010) 2039–2087). Son complementaire, l’ensemble vacant au niveau $u$, possede une transition de percolation non triviale en $u$, comme il a ete montre dans (Comm. Pure Appl. Math. 62 (2009) 831–858) et (Ann. Math. 171 (2010) 2039–2087). La composante connexe infinie, lorsqu’elle existe, est presque surement unique, voir (Ann. Appl. Probab. 19 (2009) 454–466). Dans ce papier, nous etudions les proprietes percolatives locales de l’ensemble vacant au niveau $u$ en toutes dimensions $d\geq3$ et pour un petit parametre d’intensite $u$. Nous donnons une borne exponentielle tendue sur la probabilite qu’un grand (hyper)cube contienne deux composantes macroscopiques distinctes de l’ensemble vacant au niveau $u$. Nos resultats impliquent qu’il est peu probable que les composantes connexes finies de l’ensemble vacant au niveau $u$ soient grandes. Ces resultats ont ete prouves dans (Probab. Theory Related Fields 150 (2011) 529–574) pour $d\geq5$. Notre approche est differente (de celle de (Probab. Theory Related Fields 150 (2011) 529–574)) et est valide pour $d\geq3$. L’un des ingredients principaux de la preuve est une certaine propriete d’independence conditionelle des entrelacs aleatoires, qui est interessante en elle-meme.