Presence Of Hidden Variables Research Articles

Causal Structure Learning with Conditional and Unique Information Groups-Decomposition Inequalities.

The causal structure of a system imposes constraints on the joint probability distribution of variables that can be generated by the system. Archetypal constraints consist of conditional independencies between variables. However, particularly in the presence of hidden variables, many causal structures are compatible with the same set of independencies inferred from the marginal distributions of observed variables. Additional constraints allow further testing for the compatibility of data with specific causal structures. An existing family of causally informative inequalities compares the information about a set of target variables contained in a collection of variables, with a sum of the information contained in different groups defined as subsets of that collection. While procedures to identify the form of these groups-decomposition inequalities have been previously derived, we substantially enlarge the applicability of the framework. We derive groups-decomposition inequalities subject to weaker independence conditions, with weaker requirements in the configuration of the groups, and additionally allowing for conditioning sets. Furthermore, we show how constraints with higher inferential power may be derived with collections that include hidden variables, and then converted into testable constraints using data processing inequalities. For this purpose, we apply the standard data processing inequality of conditional mutual information and derive an analogous property for a measure of conditional unique information recently introduced to separate redundant, synergistic, and unique contributions to the information that a set of variables has about a target.

The DeCAMFounder: nonlinear causal discovery in the presence of hidden variables

Abstract Many real-world decision-making tasks require learning causal relationships between a set of variables. Traditional causal discovery methods, however, require that all variables are observed, which is often not feasible in practical scenarios. Without additional assumptions about the unobserved variables, it is not possible to recover any causal relationships from observational data. Fortunately, in many applied settings, additional structure among the confounders can be expected. In particular, pervasive confounding is commonly encountered and has been utilised for consistent causal estimation in linear causal models. In this article, we present a provably consistent method to estimate causal relationships in the nonlinear, pervasive confounding setting. The core of our procedure relies on the ability to estimate the confounding variation through a simple spectral decomposition of the observed data matrix. We derive a DAG score function based on this insight, prove its consistency in recovering a correct ordering of the DAG, and empirically compare it to previous approaches. We demonstrate improved performance on both simulated and real datasets by explicitly accounting for both confounders and nonlinear effects.

Probabilistic approaches development for optimizing learning parameters procedure for testing models via the expectation-maximization method

The development and improvement of training algorithms is of special scientific and practical interest in the field of application testing. In the context of expanding the field of application of probabilistic models of Bayesian networks for solving testing problems, there is a need for stochastic assessment of hidden parameters, as well as the formation of a complete joint distribution across all variables. The growth of the dimension of probabilistic models for solving various scientific and applied problems requires the development of optimal procedures for training parameters in accordance with existing algorithms, as well as the development of new solutions. The subject of the study is testing models used to represent probabilistic relationships between individual test modules, which allow timely configuration and use of test generation to detect certain groups of program errors. The aim of the study is to develop effective stochastic algorithms for training parameters of test models, which allow you to obtain more accurate values ​ ​ of a priori distribution of parameters. The study proposes methods and algorithms for training parameters of testing models based on a stochastic gradient in combination with the Monte Carlo method using Markov chains to form a preliminary distribution over all hidden variables in accordance with the available set of training data. The results of the study showed the viability of theoretical assumptions and made it possible to implement the optimal algorithm for solving the problem of training probabilistic models of testing web applications. The proposed methods and algorithms are constructive and make it possible to increase the accuracy of training parameters of test models in the presence of hidden variables.

Learning Graphs From Smooth and Graph-Stationary Signals With Hidden Variables

Network-topology inference from (vertex) signal observations is a prominent problem across data-science and engineering disciplines. Most existing schemes assume that observations from all nodes are available, but in many practical environments, only a subset of nodes is accessible. A natural (and sometimes effective) approach is to disregard the role of unobserved nodes, but this ignores latent network effects, deteriorating the quality of the estimated graph. Differently, this paper investigates the problem of inferring the topology of a network from nodal observations while taking into account <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the presence of hidden (latent) variables</i> . Our schemes assume the number of observed nodes is considerably larger than the number of hidden variables and build on recent graph signal processing models to relate the signals and the underlying graph. Specifically, we go beyond classical correlation and partial correlation approaches and assume that the signals are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">smooth</i> and/or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stationary</i> in the sought graph. The assumptions are codified into different constrained optimization problems, with the presence of hidden variables being explicitly taken into account. Since the resulting problems are ill-conditioned and non-convex, the block matrix structure of the proposed formulations is leveraged and suitable convex-regularized relaxations are presented. Numerical experiments over synthetic and real-world datasets showcase the performance of the developed methods and compare them with existing alternatives.

Conditional probabilities via line arrangements and point configurations

We study the connection between probability distributions satisfying certain conditional independence (CI) constraints, and point and line arrangements in incidence geometry. To a family of CI statements, we associate a polynomial ideal whose algebraic invariants are encoded in a hypergraph. The primary decompositions of these ideals give a characterization of the distributions satisfying the original CI statements. Classically, these ideals are generated by 2-minors of a matrix of variables; however, in the presence of hidden variables, they contain higher degree minors. This leads to the study of the structure of determinantal hypergraph ideals whose decompositions can be understood in terms of point and line configurations in the projective space.

Robust Causal Structure Learning with Some Hidden Variables

SummaryWe introduce a new method to estimate the Markov equivalence class of a directed acyclic graph (DAG) in the presence of hidden variables, in settings where the underlying DAG among the observed variables is sparse, and there are a few hidden variables that have a direct effect on many of the observed variables. Building on the so-called low rank plus sparse framework, we suggest a two-stage approach which first removes the effect of the hidden variables and then estimates the Markov equivalence class of the underlying DAG under the assumption that there are no remaining hidden variables. This approach is consistent in certain high dimensional regimes and performs favourably when compared with the state of the art, in terms of both graphical structure recovery and total causal effect estimation.

Sparse model selection in the highly under-sampled regime

We propose a method for recovering the structure of a sparse undirected graphical model when very few samples are available. The method decides about the presence or absence of bonds between pairs of variable by considering one pair at a time and using a closed form formula, analytically derived by calculating the posterior probability for every possible model explaining a two body system using Jeffreys prior. The approach does not rely on the optimization of any cost functions and consequently is much faster than existing algorithms. Despite this time and computational advantage, numerical results show that for several sparse topologies the algorithm is comparable to the best existing algorithms, and is more accurate in the presence of hidden variables. We apply this approach to the analysis of US stock market data and to neural data, in order to show its efficiency in recovering robust statistical dependencies in real data with non-stationary correlations in time and/or space.

Inference of hidden variables in systems of differential equations with genetic programming

The data-driven modeling of dynamical systems is an important scientific activity, and many studies have applied genetic programming (GP) to the task of automatically constructing such models in the form of systems of ordinary differential equations (ODEs). These previous studies assumed that data measurements were available for all variables in the system, whereas in real-world settings, it is typically the case that one or more variables are unmeasured or "hidden." Here, we investigate the prospect of automatically constructing ODE models of dynamical systems from time series data with GP in the presence of hidden variables. Several examples with both synthetic and physical systems demonstrate the unique challenges of this problem and the circumstances under which it is possible to reverse-engineer both the form and parameters of ODE models with hidden variables.

The Latent Maximum Entropy Principle

We present an extension to Jaynes’ maximum entropy principle that incorporates latent variables. The principle oflatent maximum entropywe propose is different from both Jaynes’ maximum entropy principle and maximum likelihood estimation, but can yield better estimates in the presence of hidden variables and limited training data. We first show that solving for a latent maximum entropy model poses a hard nonlinear constrained optimization problem in general. However, we then show that feasible solutions to this problem can be obtained efficiently for the special case of log-linear models---which forms the basis for an efficient approximation to the latent maximum entropy principle. We derive an algorithm that combines expectation-maximization with iterative scaling to produce feasible log-linear solutions. This algorithm can be interpreted as an alternating minimization algorithm in the information divergence, and reveals an intimate connection between the latent maximum entropy and maximum likelihood principles. To select a final model, we generate a series of feasible candidates, calculate the entropy of each, and choose the model that attains the highest entropy. Our experimental results show that estimation based on the latent maximum entropy principle generally gives better results than maximum likelihood when estimating latent variable models on small observed data samples.

Adaptive Distributed Source Coding

We consider distributed source coding in the presence of hidden variables that parameterize the statistical dependence among sources. We derive the Slepian-Wolf bound and devise coding algorithms for a block-candidate model of this problem. The encoder sends, in addition to syndrome bits, a portion of the source to the decoder uncoded as doping bits. The decoder uses the sum-product algorithm to simultaneously recover the source symbols and the hidden statistical dependence variables. We also develop novel techniques based on density evolution (DE) to analyze the coding algorithms. We experimentally confirm that our DE analysis closely approximates practical performance. This result allows us to efficiently optimize parameters of the algorithms. In particular, we show that the system performs close to the Slepian-Wolf bound when an appropriate doping rate is selected. We then apply our coding and analysis techniques to a reduced-reference video quality monitoring system and show a bit rate saving of about 75% compared with fixed-length coding.

Estimation of causal effects using linear non-Gaussian causal models with hidden variables

The task of estimating causal effects from non-experimental data is notoriously difficult and unreliable. Nevertheless, precisely such estimates are commonly required in many fields including economics and social science, where controlled experiments are often impossible. Linear causal models (structural equation models), combined with an implicit normality (Gaussianity) assumption on the data, provide a widely used framework for this task. We have recently described how non-Gaussianity in the data can be exploited for estimating causal effects. In this paper we show that, with non-Gaussian data, causal inference is possible even in the presence of hidden variables (unobserved confounders), even when the existence of such variables is unknown a priori. Thus, we provide a comprehensive and complete framework for the estimation of causal effects between the observed variables in the linear, non-Gaussian domain. Numerical simulations demonstrate the practical implementation of the proposed method, with full Matlab code available for all simulations.

Contextualism and a puzzle about seeing

Contextualist solutions to skeptical puzzles have recently been subjected to various criticisms. In this paper, I will defend contextualism against an objection pressed by Jason Stanley. In the course of doing so, I argue that either semantic context-sensitivity is very widespread in natural language, or else Stanley's ``binding'' test for the presence of hidden variables in logical form is not a good test.

The Light of Quantum Mechanics

Although classical probability theory, as it is encapsulated in the axioms of Kolmogorov and in his criterion for the independence of two events, can consistently be employed in quantum mechanics, this can only be accomplished at an exorbitant price. By considering first the classic two-slit experiment, and then the passage of one photon through three polarizers, the applicability of Kolmogorov's last axiom is called into question, but the standard rebuff of the Copenhagen interpretation is shown to be adequate to this challenge. In the EPR experiment of Aspect, and the violation of the Bell inequalities, the matter is more delicate: it is not directly Kolmogorov's last axiom, but rather his criterion of independence that is at issue. It is explained how two events with space-like separation cannot be independent in Kolmogorov's sense, even in the presence of hidden variables. The escape route of supposing the variables to be nonlocal, with a heavy metaphysical ballast of holism, which however is cosmically censored to prevent superluminal information transfer, has all the trappings of an ad hoc makeshift. The adoption of quantum mechanical probability, which does not obey the rules of Kolmogorov, but does survive empirical testing in terms of relative frequencies of events, is more economical. The solution is simple: correlations obey the rules of quantum mechanics and probability is a theory laden concept that is tested by, but not defined in terms of, the relative frequency of selected classes of events. Quantum mechanics appears to be nonlocal only if one tries to force it into a classical, realist straitjacket. However, this is a shortcoming of the straitjacket: it is not a property of quantum mechanics. That follows from the fact that a Bell telephone does not work!