Probabilistic Inference Tasks Research Articles

DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization

Most reinforcement learning algorithms seek a single optimal strategy that solves a given task. However, it can often be valuable to learn a diverse set of solutions, for instance, to make an agent's interaction with users more engaging, or improve the robustness of a policy to an unexpected perturbance. We propose Diversity-Guided Policy Optimization (DGPO), an on-policy algorithm that discovers multiple strategies for solving a given task. Unlike prior work, it achieves this with a shared policy network trained over a single run. Specifically, we design an intrinsic reward based on an information-theoretic diversity objective. Our final objective alternately constraints on the diversity of the strategies and on the extrinsic reward. We solve the constrained optimization problem by casting it as a probabilistic inference task and use policy iteration to maximize the derived lower bound. Experimental results show that our method efficiently discovers diverse strategies in a wide variety of reinforcement learning tasks. Compared to baseline methods, DGPO achieves comparable rewards, while discovering more diverse strategies, and often with better sample efficiency.

Computational complexity of normalizing constants for the product of determinantal point processes

We consider the product of determinantal point processes (DPPs), a point process whose probability mass is proportional to the product of principal minors of multiple matrices, as a natural, promising generalization of DPPs. We study the computational complexity of computing its normalizing constant, which is among the most essential probabilistic inference tasks. Our complexity-theoretic results (almost) rule out the existence of efficient algorithms for this task unless the input matrices are forced to have favorable structures. In particular, we prove the following:•Computing ∑Sdet⁡(AS,S)p exactly for every (fixed) positive even integer p is UP-hard and Mod3P-hard, which gives a negative answer to an open question posed by Kulesza and Taskar [51].•∑Sdet⁡(AS,S)det⁡(BS,S)det⁡(CS,S) is NP-hard to approximate within a factor of 2O(|I|1−ε) or 2O(n1/ε) for any ε>0, where |I| is the input size and n is the order of the input matrix. This result is stronger than the #P-hardness for the case of two matrices derived by Gillenwater [36].•There exists a kO(k)nO(1)-time algorithm for computing ∑Sdet⁡(AS,S)det⁡(BS,S), where k is the maximum rank of A and B or the treewidth of the graph formed by nonzero entries of A and B. Such parameterized algorithms are said to be fixed-parameter tractable. These results can be extended to the fixed-size case. Further, we present two applications of fixed-parameter tractable algorithms given a matrix A of treewidth w:•We can compute a 2n2p−1-approximation to ∑Sdet⁡(AS,S)p for any fractional number p>1 in wO(wp)nO(1) time.•We can find a 2n-approximation to unconstrained maximum a posteriori inference in wO(wn)nO(1) time.

Inference and Learning with Model Uncertainty in Probabilistic Logic Programs

An issue that has so far received only limited attention in probabilistic logic programming (PLP) is the modelling of so-called epistemic uncertainty, the uncertainty about the model itself. Accurately quantifying this model uncertainty is paramount to robust inference, learning and ultimately decision making. We introduce BetaProbLog, a PLP language that can model epistemic uncertainty. BetaProbLog has sound semantics, an effective inference algorithm that combines Monte Carlo techniques with knowledge compilation, and a parameter learning algorithm. We empirically outperform state-of-the-art methods on probabilistic inference tasks in second-order Bayesian networks, digit classification and discriminative learning in the presence of epistemic uncertainty.

On Cost-Efficient Learning of Data Dependency

In this paper, we consider the problem of learning a tree graph structure that represents the statistical data dependency among nodes for a set of data samples generated by nodes, which provides the basic structure to perform a probabilistic inference task. Inference in the data graph includes marginal inference and maximum a posteriori (MAP) estimation, and belief propagation (BP) is a commonly used algorithm to compute the marginal distribution of nodes via message-passing, incurring non-negligible amount of communication cost. We inevitably have the trade-off between the inference accuracy and the message-passing cost because the learned structure of data dependency and physical connectivity graph are often highly different. In this paper, we formalize this trade-off in an optimization problem which outputs the data dependency graph that jointly considers learning accuracy and message-passing costs. We focus on two popular implementations of BP, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ASYNC-BP</monospace> and <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SYNC-BP</monospace> , which have different message-passing mechanisms and cost structures. In <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ASYNC-BP</monospace> , we propose a polynomial-time learning algorithm that is optimal, motivated by finding a maximum weight spanning tree of a complete graph. In <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SYNC-BP</monospace> , we prove the NP-hardness of the problem and propose a greedy heuristic. For both BP implementations, we quantify how the error probability that the learned cost-efficient data graph differs from the ideal one decays as the number of data samples grows, using the large deviation principle, which provides a guideline on how many samples are necessary to obtain a certain trade-off. We validate our theoretical findings through extensive simulations, which confirms that it has a good match.

A Bayesian Modeling Approach to Situated Design of Personalized Soundscaping Algorithms

Effective noise reduction and speech enhancement algorithms have great potential to enhance lives of hearing aid users by restoring speech intelligibility. An open problem in today’s commercial hearing aids is how to take into account users’ preferences, indicating which acoustic sources should be suppressed or enhanced, since they are not only user-specific but also depend on many situational factors. In this paper, we develop a fully probabilistic approach to “situated soundscaping”, which aims at enabling users to make on-the-spot (“situated”) decisions about the enhancement or suppression of individual acoustic sources. The approach rests on a compact generative probabilistic model for acoustic signals. In this framework, all signal processing tasks (source modeling, source separation and soundscaping) are framed as automatable probabilistic inference tasks. These tasks can be efficiently executed using message passing-based inference on factor graphs. Since all signal processing tasks are automatable, the approach supports fast future model design cycles in an effort to reach commercializable performance levels. The presented results show promising performance in terms of SNR, PESQ and STOI improvements in a situated setting.

From perception to inference: Utilization of probabilities as decision weights in children

In a probabilistic inference task (three probabilistic cues predict outcomes for two options), we examined decisions from 233 children (5–6 vs. 9–10 years). Contiguity (low vs. high; i.e., position of probabilistic information far vs. close to options) and demand for selectivity (low vs. high; i.e., showing predictions of desired vs. desired and undesired outcomes) were varied as configural aspects of the presentation format. Probability utilization was measured by the frequency of following the predictions of the highest validity cue in choice. High contiguity and low demand for selectivity strongly and moderately increased probability utilization, respectively. Children are influenced by presentation format when using probabilities as decision weights. They benefit from perception-like presentations that present probabilities and options as compounds.

A Comparative Study of Gamma Markov Chains for Temporal Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) has become a well-established class of methods for the analysis of non-negative data. In particular, a lot of effort has been devoted to probabilistic NMF, namely estimation or inference tasks in probabilistic models describing the data, based for example on Poisson or exponential likelihoods. When dealing with time series data, several works have proposed to model the evolution of the activation coefficients as a non-negative Markov chain, most of the time in relation with the Gamma distribution, giving rise to so-called temporal NMF models. In this paper, we review four Gamma Markov chains of the NMF literature, and show that they all share the same drawback: the absence of a well-defined stationary distribution. We then introduce a fifth process, an overlooked model of the time series literature named BGAR(1), which overcomes this limitation. These temporal NMF models are then compared in a MAP framework on a prediction task, in the context of the Poisson likelihood.

Application of junction tree clustering methods for solving dynamic Bayesian networks probabilistic inference tasks

The article covers clustering algorithms for dynamic Bayesian networks based on the construction of join tree. Investigated problems associated with simplifying the topology of dynamic Bayesian networks using clustering methods and consider various semantic approaches of junction trees construction. The paper presents a junction tree constructing algorithm for the dynamic Bayesian network that takes into account the transition and perception models applied to the process of network formation. Analyzed The relationship between the complete joint probability distribution for the dynamic Bayesian network and the complete joint distribution obtained for the junction tree. It is shown that the required complete joint distribution of the junction tree for the dynamic Bayesian network can be obtained as a product of local probability distributions for each node associated with the resulting junction tree, and proved that this distribution will be equivalent to the probability distribution of the original dynamic Bayesian network. Introduced the main structural approaches to the construction of junction tree for the discrete dynamic Bayesian network and inspected the use of junction tree for solving similar problems for continuous dynamic Bayesian networks with Gaussian and exponential distribution functions of network variables.

Autistic traits influence the strategic diversity of information sampling: Insights from two-stage decision models.

Information sampling can reduce uncertainty in future decisions but is often costly. To maximize reward, people need to balance sampling cost and information gain. Here we aimed to understand how autistic traits influence the optimality of information sampling and to identify the particularly affected cognitive processes. Healthy human adults with different levels of autistic traits performed a probabilistic inference task, where they could sequentially sample information to increase their likelihood of correct inference and may choose to stop at any moment. We manipulated the cost and evidence associated with each sample and compared participants’ performance to strategies that maximize expected gain. We found that participants were overall close to optimal but also showed autistic-trait-related differences. Participants with higher autistic traits had a higher efficiency of winning rewards when the sampling cost was zero but a lower efficiency when the cost was high and the evidence was more ambiguous. Computational modeling of participants’ sampling choices and decision times revealed a two-stage decision process, with the second stage being an optional second thought. Participants may consider cost in the first stage and evidence in the second stage, or in the reverse order. The probability of choosing to stop sampling at a specific stage increases with increasing cost or increasing evidence. Surprisingly, autistic traits did not influence the decision in either stage. However, participants with higher autistic traits inclined to consider cost first, while those with lower autistic traits considered cost or evidence first in a more balanced way. This would lead to the observed autistic-trait-related advantages or disadvantages in sampling optimality, depending on whether the optimal sampling strategy is determined only by cost or jointly by cost and evidence.

Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, Bayesian inference, and optimization

Probabilistic programming provides a convenient lingua franca for writing succinct and rigorous descriptions of probabilistic models and inference tasks. Several probabilistic programming languages, including Anglican, Church or Hakaru, derive their expressiveness from a powerful combination of continuous distributions, conditioning, and higher-order functions. Although very important for practical applications, these features raise fundamental challenges for program semantics and verification. Several recent works offer promising answers to these challenges, but their primary focus is on foundational semantics issues. In this paper, we take a step further by developing a suite of logics, collectively named PPV for proving properties of programs written in an expressive probabilistic higher-order language with continuous sampling operations and primitives for conditioning distributions. Our logics mimic the comfortable reasoning style of informal proofs using carefully selected axiomatizations of key results from probability theory. The versatility of our logics is illustrated through the formal verification of several intricate examples from statistics, probabilistic inference, and machine learning. We further show expressiveness by giving sound embeddings of existing logics. In particular, we do this in a parametric way by showing how the semantics idea of (unary and relational) ⊤⊤-lifting can be internalized in our logics. The soundness of PPV follows by interpreting programs and assertions in quasi-Borel spaces (QBS), a recently proposed variant of Borel spaces with a good structure for interpreting higher order probabilistic programs.

Attractor-like Dynamics in Belief Updating in Schizophrenia.

Subjects with a diagnosis of schizophrenia (Scz) overweight unexpected evidence in probabilistic inference: such evidence becomes "aberrantly salient." A neurobiological explanation for this effect is that diminished synaptic gain (e.g., hypofunction of cortical NMDARs) in Scz destabilizes quasi-stable neuronal network states (or "attractors"). This attractor instability account predicts that (1) Scz would overweight unexpected evidence but underweight consistent evidence, (2) belief updating would be more vulnerable to stochastic fluctuations in neural activity, and (3) these effects would correlate. Hierarchical Bayesian belief updating models were tested in two independent datasets (n = 80 male and n = 167 female) comprising human subjects with Scz, and both clinical and nonclinical controls (some tested when unwell and on recovery) performing the "probability estimates" version of the beads task (a probabilistic inference task). Models with a standard learning rate, or including a parameter increasing updating to "disconfirmatory evidence," or a parameter encoding belief instability were formally compared. The "belief instability" model (based on the principles of attractor dynamics) had most evidence in all groups in both datasets. Two of four parameters differed between Scz and nonclinical controls in each dataset: belief instability and response stochasticity. These parameters correlated in both datasets. Furthermore, the clinical controls showed similar parameter distributions to Scz when unwell, but were no different from controls once recovered. These findings are consistent with the hypothesis that attractor network instability contributes to belief updating abnormalities in Scz, and suggest that similar changes may exist during acute illness in other psychiatric conditions.SIGNIFICANCE STATEMENT Subjects with a diagnosis of schizophrenia (Scz) make large adjustments to their beliefs following unexpected evidence, but also smaller adjustments than controls following consistent evidence. This has previously been construed as a bias toward "disconfirmatory" information, but a more mechanistic explanation may be that in Scz, neural firing patterns ("attractor states") are less stable and hence easily altered in response to both new evidence and stochastic neural firing. We model belief updating in Scz and controls in two independent datasets using a hierarchical Bayesian model, and show that all subjects are best fit by a model containing a belief instability parameter. Both this and a response stochasticity parameter are consistently altered in Scz, as the unstable attractor hypothesis predicts.

BEYOND BLACK-BOXES IN BAYESIAN INVERSE PROBLEMS AND MODEL VALIDATION: APPLICATIONS IN SOLID MECHANICS OF ELASTOGRAPHY

The present paper is motivated by one of the most fundamental challenges in inverse problems, that of quantifying model discrepancies and errors. While significant strides have been made in calibrating model parameters, the overwhelming majority of pertinent methods is based on the assumption of a perfect model. Motivated by problems in solid mechanics which, as all problems in continuum thermodynamics, are described by conservation laws and phenomenological constitutive closures, we argue that in order to quantify model uncertainty in a physically meaningful manner, one should break open the black-box forward model. In particular we propose formulating an undirected probabilistic model that explicitly accounts for the governing equations and their validity. This recasts the solution of both forward and inverse problems as probabilistic inference tasks where the problem's state variables should not only be compatible with the data but also with the governing equations as well. Even though the probability densities involved do not contain any black-box terms, they live in much higher-dimensional spaces. In combination with the intractability of the normalization constant of the undirected model employed, this poses significant challenges which we propose to address with a linearly-scaling, double-layer of Stochastic Variational Inference. We demonstrate the capabilities and efficacy of the proposed model in synthetic forward and inverse problems (with and without model error) in elastography.

When knowledge activated from memory intrudes on probabilistic inferences from description - the case of stereotypes

To make decisions in probabilistic inference tasks, individuals integrate relevant information partly in an automatic manner. Thereby, potentially irrelevant stimuli that are additionally presented can intrude on the decision process (e.g., Söllner, Bröder, Glöckner, & Betsch, 2014). We investigate whether such an intrusion effect can also be caused by potentially irrelevant or even misleading knowledge activated from memory. In four studies that combine a standard information board paradigm from decision research with a standard manipulation from social psychology, we investigate the case of stereotypes and demonstrate that stereotype knowledge can yield intrusion biases in probabilistic inferences from description. The magnitude of these biases increases with stereotype accessibility and decreases with a clarification of the rational solution.

Parallel Bayesian ARTMAP and Its OpenCL Implementation

The Bayesian ARTMAP neural network, introduced by Vigdor and Lerner, is an incremental learning algorithm which can efficiently process massive datasets for classification, regression, and probabilistic inference tasks. We introduce the parallelized version of the BA neural network and implement it in OpenCL. Our implementation runs on both multi-core CPUs and GPUs architectures. We test the Parallel Bayesian ARTMAP on several classification and regression benchmarks focusing on speedup and scalability. In some cases, the parallel BA runs by an order of magnitude faster than the sequential implementation. Our implementation has the potential to scale for OpenCL devices with increasing number of compute units.

A probabilistic approach to demixing odors.

The olfactory system faces a hard problem: on the basis of noisy information from olfactory receptor neurons (the neurons that transduce chemicals to neural activity), it must figure out which odors are present in the world. Odors almost never occur in isolation, and different odors excite overlapping populations of olfactory receptor neurons, so the central challenge of the olfactory system is to demix its input. Because of noise and the large number of possible odors, demixing is fundamentally a probabilistic inference task. We propose that the early olfactory system uses approximate Bayesian inference to solve it. The computations involve a dynamical loop between the olfactory bulb and the piriform cortex, with cortex explaining incoming activity from the olfactory receptor neurons in terms of a mixture of odors. The model is compatible with known anatomy and physiology, including pattern decorrelation, and it performs better than other models at demixing odors.

The ticking time bomb: Using eye-tracking methodology to capture attentional processing during gradual time constraints.

Many decisions are made under suboptimal circumstances, such as time constraints. We examined how different experiences of time constraints affected decision strategies on a probabilistic inference task and whether individual differences in working memory accounted for complex strategy use across different levels of time. To examine information search and attentional processing, we used an interactive eye-tracking paradigm where task information was occluded and only revealed by an eye fixation to a given cell. Our results indicate that although participants change search strategies during the most restricted times, the occurrence of the shift in strategies depends both on how the constraints are applied as well as individual differences in working memory. This suggests that, in situations that require making decisions under time constraints, one can influence performance by being sensitive to working memory and, potentially, by acclimating people to the task time gradually.

Collective Graph Identification

Data describing networks—such as communication networks, transaction networks, disease transmission networks, collaboration networks, etc.—are becoming increasingly available. While observational data can be useful, it often only hints at the actual underlying process that governs interactions and attributes. For example, an email communication network provides insight into its users and their relationships, but is not the same as the “real” underlying social network. In this article, we introduce the problem of graph identification , i.e., discovering the latent graph structure underlying an observed network. We cast the problem as a probabilistic inference task, in which we must infer the nodes, edges, and node labels of a hidden graph, based on evidence. This entails solving several canonical problems in network analysis: entity resolution (determining when two observations correspond to the same entity), link prediction (inferring the existence of links), and node labeling (inferring hidden attributes). While each of these subproblems has been well studied in isolation, here we consider them as a single, collective task. We present a simple, yet novel, approach to address all three subproblems simultaneously. Our approach, which we refer to as C 3 , consists of a collection of Coupled Collective Classifiers that are applied iteratively to propagate inferred information among the subproblems. We consider variants of C 3 using different learning and inference techniques and empirically demonstrate that C 3 is superior, both in terms of predictive accuracy and running time, to state-of-the-art probabilistic approaches on four real problems.

Variational Bayesian strategies for high-dimensional, stochastic design problems

This paper is concerned with a lesser-studied problem in the context of model-based, uncertainty quantification (UQ), that of optimization/design/control under uncertainty. The solution of such problems is hindered not only by the usual difficulties encountered in UQ tasks (e.g. the high computational cost of each forward simulation, the large number of random variables) but also by the need to solve a nonlinear optimization problem involving large numbers of design variables and potentially constraints. We propose a framework that is suitable for a large class of such problems and is based on the idea of recasting them as probabilistic inference tasks. To that end, we propose a Variational Bayesian (VB) formulation and an iterative VB-Expectation-Maximization scheme that is also capable of identifying a low-dimensional set of directions in the design space, along which, the objective exhibits the largest sensitivity. We demonstrate the validity of the proposed approach in the context of two numerical examples involving $\mathcal{O}(10^3)$ random and design variables. In all cases considered the cost of the computations in terms of calls to the forward model was of the order $\mathcal{O}(10^2)$. The accuracy of the approximations provided is assessed by appropriate information-theoretic metrics.

Area-specific information processing in prefrontal cortex during a probabilistic inference task: a multivariate fMRI BOLD time series analysis.

IntroductionDiscriminating spatiotemporal stages of information processing involved in complex cognitive processes remains a challenge for neuroscience. This is especially so in prefrontal cortex whose subregions, such as the dorsolateral prefrontal (DLPFC), anterior cingulate (ACC) and orbitofrontal (OFC) cortices are known to have differentiable roles in cognition. Yet it is much less clear how these subregions contribute to different cognitive processes required by a given task. To investigate this, we use functional MRI data recorded from a group of healthy adults during a “Jumping to Conclusions” probabilistic reasoning task.MethodsWe used a novel approach combining multivariate test statistics with bootstrap-based procedures to discriminate between different task stages reflected in the fMRI blood oxygenation level dependent signal pattern and to unravel differences in task-related information encoded by these regions. Furthermore, we implemented a new feature extraction algorithm that selects voxels from any set of brain regions that are jointly maximally predictive about specific task stages.ResultsUsing both the multivariate statistics approach and the algorithm that searches for maximally informative voxels we show that during the Jumping to Conclusions task, the DLPFC and ACC contribute more to the decision making phase comprising the accumulation of evidence and probabilistic reasoning, while the OFC is more involved in choice evaluation and uncertainty feedback. Moreover, we show that in presumably non-task-related regions (temporal cortices) all information there was about task processing could be extracted from just one voxel (indicating the unspecific nature of that information), while for prefrontal areas a wider multivariate pattern of activity was maximally informative.Conclusions/SignificanceWe present a new approach to reveal the different roles of brain regions during the processing of one task from multivariate activity patterns measured by fMRI. This method can be a valuable tool to assess how area-specific processing is altered in psychiatric disorders such as schizophrenia, and in healthy subjects carrying different genetic polymorphisms.

Verbalization of Decision Strategies in Multiple‐Cue Probabilistic Inference

AbstractIn multiple‐cue probabilistic inference, people choose between alternatives based on several cues, each of which is differentially associated with an alternative's overall value. Various strategies have been proposed for probabilistic inference (e.g., weighted additive, tally, and take‐the‐best). These strategies differ in how many cue values they require to enact and in how they weight each cue. Do decision makers actually use any of these strategies? Ways to investigate this question include analyzing people's choices and the cues that they reveal. However, different strategies often predict the same decisions, and search behavior says nothing about whether or how people use the information that they acquire. In this research, we attempt to elucidate which strategies participants use in a multiple‐cue probabilistic inference task by examining verbal protocols, a high‐density source of process data. The promise of verbal data is in their utility for testing detailed information processing models. To that end, we apply protocol analysis in conjunction with computational simulations. We find converging evidence across outcome measures, search measures, and verbal reports that most participants use simplifying heuristics, namely take‐the‐best. Copyright © 2015 John Wiley &amp; Sons, Ltd.