Probabilistic Inference Problem Research Articles

Using the Fuzzy Version of the Pearl’s Algorithm for Environmental Risk Assessment Tasks

In risk assessment, numerous subfactors influence the probabilities of the main factors. These main factors reflect adverse outcomes, which are essential in risk assessment. A Bayesian network can model the entire set of subfactors and their interconnections. To assess the probabilities of all possible states of the main factors (adverse consequences), complete information about the probabilities of all relevant subfactor states in the network nodes must be utilized. This is a typical task of probabilistic inference. The algorithm proposed by J. Pearl is widely used for point estimates of relevant probabilities. However, in many practical problems, including environmental risk assessment, it is not possible to assign crisp probabilities for relevant events due to the lack of sufficient statistical data. In such situations, expert assignment of probabilities is widely used. Uncertainty in expert assessments can be successfully modeled using triangular fuzzy numbers. That is why this article proposes a fuzzy version of this algorithm, which can solve the problem of probabilistic inference on a Bayesian network when the initial probability values are given as triangular fuzzy numbers.

Learning to Learn With Variational Inference for Cross-Domain Image Classification

Learning models that can generalize to previously unseen domains to which we have no access is a fundamental yet challenging problem in machine learning. In this paper, we propose meta variational inference (MetaVI), a variational Bayesian framework of meta-learning for cross domain image classification. Within the meta learning setting, MetaVI is derived to learn a probabilistic latent variable model by maximizing a meta evidence lower bound (Meta ELBO) for knowledge transfer across domains. To enhance the discriminative ability of the model, we further introduce a Wasserstein distance based constraint to the variational objective, leading to the Wasserstein MetaVI, which largely improves classification performance. By casting into a probabilistic inference problem, MetaVI offers the first, principled variational meta-learning framework for cross domain learning. In addition, we collect a new visual recognition dataset to contribute a more challenging benchmark for cross domain learning, which will be released to the public. Extensive experimental evaluation and ablation studies on four benchmarks show that our Wasserstein MetaVI achieves new state-of-the-art performance and surpasses previous methods, demonstrating its great effectiveness.

The case against probabilistic inference: a new deterministic theory of 3D visual processing.

How the brain derives 3D information from inherently ambiguous visual input remains the fundamental question of human vision. The past two decades of research have addressed this question as a problem of probabilistic inference, the dominant model being maximum-likelihood estimation (MLE). This model assumes that independent depth-cue modules derive noisy but statistically accurate estimates of 3D scene parameters that are combined through a weighted average. Cue weights are adjusted based on the system representation of each module's output variability. Here I demonstrate that the MLE model fails to account for important psychophysical findings and, importantly, misinterprets the just noticeable difference, a hallmark measure of stimulus discriminability, to be an estimate of perceptual uncertainty. I propose a new theory, termed Intrinsic Constraint, which postulates that the visual system does not derive the most probable interpretation of the visual input, but rather, the most stable interpretation amid variations in viewing conditions. This goal is achieved with the Vector Sum model, which represents individual cue estimates as components of a multi-dimensional vector whose norm determines the combined output. This model accounts for the psychophysical findings cited in support of MLE, while predicting existing and new findings that contradict the MLE model. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

The engineering skills training process modeling using dynamic bayesian nets

The subject of research in the article is the process of intelligent computer training in engineering skills. The aim is to model the process of teaching engineering skills in intelligent computer training programs through dynamic Bayesian networks. Objectives: To propose an approach to modeling the process of teaching engineering skills. To assess the student competence level by considering the algorithms development skills in engineering tasks and the algorithms implementation ability. To create a dynamic Bayesian network structure for the learning process. To select values for conditional probability tables. To solve the problems of filtering, forecasting, and retrospective analysis. To simulate the developed dynamic Bayesian network using a special Genie 2.0-environment. The methods used are probability theory and inference methods in Bayesian networks. The following results are obtained: the development of a dynamic Bayesian network for the educational process based on the solution of engineering problems is presented. Mathematical calculations for probabilistic inference problems such as filtering, forecasting, and smoothing are considered. The solution of the filtering problem makes it possible to assess the current level of the student's competence after obtaining the latest probabilities of the development of the algorithm and its numerical calculations of the task. The probability distribution of the learning process model is predicted. The number of additional iterations required to achieve the required competence level was estimated. The retrospective analysis allows getting a smoothed assessment of the competence level, which was obtained after the task's previous instance completion and after the computation of new additional probabilities characterizing the two checkpoints implementation. The solution of the described probabilistic inference problems makes it possible to provide correct information about the learning process for intelligent computer training systems. It helps to get proper feedback and to track the student's competence level. The developed technique of the kernel of probabilistic inference can be used as the decision-making model basis for an automated training process. The scientific novelty lies in the fact that dynamic Bayesian networks are applied to a new class of problems related to the simulation of engineering skills training in the process of performing algorithmic tasks.

Learning Branching Heuristics for Propositional Model Counting

Propositional model counting, or #SAT, is the problem of computing the number of satisfying assignments of a Boolean formula. Many problems from different application areas, including many discrete probabilistic inference problems, can be translated into model counting problems to be solved by #SAT solvers. Exact #SAT solvers, however, are often not scalable to industrial size instances. In this paper, we present Neuro#, an approach for learning branching heuristics to improve the performance of exact #SAT solvers on instances from a given family of problems. We experimentally show that our method reduces the step count on similarly distributed held-out instances and generalizes to much larger instances from the same problem family. It is able to achieve these results on a number of different problem families having very different structures. In addition to step count improvements, Neuro# can also achieve orders of magnitude wall-clock speedups over the vanilla solver on larger instances in some problem families, despite the runtime overhead of querying the model.

Information networks fusion based on multi-task coordination

Information networks provide a powerful representation of entities and the relationships between them. Information networks fusion is a technique for information fusion that jointly reasons about entities, links and relations in the presence of various sources. However, existing methods for information networks fusion tend to rely on a single task which might not get enough evidence for reasoning. In order to solve this issue, in this paper, we present a novel model called MC-INFM (information networks fusion model based on multi-task coordination). Different from traditional models, MC-INFM casts the fusion problem as a probabilistic inference problem, and collectively performs multiple tasks (including entity resolution, link prediction and relation matching) to infer the final result of fusion. First, we define the intra-features and the inter-features respectively and model them as factor graphs, which can provide abundant evidence to infer. Then, we use conditional random field (CRF) to learn the weight of each feature and infer the results of these tasks simultaneously by performing the maximum probabilistic inference. Experiments demonstrate the effectiveness of our proposed model.

Rota's Fubini lectures: The first problem

In his 1998 Fubini Lectures, Rota discusses twelve problems in probability that “no one likes to bring up”. The first problem calls for a revision of the notion of a sample space, guided by the belief that mention of sample points in a probabilistic argument is bad form and that a “pointless” foundation of probability should be provided by algebras of random variables.In 1958 Chang introduced MV-algebras to prove the completeness theorem of Łukasiewicz logic Ł∞. The aim of this paper is to show that MV-algebras provide a solution of Rota's first problem.The adjunction between MV-algebras and unital commutative C*-algebras equips every MV-algebra A with a natural ring structure, as advocated by Nelson for algebras of random variables. The closed compact set S(A)⊆[0,1]A of finitely additive probability measures on A (the states of A) coincides with the set of [0,1]-valued functions on A whose finite restrictions are consistent in de Finetti's sense. MV-algebras and Ł∞ thus provide the framework for a generalization (known as ŁIPSAT) of Boole's probabilistic inference problem, and its modern reformulation known as probabilistic satisfiability, PSAT. We construct an affine homeomorphism γA of S(A) onto the weakly compact space of regular Borel probability measures on the maximal spectral space μ(A). The latter is the most general compact Hausdorff space. As a consequence, for every Kolmogorov probability space (Ω,FΩ,P), with FΩ the sigma-algebra of Borel sets of a compact Hausdorff space Ω, and P a regular probability measure on FΩ, there is an MV-algebra A and a state σ of A such that (Ω,FΩ,P)≅(μ(A),Fμ(A),γA(σ)).

Bayesian Cooperative Localization Using Received Signal Strength With Unknown Path Loss Exponent: Message Passing Approaches

We propose a Bayesian framework for the received-signal-strength-based cooperative localization problem with unknown path loss exponent. Our purpose is to infer the marginal posterior of each unknown parameter: the position or the path loss exponent. This probabilistic inference problem is solved using message passing algorithms that update messages and beliefs iteratively. To enable the numerical tractability, we combine the variable discretization and Monte-Carlo-based numerical approximation schemes. To further improve computational efficiency, we develop an auxiliary importance sampler that updates the beliefs with the help of an auxiliary variable. To sample from a normalized likelihood function, which is an important ingredient of the proposed auxiliary importance sampler, we develop a stochastic sampling strategy that mathematically interprets and corrects an existing heuristic strategy. The proposed message passing algorithms are analyzed systematically in terms of computational complexity, demonstrating the computational efficiency of the proposed auxiliary importance sampler. Various simulations are conducted to validate the overall good performance of the proposed algorithms.

A probabilistic white‐box model for PDE constrained inverse problems

AbstractInstead of employing deterministic solvers in a black‐box fashion, we seek to address the inherent challenges of uncertainty quantification by restating the solution of a PDE as a problem of probabilistic inference. In doing so, state variables are treated as random fields, constrained or mutually entangled by underlying physical laws. The resulting fully probabilistic model expresses the PDE operator as prior knowledge and allows the solution of the PDE‐constrained inverse problem simply by introducing the observed data as an additional source of information. We demonstrate the proposed framework for the solution of PDE‐constrained inverse problems in a solid mechanics setting and employ adjoint‐free, second‐order methods to infer the joint posterior over the Hilbert space of displacement, stress and unknown material parameters. The method generalizes to nonlinear problems and higher‐order function spaces, while the Maximum‐A‐Posteriori estimate recovers the results obtained by the mixed Finite Element method derived from the Hellinger‐Reissner variational principle.

Bayesian Probabilistic Power Flow Analysis Using Jacobian Approximate Bayesian Computation

A probabilistic power flow (PPF) study is an essential tool for the analysis and planning of a power system when specific variables are considered as random variables with particular probability distributions. The most widely used method for solving the PPF problem is Monte Carlo simulation (MCS). Although MCS is accurate for obtaining the uncertainty of the state variables, it is also computationally expensive, since it relies on repetitive deterministic power flow solutions. In this paper, we introduce a different perspective for the PPF problem. We frame the PPF as a probabilistic inference problem, and instead of repetitively solving optimization problems, we use Bayesian inference for computing posterior distributions over state variables. Additionally, we provide a likelihood-free method based on the approximate Bayesian computation philosophy, that incorporates the Jacobian computed from the power flow equations. Results in three different test systems show that the proposed methodologies are competitive alternatives for solving the PPF problem, and in some cases, they allow for reduction in computation time when compared to MCS.

Belief propagation for the maximum-weight independent set and minimum spanning tree problems

The belief propagation (BP) algorithm is a message-passing algorithm that is used for solving probabilistic inference problems. In practice, the BP algorithm performs well as a heuristic in many application fields. However, the theoretical understanding of BP is limited. To improve the theoretical understanding of BP, the BP algorithm has been applied to many well-understood combinatorial optimization problems. In this paper, we consider BP applied to the maximum-weight independent set (MWIS) and minimum spanning tree (MST) problems.Sanghavi et al. (2009) [12] applied the BP algorithm to the MWIS problem. We denote their algorithm by BP-MWIS. They showed that if the LP relaxation of the MWIS problem has a unique integral optimal solution and BP-MWIS converges, then BP-MWIS finds the optimal solution. Also, they showed that if the LP relaxation has a non-integral optimal solution, then BP-MWIS does not converge. In this paper, we precisely characterize the graphs for which BP-MWIS is guaranteed to find the optimal solution, regardless of the node weights.Bayati et al. (2008) [2] applied the BP algorithm to the MST problem. We denote their algorithm by BP-MST. They showed that if BP-MST converges, then it finds the optimal solution. In this paper, however, we provide an instance for which BP-MST does not converge. Also, since this instance is small and simple, we believe that BP-MST does not converge for most instances encountered in practice.

A Sparse Bayesian Learning Approach for Through-Wall Radar Imaging of Stationary Targets

Through-the-wall radar (TWR) imaging is an emerging technology that enables detection and localization of targets behind walls. In practical operations, TWR sensing faces several technical difficulties including strong wall clutter and missing data measurements. This paper proposes a sparse Bayesian learning (SBL) approach for wall-clutter mitigation and scene reconstruction from compressed data measurements. In the proposed approach, SBL is used to model both the intraantenna signal sparsity and interantenna signal correlation for estimating the antenna signals jointly. Here, the Bayesian framework provides a learning paradigm for sharing measurements among spatial positions, leading to accurate and stable antenna signal estimation. Furthermore, the task of wall-clutter mitigation is formulated as a probabilistic inference problem, where the wall-clutter subspace and its dimension are learned automatically using the mechanism of automatic relevant determination. Automatic discrimination between targets and clutter allows an effective target image formation, which is performed using Bayesian approximation. Experimental results with both real and simulated TWR data demonstrate the effectiveness of the SBL approach in indoor target detection and localization.

Heart rate from face videos under realistic conditions for advanced driver monitoring

Abstract The role of physiological signals has a large impact on driver monitoring systems, since it tells something about the human state. This work addresses the recursive probabilistic inference problem in time-varying linear dynamic systems to incorporate invariance into the task of heart rate estimation from face videos under realistic conditions. The invariance encapsulates motion as well as varying illumination conditions in order to accurately estimate vitality parameters from human faces using conventional camera technology. The solution is based on the canonical state space representation of an Itô process and a Wiener velocity model. Empirical results yield to excellent real-time and estimation performance of heart rates in presence of disturbing factors, like rigid head motion, talking, facial expressions and natural illumination conditions making the process of human state estimation from face videos applicable in a much broader sense, pushing the technology towards advanced driver monitoring systems.

Weighted positive binary decision diagrams for exact probabilistic inference

Recent work on weighted model counting has been very successfully applied to the problem of probabilistic inference in Bayesian networks. The probability distribution is encoded into a Boolean normal form and compiled to a target language. This results in a more efficient representation of local structure expressed among conditional probabilities. We show that further improvements are possible, by exploiting the knowledge that is lost during the encoding phase and by incorporating it into a compiler inspired by Satisfiability Modulo Theories. Constraints among variables are used as a background theory, which allows us to optimize the Shannon decomposition. We propose a new language, called Weighted Positive Binary Decision Diagrams, that reduces the cost of probabilistic inference by using this decomposition variant to induce an arithmetic circuit of reduced size.

A Probabilistic Weighted Archetypal Analysis Method with Earth Mover’s Distance for Endmember Extraction from Hyperspectral Imagery

A Probabilistic Weighted Archetypal Analysis method with Earth Mover’s Distance (PWAA-EMD) is proposed to extract endmembers from hyperspectral imagery (HSI). The PWAA-EMD first utilizes the EMD dissimilarity matrix to weight the coefficient matrix in the regular Archetypal Analysis (AA). The EMD metric considers manifold structures of spectral signatures in the HSI data and could better quantify the dissimilarity features among pairwise pixels. Second, the PWAA-EMD adopts the Bayesian framework and formulates the improved AA into a probabilistic inference problem by maximizing a joint posterior density. Third, the optimization problem is solved by the iterative multiplicative update scheme, with a careful initialization from the two-stage algorithm and the proper endmembers are finally obtained. The synthetic and real Cuprite Hyperspectral datasets are utilized to verify the performance of PWAA-EMD and five popular methods are implemented to make comparisons. The results show that PWAA-EMD surpasses all the five methods in the average results of spectral angle distance (SAD) and root-mean-square-error (RMSE). Especially, the PWAA-EMD obtains more accurate estimation than AA in almost all the classes of endmembers including two similar ones. Therefore, the PWAA-EMD could be an alternative choice for endmember extraction on the hyperspectral data.

Compiling Markov chain Monte Carlo algorithms for probabilistic modeling

The problem of probabilistic modeling and inference, at a high-level, can be viewed as constructing a ( model , query , inference ) tuple, where an inference algorithm implements a query on a model . Notably, the derivation of inference algorithms can be a difficult and error-prone task. Hence, researchers have explored how ideas from probabilistic programming can be applied. In the context of constructing these tuples, probabilistic programming can be seen as taking a language-based approach to probabilistic modeling and inference. For instance, by using (1) appropriate languages for expressing models and queries and (2) devising inference techniques that operate on encodings of models (and queries) as program expressions, the task of inference can be automated. In this paper, we describe a compiler that transforms a probabilistic model written in a restricted modeling language and a query for posterior samples given observed data into a Markov Chain Monte Carlo (MCMC) inference algorithm that implements the query. The compiler uses a sequence of intermediate languages (ILs) that guide it in gradually and successively refining a declarative specification of a probabilistic model and the query into an executable MCMC inference algorithm. The compilation strategy produces composable MCMC algorithms for execution on a CPU or GPU.

Population-Based MCMC on Multi-Core CPUs, GPUs and FPGAs

Markov Chain Monte Carlo (MCMC) is a method to draw samples from a given probability distribution. Its frequent use for solving probabilistic inference problems, where big-scale data are repeatedly processed, means that MCMC runtimes can be unacceptably large. This paper focuses on population-based MCMC, a popular family of computationally intensive MCMC samplers; we propose novel, highly optimized accelerators in three parallel hardware platforms (multi-core CPUs, GPUs and FPGAs), in order to address the performance limitations of sequential software implementations. For each platform, we jointly exploit the nature of the underlying hardware and the special characteristics of population-based MCMC. We focus particularly on the use of custom arithmetic precision, introducing two novel methods which employ custom precision in the largest part of the algorithm in order to reduce runtime, without causing sampling errors. We apply these methods to all platforms. The FPGA accelerators are up to 114x faster than multi-core CPUs and up to 53x faster than GPUs when doing inference on mixture models.

Real-Time Stochastic Optimal Control for Multi-Agent Quadrotor Systems

This paper presents a novel method for controlling teams of unmanned aerial vehicles using Stochastic Optimal Control (SOC) theory. The approach consists of a centralized high-level planner that computes optimal state trajectories as velocity sequences, and a platform-specific low-level controller which ensures that these velocity sequences are met. The planning task is expressed as a centralized path-integral control problem, for which optimal control computation corresponds to a probabilistic inference problem that can be solved by efficient sampling methods. Through simulation we show that our SOC approach (a) has significant benefits compared to deterministic control and other SOC methods in multimodal problems with noise-dependent optimal solutions, (b) is capable of controlling a large number of platforms in real-time, and (c) yields collective emergent behaviour in the form of flight formations. Finally, we show that our approach works for real platforms, by controlling a team of three quadrotors in outdoor conditions.

A Belief Propagation Approach to Privacy-Preserving Item-Based Collaborative Filtering

Collaborative filtering (CF) is the most popular recommendation algorithm, which exploits the collected historic user ratings to predict unknown ratings. However, traditional recommender systems run at the central servers, and thus users have to disclose their personal rating data to other parties. This raises the privacy issue, as user ratings can be used to reveal sensitive personal information. In this paper, we propose a semi-distributed belief propagation (BP) approach to privacy-preserving item-based CF recommender systems. Firstly, we formulate the item similarity computation as a probabilistic inference problem on the factor graph, which can be efficiently solved by applying the BP algorithm. To avoid disclosing user ratings to the server or other user peers, we then introduce a semi-distributed architecture for the BP algorithm. We further propose a cascaded BP scheme to address the practical issue that only a subset of users participate in BP during one time slot. We analyze the privacy of the semi-distributed BP from a information-theoretic perspective. We also propose a method that reduces the computational complexity at the user side. Through experiments on the MovieLens dataset, we show that the proposed algorithm achieves superior accuracy.

Projection, consistency, and George Boole

Although best known for his work in symbolic logic, George Boole made seminal contributions in the logic of probabilities. He solved the probabilistic inference problem with a projection method, leading to the insight that inference (as well as optimization) is essentially a projection problem. This unifying perspective has applications in constraint programming, because consistency maintenance is likewise a form of inference that can be conceived as projection. Viewing consistency in this light suggests a concept of J-consistency, which is achieved by projection onto a subset J of variables. We show how this projection problem can be solved for the satisfiability problem by logic-based Benders decomposition. We also solve it for among, sequence, regular, and all-different constraints. Maintaining J-consistency for global constraints can be more effective than maintaining traditional domain and bounds consistency when propagating through a richer structure than a domain store, such as a relaxed decision diagram. This paper is written in recognition of Boole's 200th birthday.