Loopy Belief Propagation Algorithm Research Articles

Local Minimum Energy of LBP Algorithm Concrete CT Image Segmentation

Concrete CT image segmentation is a hot civil engineering research field in recent years. An algorithm of minimizing the energy of local region is proposed to solve the problem of interaction fails to capture statistical property of natural image and to segment concrete CT image. At first, the algorithm utilizes local region information of image to construct the energy model of local region, establishes a segmentation model of local interaction region based on MRF. Then using loopy belief propagation (LBP) algorithm and other algorithms optimizes global energy. In process of optimization, local region energy is converged, and the label of local region is estimated based on MAP algorithm. Then the information of local region is transferred to the region of neighborhood. The result of experiment shows that comparing with standard LBP algorithm, the new algorithm has a better segmentation result, and efficiently restrained effect on image noise and texture for segmentation. Result of concrete CT image segmentation will simplify the following CT statistical analysis and provide an important method for reaching real meso-structure of concrete’s finite element network.

Image segmentation using adaptive loopy belief propagation

Loopy belief propagation (LBP) algorithm over pairwise-connected Markov random fields (MRFs) has become widely used for low-level vision problems. However, Pairwise MRF is often insufficient to capture the statistics of natural images well, and LBP is still extremely slow for application on an MRF with large discrete label space. To solve these problems, the present study proposes a new segmentation algorithm based on adaptive LBP. The proposed algorithm utilizes local region information to construct a local region model, as well as a local interaction region MRF model for image segmentation. The adaptive LBP algorithm maximizes the global probability of the proposed MRF model, which employs two very important strategies, namely, “message self-convergence” and “adaptive label pruning”. Message self-convergence can improve the reliability of a pixel in choosing a label in local region, and label pruning can dismiss impossible labels for every pixel. Thus, the most reliable information messages transfer through the LBP algorithm. The experimental results show that the proposed algorithm not only obtains more accurate segmentation results but also greater speed.

A nonparametric belief propagation method for uncertainty quantification with applications to flow in random porous media

A probabilistic graphical model approach to uncertainty quantification for flows in random porous media is introduced. Model reduction techniques are used locally in the graph to represent the random permeability. Then the conditional distribution of the multi-output responses on the low dimensional representation of the permeability field is factorized into a product of local potential functions. An expectation–maximization algorithm is used to learn the nonparametric representation of these potentials using the given input/output data. We develop a nonparametric belief propagation method for uncertainty quantification by employing the loopy belief propagation algorithm. The nonparametric nature of our model is able to capture non-Gaussian features of the response. The proposed framework can be used as a surrogate model to predict the responses for new input realizations as well as our confidence on these predictions. Numerical examples are presented to demonstrate the accuracy and efficiency of the proposed framework for solving uncertainty quantification problems in flows through porous media using stationary and non-stationary permeability fields.

Learning Topic Models by Belief Propagation

Latent Dirichlet allocation (LDA) is an important hierarchical Bayesian model for probabilistic topic modeling, which attracts worldwide interest and touches on many important applications in text mining, computer vision and computational biology. This paper represents the collapsed LDA as a factor graph, which enables the classic loopy belief propagation (BP) algorithm for approximate inference and parameter estimation. Although two commonly used approximate inference methods, such as variational Bayes (VB) and collapsed Gibbs sampling (GS), have gained great success in learning LDA, the proposed BP is competitive in both speed and accuracy, as validated by encouraging experimental results on four large-scale document datasets. Furthermore, the BP algorithm has the potential to become a generic scheme for learning variants of LDA-based topic models in the collapsed space. To this end, we show how to learn two typical variants of LDA-based topic models, such as author-topic models (ATM) and relational topic models (RTM), using BP based on the factor graph representations.

Pooling Design and Bias Correction in DNA Library Screening

We study the group test for DNA library screening based on probabilistic approach. Group test is a method of detecting a few positive items from among a large number of items, and has wide range of applications. In DNA library screening, positive item corresponds to the clone having a specified DNA segment, and it is necessary to identify and isolate the positive clones for compiling the libraries. In the group test, a group of items, called pool, is assayed in a lump in order to save the cost of testing, and positive items are detected based on the observation from each pool. It is known that the design of grouping, that is, pooling design is important to %reduce the estimation bias and achieve accurate detection. In the probabilistic approach, positive clones are picked up based on the posterior probability. Naive methods of computing the posterior, however, involves exponentially many sums, and thus we need a device. Loopy belief propagation (loopy BP) algorithm is one of popular methods to obtain approximate posterior probability efficiently. There are some works investigating the relation between the accuracy of the loopy BP and the pooling design. Based on these works, we develop pooling design with small estimation bias of posterior probability, and we show that the balanced incomplete block design (BIBD) has nice property for our purpose. Some numerical experiments show that the bias correction under the BIBD is useful to improve the estimation accuracy.

Segmentation of objects in a detection window by Nonparametric Inhomogeneous CRFs

This paper presents a method for segmenting objects of a specific class in a given detection window. The task is to label each pixel as belonging to the foreground or the background. We pose the problem as that of finding the maximum a posterior (MAP) estimation in a modified form of Conditional Random Field model that we call a Nonparametric Inhomogeneous CRF (NICRFs). An NICRF, like a conventional CRF, has nodes representing pixels and pairwise links connecting neighboring pixels; however, both the unary and pairwise energy terms are inhomogeneous in the sense of being dependent on pixel positions to account for prior information of the known object class. It differs from earlier methods in that position information is in form of unique term functions for each individual pixel, rather than the same parametric function but with varying parameters. Unary terms are given by a learned boosted classifier based on novel Adaptive Edgelet Features (AEFs) for inferring probability of a pixel being foreground; pairwise terms are learned by joint probabilities for neighboring pixels as a function of contrast; a monotonicity constraint is used to reduce possible over-fit effects. We expand the neighborhood used for pairwise terms, and add inhomogeneous weighting factors for different pairwise terms. We use the Loopy Belief Propagation (LBP) algorithm for MAP estimation. A local search process is proposed to deal with inaccurate detection windows. We evaluate our approach on examples of pedestrians and cars and demonstrate significant improvements compared to earlier methods.

JIGSAW: Joint Inhomogeneity Estimation via Global Segment Assembly for Water–Fat Separation

Water-fat separation in magnetic resonance imaging (MRI) is of great clinical importance, and the key to uniform water-fat separation lies in field map estimation. This work deals with three-point field map estimation, in which water and fat are modelled as two single-peak spectral lines, and field inhomogeneities shift the spectrum by an unknown amount. Due to the simplified spectrum modelling, there exists inherent ambiguity in forming field maps from multiple locally feasible field map values at each pixel. To resolve such ambiguity, spatial smoothness of field maps has been incorporated as a constraint of an optimization problem. However, there are two issues: the optimization problem is computationally intractable and even when it is solved exactly, it does not always separate water and fat images. Hence, robust field map estimation remains challenging in many clinically important imaging scenarios. This paper proposes a novel field map estimation technique called JIGSAW. It extends a loopy belief propagation (BP) algorithm to obtain an approximate solution to the optimization problem. The solution produces locally smooth segments and avoids error propagation associated with greedy methods. The locally smooth segments are then assembled into a globally consistent field map by exploiting the periodicity of the feasible field map values. In vivo results demonstrate that JIGSAW outperforms existing techniques and produces correct water-fat separation in challenging imaging scenarios.

Probabilistic models for concurrent chatting activity recognition

Recognition of chatting activities in social interactions is useful for constructing human social networks. However, the existence of multiple people involved in multiple dialogues presents special challenges. To model the conversational dynamics of concurrent chatting behaviors, this article advocates Factorial Conditional Random Fields (FCRFs) as a model to accommodate co-temporal relationships among multiple activity states. In addition, to avoid the use of inefficient Loopy Belief Propagation (LBP) algorithm, we propose using Iterative Classification Algorithm (ICA) as the inference method for FCRFs. We designed experiments to compare our FCRFs model with two dynamic probabilistic models, Parallel Condition Random Fields (PCRFs) and Hidden Markov Models (HMMs), in learning and decoding based on auditory data. The experimental results show that FCRFs outperform PCRFs and HMMs-like models. We also discover that FCRFs using the ICA inference approach not only improves the recognition accuracy but also takes significantly less time than the LBP inference method.

Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems

This paper describes a statistically motivated framework for performing real-time dialogue state updates and policy learning in a spoken dialogue system. The framework is based on the partially observable Markov decision process (POMDP), which provides a well-founded, statistical model of spoken dialogue management. However, exact belief state updates in a POMDP model are computationally intractable so approximate methods must be used. This paper presents a tractable method based on the loopy belief propagation algorithm. Various simplifications are made, which improve the efficiency significantly compared to the original algorithm as well as compared to other POMDP-based dialogue state updating approaches. A second contribution of this paper is a method for learning in spoken dialogue systems which uses a component-based policy with the episodic Natural Actor Critic algorithm. The framework proposed in this paper was tested on both simulations and in a user trial. Both indicated that using Bayesian updates of the dialogue state significantly outperforms traditional definitions of the dialogue state. Policy learning worked effectively and the learned policy outperformed all others on simulations. In user trials the learned policy was also competitive, although its optimality was less conclusive. Overall, the Bayesian update of dialogue state framework was shown to be a feasible and effective approach to building real-world POMDP-based dialogue systems.

Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling

In this paper, we formulate a stereo matching algorithm with careful handling of disparity, discontinuity and occlusion. The algorithm works with a global matching stereo model based on an energy-minimization framework. The global energy contains two terms, the data term and the smoothness term. The data term is first approximated by a color-weighted correlation, then refined in occluded and low-texture areas in a repeated application of a hierarchical loopy belief propagation algorithm. The experimental results are evaluated on the Middlebury data sets, showing that our algorithm is the top performer among all the algorithms listed there.

Accuracy of Loopy belief propagation in Gaussian models

This paper considers the loopy belief propagation (LBP) algorithm applied to Gaussian graphical models. It is known for Gaussian belief propagation that, if LBP converges, LBP computes the exact posterior means but incorrect variances. In this paper, we analytically derive the posterior variances for some special structured graphs and clarify the accuracy of LBP. For the graphs of a single cycle, we derive a rigorous solution for the posterior variances and thereby find the quantity that determines the accuracy of LBP. Based on this result, we state a necessary condition for LBP convergence. The quantity above also plays an important role in graphs of a single cycle with arbitrary trees. For arbitrary topological graphs, we consider the situation where correlations between any pair of nodes are comparatively small and show analytically the principal values that determine the accuracy of LBP.

THE DESIGN AND TESTING OF A FIRST-ORDER LOGIC-BASED STOCHASTIC MODELING LANGUAGE

We have created a logic-based, Turing-complete language for stochastic modeling. Since the inference scheme for this language is based on a variant of Pearl's loopy belief propagation algorithm, we call it Loopy Logic. Traditional Bayesian networks have limited expressive power, basically constrained to finite domains as in the propositional calculus. Our language contains variables that can capture general classes of situations, events and relationships. A first-order language is also able to reason about potentially infinite classes and situations using constructs such as hidden Markov models(HMMs). Our language uses an Expectation-Maximization (EM) type learning of parameters. This has a natural fit with the Loopy Belief Propagation used for inference since both can be viewed as iterative message passing algorithms. We present the syntax and theoretical foundations for our Loopy Logic language. We then demonstrate three examples of stochastic modeling and diagnosis that explore the representational power of the language. A mechanical fault detection example displays how Loopy Logic can model time-series processes using an HMM variant. A digital circuit example exhibits the probabilistic modeling capabilities, and finally, a parameter fitting example demonstrates the power for learning unknown stochastic values.

Kikuchi approximation method for joint decoding of LDPC codes and partial-response channels

In this letter, we apply the Kikuchi approximation method to the problem of joint decoding of a low-density parity-check code and a partial-response channel. The Kikuchi method is, in general, more powerful than the conventional loopy belief propagation (BP) algorithm, and can produce better approximations to an underlying inference problem. We will first review the Kikuchi approximation method and the generalized BP algorithm, which is an iterative message-passing algorithm based on this method. We will then report simulation results which show that the Kikuchi method outperforms the best conventional iterative method

Diagnosis using a first-order stochastic language that learns

We have created a diagnostic/prognostic software tool for the analysis of complex systems, such as monitoring the ''running health'' of helicopter rotor systems. Although our software is not yet deployed for real-time in-flight diagnosis, we have successfully analyzed the data sets of actual helicopter rotor failures supplied to us by the US Navy. In this paper, we discuss both critical techniques supporting the design of our stochastic diagnostic system as well as issues related to its full deployment. We also present four examples of its use. Our diagnostic system, called DBAYES, is composed of a logic-based, first-order, and Turing-complete set of software tools for stochastic modeling. We use this language for modeling time-series data supplied by sensors on mechanical systems. The inference scheme for these software tools is based on a variant of Pearl's loopy belief propagation algorithm [Pearl, P. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Francisco, CA: Morgan Kaufmann]. Our language contains variables that can capture general classes of situations, events, and relationships. A Turing-complete language is able to reason about potentially infinite classes and situations, similar to the analysis of dynamic Bayesian networks. Since the inference algorithm is based on a variant of loopy belief propagation, the language includes expectation maximization type learning of parameters in the modeled domain. In this paper we briefly present the theoretical foundations for our first-order stochastic language and then demonstrate time-series modeling and learning in the context of fault diagnosis.

On the Convergence of Loopy Belief Propagation Algorithm for Different Update Rules

The belief propagation (BP) algorithm is a tool with which one can calculate beliefs, marginal probabilities, of probabilistic networks without loops (e.g., Bayesian networks) in a time proportional to the number of nodes. For networks with loops, it may not converge and, even if it converges, beliefs may not be equal to exact marginal probabilities although its application is known to give remarkably good results such as in the coding theory. Tatikonda and Jordan show a theoretical result on the convergence of the algorithm for probabilistic networks with loops in terms of the theory of Markov random fields on trees and give a sufficient condition of the convergence of the algorithm. In this paper, we discuss the update rule as well as the update rule discussed in Tatikonda and Jordan. In the viewpoint of the theory of Markov random fields, it is shown that the rule for updating both gives essentially the same results and the impatient update rule is expected to converge faster than the lazy one. Numerical experiments are also given.

On the properties of the Bethe approximation and loopy belief propagation on binarynetworks

We analyse the local stability of the high-temperature fixed point of the loopy beliefpropagation (LBP) algorithm and how this relates to the properties of the Bethe freeenergy which LBP tries to minimize. We focus on the case of binary networks with pairwiseinteractions. In particular, we state sufficient conditions for convergence of LBP to a uniquefixed point and show that these are sharp for purely ferromagnetic interactions. In contrast,in the purely antiferromagnetic case, the undamped parallel LBP algorithm issuboptimal in the sense that the stability of the fixed point breaks down much earlierthan for damped or sequential LBP; we observe that the onset of instability forthe latter algorithms is related to the properties of the Bethe free energy. Forspin-glass interactions, damping LBP only helps slightly. We estimate analytically thetemperature at which the high-temperature LBP fixed point becomes unstablefor random graphs with arbitrary degree distributions and random interactions.