Standard Belief Propagation Algorithm Research Articles

Inference With Aggregate Data in Probabilistic Graphical Models: An Optimal Transport Approach

We consider inference (filtering) problems over probabilistic graphical models with aggregate data generated by a large population of individuals. We propose a new efficient belief propagation type algorithm over tree graphs with polynomial computational complexity as well as a global convergence guarantee. This is in contrast to previous methods that either exhibit prohibitive complexity as the population grows or do not guarantee convergence. Our method is based on optimal transport, or more specifically, multimarginal optimal transport theory. In particular, we consider an inference problem with aggregate observations, that can be seen as a structured multimarginal optimal transport problem where the cost function decomposes according to the underlying graph. Consequently, the celebrated Sinkhorn/iterative scaling algorithm for multi-marginal optimal transport can be leveraged together with the standard belief propagation algorithm to establish an efficient inference scheme which we call Sinkhorn belief propagation (SBP). We further specialize the SBP algorithm to cases associated with hidden Markov models due to their significance in control and estimation. We demonstrate the performance of our algorithm on applications such as inferring population flow from aggregate observations. We also show that in the special case where the aggregate observations are in Dirac form, our algorithm naturally reduces to the standard belief propagation algorithm.

Learning to Decode Polar Codes With One-Bit Quantizer

A deep learning method for improving the performance of polar belief propagation (BP) decoder equipped with a one-bit quantizer is proposed. The method generalizes the standard polar BP algorithm by assigning weights to the layers of the unfolded factor graph. These weights can be learned autonomously using deep learning techniques. We prove that the improved polar BP decoder has a symmetric structure, so that the weights can be trained by an all-zero codeword rather than an exponential number of codewords. In order to accelerate the training convergence, a layer-based weight assignment scheme is designed, which decreases the amount of trainable weights. Simulation results show that the improved polar BP decoder with a one-bit quantizer outperforms the standard polar BP decoder with a 2-bit quantizer and achieves faster convergence.

Novel Belief Propagation Algorithm for Stereo Matching With a Robust Cost Computation

Stereo matching is a challenging problem in the field of computer vision and has recently received extensive attention. However, the traditional methods are labor intensive and premised on specific conditions. In this paper, we propose a robust stereo matching cost algorithm that relies on refined features extracted by a stack auto-encoder. These features are robust for different types of image pairs, which significantly improve the generality of the proposed matching algorithm. In addition, we smoothed the belief volume with a guided filter to improve the performance of the belief propagation algorithm on edge regions. To deal with the time consumption issue of the standard belief propagation algorithm, we also proposed a constraint method to speed it up. This constraint does not degrade the matching results and can easily be generalized to other message passing algorithms. The experiments conducted on a Middlebury benchmark dataset demonstrate the effectiveness of the proposed algorithms.

Graph‐based low complexity detection algorithms in multiple‐input–multiple‐out systems: an edge selection approach

In this study, the problem of low complexity multiple-input-multiple-out signal detection based on belief propagation (BP) is addressed. The authors propose an edge selection approach that works on factor graph model to cut down the number of circles and high complexity of standard BP algorithm. The message passing from factor nodes to variable nodes is updated by only partial edges, and the mean feedback method is designed to compensate the information loss brought by the edge selection. Both binary and high-order modulations are considered, and the scheme of mapping between bit soft output and modulation symbols when computing the feedback information is discussed. In addition, a minimum mean-square error filter initialised algorithm is proposed, in which the initial message of BP detection is exploited. Both binary and high-order modulations are discussed as well when the authors design this initial message. Simulation results along with convergence and complexity analyses verify that the proposed edge selection approach can achieve good performance with low complexity, and significantly outperform the existing methods with comparative complexity. Moreover, our approach has asymptotic optimality and is a self-adapting scheme, which can achieve the trade-off between performance and complexity by varying the number of selected edges.

저밀도 패리티 검사부호의 복호 알고리즘에 따른 성능 비교 분석

LDPC (low density parity check) code shows near Shannon limit performance by iterative decoding based on sum-product algorithm (SPA). Message updating procedure between variable and check nodes in SPA is done by a scheduling method. LDPC code shows different performance according to scheduling schemes. The conventional researches have been shown that the shuffled BP (belief propagation) algorithm shows better performance than the standard BP algorithm although it needs less number of iterations. However the reason is not analyzed clearly. Therefore the reason of difference in performance according to LDPC decoding algorithms is analyzed in this paper. 4 cases according to satisfaction of parity check condition are considered and compared. As results, the difference in the updating procedure in a cycle in the parity check matrix is considered to be the main reason of performance difference.

Wavelet Bayesian Network Image Denoising

From the perspective of the Bayesian approach, the denoising problem is essentially a prior probability modeling and estimation task. In this paper, we propose an approach that exploits a hidden Bayesian network, constructed from wavelet coefficients, to model the prior probability of the original image. Then, we use the belief propagation (BP) algorithm, which estimates a coefficient based on all the coefficients of an image, as the maximum-a-posterior (MAP) estimator to derive the denoised wavelet coefficients. We show that if the network is a spanning tree, the standard BP algorithm can perform MAP estimation efficiently. Our experiment results demonstrate that, in terms of the peak-signal-to-noise-ratio and perceptual quality, the proposed approach outperforms state-of-the-art algorithms on several images, particularly in the textured regions, with various amounts of white Gaussian noise.

Enhanced Feedback Iterative Decoding of Sparse Quantum Codes

Decoding sparse quantum codes can be accomplished by syndrome-based decoding using a belief propagation (BP) algorithm.We significantly improve this decoding scheme by developing a new feedback adjustment strategy for the standard BP algorithm. In our feedback procedure, we exploit much of the information from stabilizers, not just the syndrome but also the values of the frustrated checks on individual qubits of the code and the channel model. Furthermore we show that our decoding algorithm is superior to belief propagation algorithms using only the syndrome in the feedback procedure for all cases of the depolarizing channel. Our algorithm does not increase the measurement overhead compared to the previous method, as the extra information comes for free from the requisite stabilizer measurements.

Jointly-check iterative decoding algorithm for quantum sparse graph codes

For quantum sparse graph codes with stabilizer formalism, the unavoidable girth-four cycles in their Tanner graphs greatly degrade the iterative decoding performance with a standard belief-propagation (BP) algorithm. In this paper, we present a jointly-check iterative algorithm suitable for decoding quantum sparse graph codes efficiently. Numerical simulations show that this modified method outperforms the standard BP algorithm with an obvious performance improvement.

Lossy Source Compression Using Low-Density Generator Matrix Codes: Analysis and Algorithms

We study the use of low-density generator matrix (LDGM) codes for lossy compression of the Bernoulli symmetric source. First, we establish rigorous upper bounds on the average distortion achieved by check-regular ensemble of LDGM codes under optimal minimum distance source encoding. These bounds establish that the average distortion using such bounded degree families rapidly approaches the Shannon limit as the degrees are increased. Second, we propose a family of message-passing algorithms, ranging from the standard belief propagation algorithm at one extreme to a variant of survey propagation algorithm at the other. When combined with a decimation subroutine and applied to LDGM codes with suitably irregular degree distributions, we show that such a message-passing/decimation algorithm yields distortion very close to the Shannon rate-distortion bound for the binary symmetric source.

A Modified BP Algorithm for LDPC Decoding Based on Minimum Mean Square Error Criterion

In this paper, a modified Belief Propagation (BP) decoding algorithm for low-density parity check (LDPC) codes based on minimum mean square error (MMSE) criterion is proposed. This modified algorithm uses linear equation to replace the hyperbolic function in the original BP algorithm and optimizes the linear approximation error based on MMSE criterion. As a result, compared with the standard BP algorithm the computational complexity is reduced significantly as the modified algorithm requires only addition operations to implement. Besides that simulation results show our modified algorithm can achieve an error performance very close to the BP algorithm on the additive white Gaussian noise channel.

Low-density parity-check codes with 2-state trellis decoding

A class of low-density parity-check (LDPC) codes with a simple 2-state trellis structure is presented. For LDPC decoding, the conventional belief propagation (BP) algorithm consists of numerous sub-decoders of single-parity check codes and exchanges information between sub-decoders in an iterative manner. If the single-parity check codes can be constructed and grouped in a proper way, the decoder can be decomposed into few identical 2-state trellis decoders. Therefore, instead of numerous sub-decoders of single-parity check codes, an iterative decoding algorithm based on few sub-decoders over 2-state trellis is proposed. The proposed decoding algorithm improves the efficiency of message passing between sub-decoders and hence provides a fast convergent rate as compared to the standard BP algorithm. Simulation results show that the proposed scheme provides a better performance and a fast convergent rate as compared to those of standard BP algorithm. The result also shows that the proposed algorithm has a similar performance as that of asynchronous replica shuffled BP algorithm and has a slightly inferior performance than that of synchronous replica shuffled BP algorithm. However, complexity analysis shows that our proposed algorithm has complexity that is lower than that of the replica shuffled BP algorithm.

On the iterative decoding of sparse quantum codes

We address the problem of decoding sparse quantum error correction codes. For Pauli channels, this task can be accomplished by a version of the belief propagation algorithm used for decoding sparse classical codes. Quantum codes pose two new challenges however. Firstly, their Tanner graph unavoidably contain small loops which typically undermines the performance of belief propagation. Secondly, sparse quantum codes are by definition highly degenerate. The standard belief propagation algorithm does not exploit this feature, but rather it is impaired by it. We propose heuristic methods to improve belief propagation decoding, specifically targeted at these two problems. While our results exhibit a clear improvement due to the proposed heuristic methods, they also indicate that the main source of errors in the quantum coding scheme remains in the decoding.

On the iterative decoding of sparse quantum codes

We address the problem of decoding sparse quantum error correction codes. For Pauli channels, this task can be accomplished by a version of the belief propagation algorithm used for decoding sparse classical codes. Quantum codes pose two new challenges however. Firstly, their Tanner graph unavoidably contain small loops which typically undermines the performance of belief propagation. Secondly, sparse quantum codes are by definition highly degenerate. The standard belief propagation algorithm does not exploit this feature, but rather it is impaired by it. We propose heuristic methods to improve belief propagation decoding, specifically targeted at these two problems. While our results exhibit a clear improvement due to the proposed heuristic methods, they also indicate that the main source of errors in the quantum coding scheme remains in the decoding.

Image Completion Using Efficient Belief Propagation Via Priority Scheduling and Dynamic Pruning

In this paper, a new exemplar-based framework is presented, which treats image completion, texture synthesis, and image inpainting in a unified manner. In order to be able to avoid the occurrence of visually inconsistent results, we pose all of the above image-editing tasks in the form of a discrete global optimization problem. The objective function of this problem is always well-defined, and corresponds to the energy of a discrete Markov random field (MRF). For efficiently optimizing this MRF, a novel optimization scheme, called priority belief propagation (BP), is then proposed, which carries two very important extensions over the standard BP algorithm: "priority-based message scheduling" and "dynamic label pruning." These two extensions work in cooperation to deal with the intolerable computational cost of BP, which is caused by the huge number of labels associated with our MRF. Moreover, both of our extensions are generic, since they do not rely on the use of domain-specific prior knowledge. They can, therefore, be applied to any MRF, i.e., to a very wide class of problems in image processing and computer vision, thus managing to resolve what is currently considered as one major limitation of the BP algorithm: its inefficiency in handling MRFs with very large discrete state spaces. Experimental results on a wide variety of input images are presented, which demonstrate the effectiveness of our image-completion framework for tasks such as object removal, texture synthesis, text removal, and image inpainting.

Modified belief-propagation algorithm for decoding of irregular low-density parity-check codes

A modified belief-propagation algorithm is proposed for decoding of irregular low-density parity-check codes. This algorithm effectively reduces the decoding complexity while achieving a BER performance near to or even better than that of the standard belief-propagation algorithm.