Belief Propagation Algorithm Research Articles

Polar-Coded OFDM With Index Modulation

In this paper, we apply polar codes to a modified version of OFDM systems with index modulation which is called OFDM with in-phase/quadrature index modulation (OFDM-I/Q-IM). We provide general design guidelines for the proposed polar-coded OFDM-I/Q-IM systems. In the proposed system, at the transmitter, we employ a random frozen bits appending scheme which not only makes the polar code compatible with OFDM-I/Q-IM but also improves the bit error rate (BER) performance of the system. Furthermore, at the receiver, it is shown that the a posteriori information for each index provided by the index detector is essential for the iterative decoding of polar codes by the belief propagation (BP) algorithm. Simulation results show that the proposed polar-coded OFDM-I/Q-IM system outperforms its OFDM counterpart in terms of BER performance.

Discovering latent structures with integrated propagation algorithms in geographical information networks

The real-world exists various kinds of complex network systems, including geographical information systems (GIS) as the important components. However, it is hard to understand the functions scientifically because that the structures are latent for our eyes in these systems. In this paper, a specific model is proposed to deal with the expression of geographical information networks. And by integrating the belief propagation algorithm and label propagation algorithm, our algorithm is able to discover the latent structures and achieve some understandings of the networks without knowing the number of communities. We test the algorithm on the benchmark networks as well as the real-world subway-road networks. The results show the availability for the applications of our algorithm to the geographical information networks.

Analytical Determination of Thresholds of LDPC Codes in Free Space Optical Channel

Free-space optical (FSO) communication is crucial for the next-generation 5G+ wireless networks. The FSO links suffer from atmospheric-turbulence-induced bit errors. For the increasing link's performance, low-density parity-check (LDPC) codes, complemented by the belief-propagation (BP) algorithm, are an excellent option. The bit error rate (BER) of the LDPC code is characterized by a parameter called the threshold. The threshold is the signal-to-noise ratio (SNR), after which the BER falls arbitrarily and becomes close to zero. We derive the threshold for the LDPC codes under the BP algorithm for an uncorrelated flat FSO channel. The determination of the FSO channel threshold is a tedious task as the density of the log-likelihood ratio from the FSO channel cannot be assumed as Gaussian and is available only in a numerical form. It, thus, requires testing different values of SNR as a possible threshold systematically. Therefore, we propose the divide and conquer algorithm. The threshold depends on the degree distributions, channel state information (CSI), and the turbulence level. When CSI is known, we obtain the threshold at an SNR of 8.10 dB in high turbulence for a regular (3, 6) LDPC code. This threshold steps up to 12.48 dB when the CSI is unknown at the receiver. We evaluate the threshold values for various degree distributions (regular and irregular LDPC codes) under high, moderate, and low turbulence levels for both channel models (CSI known and unknown at the receiver). We also confirm the derived threshold values with MATLAB simulations.

Outdoor image restoration based on belief propagation algorithm and formalized MTF

Single image dehazing technology can be applied to many fields. In order to solve this problem, an improved framework is proposed here, which can be used to estimate the light transmittance t in a given single input image and then the belief propagation (PB) algorithm is applied to image dehazing. At first, a coarse transmission map is calculated through Dark Channel Prior (DCP) knowledge. Next, the belief propagation methodology is introduced to correct the transmission map when the object in degraded image is similar to the airlight over a large local region. As a result, the transmission map can be estimated better and the important drawback of artifacts phenomenon can be avoid to some extent. At last, through this effective estimation to transmittance, the formalized MTF can be predicted to restore degraded images, then the haze-free scene contrasts can be better recovered. The experimental analysis results show that compared with defogged images acquired through DCP algorithms, our proposed algorithm can provide good results. Our method is more effective and robust.

RFBP: Replicated Focusing Belief Propagation algorithm

rFBP: Replicated Focusing Belief Propagation algorithm

Excess rate for model selection in interactive compression using belief propagation decoding

Interactive compression refers to the problem of compressing data while sending only the part requested by the user. In this context, the challenge is to perform the extraction in the compressed domain directly. Theoretical results exist, but they assume that the true distribution is known. In practical scenarios instead, the distribution must be estimated. In this paper, we first formulate the model selection problem for interactive compression and show that it requires to estimate the excess rate incurred by mismatched decoding. Then, we propose a new expression to evaluate the excess rate of mismatched decoding in a practical case of interest: when the decoder is the belief propagation algorithm. We also propose a novel experimental setup to validate this closed-form formula. We show a good match for practical interactive compression schemes based on fixed-length Low-Density Parity-Check (LDPC) codes. This new formula is of great importance to perform model and rate selection.

Phase transitions and optimal algorithms for semisupervised classifications on graphs: From belief propagation to graph convolution network

We perform theoretical and algorithmic studies for the problem of clustering and semi-supervised classification on graphs with both pairwise relational information and single-point feature information, upon a joint stochastic block model for generating synthetic graphs with both edges and node features. Asymptotically exact analysis based on the Bayesian inference of the underlying model are conducted, using the cavity method in statistical physics. Theoretically, we identify a phase transition of the generative model, which puts fundamental limits on the ability of all possible algorithms in the clustering task of the underlying model. Algorithmically, we propose a belief propagation algorithm that is asymptotically optimal on the generative model, and can be further extended to a belief propagation graph convolution neural network (BPGCN) for semi-supervised classification on graphs. For the first time, well-controlled benchmark datasets with asymptotially exact properties and optimal solutions could be produced for the evaluation of graph convolution neural networks, and for the theoretical understanding of their strengths and weaknesses. In particular, on these synthetic benchmark networks we observe that existing graph convolution neural networks are subject to an sparsity issue and an ovefitting issue in practice, both of which are successfully overcome by our BPGCN. Moreover, when combined with classic neural network methods, BPGCN yields extraordinary classification performances on some real-world datasets that have never been achieved before.

Statistical mechanics of the directed 2-distance minimal dominating set problem

The directed L-distance minimal dominating set (MDS) problem has wide practical applications in the fields of computer science and communication networks. Here, we study this problem from the perspective of purely theoretical interest. We only give results for an Erdós Rényi (ER) random graph and regular random (RR) graph, but this work can be extended to any type of network. We develop spin glass theory to study the directed 2-distance MDS problem. First, we find that the belief propagation (BP) algorithm does not converge when the inverse temperature β exceeds a threshold on either an ER random network or RR network. Second, the entropy density of replica symmetric theory has a transition point at a finite β on a regular random graph when the arc density exceeds 2 and on an ER random graph when the arc density exceeds 3.3; there is no entropy transition point (or ) in other circumstances. Third, the results of the replica symmetry (RS) theory are in agreement with those of BP algorithm while the results of the BP decimation algorithm are better than those of the greedy heuristic algorithm.

Bayesian Inference of Infected Patients in Group Testing with Prevalence Estimation

Group testing is a method of identifying infected patients by performing tests on a pool of specimens collected from patients. For the case in which the test returns a false result with finite probability, we propose Bayesian inference and a corresponding belief propagation (BP) algorithm to identify the infected patients from the results of tests performed on the pool. We show that the true-positive rate is improved by taking into account the credible interval of a point estimate of each patient. Further, the prevalence and the error probability in the test are estimated by combining an expectation-maximization method with the BP algorithm. As another approach, we introduce a hierarchical Bayes model to identify the infected patients and estimate the prevalence. By comparing these methods, we formulate a guide for practical usage.

Joint Distributed Link Scheduling and Power Allocation for Content Delivery in Wireless Caching Networks

In wireless caching networks, the design of the content delivery method must consider random user requests, caching states, network topology, and interference management. In this article, we establish a general framework for content delivery in wireless caching networks without stringent assumptions that restrict the network structure and interference model. Based on the framework, we propose a dynamic and distributed link scheduling and power allocation scheme for content delivery that is assisted by belief-propagation (BP) algorithms. The proposed scheme achieves three critical purposes of wireless caching networks: 1) limiting the delay of user request satisfactions, 2) maintaining the power efficiency of caching nodes, and 3) managing interference among users. In addition, we address the intrinsic problem of the BP algorithm in our network model, proposing a matching algorithm for one-to-one link scheduling. Simulation results show that the proposed scheme provides almost the same delay performance as the optimal scheme found through an exhaustive search at the expense of a little additional power consumption and does not require a clustering method and orthogonal resources in a large-scale D2D network.

Cauliflower Seedling Recognition Using Stereo Vision and K-Medoids Algorithm under Weedy Conditions

The natural field environment is unstructured where the illumination conditions are uneven, the soil background is complex and changeable. Also, the improper of field management is likely to cause weeds whose colour and shape are similar to crops. Those factors make it hard to identify crops quickly and stably by traditional 2D image processing technologies. So the cauliflower seedling recognition method using binocular stereo vision and K-medoids was proposed in this paper. For the disparity map the Sum of Squared Differences (SSD) algorithm was used in the step of stereo matching. For the cauliflower seedling recognition the K-medoids algorithm was applied in the step of clustering. The experiment results showed that the average operation time of the SSD algorithm was 190ms. The real-time performance and matching effect of the SSD algorithm were superior to Belief Propagation algorithm (BP) and Dynamic Programming algorithm (DP). The average running time of the proposed method for processing the left image and its corresponding right images was 379ms, which meet the real-time requirements. The correct recognition rate of the method proposed was 98.75% of 240 pairs of cauliflower seedling images.

Distributed weighted least-squares estimation for networked systems with edge measurements

This paper studies the problem of distributed weighted least-squares (WLS) estimation for an interconnected linear measurement network with additive noise. Two types of measurements are considered: self measurements for individual nodes, and edge measurements for the connecting nodes. Each node in the network carries out distributed estimation by using its own measurement and information transmitted from its neighbours. We study two distributed estimation algorithms: a recently proposed distributed WLS algorithm and the so-called Gaussian Belief Propagation (BP) algorithm. We first establish the equivalence of the two algorithms. We then prove a key result which shows that the information matrix is always generalized diagonally dominant, under some very mild condition. Using these two results and some known convergence properties of the Gaussian BP algorithm, we show that the aforementioned distributed WLS algorithm computes the exact WLS solution asymptotically. A bound on its convergence rate is also presented.

Designing Finite Alphabet Iterative Decoders of LDPC Codes Via Recurrent Quantized Neural Networks

In this paper, we propose a new approach to design finite alphabet iterative decoders (FAIDs) for Low-Density Parity Check (LDPC) codes over binary symmetric channel (BSC) via recurrent quantized neural networks (RQNN). We focus on the linear FAID class and use RQNNs to optimize the message update look-up tables by jointly training their message levels and RQNN parameters. Existing neural networks for channel coding work well over Additive White Gaussian Noise Channel (AWGNC) but are inefficient over BSC due to the finite channel values of BSC fed into neural networks. We propose the bit error rate (BER) as the loss function to train the RQNNs over BSC. The low precision activations in the RQNN and quantization in the BER cause a critical issue that their gradients vanish almost everywhere, making it difficult to use classical backward propagation. We leverage straight-through estimators as surrogate gradients to tackle this issue and provide a joint training scheme. We show that the framework is flexible for various code lengths and column weights. Specifically, in high column weight case, it automatically designs low precision linear FAIDs with superior performance, lower complexity, and faster convergence than the floating-point belief propagation algorithms in waterfall region.

Distribution line parameter estimation considering dynamic operating states with a probabilistic graphical model

Accurate distribution line parameters are critical for power system control, operation and management. Adjustment of the operating modes makes the estimation of real-time distribution line parameters dynamic and difficult. To calibrate the distribution line parameters from collected operating data, this paper proposes a Discrete Dynamic Bayesian Network (DDBN)-based distribution line parameter estimation model. The maximum likelihood estimation algorithm is utilized to train the DDBN model. The belief propagation (BP) algorithm is used to infer distribution line parameters with the DDBN model. The accuracy of the proposed model and the inference algorithm is investigated by carrying out experiments in a real-world medium voltage distribution network state estimation environment. The experimental results demonstrate that the proposed DDBN model performs better than existing approaches in providing accurate real-time line parameters.

Artificial-Intelligence-Based Distributed Belief Propagation and Recurrent Neural Network Algorithm for Wide-Area Monitoring Systems

To monitor the power grid over a wide area, the wide area monitoring systems (WAMSs) has been developed. At each substation, the Global Positioning System (GPS) receiving system resides to provide trusted timing. Thus, it is critical for the WAMS to maintain authentic GPS timing over a wide area. However, GPS timing is susceptible to spoofing due to the unencrypted signal structure and its low signal power. Thus, to obtain trusted GPS timing from spoofing, a new wide-area monitoring algorithm, which comprises distributed belief propagation (BP) and a bidirectional recurrent neural network (RNN), is developed under the framework of artificial intelligence (AI). This joint BP-RNN algorithm authenticates each power substation by evaluating the estimated GPS timing error by its distributed processing capability. Specifically, the bidirectional RNN provides fast timing error estimation under the framework of AI. Simulation results demonstrate a fast detection time over the Kullback-Leibler divergence-based approach, and timing error estimation accuracy over the limit provided by the IEEE C37.118.1-2011 standard.

Convergence rate analysis of Gaussian belief propagation for Markov networks

Gaussian belief propagation algorithm ( GaBP ) is one of the most important distributed algorithms in signal processing and statistical learning involving Markov networks. It is well known that the algorithm correctly computes marginal density functions from a high dimensional joint density function over a Markov network in a finite number of iterations when the underlying Gaussian graph is acyclic. It is also known more recently that the algorithm produces correct marginal means asymptotically for cyclic Gaussian graphs under the condition of walk summability ( or generalised diagonal dominance ) . This paper extends this convergence result further by showing that the convergence is exponential under the generalised diagonal dominance condition, and provides a simple bound for the convergence rate. Our results are derived by combining the known walk summability approach for asymptotic convergence analysis with the control systems approach for stability analysis.

Smart Grid Security Enhancement by Using Belief Propagation

False data injection attack (FDIA) is a critical cyber-attack that can cause disrupt operations and subsequently blackouts in smart grid networks. Cleverly constructed stealthy false measurement vectors can circumvent the bad data detector unit and mislead the state estimation process. This article proposes a novel belief propagation (BP)-based algorithm to detect both random and stealthy-type FDIAs in smart grids with higher detection rate than the state-of-the-art machine learning classifiers such as Naive Bayes, support vector machines, Random Forest, OneR, and AdaBoost. Another novel feature of the proposed algorithm is to detect FDIAs without using any historical cyber-attack data, which are sketchy due to security constraints and infinitesimal in occurrence numbers. The proposed BP method utilizes local sensor measurement data to calculate local belief and send it as a message signal to the control center. Then, the control center determines final/global belief and compares the result with a predefined threshold value derived from the uncompromised measurement database. The real-time steady-state load data are utilized for dc state estimation. From the obtained results, performance parameters such as detection rate, receiver operating characteristic curve, precision, recall, and F-measure of the proposed BP algorithm are found superior to the aforementioned state-of-the-art machine learning algorithms.

Contamination source detection in water distribution networks using belief propagation

We present a Bayesian approach for the Contamination Source Detection problem in water distribution networks. Assuming that contamination is a rare event (in space and time), we try to locate the most probable source of such events after reading contamination patterns in few sensed nodes. The method relies on strong simplifications considering binary clean/contaminated states for nodes in discrete time, and therefore focuses on the time structure of the sensed patterns rather than on the concentration levels. As a result, a posterior probability over discrete variables is written, and posterior marginals are computed using belief propagation algorithm. The resulting algorithm runs once on a given observation and reports probabilities for each node being the source and for the contamination patterns altogether. We test it on Anytown model, proving its efficacy even when only a single sensed node is known.

Loopy belief propagation based data association for extended target tracking

The data association problem of multiple extended target tracking is very challenging because each target may generate multiple measurements. Recently, the belief propagation based multiple target tracking algorithms with high efficiency have been a research focus. Different from the belief propagation based Extended Target tracking based on Belief Propagation (ET-BP) algorithm proposed in our previous work, a new graphical model formulation of data association for multiple extended target tracking is proposed in this paper. The proposed formulation can be solved by the Loopy Belief Propagation (LBP) algorithm. Furthermore, the simplified measurement set in the ET-BP algorithm is modified to improve tracking accuracy. Finally, experiment results show that the proposed algorithm has better performance than the ET-BP and joint probabilistic data association based on the simplified measurement set algorithms in terms of accuracy and efficiency. Additionally, the convergence of the proposed algorithm is verified in the simulations.

Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation

Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields. Nevertheless, it remains unclear how probabilistic inference can be implemented by a network of spiking neurons in the brain. Previous studies have tried to relate the inference equation of binary Markov random fields to the dynamic equation of spiking neural networks through belief propagation algorithm and reparameterization, but they are valid only for Markov random fields with limited network structure. In this paper, we propose a spiking neural network model that can implement inference of arbitrary binary Markov random fields. Specifically, we design a spiking recurrent neural network and prove that its neuronal dynamics are mathematically equivalent to the inference process of Markov random fields by adopting mean-field theory. Furthermore, our mean-field approach unifies previous works. Theoretical analysis and experimental results, together with the application to image denoising, demonstrate that our proposed spiking neural network can get comparable results to that of mean-field inference.