Belief Propagation Algorithm Research Articles

Efficient approximations of the multi-sensor labelled multi-Bernoulli filter

In this paper, we propose two efficient, approximate formulations of the multi-sensor labelled multi-Bernoulli (LMB) filter, which both allow the sensors’ measurement updates to be computed in parallel. Our first filter is based on the direct mathematical manipulation of the multi-sensor, multi-object Bayes filter’s posterior distribution. Unfortunately, it requires the division of probability distributions and its extension beyond linear Gaussian applications is not obvious. Our second filter approximates the multi-sensor, multi-object Bayes filter’s posterior distribution using the geometric average of each sensor’s measurement-updated distribution. This filter can be used under non-linear conditions; however, it is not as accurate as our first filter. In both cases, we approximate the LMB filter’s measurement update using an existing loopy belief propagation algorithm. Both filters have a constant complexity in the number of sensors, and linear complexity in both number of measurements and objects. This is an improvement on an iterated-corrector LMB (IC-LMB) filter, which has linear complexity in the number of sensors. The proposed filters are of interest when tracking many objects using several sensors, where filter run-time is more important than filter accuracy. Simulations indicate that the filters’ loss of accuracy compared to the IC-LMB filter is not significant.

Joint Device Detection, Channel Estimation, and Data Decoding With Collision Resolution for MIMO Massive Unsourced Random Access

In this paper, we investigate a joint device activity detection (DAD), channel estimation (CE), and data decoding (DD) algorithm for multiple-input multiple-output (MIMO) massive unsourced random access (URA). Different from the state-of-the-art slotted transmission scheme, the data in the proposed framework is split into only two parts. A portion of the data is coded by compressed sensing (CS) and the rest is low-density-parity-check (LDPC) coded. In addition to being part of the data, information bits in the CS phase also undertake the task of interleaving pattern design and CE. The principle of interleave-division multiple access (IDMA) is exploited to reduce the interference among devices in the LDPC phase. Based on the belief propagation (BP) algorithm, a low-complexity iterative message passing (MP) algorithm is utilized to decode the data embedded in these two phases separately. Moreover, combined with successive interference cancellation (SIC), the proposed joint DAD-CE-DD algorithm is performed to further improve performance by utilizing the belief of each other. Additionally, based on the energy detection (ED) and sliding window protocol (SWP), we develop a collision resolution protocol to handle the codeword collision, a common issue in the URA system. In addition to the complexity reduction, the proposed algorithm exhibits a substantial performance enhancement compared to the state-of-the-art in terms of efficiency and accuracy.

Belief propagation for supply networks: efficient clustering of their factor graphs

We consider belief propagation (BP) as an efficient and scalable tool for state estimation and optimization problems in supply networks such as power grids. BP algorithms make use of factor graph representations, whose assignment to the problem of interest is not unique. It depends on the state variables and their mutual interdependencies. Many short loops in factor graphs may impede the accuracy of BP. We propose a systematic way to cluster loops of naively assigned factor graphs such that the resulting transformed factor graphs have no additional loops as compared to the original network. They guarantee an accurate performance of BP with only slightly increased computational effort, as we demonstrate by a concrete and realistic implementation for power grids. The method outperforms existing alternatives to handle the loops. We point to other applications to supply networks such as gas-pipeline or other flow networks that share the structure of constraints in the form of analogues to Kirchhoff’s laws. Whenever small and abundant loops in factor graphs are systematically generated by constraints between variables in the original network, our factor-graph assignment in BP complements other approaches. It provides a fast and reliable algorithm to perform marginalization in tasks like state determination, estimation, or optimization issues in supply networks.Graphical abstract

Simplify Belief Propagation and Variation Expectation Maximization for Distributed Cooperative Localization

Only a specific location can make sensor data useful. The paper presents an simplify belief propagation and variation expectation maximization (SBPVEM) algorithm to achieve node localization by cooperating with another target node while lowering communication costs in a challenging environment where the anchor is sparse. A simplified belief propagation algorithm is proposed as the overall reasoning framework by modeling the cooperative localization problem as a graph model. The high-aggregation sampling and variation expectation–maximization algorithm is applied to sample and fit the complicated distribution. Experiments show that SBPVEM can obtain accurate node localization equal to NBP and SPAWN in a challenging environment while reducing bandwidth requirements. In addition, the SBPVEM has a better expressive ability than PVSPA, for SBPVEM is efficient in challenging environments.

A Polygonal Line Min-Sum Decoding Scheme for Low Density Parity Check Codes

Low-density parity-check (LDPC) codes are widely used as error correction codes in new generation digital TV standards, such as the second generation of terrestrial digital video broadcasting standard (DVB-T2), Advanced Television Systems Committee (ATSC) 3.0, etc. The nonlinear belief propagation (BP) algorithm has excellent decoding performance for LDPC codes, but is often simplified in hardware implementations by linear min-sum (MS) algorithm due to its high complexity. This simplification also leads to over-estimation problems, which can be corrected by adding factors in conventional algorithms (e.g., normalized min-sum (NMS), offset min-sum (OMS), and variable scaling normalized min-sum (VMS) algorithms). However, the correction factors of these modified MS algorithms cannot adapt to different channels and modulations, and the performance needs further improvement. In this paper, the concepts of over-estimation value (OEV) and over-estimation rate (OER) are introduced to describe the over-estimation problem of the MS algorithm. Then, under the guidance of OEV and OER, a polygonal line min-sum (PMS) algorithm with correction factors adapted to different channels and modulations is proposed according to LLR distribution. Following the properties of OEV and OER, PMS algorithm is further simplified into Simplified PMS (SPMS) algorithm. LDPC codes from ATSC 3.0 are adopted in this paper to evaluate SPMS algorithm in comparison with the conventional algorithms. Extensive simulation results show that the SPMS algorithm for ATSC 3.0 LDPC decoder has at most 1.61dB, 0.24dB and 0.36dB gain over NMS, OMS and VMS algorithms respectively when frame error rate (FER) is at 10⁻⁴ level over additive white Gaussian noise (AWGN) channel with QPSK modulation. More importantly, the simulation results show that the SPMS algorithm can achieve much better performance than these modified MS algorithms over AWGN and Rayleigh channel with higher-order modulations or under limited maximum iteration number.

A distributed message passing algorithm for the capacitated directed Chinese postman problem

Message passing is a class of extremely powerful distributed iterative algorithms based on probabilistic graphic model, in which little computation performed per iteration on minimal data structure. As a prototypical message-passing algorithm, belief propagation (BP) algorithm has wide applications in various fields of coding theory, machine learning and combinatorial optimization. Due to its distributed and iterative nature, BP algorithm can run effectively and fast on large data networks. In this paper, we study the behavior of Min-Sum BP algorithm for the capacitated directed Chinese postman problem (CDCP). We derive the iterative process of message passing for solving CDCP. As the main result, for any weighted digraph G of size n, if the weight on each edge is nonnegative integral, then our algorithm converges to the optimal solution of CDCP after O(w∗n2) iterations, provided that CDCP has a unique optimal solution, where w∗=max{we:e∈E(G)}.

A Belief Propagation Based State Estimator for Semi-Intrusive Load Monitoring System

For industrial application of load monitoring techniques, it is important to establish high-performance state estimators of low-cost and low frequency smart meters (SM) and sensors in a power system, which can run under resource-constrained computing units. Because household electronic appliances often tap power from fixed sockets, a finite state table for the corresponding sensors is suitable and convenient. However, SM in the main line may have an enormous state table. In this study, we propose a belief propagation (BP) algorithm to calculate the power consumption of electronic appliances in a semi-intrusive load monitoring (SILM) system whose SM and sensors have state tables with sizes varying largely. The novelty of the proposed method lies in a continuous approximation to a large state table and a switching scheme between discrete and continuous parts of the SILM system. With datasets from numerical simulations and a real-world experimental SILM system in a set of high-density school buildings within a secondary distribution network, the proposed BP algorithm is compared with relevant state-of-the-art algorithms. The results show that the proposed algorithm achieves a percentage of error (8%), which outperforms the percentage achieved by the other methods, a linear state estimation of 99%, a hidden Markov model of 21%, and a full-discrete BP algorithm of 11%. In addition, the complexity of the proposed algorithm is the least of all methods, and the proposed algorithm can run by SoC on concentrators.

Adaptive Kernel Kalman Filter Based Belief Propagation Algorithm for Maneuvering Multi-Target Tracking

This letter incorporates the adaptive kernel Kalman filter (AKKF) into the belief propagation (BP) algorithm for multi-target tracking (MTT) in single-sensor systems. The algorithm is capable of tracking an unknown and time-varying number of targets, in the presence of false alarms, clutter and measurement-to-target association uncertainty. Experiment results reveal that the proposed method has a favourable tracking performance using the generalized optimal sub-patten assignment (GOSAP) metrics at substantially less computation cost than the particle filter (PF) based multi-target tracking (MTT) BP algorithm.

Deployment of Polar Codes for Mission-Critical Machine-Type Communication Over Wireless Networks

Mission critical Machine-type Communication, also referred to as Ultra-reliable Low Latency Communication is primarily characterized by communication that provides ultra-high reliability and very low latency to concurrently transmit short commands to a massive number of connected devices. While the reduction in PHY layer overhead and improvement in channel coding techniques are pivotal in reducing latency and improving reliability, the current wireless standards dedicated to support mcMTC rely heavily on adopting the bottom layers of general-purpose wireless standards and customizing only the upper layers. The mcMTC has a significant technical impact on the design of all layers of the communication protocol stack. In this paper, an innovative bottom-up approach has been proposed for mcMTC applications through PHY layer targeted at improving the transmission reliability by implementing ultra-reliable channel coding scheme in the PHY layer of IEEE 802.11a bearing in mind short packet transmission system. To achieve this aim, we analyzed and compared the channel coding performance of convolutional codes, LDPC codes, and polar codes in wireless network on the condition of short data packet transmission. The Viterbi decoding algorithm, logarithmic belief propagation algorithm, and cyclic redundancy check - successive cancellation list decoding algorithm were adopted to CC, LDPC codes, and polar codes, respectively. Consequently, a new PHY layer for mcMTC has been proposed. The reliability of the proposed approach has been validated by simulation in terms of Bit error rate vs. SNR. The simulation results demonstrate that the reliability of IEEE 802.11a standard has been significantly improved to be at PER less 10e-5 with the implementation of polar codes. The results also show that the general-purpose wireless networks are prominent in providing short packet mcMTC with the modification needed.

Design of Code Pair for Protograph-LDPC Codes-Based JSCC System With Joint Shuffled Scheduling Decoding Algorithm

Although there are many studies on code optimization of the joint source-channel coding (JSCC) system based on double protograph low-density parity-check codes with the joint belief propagation (JBP) algorithm, but it is still unknown whether the source code and channel code (as a code pair) can perform well when the joint shuffled scheduling decoding (JSSD) algorithm is adopted. In this letter, two decoding threshold analysis algorithms, including joint shuffled protograph extrinsic information transfer (PEXIT) and source shuffled PEXIT algorithm, are proposed to calculate joint/source decoding thresholds for this system with the JSSD algorithm. With the proposed algorithms, it is found that the optimized code pairs for this system with the JBP algorithm may not perform well with the JSSD algorithm, implying that code pair with the JSSD algorithm needs to be redesigned. Then, a two-stage optimized framework is proposed to design the code pair for this system with the JSSD algorithm. Simulations and decoding threshold analysis both show that the proposed code pair for this system with the JSSD algorithm can obtain lower error floor and better waterfall performance than the existing code pairs.

Reduced-complexity decoding implementation of QC-LDPC codes with modified shuffling

Layered decoding (LD) facilitates a partially parallel architecture for performing belief propagation (BP) algorithm for decoding low-density parity-check (LDPC) codes. Such a schedule for LDPC codes has, in general, reduced implementation complexity compared to a fully parallel architecture and higher convergence rate compared to both serial and parallel architectures, regardless of the codeword length or code-rate. In this paper, we introduce a modified shuffling method which shuffles the rows of the parity-check matrix (PCM) of a quasi-cyclic LDPC (QC-LDPC) code, yielding a PCM in which each layer can be produced by the circulation of its above layer one symbol to the right. The proposed shuffling scheme additionally guarantees the columns of a layer of the shuffled PCM to be either zero weight or single weight. This condition has a key role in further decreasing LD complexity. We show that due to these two properties, the number of occupied look-up tables (LUTs) on a field programmable gate array (FPGA) reduces by about 93% and consumed on-chip power by nearly 80%, while the bit error rate (BER) performance is maintained. The only drawback of the shuffling is the degradation of decoding throughput, which is negligible for low values of E_b/N_0 until the BER of 1e−6.

Flexible soft-output decoding of polar codes

In this research, we study soft-output decoding of polar codes. Two representative soft-output decoding algorithms are belief propagation (BP) and soft cancellation (SCAN). The BP algorithm has low latency but suffers from high computational complexity. On the other hand, the SCAN algorithm, which is proposed for reduced complexity of soft-output decoding, achieves good decoding performance but suffers from long latency. These two algorithms are suitable only for two extreme cases that need very low latency (but with high complexity) or very low complexity (but with high latency). However, many practical systems may need to work for the moderate cases (i.e., not too high latency and not too high complexity) rather than two extremes. To adapt to the various needs of the systems, we propose a very flexible soft-output decoding framework of polar codes. Depending on which system requirement is most crucial, the proposed scheme can adapt to the systems by controlling the level of parallelism. Numerical results demonstrate that the proposed scheme can effectively adapt to various system requirements by changing the level of parallelism.

Social event planning using hybrid pairwise Markov random fields

Event-based social networks (EBSNs) have become increasingly popular, which provide online social event management platforms for event organizers to publish and share social events (e.g., outdoor activities). In EBSNs, a major challenge for a social event organizer is how to plan a social event to attract the maximum number of attendance. To organize an event, three essential elements are required, namely, what (i.e., event content), where (i.e., event location), and when (i.e., event time). In this paper, we focus on the social event planning problem, which selects a location and time to hold a social event for the organizer with the given event content, to maximize the total number of participants. The solution of the social event planning problem could support decision-making for social event organizers. For simplicity, we denote a location and time pair as an item in this paper. To solve the social event planning problem, we present a hybrid pairwise Markov random field (H-PMRF) model which takes latent preferences of users, latent attributes of items, similarities between users and similarities between items into consideration. In particular, we construct an undirected graph where each node represents a user's decision on a specific item and each edge represents the relationship between the nodes, define the node potentials and edge potentials which model the dependency relationships between nodes, and give a joint probability distribution over the graph. Further, we adopt the Loopy Belief Propagation algorithm to compute the posterior probability distribution of each node in H-PMRF and select the location and time to hold the event which could attract the maximum number of participants. We collect real-world data set from DoubanEvent website and conduct extensive experiments on it. Experimental results show that the proposed model outperforms several baselines.

Self-Corrected Belief-Propagation Decoder for Source Coding With Unknown Source Statistics

This letter describes a practical Slepian-Wolf source coding scheme based on Low Density Parity Check (LDPC) codes. It considers the realistic setup where the parameters of the statistical model between the source and the side information are unknown. A novel Self-Corrected Belief-Propagation (SC-BP) algorithm is proposed in order to make the coding scheme robust to incorrect model parameters by introducing some memory inside the LDPC decoder. A Two Dimensional Density Evolution (2D-DE) analysis is then developed to predict the theoretical performance of the SC-BP decoder. Both the 2D-DE analysis and Monte-Carlo simulations confirm the robustness of the SC-BP decoder. The proposed solution allows for an important complexity reduction and shows a performance very close to existing methods which jointly estimate the model parameters and the source sequence.

Iterative Soft-Input Soft-Output Bit-Level Reed-Solomon Decoder Based on Information Set Decoding

In this paper, a bit-level decoder is presented for soft-input soft-output iterative decoding of Reed-Solomon (RS) codes. The main aim for the development of the proposed algorithm is to reduce the complexity of the decoding process, while yielding a relatively good error correction performance, for the efficient use of RS codes. The decoder utilises information set decoding techniques to reduce the computational complexity cost by lowering the iterative convergence rate during the decoding process. As opposed to most iterative bit-level soft-decision decoders for RS codes, the proposed algorithm is also able to avoid the use of belief propagation in the iterative decoding of the soft bit information, which also contributes to the reduction in the computational complexity cost of the decoding process. The performance of the proposed decoder is investigated when applied to short RS codes. The error correction simulations show the proposed algorithm is able to yield a similar performance to that of the Adaptive Belief Propagation (ABP) algorithm, while being a less complex decoder.

Distributed time synchronization algorithm based on sequential belief propagation in wireless sensor networks

In the context of the non-Gaussian delay model, the fully distributed time synchronization approach based on Gaussian belief propagation will lead to the decline of synchronization accuracy. This paper proposes a distributed time synchronization algorithm based on sequential belief propagation (SBP-DTS), which assumes that the network delay is unknown. SBP-DTS first establishes a factor graph (FG) model for the time synchronization problem of wireless sensor networks (WSNs), and then uses sequential belief propagation (BP) algorithms to estimate node clock parameters under an unknown random delay model. At the same time, in order to reduce the amount of data exchanged between nodes during the execution of sequential belief propagation algorithm, the weighted expectation–maximization (EM) algorithm is used to reduce the number of Gaussian mixture components in the message. At last, the performance of SBP-DTS is evaluated under asymmetric Gaussian and exponential delay models.

Data-Driven Approach for Analyzing Spatiotemporal Price Elasticities of EV Public Charging Demands Based on Conditional Random Fields

With the increase of electric vehicle (EV) sales, the pricing strategies of public charging stations have significant impacts on their revenues and the spatiotemporal distribution of charging loads. In this paper, we quantify three kinds of price elasticity of charging demands based on the historical charging data of multiple public charging stations with different pricing schemes. The relationship between the volume-weighted average price (VWAP) and the corresponding total charging demand within a zone is studied, which does not require changing the charging prices to estimate elasticity. To evaluate the shifting of charging demands in different periods and zones, a conditional random field (CRF) model is built, which depicts the spatiotemporal correlations of charging demands. In this model, the VWAPs and the total charging demands are taken as observed variables and hidden variables, respectively. The loopy belief propagation algorithm is used to infer the loopy graph approximately, and the learning algorithm with forgetting factors is used to estimate the unknown parameters of the CRF model. The price elasticities are derived from CRF, and the elasticity matrices of charging demands are obtained. Computational results based on historical charging data verify the validity of the proposed model and method.

Non-rigid registration of biomedical image for radiotherapy based on adaptive feature density flow

This paper presents a non-rigid biomedical image registration method based on Adaptive feature density flow (AFDF). Considering the muhiplicative characteristics of metadata signal for instance Magnetic Resonance Imaging (MRI) with high-resolution and high-sensitivity, the Non-Local Means Filtering Adapted to Rician Noise (NLMr) is employed to remove Rician noise rather than Gaussian noise with higher Peak Signal to Noise Ratio (PSNR). The deformation information and local shape information are homogeneous, so a flexible dimension of Fast Local Self-Similarity (FLSS) feature descriptor can be constructed to obtain adaptive FLSS (AFLSS) feature descriptor with adaptive Angle and Radial intervals. On the basis of AFLSS, the Adaptive feature density flow (AFDF) is proposed by calculating the dense feature flow field via Belief Propagation algorithm and thereby further optimized by the Pyramid iterative method. Finally, the image registration is completed by using the bicubic interpolation algorithm according to the dense feature flow field. Aiming at radiotherapy positioning control through precise robust registration of biomedical images, experiments of human brain MR images are carried out. The comparative results with PSNR, Root Mean Square Error (RMSE) and normalize mutual information (NMI) show that AFDF performs better than the optical flow method and Scale Invariant Feature Transform (SIFT) flow method, especially for the biomedical scenario with unconscious creep deformation.

A 7.8–13.6 pJ/b Ultra-Low Latency and Reconfigurable Neural Network-Assisted Polar Decoder With Multi-Code Length Support

Polar codes have been officially selected as the channel coding in 5G standard. To meet the requirements of enhanced mobile broadband (eMBB), most published polar decoder chips aim to improve throughput rate and error-correction performance. However, to meet with the requirements of another two 5G new radio (NR) application scenarios, ultra-reliable low-latency communications (URLLC), and massive machine-type communications (mMTC), the design features of low latency and energy efficiency are also desirable. In this article, we present a 7.8-13.6 pJ/b ultra-low latency and energy-efficient polar decoder fabricated in 40nm CMOS technology. By adopting the decoding algorithm of recurrent neural network-assisted belief propagation (RNN-BP), the learned scaling parameters can improve the convergence rate by 8 times with reasonable hardware and memory overhead. Then, by taking advantage of BP's regular structure, we propose a fully-reconfigurable RNN-BP decoder architecture to support multiple code lengths with negligible hardware complexity. It contributes to 2- 8× improved hardware utilization rate while providing a flexible adjustment between throughput and error-correction performance. At the architectural level, two optimization techniques for the design of the processing element (PE) are proposed to jointly reduce the chip's area and power by 73% and 67%, respectively. From the measurement results, our reconfigurable RNN-BP polar decoder chip has 2.3×, 2.3×, and 10.0× enhancement over prior designs in terms of latency, throughput rate, and energy efficiency. Consequently, our reconfigurable design has great potential to meet various 5G NR applications.

Tensor networks contraction and the belief propagation algorithm

Belief Propagation is a well-studied message-passing algorithm that runs over graphical models and can be used for approximate inference and approximation of local marginals. The resulting approximations are equivalent to the Bethe-Peierls approximation of statistical mechanics. Here we show how this algorithm can be adapted to the world of PEPS tensor networks and used as an approximate contraction scheme. We further show that the resultant approximation is equivalent to the ``mean field'' approximation that is used in the Simple-Update algorithm, thereby showing that the latter is a essentially the Bethe-Peierls approximation. This shows that one of the simplest approximate contraction algorithms for tensor networks is equivalent to one of the simplest schemes for approximating marginals in graphical models in general, and paves the way for using improvements of BP as tensor networks algorithms.