Message-passing Algorithm Research Articles

Capacity Analysis and Data Detection of OvTDM-MIMO System

It is well known that the channel capacity of traditional MIMO systems is positively correlated with the number of antennas, so MIMO systems usually use more antennas to meet higher system capacity requirements, which is increasingly conflicting with the miniaturization requirements of portable devices. To overcome the above problems, a new OvTDM-MIMO system is proposed, which can effectively improve the capacity of the system. This system introduces the overlapped time division multiplexing (OvTDM) technology into the MIMO system, which can further improve the channel capacity of the MIMO system by using a smaller number of antennas. This paper introduces the transceiver model of the OvTDM-MIMO system in detail and derives the channel capacity of the system based on mutual information theory. Compared with a conventional MIMO system, the OvTDM-MIMO system has the ability to improve channel capacity. Then, this paper proposes an OvTDM-MIMO symbol detection scheme based on an improved low-complexity Orthogonal Approximate Message Passing (OAMP) algorithm to solve the symbol detection problem caused by symbol correlation. The main design idea of this scheme is to use symbol correlation as the detection constraint and realizes joint symbol detection by combining adjacent symbols, avoiding the problem of excessive matrix size caused by Kronecker product operation. The simulation results show that the proposed detection algorithm can achieve similar performance to traditional MIMO system detection algorithms with low computational complexity.

Algebra of L-Banded Matrices

Convergence is a crucial issue in iterative algorithms. Damping is commonly employed to ensure the convergence of iterative algorithms. The conventional ways of damping are scalar-wise, and either heuristic or empirical. Recently, an analytically optimized vector damping was proposed for memory message-passing (iterative) algorithms. As a result, it yields a special class of covariance matrices called L-banded matrices. In this paper, we show these matrices have broad algebraic properties arising from their L-banded structure. In particular, compact analytic expressions for the LDL decomposition, the Cholesky decomposition, the determinant after a column substitution, minors, and cofactors are derived. Furthermore, necessary and sufficient conditions for an L-banded matrix to be definite, a recurrence to obtain the characteristic polynomial, and some other properties are given. In addition, we give new derivations of the determinant and the inverse.

Message Passing-Based 9-D Cooperative Localization and Navigation With Embedded Particle Flow

Cooperative localization (CL) is an important technology for innovative services such as location-aware communication networks, modern convenience, and public safety. We consider wireless networks with mobile agents that aim to localize themselves by performing pairwise measurements amongst agents and exchanging their location information. Belief propagation (BP) is a state-of-the-art Bayesian method for CL. In CL, particle-based implementations of BP often are employed that can cope with non-linear measurement models and state dynamics. However, particle-based BP algorithms are known to suffer from particle degeneracy in large and dense networks of mobile agents with high-dimensional states. This paper derives the messages of BP for CL by means of particle flow, leading to the development of a distributed particle-based message-passing algorithm which avoids particle degeneracy. Our combined particle flow-based BP approach allows the calculation of highly accurate proposal distributions for agent states with a minimal number of particles. It outperforms conventional particle-based BP algorithms in terms of accuracy and runtime. Furthermore, we compare the proposed method to a centralized particle flow-based implementation, known as the exact Daum-Huang filter, and to sigma point BP in terms of position accuracy, runtime, and memory requirement versus the network size. We further contrast all methods to the theoretical performance limit provided by the posterior Cramér-Rao lower bound (PCRLB). Based on three different scenarios, we demonstrate the superiority of the proposed method.

Replica analysis of the lattice-gas restricted Boltzmann machine partition function

We study the expectation value of the logarithm of the partition function of large binary-to-binary lattice-gas restricted Boltzmann machines (RBMs) within a replica-symmetric ansatz, averaging over the disorder represented by the parameters of the RBM Hamiltonian. Averaging over the Hamiltonian parameters is done with a diagonal covariance matrix. Due to the diagonal form of the parameter covariance matrix not being preserved under the isomorphism between the Ising and lattice-gas forms of the RBM, we find differences in the behaviour of the quenched log partition function of the lattice-gas RBM compared to that of the Ising RBM form usually studied. We obtain explicit expressions for the expectation and variance of the lattice-gas RBM log partition function per node in the thermodynamic limit. We also obtain explicit expressions for the leading order finite size correction to the expected log partition function per node, and the threshold for the stability of the replica-symmetric approximation. We show that the stability threshold of the replica-symmetric approximation is equivalent, in the thermodynamic limit, to the stability threshold of a recent message-passing algorithm used to construct a mean-field Bethe approximation to the RBM free energy. Given the replica-symmetry assumption breaks down as the level of disorder in the spin-spin couplings increases, we obtain asymptotic expansions, in terms of the variance controlling this disorder, for the replica-symmetric log partition function and the replica-symmetric stability threshold. We confirm the various results derived using simulation.

Research on improved multiple users detection algorithm in pattern division multiple access system

Receiver detection algorithm has always been a research hotspot in communication system and the key point to determine the performance of 5G system. Both industry and academia have been committed to finding detection algorithms with lower complexity and higher detection performance. This paper will study how to solve the problems of low detection performance and high complexity of classical algorithms. Taking Pattern Division Multiple Access (PDMA) as an example, this paper improves the existing receiver detection algorithms such as Minimum Mean Square Error-Serial Interference Cancellation (MMSE-SIC), Message Passing Algorithm (MPA) and Expectation Propagation Algorithm (EPA), and puts forward corresponding improved algorithms with specific formulas, i.e. Enhanced-MMSE-SIC, Turbo-like MPA (MPA-Turbo) and Turbo-like EPA with hybrid parallel interference cancellation (EPA-Turbo-PIC). Link-level simulation results and analysis show that the proposed improved algorithms have performance advantages over the corresponding traditional algorithms.

On Near-Optimal Codebook and Receiver Designs for MIMO-SCMA Schemes

Sparse code multiple access (SCMA) is a promising code-domain non-orthogonal multiple access (NOMA) technique that supports high-level connectivity. Through the combination of SCMA and multiple-input multiple-output (MIMO) techniques, MIMO-SCMA schemes are capable of enhancing the overall spectral efficiency. This paper proposes novel MIMO-SCMA designs for both uplink and downlink Rayleigh fading channels. Rather than directly combining the SCMA and spatial multiplexing techniques presented in the previous literature, the proposed designs consider not only the frequency or time diversity, but also simultaneously include space diversity. Codebook design criteria for different transmission scenarios are respectively investigated, and a low-complexity codebook design algorithm is proposed based on the cross-entropy method. In addition, since the new designs suggest a dense factor graph, conventional message-passing algorithm (MPA)-based detectors are less suitable. Consequently, low-complexity MIMO-SCMA codeword detectors based on depth-first and breadth-first tree-search algorithms are respectively investigated. It is shown that the overall designs are able to simultaneously achieve a higher diversity order, enhanced BER results, and lower levels of detection complexity for both uplink and downlink scenarios, when compared to conventional MIMO-SCMA schemes.

On the Convergence of Orthogonal/Vector AMP: Long-Memory Message-Passing Strategy

Orthogonal/vector approximate message-passing (AMP) is a powerful message-passing (MP) algorithm for signal reconstruction in compressed sensing. This paper proves the convergence of Bayes-optimal orthogonal/vector AMP in the large system limit. The proof strategy is based on a novel long-memory (LM) MP approach: A first step is a construction of LM-MP that is guaranteed to converge systematically. A second step is a large-system analysis of LM-MP via an existing framework of state evolution. A third step is to prove the convergence of state evolution recursions for Bayes-optimal LM-MP via a new statistical interpretation of existing LM damping. The last is an exact reduction of the state evolution recursions for Bayes-optimal LM-MP to those for Bayes-optimal orthogonal/vector AMP. The convergence of the state evolution recursions for Bayes-optimal LM-MP implies that for Bayes-optimal orthogonal/vector AMP. Numerical simulations are presented to show the verification of state evolution results for damped orthogonal/vector AMP and a negative aspect of LM-MP in finite-sized systems.

Network Pruning Using Adaptive Exemplar Filters.

Popular network pruning algorithms reduce redundant information by optimizing hand-crafted models, and may cause suboptimal performance and long time in selecting filters. We innovatively introduce adaptive exemplar filters to simplify the algorithm design, resulting in an automatic and efficient pruning approach called EPruner. Inspired by the face recognition community, we use a message-passing algorithm Affinity Propagation on the weight matrices to obtain an adaptive number of exemplars, which then act as the preserved filters. EPruner breaks the dependence on the training data in determining the "important" filters and allows the CPU implementation in seconds, an order of magnitude faster than GPU-based SOTAs. Moreover, we show that the weights of exemplars provide a better initialization for the fine-tuning. On VGGNet-16, EPruner achieves a 76.34%-FLOPs reduction by removing 88.80% parameters, with 0.06% accuracy improvement on CIFAR-10. In ResNet-152, EPruner achieves a 65.12%-FLOPs reduction by removing 64.18% parameters, with only 0.71% top-5 accuracy loss on ILSVRC-2012. Our code is available at https://github.com/lmbxmu/EPruner.

Analysis and Design of Compressive Pulsed Radar Based on Adaptive Pipelined Algorithm

Nowadays, applying the theory of compressive sensing in radar systems based on the traditional Complex Approximate Message Passing (CAMP) algorithm has many advantages such as; memory size, used lower sampling rate, and low hardware complexity compared with other techniques, and consequently lower processing time. In this paper, an analysis of the adaptive CAMP algorithm is illustrated for pulsed radar signals depending on the adaptive CAMP threshold. The performance of the proposed adaptive CAMP algorithm is compared with that of the traditional one with respect to detection performance using Receiver Operating Characteristic (ROC) curves, resolution in range, and the possible number of detected targets. Design and implementation of the proposed algorithm are achieved using FPGA based on parallel and pipeline processes which has a good effect in processing time enhancement and consequently reduction in hardware complexity compared with the traditional algorithm.

Asynchronous Partial Gaussian Approximation Detection Algorithm for Uplink-Grouped MIMO-SCMA System

To relieve the impact caused by different arrival times of multiple users in the uplink MIMO-SCMA system, a grouped multiuser transmission model, according to different delays and its corresponding detection scheme, is proposed in this paper. Assuming perfect synchronization in one group, a message-passing algorithm based on serial propagation (SP-MPA) is proposed to reduce the bit error rate (BER), which could transfer the updated information to the next symbol as its initial probability. Furthermore, with a more practical case, the partial Gaussian approximation method (PGA) is designed to decrease the interference resulting from the imperfect synchronization in one group. As the result, the computing complexity of the proposed PGA method could be decreased by at least 20% compared with SP-MPA and the BER could be improved by about 10%.

Joint Channel Estimation and Data Detection for Hybrid RIS Aided Millimeter Wave OTFS Systems

For high mobility communication scenario, the recently emerged orthogonal time frequency space (OTFS) modulation introduces a new delay-Doppler domain signal space, and can provide better communication performance than traditional orthogonal frequency division multiplexing system. This article focuses on the joint channel estimation and data detection (JCEDD) for hybrid reconfigurable intelligent surface (HRIS) aided millimeter wave (mmWave) OTFS systems. Firstly, a new transmission structure is designed. Within the pilot durations of the designed structure, partial HRIS elements are alternatively activated. The time domain channel model is then exhibited. Secondly, the received signal model for both the HRIS over time domain and the base station over delay-Doppler domain are studied. Thirdly, by utilizing channel parameters acquired at the HRIS, an HRIS beamforming design strategy is proposed. For the OTFS transmission, we propose a JCEDD scheme over delay-Doppler domain. In this scheme, message passing (MP) algorithm is designed to simultaneously obtain the equivalent channel gain and the data symbols. On the other hand, the channel parameters, i.e., the Doppler shift, the channel sparsity, and the channel variance, are updated through expectation-maximization (EM) algorithm. By iteratively executing the MP and EM algorithm, both the channel and the unknown data symbols can be accurately acquired. Finally, simulation results are provided to validate the effectiveness of our proposed JCEDD scheme.

Low-complexity hybrid interference cancellation for sparse code multiple access

Sparse Code Multiple Access (SCMA) is one of Non Orthogonal Multiple Access (NOMA) candidates for 5G, the fifth generation mobile network. Multiple overlapped blocks of SCMA code are transmitted per frequency layer which compels the receiver to cancel unwanted SCMA code blocks. Thanks to the sparsity of SCMA, Message Passing Algorithm (MPA) achieves near-optimal performance. Nevertheless, this process is time-consuming and has a high degree of complexity even if Max-Log Algorithm is used. The major contribution of this paper is to propose a new decoding scheme called Hybrid Interference Cancellation (HIC) to reduce time processing without degrading a target Bit Error Rate (BER). The HIC decodes and removes the strongest user, reducing the complexity of Log Domain MPA decoding of remaining users, followed by a Low Density Parity Code (LDPC) decoder. The interference cancellation decoder is either a hard interference cancellation or a soft interference cancellation. The originality of the paper is to calculate a threshold value based on power measurements to decide whether hard decoder or soft decoder should be used to reach the BER target with a lower complexity. Results show a significant reduction in terms of complexity as compared to the traditional MPA process for the same values of BER.

Quantum message-passing algorithm for optimal and efficient decoding

Recently, Renes proposed a quantum algorithm called belief propagation with quantum messages (BPQM) for decoding classical data encoded using a binary linear code with tree Tanner graph that is transmitted over a pure-state CQ channel \cite{renes_2017}, i.e., a channel with classical input and pure-state quantum output. The algorithm presents a genuine quantum counterpart to decoding based on the classical belief propagation algorithm, which has found wide success in classical coding theory when used in conjunction with LDPC or Turbo codes. More recently Rengaswamy etal. \cite{rengaswamy_2020} observed that BPQM implements the optimal decoder on a small example code, in that it implements the optimal measurement that distinguishes the quantum output states for the set of input codewords with highest achievable probability. Here we significantly expand the understanding, formalism, and applicability of the BPQM algorithm with the following contributions. First, we prove analytically that BPQM realizes optimal decoding for any binary linear code with tree Tanner graph. We also provide the first formal description of the BPQM algorithm in full detail and without any ambiguity. In so doing, we identify a key flaw overlooked in the original algorithm and subsequent works which implies quantum circuit realizations will be exponentially large in the code dimension. Although BPQM passes quantum messages, other information required by the algorithm is processed globally. We remedy this problem by formulating a truly message-passing algorithm which approximates BPQM and has quantum circuit complexity O(poly n,polylog 1ϵ), where n is the code length and ϵ is the approximation error. Finally, we also propose a novel method for extending BPQM to factor graphs containing cycles by making use of approximate cloning. We show some promising numerical results that indicate that BPQM on factor graphs with cycles can significantly outperform the best possible classical decoder.

Phase Transition of Total Variation Based on Approximate Message Passing Algorithm

In compressed sensing (CS), one seeks to down-sample some high-dimensional signals and recover them accurately by exploiting the sparsity of the signals. However, the traditional sparsity assumption cannot be directly satisfied in most practical applications. Fortunately, many signals-of-interest do at least exhibit a low-complexity representation with respect to a certain transformation. Particularly, total variation (TV) minimization is a notable example when the transformation operator is a difference matrix. Presently, many theoretical properties of total variation have not been completely explored, e.g., how to estimate the precise location of phase transitions and their rigorous understanding is still in its infancy. So far, the performance and robustness of the existing algorithm for phase transition prediction of TV model are not satisfactory. In this paper, we design a new approximate message passing algorithm to solve the above problems, called total variation vector approximate message passing (TV-VAMP) algorithm. To be specific, we first consider the problem from the Bayesian perspective, and formulate it as an optimization problem. Then, the vector factor graph for the TV model is constructed based on the formulized problem. Finally, the TV-VAMP algorithm is derived according to the idea of probabilistic inference in machine learning. Compared with the existing algorithm, our algorithm can be applied to a wider range of measurements distributions, including the non-zero-mean Gaussian distribution measurements matrix and ill-conditioned measurements matrix. Furthermore, in experiments with various settings, including different measurement distribution matrices, signal gradient sparsity, and measurement times, the proposed algorithm can almost reach the target mean squared error (−60 dB) with fewer iterations and better fit the empirical phase transition curve.

Finite-time consensus for leader-follower and leaderless swarms in the presence of malicious agents

A finite-time resilient consensus protocol (RCP) is developed for a connected network of agents, where communication between agents occurs locally, a few of the agents are malicious (MA), and the non-malicious or cooperating (CO) agents do not know the locations of the MA ones. Networks with a single leader and several followers as well as leaderless networks are considered. Agents are modelled with first-order dynamics, and the inputs to each CO agent that enable consensus are designed using the principles of sliding mode control (SMC). An SMC-based consensus protocol (CP), derived using the Laplacian matrix of the graph describing the network that permits consensus amongst CO agents, is first illustrated; that MA agents can prevent consensus with the use of this CP is also discussed. The SMC-based RCP proposed in this paper requires that the CO agents know the bounds, defined by a statistical distribution, of other CO agents, and that the subgraph of the CO agents be connected; knowledge of MA agents is not needed. With these assumptions, in the case of networks with a single MA agent, CO agents can reach a consensus by disregarding information transmitted by this agent if its state violates some statistical properties. On the other hand, multiple MA agents can be disregarded without the need for these assumptions, owing to the relations that hold with the occurrence of sliding mode. The RCP consists of a message-passing algorithm whose basis can be found in the distributed computing literature. The proposed RCP can also be applied for a leader-follower network.

Optical SCMA coding technique based on fully interleaved 3D codebook

In the paper, an optical Sparse code division multiple access (SCMA) coding technique based on fully interleaved 3D codebook is proposed. We design a completely interleaved 3D codebook, the projection of all constellation points in the joint constellation on the XY, XZ and YZ planes are all interleaved, which is beneficial for making the complexity of message passing algorithm (MPA) lower. We used 3D amplitude and phase modulation (3D-CAP) to modulate 3D-SCMA signal, and then transmitted the signal with 7-core fiber. The feasibility of 3D-SCMA is proved by experiments. Our proposed 3D-SCMA has the same resource block efficiency as traditional two-dimensional SCMA: 150%. Due to the larger Euclidean distance of the 3D joint constellation, 3D-SCMA has lower decoding complexity and better reception sensitivity. Compared with 2D-SCMA, the complexity of 3D-SCMA is 66.66% lower and the receiving sensitivity is 3.5 dBm higher, which indicates that it not only reduces the complexity of the MPA algorithm, but also reduces the energy consumption of the system and saves the transmission cost. In this paper, optical 3D-SCMA is proposed creatively, which is an effective improvement on the traditional optical 2D-SCMA. This research can be applied in the future access network.

Deep learning via message passing algorithms based on belief propagation

Message-passing algorithms based on the belief propagation (BP) equations constitute a well-known distributed computational scheme. They yield exact marginals on tree-like graphical models and have also proven to be effective in many problems defined on loopy graphs, from inference to optimization, from signal processing to clustering. The BP-based schemes are fundamentally different from stochastic gradient descent (SGD), on which the current success of deep networks is based. In this paper, we present and adapt to mini-batch training on GPUs a family of BP-based message-passing algorithms with a reinforcement term that biases distributions towards locally entropic solutions. These algorithms are capable of training multi-layer neural networks with performance comparable to SGD heuristics in a diverse set of experiments on natural datasets including multi-class image classification and continual learning, while being capable of yielding improved performances on sparse networks. Furthermore, they allow to make approximate Bayesian predictions that have higher accuracy than point-wise ones.

Compressed Sensing With Upscaled Vector Approximate Message Passing

The Recently proposed Vector Approximate Message Passing (VAMP) algorithm demonstrates a great reconstruction potential at solving compressed sensing related linear inverse problems. VAMP provides high per-iteration improvement, can utilize powerful denoisers like BM3D, has rigorously defined dynamics and is able to recover signals measured by highly undersampled and ill-conditioned linear operators. Yet, its applicability is limited to relatively small problem sizes due to the necessity to compute the expensive LMMSE estimator at each iteration. In this work we consider the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator. We propose a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction. To further improve the performance of CG-VAMP, we design a warm-starting scheme for CG and develop theoretical models for the Onsager correction and the State Evolution of Warm-Started CG-VAMP (WS-CG-VAMP). Additionally, we develop robust and accurate methods for implementing the WS-CG-VAMP algorithm. The numerical experiments on large-scale image reconstruction problems demonstrate that WS-CG-VAMP requires much fewer CG iterations compared to CG-VAMP to achieve the same or superior level of reconstruction.

A Deep Coordination Graph Convolution Reinforcement Learning for Multi-Intelligent Vehicle Driving Policy

With the growing up of Internet of Things technology, the application of Internet of Things has been popularized in the field of intelligent vehicles. Therefore, more artificial intelligence algorithms, especially DRL methods, are more widely used in autonomous driving. A large number of deep reinforcement learning (RL) technologies are continuously applied to the behavior planning module of single-vehicle autonomous driving in early. However, autonomous driving is an environment where multi-intelligent vehicles coexist, interact with each other, and dynamically change. In this environment, multiagent RL technology is one of the most promising technologies for solving the coordination behavior planning problem of multivehicles. However, the research related to this topic is rare. This paper introduces a dynamic coordination graph (CG) convolution technology for the cooperative learning of multi-intelligent vehicles. This method dynamically constructs a CG model among multiple vehicles, effectively reducing the impact of unrelated intelligent vehicles and simplifying the learning process. The relationship between intelligent vehicles is refined using the attention mechanism, and the graph convolution RL technology is used to simulate the message-passing aggregation algorithm to maximize the local utility and obtain the maximum joint utility to guide coordination learning. Driving samples are used as training data, and the model guided by reward shaping is combined with the model of the free graph convolution RL method, which enables our proposed method to achieve high gradualness and improve its learning efficiency. In addition, as the graph convolutional RL algorithm shares parameters between agents, it can easily build scales that are suitable for large-scale multiagent systems, such as traffic environments. Finally, the proposed algorithm is tested and verified for the multivehicle cooperative lane-changing problem in the simulation environment of autonomous driving. Experimental results show that our proposed method has better value function representation in that it can learn better coordination driving policies than traditional dynamic coordination algorithms.

NoC Application Mapping Optimization Using Reinforcement Learning

Application mapping is one of the early stage design processes aimed to improve the performance of Network-on-Chip. Mapping is an NP-hard problem. A massive amount of high-quality supervised data is required to solve the application mapping problem using traditional neural networks. In this article, a reinforcement learning–based neural framework is proposed to learn the heuristics of the application mapping problem. The proposed reinforcement learning–based mapping algorithm (RL-MAP) has actor and critic networks. The actor is a policy network, which provides mapping sequences. The critic network estimates the communication cost of these mapping sequences. The actor network updates the policy distribution in the direction suggested by the critic. The proposed RL-MAP is trained with unsupervised data to predict the permutations of the cores to minimize the overall communication cost. Further, the solutions are improved using the 2-opt local search algorithm. The performance of RL-MAP is compared with a few well-known heuristic algorithms, the Neural Mapping Algorithm (NMA) and message-passing neural network-pointer network-based genetic algorithm (MPN-GA). Results show that the communication cost and runtime of the RL-MAP improved considerably in comparison with the heuristic algorithms. The communication cost of the solutions generated by RL-MAP is nearly equal to MPN-GA and improved by 4.2% over NMA, while consuming less runtime.