Expectation Propagation Algorithm Research Articles

EPSOM-Hyb: A General Purpose Estimator of Log-Marginal Likelihoods with Applications in Probabilistic Graphical Models

We consider the estimation of the marginal likelihood in Bayesian statistics, with primary emphasis on Gaussian graphical models, where the intractability of the marginal likelihood in high dimensions is a frequently researched problem. We propose a general algorithm that can be widely applied to a variety of problem settings and excels particularly when dealing with near log-concave posteriors. Our method builds upon a previously posited algorithm that uses MCMC samples to partition the parameter space and forms piecewise constant approximations over these partition sets as a means of estimating the normalizing constant. In this paper, we refine the aforementioned local approximations by taking advantage of the shape of the target distribution and leveraging an expectation propagation algorithm to approximate Gaussian integrals over rectangular polytopes. Our numerical experiments show the versatility and accuracy of the proposed estimator, even as the parameter space increases in dimension and becomes more complicated.

Expectation Propagation Aided Model Driven Learning for OTFS Signal Detection

Orthogonal time frequency space (OTFS) modulation, which modulates constellation symbols in the delay-Doppler (DD) domain, is capable of outperforming conventional multi-carrier schemes in the context of high mobility communication scenarios. However, due to its special structure, it poses serious challenge to its signal detection, especially in a rich-scattering environment. To address this issue, in this paper, an expectation propagation (EP) aided model-driven deep learning framework is proposed for OTFS signal detection. Specifically, each iteration of the EP algorithm is unfolded into a layer-wise network and trainable parameters are added to this network to accelerate the convergence of the algorithm, as well as improve the detection performance. Simulation results show that the proposed EP aided model-driven network is capable of providing significant performance gain over conventional EP counterparts.

Clustered Sparse Channel Estimation for Massive MIMO Systems by Expectation Maximization-Propagation (EM-EP)

We study the problem of downlink channel estimation in multi-user massive multiple input multiple output (MIMO) systems. To this end, we consider the use of Bayesian compressive sensing approach in which the clustered sparse structure of the channel in the angular domain is employed to reduce the pilot overhead. To capture the clustered structure, we employ a conditionally independent identically distributed Bernoulli-Gaussian prior on the sparse vector representing the channel, and a Markov prior on its support vector. An expectation propagation (EP) algorithm is developed to approximate the intractable joint distribution of the sparse vector and its support with a distribution from an exponential family. The approximate distribution is then used for direct estimation of the channel. The EP algorithm assumes that the model parameters are known a priori. Since these parameters are unknown, we estimate these parameters using the expectation maximization (EM) algorithm. The combination of EM and EP referred to as EM-EP algorithm is reminiscent of the variational EM approach. Simulation results show that the proposed EM-EP algorithm outperforms several recently-proposed algorithms in the literature.

OTFS Signaling for SCMA With Coordinated Multi-Point Vehicle Communications

This paper investigates an uplink coordinated multi-point (CoMP) coverage scenario, in which multiple mobile users are grouped for sparse code multiple access (SCMA), and served by the remote radio head (RRH) in front of them and the RRH behind them simultaneously. We apply orthogonal time frequency space (OTFS) modulation for each user to exploit the degrees of freedom arising from both the delay and Doppler domains. As the signals received by the RRHs in front of and behind the users experience respectively positive and negative Doppler frequency shifts, our proposed OTFS-based SCMA (OBSCMA) with CoMP system can effectively harvest extra Doppler and spatial diversity for better performance. Based on maximum likelihood (ML) detector, we analyze the single-user average bit error rate (ABER) bound as the benchmark of the ABER performance for our proposed OBSCMA with CoMP system. We also develop a customized Gaussian approximation with expectation propagation (GAEP) algorithm for multi-user detection and propose efficient algorithm structures for centralized and decentralized detectors. Our proposed OBSCMA with CoMP system leads to stronger performance than the existing solutions. The proposed centralized and decentralized detectors exhibit effective reception and robustness under channel state information uncertainty.

Residual Transceiver Impairments Aware Expectation Propagation Algorithm Receiver for PDMA Systems

Pattern division multiple access (PDMA) is a promising non-orthogonal multiple access technology for the next generation wireless communication systems. In PDMA systems, the inherent residual transceiver impairments introduced by nonideal hardwares incur significant performance degradation. In this paper, we propose a residual-transceiver-impairments-aware (RTIA) expectation propagation algorithm (EPA) receiver to mitigate the detrimental influences of the nonideal hardware imperfections for uplink PDMA systems. The user-specific transmitter distortion powers are considered in the message passing procedure of the EPA detection. The parallel interference cancellation (PIC) scheme is incorporated into the proposed RTIA-EPA receiver to further improve the system performance and reduce the computational complexity. Simulation results illustrate that the proposed RTIA-EPA and RTIA-EPA-PIC receivers achieve significant performance gains over the conventional residual-transceiver-impairments-agnostic counterparts with negligible computational complexity overhead.

Decentralized Groupwise Expectation Propagation Detector for Uplink Massive MU-MIMO Systems

With the proliferation of emerging Internet of Things (IoT) applications, massive multiuser multiple-input–multiple-output (MU-MIMO) is a promising technology to support the massive connectivity requirement with limited spectral resources. The existing detection algorithms are mainly designed based on a centralized baseband processing architecture, which requires extremely high raw baseband data rates transferred between base-station antennas and the central processing unit (CPU). Decentralized baseband processing (DBP) architecture has recently been proposed to alleviate high interconnect data rates and chip input/output bandwidth bottlenecks. Since the information is not fully shared among each antenna cluster, conventional decentralized detectors suffer from significant performance loss, especially for systems with high overload ratios and/or spatially correlated channels. In this work, we first propose a decentralized groupwise detection paradigm for the star architecture by dividing users into multiple user groups, which can be effectively exploited by factor graph-based message passing algorithms, such as the expectation propagation (EP) algorithm. Then, an efficient message fusion rule is devised based on the product principle of the multivariate Gaussian probability density function at the CPU. To reduce the computational complexity, an approximate groupwise EP (AGW-EP) method is proposed by judiciously selecting reliable constellation vectors during the symbol belief calculation phase. In addition, we extend the proposed methods to the daisy-chain architecture, which requires constant interconnect data rates. Simulation results demonstrate that the proposed groupwise paradigm greatly enhances the performance of the conventional EP detector. Moreover, the proposed decentralized groupwise EP and AGW-EP detection schemes outperform their counterparts, particularly in correlated MIMO channels.

Iterative receiver for orthogonal time frequency space with index modulation via structured prior-based hybrid belief and expectation propagation

Orthogonal Time Frequency Space (OTFS) signaling with index modulation (IM) is a promising transmission scheme characterized by high transmission efficiency for high mobility scenarios. In this paper, we study the receiver for coded OTFS-IM system. First, we construct the corresponding factor graph, on which the structured prior incorporating activation pattern constraint and channel coding is devised. Then we develop a iterative receiver via structured prior-based hybrid belief propagation (BP) and expectation propagation (EP) algorithm, named as Str-BP-EP, for the coded OTFS-IM system. To reduce the computational complexity of discrete distribution introduced by structured prior, Gaussian approximation conducted by EP is adopted. To further reduce the complexity, we derive two variations of the proposed algorithm by using some approximations. Simulation results validate the superior performance of the proposed algorithm.

Graph Neural Network-Enhanced Expectation Propagation Algorithm for MIMO Turbo Receivers

Graph Neural Network-Enhanced Expectation Propagation Algorithm for MIMO Turbo Receivers

Research on improved multiple users detection algorithm in pattern division multiple access system

Receiver detection algorithm has always been a research hotspot in communication system and the key point to determine the performance of 5G system. Both industry and academia have been committed to finding detection algorithms with lower complexity and higher detection performance. This paper will study how to solve the problems of low detection performance and high complexity of classical algorithms. Taking Pattern Division Multiple Access (PDMA) as an example, this paper improves the existing receiver detection algorithms such as Minimum Mean Square Error-Serial Interference Cancellation (MMSE-SIC), Message Passing Algorithm (MPA) and Expectation Propagation Algorithm (EPA), and puts forward corresponding improved algorithms with specific formulas, i.e. Enhanced-MMSE-SIC, Turbo-like MPA (MPA-Turbo) and Turbo-like EPA with hybrid parallel interference cancellation (EPA-Turbo-PIC). Link-level simulation results and analysis show that the proposed improved algorithms have performance advantages over the corresponding traditional algorithms.

An Enhanced Fully Decentralized Detector for the Uplink M-MIMO System

Decentralized baseband processing (DBP) architectures can alleviate the extremely high interconnect data rates and chip input/output (I/O) bandwidth bottlenecks of the conventional centralized massive multiple-input multiple-output (M-MIMO) system. Nonetheless, fully decentralized (FD) detectors suffer significant performance loss compared with the centralized counterparts due to the constrained information-sharing between antenna clusters, especially for systems with a large number of antenna clusters and/or high-order modulation. In this article, we propose an enhanced message-passing-based FD detection method by utilizing an information-lossless factor graph (FG) transformation scheme. As an illustrative example, the expectation propagation (EP) algorithm is considered in this work. In each antenna cluster, the FG is transformed to a less-loopy one and the effective interference of users is suppressed. Based on the message-passing principle of the EP algorithm, we devise an efficient non-linear fusion scheme in the central processing unit (CPU). In addition, a compensation factor accounting for the cluster profile is introduced to mitigate the inaccuracy of message-passing caused by DBP, thus further enhancing the detection performance. Numerical results demonstrate that the proposed enhanced FD detector obtains significantly better performance than the counterpart with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">even</i> lower computational complexity, especially in a large number of antenna clusters and/or high-order modulation scenarios.

Genomic data analysis using a two stage expectation propagation algorithm for analysis of sparse Bayesian high-dimensional instrumental variables regression

Simultaneous analysis of gene expression data and genetic variants is highly of interest, especially when the number of gene expressions and genetic variants are both greater than the sample size. Association of both causal genes and effective SNPs makes the use of sparse modeling of such genetic data sets, highly important. The high-dimensional sparse instrumental variables models are one of such useful association models, which models the simultaneous relation of the gene expressions and genetic variants with complex traits. From a Bayesian viewpoint, the sparsity can be favored using sparsity-enforcing priors such as spike-and-slab priors. A two-stage modification of the expectation propagation (EP) algorithm is proposed and examined for approximate inference in high-dimensional sparse instrumental variables models with spike-and-slab priors. This method is an adoption of the classical two-stage least squares method, to be used with the Bayes context. A simulation study is performed to examine the performance of the methods. The proposed method is applied to analysis of the mouse obesity data.

An Improved EPA-Based Receiver Design for Uplink LDPC Coded SCMA System

Sparse code multiple access (SCMA) is an emerging paradigm for efficient enabling of massive connectivity in future machine-type communications (MTC). In this letter, we conceive the uplink transmissions of the low-density parity check (LDPC) coded SCMA system. Traditional receiver design of LDPC-SCMA system, which is based on message passing algorithm (MPA) for multiuser detection followed by individual LDPC decoding, may suffer from the drawback of the high complexity and large decoding latency, especially when the system has large codebook size and/or high overloading factor. To address this problem, we introduce a novel receiver design by applying the expectation propagation algorithm (EPA) to the joint detection and decoding (JDD) involving an aggregated factor graph of LDPC code and sparse codebooks. Our numerical results demonstrate the superiority of the proposed EPA based JDD receiver over the conventional Turbo receiver in terms of both significantly lower complexity and faster convergence rate without noticeable error rate performance degradation.

Improved Approximate Expectation Propagation Massive MIMO Detector with Second-Order Richardson Iteration

The expectation propagation (EP) detector achieves significantly better performance than the linear detectors (such as minimum mean squared error detector) in massive MIMO systems, which has drawn great attention recently. EP’s approximation (EPA) algorithm simplifies the update formula of the EP algorithm by reexpressing the moment matching condition so that the number of matrix inversions in the EP algorithm is reduced to one. However, the expense is that the EPA algorithm requires higher accuracy for this inversion; otherwise, the bit-error-rate (BER) performance will suffer serious losses. To tackle this issue, the SORI iterative algorithm is introduced to obtain the high-precision result of this inversion to ensure the good BER performance of the EPA algorithm. First, the new expression of the SORI iterative algorithm is derived under the equivalent real-valued system. Second, the improved EPA-SORI algorithm is then introduced by the SORI algorithm, which is used to approximate the initial value of the EPA algorithm under the real-valued system. Finally, by designing the initial solution and the relaxation factor of the EPA-SORI iterative algorithm, the convergence rate can be quickly increased without increasing the complexity. Simulation and complexity results exhibit that in various massive MIMO system configurations, the proposed EPA-SORI algorithm can achieve the same BER performance as the Exact EP algorithm with significantly lower complexity. At the same time, compared with MMSE and the existing EPA algorithms, the proposed EPA-SORI algorithm has a better performance-complexity trade-off advantage, which is more obvious in scenarios with high modulation order and a large number of users.

Low Complexity Receiver via Expectation Propagation for OTFS Modulation

Orthogonal time frequency space (OTFS) modulation has recently received considerable attention due to its capabilities to exploit full time-frequency diversity and deal with channel dynamics in high-mobility scenarios. However, the implementation of high-performance receivers in the delay-Doppler (DD) domain poses serious challenges in practice, especially in rich-scattering environments and when the fractional Doppler is considered. In this letter, we first propose an iterative receiver under the framework of an expectation propagation (EP) algorithm for the OTFS system. Then, a channel-coefficients-aware scheme is incorporated into the message passing by bundling multiple graph edges into one edge on the factor graph based on the sizes of associated channel coefficients to further reduce the computational complexity. Simulation results illustrate that the proposed approximate EP receiver achieves a good tradeoff between performance and complexity.

Improving Approximate Expectation Propagation Massive MIMO Detector With Deep Learning

In this letter, an efficient model-driven deep learning (DL) based massive multiple-input multiple-output (MIMO) detector is proposed by improving the approximate expectation propagation (EPA) algorithm, named EPANet. Specifically, EPANet is constructed by unfolding the iterative EPA detector and adding learnable parameters to enhance the performance and convergence robustness through the DL approach. Only one training procedure is required in advance for EPANet to be reused for multiple detection tasks with different antenna configurations. Numerical results indicate that DL can bring significant performance improvement to EPA with various antenna settings. Besides, the proposed EPANet can outperform state-of-the-art (SOA) DL-based detectors with hardware-friendly complexity, especially under highly-correlated channels.

Expectation propagation on the diluted Bayesian classifier.

Efficient feature selection from high-dimensional datasets is a very important challenge in many data-driven fields of science and engineering. We introduce a statistical mechanics inspired strategy that addresses the problem of sparse feature selection in the context of binary classification by leveraging a computational scheme known as expectation propagation (EP). The algorithm is used in order to train a continuous-weights perceptron learning a classification rule from a set of (possibly partly mislabeled) examples provided by a teacher perceptron with diluted continuous weights. We test the method in the Bayes optimal setting under a variety of conditions and compare it to other state-of-the-art algorithms based on message passing and on expectation maximization approximate inference schemes. Overall, our simulations show that EP is a robust and competitive algorithm in terms of variable selection properties, estimation accuracy, and computational complexity, especially when the student perceptron is trained from correlated patterns that prevent other iterative methods from converging. Furthermore, our numerical tests demonstrate that the algorithm is capable of learning online the unknown values of prior parameters, such as the dilution level of the weights of the teacher perceptron and the fraction of mislabeled examples, quite accurately. This is achieved by means of a simple maximum likelihood strategy that consists in minimizing the free energy associated with the EP algorithm.

Low-Complexity EP Receiver Based on Location-Aware and Reliability-Aware Strategy

The expectation propagation (EP) algorithm has received significant attention recently due to its good performance and manageable computational complexity for large-scale inference problems, such as detection for massive multiple-input multiple-output (MIMO) systems. Its computational complexity depends mainly on how messages are represented for computation and passing. In this letter, we propose a low-complexity EP algorithm based on location-aware and reliability-aware strategy, shortened as LR-EP, to efficiently implement the message computation and passing. At each iteration, only the most probable symbol candidates are selected to perform message computation based on a low-complexity quadrant detection and predetermined lookup table. In the iterative process, reliable variable nodes with sufficiently high posterior probabilities are kept intact, thus reducing the less efficient message passing. Numerical results illustrate that the proposed LR-EP receiver reduces computational complexity significantly compared to the conventional EP receiver with negligible performance degradation.

Unifying Message Passing Algorithms Under the Framework of Constrained Bethe Free Energy Minimization

Variational message passing (VMP), belief propagation (BP) and expectation propagation (EP) have found their wide applications in complex statistical signal processing problems. In addition to viewing them as a class of algorithms operating on graphical models, this article unifies them under an optimization framework, namely, Bethe free energy minimization with differently and appropriately imposed constraints. This new perspective in terms of constraint manipulation can offer additional insights on the connection between different message passing algorithms and is valid for a generic statistical model. It also founds a theoretical framework to systematically derive message passing variants. Taking the sparse signal recovery (SSR) problem as an example, a low-complexity EP variant can be obtained by simple constraint reformulation, delivering better estimation performance with lower complexity than the standard EP algorithm. Furthermore, we can resort to the framework for the systematic derivation of hybrid message passing for complex inference tasks. Notably, a hybrid message passing algorithm is exemplarily derived for joint SSR and statistical model learning with near-optimal inference performance and scalable complexity.

An Improved Expectation Propagation Based Detection Scheme for MIMO Systems

Multiple-input Multiple-output (MIMO) technologies play an important role in modern and future wireless communication systems as they can improve the capacity without increasing the bandwidth. MIMO detection is one of the key technologies for MIMO system designs. MIMO detection schemes based on Message Passing (MP) algorithms have attracted extensive attention in recent years. The MIMO detection scheme based on Expectation Propagation (EP), which is also a kind of MP algorithms, has been proved to achieve the Bayes-optimal performance under the conditions of large system limit and compression rate threshold. However, there is improvement space for the detection performance of the EP algorithm when the conditions are not satisfied, which are the common cases in the practical applications. In this paper, when the conditions of the large system limit and compression rate threshold are not satisfied, we analyze the influences of two factors, the initial parameters selection and the moment matching strategy, on the performance of EP based detection scheme. As a result, we propose an improved EP detection scheme based on optimizing the two factors. Simulation results show that the proposed scheme outperforms the original EP detection scheme in different MIMO scenarios.

Approximate Inference and Learning of State Space Models With Laplace Noise

State space models have been extensively applied to model and control dynamical systems in disciplines including neuroscience, target tracking, and audio processing. A common modeling assumption is that both the state and data noise are Gaussian because it simplifies the estimation of the system's state and model parameters. However, in many real-world scenarios where the noise is heavy-tailed or includes outliers, this assumption does not hold, and the performance of the model degrades. In this paper, we present a new approximate inference algorithm for state space models with Laplace-distributed multivariate data that is robust to a wide range of non-Gaussian noise. Exact inference is combined with an expectation propagation algorithm, leading to filtering and smoothing that outperforms existing approximate inference methods for Laplace-distributed data, while retaining a fast speed similar to the Kalman filter. Further, we present a maximum posterior expectation-maximization (EM) algorithm that learns the parameters of the model in an unsupervised way, automatically avoids over-fitting the data, and provides better model estimation than existing methods for the Gaussian model. The quality of the inference and learning algorithms are exemplified through a diverse set of experiments and an application to non-linear tracking of audio frequency.