Superposition Codes Research Articles

Block Orthogonal Sparse Superposition Codes for Ultra-Reliable Low-Latency Communications

Block Orthogonal Sparse Superposition Codes for Ultra-Reliable Low-Latency Communications

Hyper-Dimensional Modulation for Robust Short Packets in Massive Machine-Type Communications

In this paper, we introduce Hyper-Dimensional Modulation (HDM) for massive machine-type communications (mMTC). HDM enables robust communication of short packets by spreading information bits across many elements in a hyper-dimensional vector and superimposing a set of such non-orthogonal vectors. The proposed CRC-aided K-best decoding algorithm for HDM can achieve a very low packet error rate (PER) in additive white Gaussian noise (AWGN) channels for short packets. Furthermore, extended decoding algorithms are proposed to combat overwhelming interference in an mMTC network. Comprehensive simulation and real-world experiment results show that HDM outperforms sparse superposition codes in AWGN channels and state-of-the-art short codes such as polar and tail-biting convolutional codes in interference-heavy channels for short packet transmissions.

Deep Learning of Damped AMP Decoding Networks for Sparse Superposition Codes via Annealing

This paper addresses short-length sparse superposition codes (SSCs) over the additive white Gaussian noise channel. Damped approximate message-passing (AMP) is used to decode short SSCs with zero-mean independent and identically distributed Gaussian dictionaries. To design damping factors in AMP via deep learning, this paper constructs deep-unfolded damped AMP decoding networks. An annealing method for deep learning is proposed for designing nearly optimal damping factors with high probability. In annealing, damping factors are first optimized via deep learning in the low signal-to-noise ratio (SNR) regime. Then, the obtained damping factors are set to the initial values in stochastic gradient descent, which optimizes damping factors for slightly larger SNR. Repeating this annealing process designs damping factors in the high SNR regime. Numerical simulations show that annealing mitigates fluctuation in learned damping factors and outperforms exhaustive search based on an iteration-independent damping factor.

Capacity-Achieving Spatially Coupled Sparse Superposition Codes With AMP Decoding

Sparse superposition codes, also called sparse regression codes (SPARCs), are a class of codes for efficient communication over the AWGN channel at rates approaching the channel capacity. In a standard SPARC, codewords are sparse linear combinations of columns of an i.i.d. Gaussian design matrix, while in a spatially coupled SPARC the design matrix has a block-wise structure, where the variance of the Gaussian entries can be varied across blocks. A well-designed spatial coupling structure can significantly enhance the error performance of iterative decoding algorithms such as Approximate Message Passing (AMP). In this paper, we obtain a non-asymptotic bound on the probability of error of spatially coupled SPARCs with AMP decoding. Applying this bound to a simple band-diagonal design matrix, we prove that spatially coupled SPARCs with AMP decoding achieve the capacity of the AWGN channel. The bound also highlights how the decay of error probability depends on each design parameter of the spatially coupled SPARC. An attractive feature of AMP decoding is that its asymptotic mean squared error (MSE) can be predicted via a deterministic recursion called state evolution. Our result provides the first proof that the MSE concentrates on the state evolution prediction for spatially coupled designs. Combined with the state evolution prediction, this result implies that spatially coupled SPARCs with the proposed band-diagonal design are capacity-achieving. Using the proof technique used to establish the main result, we also obtain a concentration inequality for the MSE of AMP applied to compressed sensing with spatially coupled design matrices. Finally we provide numerical simulation results that demonstrate the finite length error performance of spatially coupled SPARCs. The performance is compared with coded modulation schemes that use LDPC codes from the DVB-S2 standard.

Modulated Sparse Superposition Codes for the Complex AWGN Channel

This paper studies a generalization of sparse superposition codes (SPARCs) for communication over the complex additive white Gaussian noise (AWGN) channel. In a SPARC, the codebook is defined in terms of a design matrix, and each codeword is a generated by multiplying the design matrix with a sparse message vector. In the standard SPARC construction, information is encoded in the locations of the non-zero entries of the message vector. In this paper we generalize the construction and consider modulated SPARCs, where information in encoded in both the locations and the values of the non-zero entries of the message vector. We focus on the case where the non-zero entries take values from a phase-shift keying (PSK) constellation. We propose a computationally efficient approximate message passing (AMP) decoder, and obtain analytical bounds on the state evolution parameters which predict the error performance of the decoder. Using these bounds we show that PSK-modulated SPARCs are asymptotically capacity achieving for the complex AWGN channel, with either spatial coupling or power allocation. We also provide numerical simulation results to demonstrate the error performance at finite code lengths. These results show that introducing modulation to the SPARC design can significantly reduce decoding complexity without sacrificing error performance.

Adaptive Path Interpolation Method for Sparse Systems: Application to a Censored Block Model

Recently, a new adaptive path interpolation method has been developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and generalized estimation, sparse superposition codes, and low-rank matrix / tensor estimation. For all these systems, the adaptive interpolation method directly proves that the replica-symmetric prediction is exact, in a simple and unified manner. When the underlying factor graph of the inference problem is sparse the replica prediction is considerably more complicated, and rigorous results are often lacking or obtained by rather complicated methods. In this work we show how to extend the adaptive path interpolation method to sparse systems. We concentrate on a censored block model, where hidden variables are measured through a binary erasure channel, for which we fully prove the replica prediction.

Symbol error rate analysis of non-orthogonal multiple access systems

Non-orthogonal multiple access (NOMA) systems use superposition coding (SC) at transmitter and successive interference cancellation (SIC) at the receiver in order to simultaneously transmit the messages of multiple users. In this paper, closed from expressions for the symbol error rate (SER) of superposition codes are derived. Moreover, an approximate expression is presented and it is shown that it closely follows the exact symbol error rate. The derived formulas are applicable to general scenarios with N users when each individual user uses pulse amplitude modulation (PAM) or quadrature amplitude modulation (QAM) signalling with arbitrary constellation sizes. While the error expressions are for additive white Gaussian noise (AWGN) channels, it will be shown that they can be extended to fading channels. Numerical simulations verify the validity of both expressions.

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a number of works considering the mutual information for this problem using the replica method from statistical physics. Here we put these considerations on a firm rigorous basis. First, we show, using a Guerra-Toninelli type interpolation, that the replica formula yields an upper bound to the exact mutual information. Secondly, for many relevant practical cases, we present a converse lower bound via a method that uses spatial coupling, state evolution analysis and the I-MMSE theorem. This yields a single letter formula for the mutual information and the minimal-mean-square error for random Gaussian linear estimation of all discrete bounded signals. In addition, we prove that the low complexity approximate message-passing algorithm is optimal outside of the so-called hard phase, in the sense that it asymptotically reaches the minimal-mean-square error. In this work spatial coupling is used primarily as a proof technique. However our results also prove two important features of spatially coupled noisy linear random Gaussian estimation. First there is no algorithmically hard phase. This means that for such systems approximate message-passing always reaches the minimal-mean-square error. Secondly, in a proper limit the mutual information associated to such systems is the same as the one of uncoupled linear random Gaussian estimation.

Universal superposition codes: Capacity regions of compound quantum broadcast channel with confidential messages

We derive universal codes for transmission of broadcast and confidential messages over classical-quantum–quantum and fully quantum channels. These codes are robust to channel uncertainties considered in the compound model. To construct these codes, we generalize random codes for transmission of public messages to derive a universal superposition coding for the compound quantum broadcast channel. As an application, we give a multi-letter characterization of regions corresponding to the capacity of the compound quantum broadcast channel for transmitting broadcast and confidential messages simultaneously. This is done for two types of broadcast messages, one called public and the other common.

An improved analysis of least squares superposition codes with bernoulli dictionary

For the additive white Gaussian noise channel with average power constraint, sparse superposition codes (or sparse regression codes), proposed by Barron and Joseph in 2010, achieve the capacity. While the codewords of the original sparse superposition codes are made with a dictionary matrix drawn from a Gaussian distribution, we consider the case that it is drawn from a Bernoulli distribution. We show an improved upper bound on its block error probability with least squares decoding, which is fairly simplified and tighter bound than our previous result in 2014.

Universal Sparse Superposition Codes With Spatial Coupling and GAMP Decoding

Sparse superposition codes, or sparse regression codes, constitute a new class of codes, which was first introduced for communication over the additive white Gaussian noise (AWGN) channel. It has been shown that such codes are capacity-achieving over the AWGN channel under optimal maximum-likelihood decoding as well as under various efficient iterative decoding schemes equipped with power allocation or spatially coupled constructions. Here, we generalize the analysis of these codes to a much broader setting that includes all memoryless channels. We show, for a large class of memoryless channels, that spatial coupling allows an efficient decoder, based on the generalized approximate message-passing (GAMP) algorithm, to reach the potential (or Bayes optimal) threshold of the underlying (or uncoupled) code ensemble. Moreover, we argue that spatially coupled sparse superposition codes universally achieve capacity under GAMP decoding by showing, through analytical computations, that the error floor vanishes and the potential threshold tends to capacity, as one of the code parameters goes to infinity. Furthermore, we provide a closed-form formula for the algorithmic threshold of the underlying code ensemble in terms of Fisher information. Relating an algorithmic threshold to a Fisher information has theoretical as well as practical importance. Our proof relies on the state evolution analysis and uses the potential method developed in the theory of low-density parity-check (LDPC) codes and compressed sensing.

The Error Probability of Sparse Superposition Codes With Approximate Message Passing Decoding

Sparse superposition codes, or sparse regression codes (SPARCs), are a recent class of codes for reliable communication over the AWGN channel at rates approaching the channel capacity. Approximate message passing (AMP) decoding, a computationally efficient technique for decoding SPARCs, has been proven to be asymptotically capacity-achieving for the AWGN channel. In this paper, we refine the asymptotic result by deriving a large deviations bound on the probability of AMP decoding error. This bound gives insight into the error performance of the AMP decoder for large but finite problem sizes, giving an error exponent as well as guidance on how the code parameters should be chosen at finite block lengths. For an appropriate choice of code parameters, we show that for any fixed rate less than the channel capacity, the decoding error probability decays exponentially in $n/(\log n)^{2T}$, where $T$, the number of AMP iterations required for successful decoding, is bounded in terms of the gap from capacity.

Clipping Can Improve the Performance of Spatially Coupled Sparse Superposition Codes

Gaussian signaling of spatially coupled sparse superposition codes incurs a high peak-to-average-power ratio (PAPR) problem. Clipping can be used to reduce PAPR. At the receiver side, generalized approximate message passing algorithm is employed to handle the non-linear clipping distortion. Surprisingly, we observe empirically that clipping can improve the performance in low-to-medium rate region. We provide an explanation of this observation based on potential function analysis.

Techniques for Improving the Finite Length Performance of Sparse Superposition Codes

Sparse superposition codes are a recent class of codes introduced by Barron and Joseph for efficient communication over the AWGN channel. With an appropriate power allocation, these codes have been shown to be asymptotically capacity-achieving with computationally feasible decoding. However, a direct implementation of the capacity-achieving construction does not give good finite length error performance. In this paper, we consider sparse superposition codes with approximate message passing (AMP) decoding, and describe a variety of techniques to improve their finite length performance. These include an iterative algorithm for SPARC power allocation, guidelines for choosing codebook parameters, and estimating a critical decoding parameter online instead of pre-computation. We also show how partial outer codes can be used in conjunction with AMP decoding to obtain a steep waterfall in the error performance curves. We compare the error performance of AMP-decoded sparse superposition codes with coded modulation using LDPC codes from the WiMAX standard.

A novel physical layer scheme based on superposition codes

The recently proposed superposition codes (SCs) have been mathematically proved to be decoded at any rate below the capacity, for additive white Gaussian noise (AWGN) channels. The main objective of this paper is to study the feasibility of a novel SC approach as an alternative to the traditional way of designing modern physical (PHY) layer schemes. Indeed, currently, PHY solutions are based on the decomposition into two separate problems of modulation shaping and coding over finite alphabets. Since superposition codes are defined over real numbers, modulation and coding can be jointly realized. Moreover, a fast decoding method is developed and tested by comparing the obtained results with both the uncoded system performance and two approximate message passing (AMP) algorithms. Finally, possible perspective to fifth generation (5G) applications exploiting SC solutions are outlined in the paper, and some interesting relations with sparse signal recovery are analyzed for further future research lines.

Implementation of Sparse Superposition Codes

Sparse superposition codes (SSCs) are capacity achieving codes whose decoding process is a linear sensing problem. Decoding approaches thus exploit the approximate message passing algorithm, which has been proven to be effective in compressing sensing. Previous work from the authors has evaluated the error correction performance of SSCs under finite precision and finite code length. This paper proposes the first SSC encoder and decoder architectures in the literature. The architectures are parametrized and applicable to all SSCs: A set of wide-ranging case studies is then considered, and code-specific approximations, along with implementation results in 65 nm CMOS technology, are then provided. The encoding process can be carried out with low power consumption ( $\leq \text{2.103}$ mW), while the semi-parallel decoder architecture can reach a throughput of 1.3 Gb/s with a $768\times 6$ -bit SSC codeword and an area occupation of 2.43 mm $^2$.

Capacity-Achieving Sparse Superposition Codes via Approximate Message Passing Decoding

Sparse superposition codes were recently introduced by Barron and Joseph for reliable communication over the AWGN channel at rates approaching the channel capacity. The codebook is defined in terms of a Gaussian design matrix, and codewords are sparse linear combinations of columns of the matrix. In this paper, we propose an approximate message passing decoder for sparse superposition codes, whose decoding complexity scales linearly with the size of the design matrix. The performance of the decoder is rigorously analyzed and it is shown to asymptotically achieve the AWGN capacity with an appropriate power allocation. Simulation results are provided to demonstrate the performance of the decoder at finite blocklengths. We introduce a power allocation scheme to improve the empirical performance, and demonstrate how the decoding complexity can be significantly reduced by using Hadamard design matrices.

Switched-Power Two-Layer Superposition Coding in Cooperative Decode-Forward Relay Systems

In this paper, we consider a decode-forward relay system with a source, a relay, and a destination, where two-layer superposition codes are used at the source and the relay. An equivalent squared minimum distance (ESMD) that determines the error performance is derived by using an upper bound on the pair-wise error probability. Without deriving error probabilities, the error performance level for each of superimposed symbols can be shown in a straightforward manner by the ESMD. An optimal superposition-coded relay scheme and a suboptimal switched-power superposition coding scheme are proposed by improving the ESMD. Closed-form power allocation that maximizes the ESMD for the switched scheme is derived for 2-ary pulse amplitude modulation (PAM). An $M$ -ary PAM generalization for the switched-power superposition-coded relay scheme is also presented. Simulation results show that significant signal-to-noise ratio gains are achieved in the optimal and switched-power superposition coding strategies for 2-ary and 4-ary PAM over the Rayleigh fading channel.

Approximate message-passing with spatially coupled structured operators, with applications to compressed sensing and sparse superposition codes

We study the behavior of approximate message-passing (AMP), a solver for linear sparse estimation problems such as compressed sensing, when the i.i.d matrices—for which it has been specifically designed—are replaced by structured operators, such as Fourier and Hadamard ones. We show empirically that after proper randomization, the structure of the operators does not significantly affect the performances of the solver. Furthermore, for some specially designed spatially coupled operators, this allows a computationally fast and memory efficient reconstruction in compressed sensing up to the information-theoretical limit. We also show how this approach can be applied to sparse superposition codes, allowing the AMP decoder to perform at large rates for moderate block length.

Least Squares Superposition Codes With Bernoulli Dictionary are Still Reliable at Rates up to Capacity

For the additive white Gaussian noise channel with average power constraint, sparse superposition codes with least squares decoding are proposed by Barron and Joseph in 2010. The codewords are designed by using a dictionary each entry of which is drawn from a Gaussian distribution. The error probability is shown to be exponentially small for all rates up to the capacity. This paper proves that when each entry of the dictionary is drawn from a Bernoulli distribution, the error probability is also exponentially small for all rates up to the capacity. The proof is via a central limit theorem-type inequality, which we show for this analysis.