Asymptotic Mutual Information Research Articles

Asymptotic Mutual Information Analysis for Double-Scattering MIMO Channels: A New Approach by Gaussian Tools

The asymptotic mutual information (MI) analysis for multiple-input multiple-output (MIMO) systems over double-scattering channels has achieved engaging results, but the convergence rates of the mean, variance, and the distribution of the MI are not yet available in the literature. In this paper, by utilizing the large random matrix theory (RMT), we give a central limit theory (CLT) for the MI and derive the closed-form approximation for the mean and the variance by a new approach—Gaussian tools. The convergence rates of the mean, variance, and the characteristic function are proved to be <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> ) for the first time, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> is the number of receive antennas. Furthermore, the impact of the number of effective scatterers on the mean and variance was investigated in the moderate-to-high SNR regime with some interesting physical insights. The proposed evaluation framework can be utilized for the asymptotic performance analysis of other systems over double-scattering channels.

Information theoretic limits of learning a sparse rule

We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signal. We prove a variational formula for the asymptotic mutual information per sample when the system size grows to infinity. This result allows us to derive an expression for the minimum mean-square error (MMSE) of the Bayesian estimator when the signal entries have a discrete distribution with finite support. We find that, for such signals and suitable vanishing scalings of the sparsity and sampling rate, the MMSE is nonincreasing piecewise constant. In specific instances the MMSE even displays an all-or-nothing phase transition, that is, the MMSE sharply jumps from its maximum value to zero at a critical sampling rate. The all-or-nothing phenomenon has previously been shown to occur in high-dimensional linear regression. Our analysis goes beyond the linear case and applies to learning the weights of a perceptron with general activation function in a teacher-student scenario. In particular, we discuss an all-or-nothing phenomenon for the generalization error with a sublinear set of training examples.

Adaptive Path Interpolation Method for Sparse Systems: Application to a Censored Block Model

Recently, a new adaptive path interpolation method has been developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and generalized estimation, sparse superposition codes, and low-rank matrix / tensor estimation. For all these systems, the adaptive interpolation method directly proves that the replica-symmetric prediction is exact, in a simple and unified manner. When the underlying factor graph of the inference problem is sparse the replica prediction is considerably more complicated, and rigorous results are often lacking or obtained by rather complicated methods. In this work we show how to extend the adaptive path interpolation method to sparse systems. We concentrate on a censored block model, where hidden variables are measured through a binary erasure channel, for which we fully prove the replica prediction.

Mutual information for low-rank even-order symmetric tensor estimation

Abstract We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its asymptotic mutual information when the tensor is of even order. The proof applies the adaptive interpolation method originally invented for rank-one factorization. Here we show how to extend the adaptive interpolation to finite-rank and even-order tensors. This requires new non-trivial ideas with respect to the current analysis in the literature. We also underline where the proof falls short when dealing with odd-order tensors.

Receive Antenna Selection Under Discrete Inputs: Approximation and Applications

To analyze the achievable performance of antenna selection (AS) in practical multi-antenna systems, this paper studies the receive antenna selection (RAS) in single-input multiple-output (AS-SIMO) systems under discrete inputs. We first propose an approximate expression to evaluate the instantaneous mutual information (MI) of $M$ -ary quadrature amplitude modulation ( $M$ -QAM) signaling over additive white Gaussian noise (AWGN) channels. Then, by exploiting this approximate formula, we develop a closed-form formula for the ergodic MI in AS-SIMO systems with $M$ -QAM signaling. Additionally, we also analyze the asymptotic MI for a large number of receive antennas $N_{\mathrm{r}}$ . This asymptotic analysis suggests that the scaling rate of the MI with $N_{\mathrm{r}}$ becomes zero rate in contrast to the double logarithmic rate under Gaussian inputs. Besides, our result is also extended to discuss the mutual information of multiple-input multiple-output (MIMO) systems having discrete inputs with receive antenna selection, and an upper bound for the MI is derived. Finally, the derived result is applied to analyze several performance measures of the discrete inputs driven AS-SIMO systems. Specifically, it is first used to discuss the relationship between the ergodic MI and the number of active antennas. Our investigation shows that this relationship follows Pareto principle, i.e., 80% of the MI of full-antenna selection can be achieved via 20% of the total antennas. Then, our proposed approximation is employed to analytically study the effective MI which takes channel estimation (CE) into consideration, indicating that CE is a main limit of large-scale systems. Moreover, the energy efficiency (EE) is explored on the basis of our results, and we find there exists an optimal number of active antennas to maximize the energy efficiency. In addition to theoretical derivations, all the analytical results are validated by numerical simulations.

The Replica-Symmetric Prediction for Random Linear Estimation With Gaussian Matrices Is Exact

This paper considers the fundamental limit of random linear estimation for i.i.d. signal distributions and i.i.d. Gaussian measurement matrices. Its main contribution is a rigorous characterization of the asymptotic mutual information (MI) and minimum mean-square error (MMSE) in this setting. Under mild technical conditions, our results show that the limiting MI and MMSE are equal to the values predicted by the replica method from statistical physics. This resolves a well-known problem that has remained open for over a decade.

Asymptotic mutual information for the balanced binary stochastic block model

We develop an information-theoretic view of the stochastic block model, a popular statistical model for the large-scale structure of complex networks. A graph |$G$| from such a model is generated by first assigning vertex labels at random from a finite alphabet, and then connecting vertices with edge probabilities depending on the labels of the endpoints. In the case of the symmetric two-group model, we establish an explicit ‘single-letter’ characterization of the per-vertex mutual information between the vertex labels and the graph, when the mean vertex degree diverges. The explicit expression of the mutual information is intimately related to estimation-theoretic quantities, and —in particular— reveals a phase transition at the critical point for community detection. Below the critical point the per-vertex mutual information is asymptotically the same as if edges were independent. Correspondingly, no algorithm can estimate the partition better than random guessing. Conversely, above the threshold, the per-vertex mutual information is strictly smaller than the independent-edges upper bound. In this regime, there exists a procedure that estimates the vertex labels better than random guessing.

Asymptotic Mutual Information Statistics of MIMO Channels and CLT of Sample Covariance Matrices

In this paper, we consider the fluctuation of mutual information statistics of a multiple input multiple output channel communication systems without assuming that the entries of the channel matrix have zero pseudovariance. To this end, we also establish a central limit theorem of the linear spectral statistics for sample covariance matrices under general moment conditions by removing the restrictions imposed on the second moment and fourth moment on the matrix entries in Bai and Silverstein (2004).

Further Results on the Asymptotic Mutual Information of Rician Fading MIMO Channels

Rician fading multiple-input multiple-output (MIMO) channels with Kronecker-decomposable covariance structure are considered in this study to derive information measures characterizing their achievable rate. Analytic results are obtained in the asymptotic setting corresponding to infinitely many transmit and receive antennas, whose numbers approach a finite limiting ratio. More specifically, the asymptotic mean and variance of the mutual information are derived in this asymptotic regime, and it is shown that the corresponding probability distribution converges to a Gaussian distribution. All these results are based on the replica method, whose mathematical principles are discussed in this study. The ergodic capacity is also addressed by deriving an iterative water-filling algorithm yielding the capacity-achieving input covariance matrix (under an average power constraint). Resorting to the asymptotic Gaussianity of the mutual information distribution, an analytic expression of the outage capacity is derived based on the mean and variance of the mutual information. Finally, numerical results are reported to assess the level of approximation of the analytic results in comparison with those obtained by Monte Carlo simulation.

Robust Transmitter Design for Amplify-and-Forward MIMO Relay Systems Exploiting Only Channel Statistics

In this paper, we address statistically robust transmit design problems that maximize the average mutual information of amplify-and-forward (AF) multiple-input multiple-output (MIMO) two-hop relay channels having perfect channel state information at the receiver (CSIR) and statistical channel state information at the transmitter (CSIT). In the selected scenario, the source, relay, and destination terminals are equipped with correlated antennas where a direct link between the source and the destination terminals can be found. Moreover, the statistical CSIT consists of the channel means and covariance matrices of various links. The design problem is taken into account for regimes in large systems because there are no closed-form expressions of the average mutual information in the MIMO relay channel. The contribution of this study includes the derivation of asymptotic mutual information expressions for the MIMO relay channel in the large system limit. Additionally, an efficient optimization algorithm based on the asymptotic mutual information is proposed in order to find the asymptotically optimal source covariance matrix and the relay amplifying matrix. Numerical simulation results show that the new approach achieves indistinguishable performance compared to those of the maximization approaches in finite-dimensional systems, even for a small number of antennas at each link. The impact of the CSI of various links on the throughput of the MIMO relay channel is studied using the new approach.

On the Asymptotic Properties of Amplify-and-Forward MIMO Relay Channels

This paper studies the asymptotic properties of amplify-and-forward (AF) multiple-input multiple-output (MIMO) two-hop relay channels, where the source terminal (ST), relay terminal (RT) and destination terminal (DT) are equipped with a number of correlated antennas and there is a direct link between the ST and DT. The analysis is based on the Kronecker correlated fading model where the correlation properties of the MIMO channel are modeled at the transmitter and receiver ends separately. Our first main contribution is the derivation of the asymptotic mutual information expressions for the MIMO-AF relay channel for any given signaling input distributions (not necessarily Gaussian) in the large-system regime, which is particularly relevant if practical modulation schemes are used. In addition to the formula that uses joint optimal decoding (JOD) at the DT, we also obtain the asymptotic result for a more practical system which performs separate detection and decoding (SDD) (i.e., spatial detection followed by a temporal error-correction decoder). Another major contribution of this paper is an efficient algorithm that can compute the optimal source covariance matrix and the relay precoding matrix to maximize the asymptotic mutual information of the MIMO-AF relay system if channel covariance information (CCI) is known at the ST and RT.

Mutual Information Statistics and Beamforming Performance Analysis of Optimized LoS MIMO Systems

This paper provides a systematic mutual information (MI) and multichannel beamforming (MBF) characterization of optimized multiple-input multiple-output (MIMO) communication systems operating in Ricean fading. These optimized configurations are of high practical importance since, contrary to the common belief, benefit from the presence of direct Line-of-Sight (LoS) components and deliver maximum multiplexing gains, by deploying specifically designed antenna arrays at both ends. In the following, using elements from random matrix theory, novel analytical expressions are derived for the exact and asymptotic MI statistics while the prevalent Gaussian approximation is examined. Moreover, new explicit expressions for the marginal eigenvalues are deduced which are thereafter used to analyze the BF performance of the associated eigenmodes in terms of Signal-to-Noise ratio (SNR) outage probability. We note that all derived formulas are given in tractable determinant form and therefore allow for fast and efficient computation and also yield an excellent match with Monte-Carlo simulations, under different fading scenarios and model parameters.

Asymptotic Mutual Information for Rician MIMO-MA Channels with Arbitrary Inputs: A Replica Analysis

Using the replica method, originally developed in statistical physics, we derive the asymptotic mutual information (MI) of a spatially correlated Rician multiple-input multiple-output (MIMO) multiple-access (MA) channel for any given signaling input distributions (not necessarily Gaussian) in the large-system regime where the numbers of antennas at the transmitters and the receiver go to infinity with a constant ratio. In addition to the result that uses joint optimum decoding (JOD) at the central receiver, we also obtain asymptotic characteristics of a more practical system which performs separate detection and decoding (SDD) (i.e., spatial detection followed by a temporal error-correction decoder). Simulation results reveal that the asymptotic results provide promising estimates for the average performance metrics (e.g., the error probability and the MI) even with only a few antenna elements at the transceivers.

Asymptotic Mutual Information Statistics of Separately Correlated Rician Fading MIMO Channels

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The asymptotic probability distribution of the mutual information of a separately correlated Rician fading multiple-input multiple-output (MIMO) channel is addressed. The mean and variance of the mutual information are derived when the number of transmit and receive antennas grows asymptotically large while their ratio approaches a finite constant. This derivation is based on the <emphasis emphasistype="boldital">replica method</emphasis>, widely used in theoretical physics and, more recently, in the analysis of communication systems (code-division multiple access (CDMA) and MIMO). Though the replica method allows to analyze complex systems in a comparatively simple way, some authors pointed out that its assumptions are not always rigorous. It is shown that the mutual information converges asymptotically to a Gaussian distribution under mild technical conditions, which are tantamount to assuming that the spatial correlation structure has no asymptotically dominant eigenmodes. The accuracy of the asymptotic approach is assessed by numerical results. It is shown that the approximation is very accurate in a wide range of system settings, even when the number of transmit and receive antennas is as small as a few units. </para>