Instantaneous Mutual Information Research Articles

Behavior of information flow near criticality.

Recent experiments have indicated that many biological systems self-organize near their critical point, which hints at a common design principle. While it has been suggested that information transmission is optimized near the critical point, it remains unclear how information transmission depends on the dynamics of the input signal, the distance over which the information needs to be transmitted, and the distance to the critical point. Here we employ stochastic simulations of a driven two-dimensional Ising system and study the instantaneous mutual information and the information transmission rate between a driven input spin and an output spin. The instantaneous mutual information varies nonmonotonically with the temperature but increases monotonically with the correlation time of the input signal. In contrast, there exists not only an optimal temperature but also an optimal finite input correlation time that maximizes the information transmission rate. This global optimum arises from a fundamental trade-off between the need to maximize the frequency of independent input messages, the necessity to respond fast to changes in the input, and the need to respond reliably to these changes. The optimal temperature lies above the critical point but moves toward it as the distance between the input and output spin is increased.

Receive Antenna Selection Under Discrete Inputs: Approximation and Applications

To analyze the achievable performance of antenna selection (AS) in practical multi-antenna systems, this paper studies the receive antenna selection (RAS) in single-input multiple-output (AS-SIMO) systems under discrete inputs. We first propose an approximate expression to evaluate the instantaneous mutual information (MI) of $M$ -ary quadrature amplitude modulation ( $M$ -QAM) signaling over additive white Gaussian noise (AWGN) channels. Then, by exploiting this approximate formula, we develop a closed-form formula for the ergodic MI in AS-SIMO systems with $M$ -QAM signaling. Additionally, we also analyze the asymptotic MI for a large number of receive antennas $N_{\mathrm{r}}$ . This asymptotic analysis suggests that the scaling rate of the MI with $N_{\mathrm{r}}$ becomes zero rate in contrast to the double logarithmic rate under Gaussian inputs. Besides, our result is also extended to discuss the mutual information of multiple-input multiple-output (MIMO) systems having discrete inputs with receive antenna selection, and an upper bound for the MI is derived. Finally, the derived result is applied to analyze several performance measures of the discrete inputs driven AS-SIMO systems. Specifically, it is first used to discuss the relationship between the ergodic MI and the number of active antennas. Our investigation shows that this relationship follows Pareto principle, i.e., 80% of the MI of full-antenna selection can be achieved via 20% of the total antennas. Then, our proposed approximation is employed to analytically study the effective MI which takes channel estimation (CE) into consideration, indicating that CE is a main limit of large-scale systems. Moreover, the energy efficiency (EE) is explored on the basis of our results, and we find there exists an optimal number of active antennas to maximize the energy efficiency. In addition to theoretical derivations, all the analytical results are validated by numerical simulations.

Predictive Strategy for Energy Harvesting MIMO Systems With Finite Alphabet Inputs and Limited Feedback

Due to the lack of explicit expression of instantaneous mutual information (IMI), online throughput optimization for energy harvesting (EH) MIMO systems with finite alphabet inputs suffers from heavy computation burden and large feedback overhead. We propose a two-layer predictive strategy with the aid of context information, by constructing a precoding codebook offline at both the transmitter and receiver, to relax the time pressure for the online strategy. Two schemes were provided to accelerate the codebook construction. One is to estimate IMI during optimization by using a reference function. The other is by starting the optimization of each precoder from an adjacent precoder. With the channel distribution information (CDI), the feedback overhead is reduced by minimizing the codebook size. By the methods above, the building speed of the precoding codebook can be up to two orders of magnitude faster than no accelerating strategies. Meanwhile, through topology analysis, we propose a higher layer predictive strategy, which determines the index of allocated energy at each slot within a frame. The higher layer strategy takes full advantage of the constructed precoding codebook and context information when estimating the future throughput. Simulation results demonstrate the effectiveness and practicality of the proposed two-layer strategy.

Iterative Search Algorithm to Maximize System Capacity in Time-Varying MIMO DAS

In this paper, an iterative search algorithm to maximize system capacity in a time-varying MIMO distributed antenna system (DAS) is proposed. A common DAS model is employed, in which the transmit antennas are distributively located and the receiver can move arbitrarily in the region. Due to the small-scale fading effect and the receiver movement, the transmitters and receiver cannot obtain accurate channel state information (CSI) or optimize the system capacity. Therefore, we present a time-varying MIMO DAS model and corresponding channel evolution model to predict the time-varying channel. With the channel evolution model, the proposed iterative search algorithm can calculate the precoding matrix, and an iterative search process is employed to determine the optimal precoding matrix from a precoding matrix set. Meanwhile, error analysis show the error on the instantaneous mutual information with the proposed algorithm has a close relationship with the element number of the precoding matrix set and small-scale fading evolution coefficient. We also propose a non-uniform power allocation strategy which can improve the system capacity. Simulation results are presented to verity the analysis above and demonstrate the performance of system capacity with the proposed iterative search algorithm.

An Opportunistic Source Selection Spectrum Sharing Protocol for Overlay Cognitive Two-Way Relaying System

In this paper, an overlay two-way relaying spectrum sharing protocol is proposed by employing an opportunistic source selection strategy. In this protocol, a secondary user acts as a relay and allocates a fraction of its transmit power to assist two-way communication between two primary users (PUs), thus, gaining spectrum sharing as long as without degrading the primary system performance. Moreover, to exploit the multiuser diversity inherent in the multiuser primary systems, the two PUs transmit their own messages in an opportunistic manner, depending on the instantaneous channel conditions. Specifically, at each transaction, only one of the PUs is selected for transmission such that the instantaneous mutual information on the whole primary network is maximized and as a result, the multiuser diversity gain is achieved by the primary system. Then, we derive the exact closed-form outage probabilities for both the primary and secondary systems. To gain more insights, closed-form asymptotic outage probability expressions are also derived. Based on the approximations, we further obtain analytical results on the diversity order of the primary system and the power-allocation threshold above which the spectrum sharing is allowed for secondary transmission. Numerical results are presented to verify our analysis and highlight the advantages of our proposed scheme compared to another two-phase spectrum sharing protocol based on analog network coding.

What we learn from the learning rate

The learning rate is an information-theoretical quantity for bipartite Markov chains describing two coupled subsystems. It is defined as the rate at which transitions in the downstream subsystem tend to increase the mutual information between the two subsystems, and is bounded by the dissipation arising from these transitions. Its physical interpretation, however, is unclear, although it has been used as a metric for the sensing performance of the downstream subsystem. In this paper, we explore the behaviour of the learning rate for a number of simple model systems, establishing when and how its behaviour is distinct from the instantaneous mutual information between subsystems. In the simplest case, the two are almost equivalent. In more complex steady-state systems, the mutual information and the learning rate behave qualitatively distinctly, with the learning rate clearly now reflecting the rate at which the downstream system must update its information in response to changes in the upstream system. It is not clear whether this quantity is the most natural measure for sensor performance, and, indeed, we provide an example in which optimising the learning rate over a region of parameter space of the downstream system yields an apparently sub-optimal sensor.

Optimal Power Allocation in Spatial Modulation Systems

In spatial modulation (SM) systems, when channel state information is available at the transmitter, adaptive power allocation can be exploited to enhance the SM performance. In this paper, optimal power allocation in SM systems is considered from an information-theoretical view. First, as there is no closed-form expression of the SM capacity, the instantaneous mutual information is formulated in an analytic form for SM systems with two transmit antennas. Then, two power-allocation algorithms are proposed to maximize the SM mutual information. One is a sub-optimal power-allocation algorithm that is established in closed form to maximize the upper bound of SM mutual information. The other is an iterative algorithm for optimal power allocation that is proposed to achieve the maximum mutual information of SM systems while dramatically reducing the operational complexity compared with global computer searching. To evaluate the performance of the proposed power-allocation algorithms, numerical and simulation results are reported in terms of outage capacity and ergodic capacity, which demonstrates that the proposed power-allocation algorithms significantly improve the SM performance, compared with conventional equal-power allocation.

Joint source and relay precoding for generally correlated MIMO with full and partial CSIT

In this paper, we jointly design linear source and relay precoders for two-hop MIMO relaying. The involved channels encounter spatially correlated fading, the source data symbols are mutually correlated, and the noises are colored not only at the destination but also at the relay. Two different scenarios of channel state information (CSI) are assumed to be available at the transmitters: full CSI of both hops (the full CSIT) and full CSI of the source-relay hop only plus the covariance information of the relay-destination hop (the partial CSIT). First, with the full CSIT, we derive optimal precoders by maximizing the instantaneous mutual information (MI). Secondly, with the partial CSIT knowledge we derive suboptimal precoders by maximizing the average MI. For both the CSIT cases, we propose an iterative algorithm to perform power allocation iteratively and alternatively between the source antennas and the relay antennas. Its simplified version in which the power allocation is performed separately between the source antennas and the relay antennas is also developed. Simulation results show that our proposed precoding schemes with the full CSIT provide significantly higher capacity than the existing schemes. Besides, the proposed schemes with the partial CSIT also perform well especially when the channels are spatially correlated at the transmit sides and at medium-to-high signal-to-noise ratios (SNRs), while they require much lower computational complexity and less feedback overhead.

A Closed-Form Mutual Information Approximation for Multiple-Antenna Systems With Spatial Modulation

In this letter, a low-complexity approximation for instantaneous mutual information (IMI) of spatial modulation multiple-input single-output systems is presented. Based on this low-complexity approximation, a closed-form approximation for the average mutual information (MI) is derived. The approximation closely follows exact average MI values for different numbers of antennas and a wide range of signal-to-noise ratios. The proposed approximation for the IMI has a considerably lower computational complexity compared to previously proposed bounds. Also, the closed-form approximation of the average MI is considerably less complex than previously proposed bounds that require numerical averaging over randomly generated channel realizations.

On the Transmission Rate Strategies in Cognitive Radios

We investigate instantaneous transmission rate strategies for secondary users in cognitive radio networks by analyzing their effective capacity performance in different signal-to-noise ratio regimes with different quality-of-service constraints and transmission block sizes. Describing a channel model with one secondary transmitter and one secondary receiver with the potential presence of primary users, we present an interference power constraint that limits the transmission power of secondary users not only when a channel is sensed as busy but also when a channel is sensed as idle. Calling the existing transmission rate strategy Optimistic Policy , we introduce two other strategies, particularly Conservative Policy and Greedy Policy . Secondary users in Optimistic Policy set the instantaneous transmission rate to the instantaneous mutual information assuming the correctness of channel sensing results, whereas they set the instantaneous transmission rate to the instantaneous mutual information regarding possible transmission outages in Conservative Policy and disregarding possible transmission outages in Greedy Policy . We construct a state transition diagram and formulate the effective capacity employing these policies. We calculate the minimum energy-per-bit requirements and the high signal-to-noise ratio slope in order to explore performance variations in low and high signal-to-noise ratio regimes, respectively. Correspondingly, we show that Optimistic Policy is, in general, more favorable in secondary users when the quality-of-service constraints are loose, the transmission blocks are shorter, and the signal-to-noise ratio is low. On the other hand, Conservative Policy is better when the quality-of-service constraints are strict, the transmission blocks are longer, and the signal-to-noise ratio is high.

Second-order statistics of the instantaneous mutual information in time-varying underwater acoustic particle velocity channels

Instantaneous mutual information (IMI) is the amount of information that a time-varying channel can convey for the given time instant. In this paper, second-order statistics of IMI are studied in time-varying underwater acoustic particle velocity channels. First, the autocorrelation function, correlation coefficient, level crossing rate, and average outage duration of IMI are provided in a time-varying fading channel. Exact expressions are given in terms of the summation of special functions, which facilitate numerical calculations. Then, accurate approximations for the autocorrelation function and correlation coefficient are presented for low and high signal-to-noise ratios. Moreover, analytical and numerical results are provided for the correlation and level-crossing characteristics of IMI in underwater particle velocity channels. The results shed light on the dynamic behavior of mutual information in underwater acoustic particle velocity channels. [Work supported in part by the National Science Foundation (NSF), Grant CCF-0830190.]

On Power Allocation for Incremental Redundancy Hybrid ARQ

We study the power allocation for incremental redundancy (IR) hybrid automatic repeat request (HARQ) in block fading channel where causal channel state information is known at both the transmitter and the receiver. In IR-HARQ, the traffic is assigned into consecutive transmission rounds, and at each round, the instantaneous mutual information that is required for successful decoding is upper bounded by the data rate that is to be delivered. We propose an HARQ power allocation method that maximizes the average of incremental mutual information at each round, and its throughput quickly converges to the ergodic capacity as the number of retransmissions increases. The numerical results show that the proposed power allocation method achieves almost the full channel capacity with moderate average transmission delay and that it maintains good throughput under stringent delay requirement.

An information theoretic sparse kernel algorithm for online learning

Kernel-based algorithms have been proven successful in many nonlinear modeling applications. However, the computational complexity of classical kernel-based methods grows superlinearly with the increasing number of training data, which is too expensive for online applications. In order to solve this problem, the paper presents an information theoretic method to train a sparse version of kernel learning algorithm. A concept named instantaneous mutual information is investigated to measure the system reliability of the estimated output. This measure is used as a criterion to determine the novelty of the training sample and informative ones are selected to form a compact dictionary to represent the whole data. Furthermore, we propose a robust learning scheme for the training of the kernel learning algorithm with an adaptive learning rate. This ensures the convergence of the learning algorithm and makes it converge to the steady state faster. We illustrate the performance of our proposed algorithm and compare it with some recent kernel algorithms by several experiments.

Soft forwarding technique for unitary space‐time modulation

The authors propose two soft forwarding schemes for unitary space-time modulation, namely estimate-and-forward (EF) and mutual information-based forwarding (MIF), which are ideally suited for rapid piecewise-constant fading scenarios with no channel state information both at transmitters and receivers. In the EF scheme, the relay regenerates signal via the conditional expectation of the source unitary space-time (UST) signal given the received matrix at the relay node instead of hard decisions. The MIF scheme employs the instantaneous mutual information between the source UST signal and received matrix at the relay node as a measurement for maximum-likelihood detection. The authors provide the relay forwarding functions, the normalisation factors, and the detection methods at the destination firstly. Above all, the generalised signal-to-noise ratio (SNR) at the destination for our unitary space-time modulated relaying system is analysed and simulated firstly. Simulation results reveal that the proposed EF and MIF schemes can outperform the existing amplify-and-forward (AF) and detect-and-forward (DF) schemes in terms of bit-error-rate performances in most scenarios. For a parallel relay network with two relays, the proposed EF scheme achieves 2 and 3 dB SNR gain over AF and DF schemes, respectively. Moreover, the MIF scheme can further give about 1.5 dB SNR gain.

Outage minimization for parallel fading channels with limited feedback

We address an optimal power allocation problem for minimizing the outage probability for M parallel block-Nakagami-fading channels under a long-term average sum transmit power constraint with finite rate feedback of channel state information (CSI). A simulation-based optimization technique called simultaneous perturbation stochastic approximation algorithm (SPSA) is employed first to numerically derive a locally optimal power codebook. Due to the high computational complexity and long convergence time of SPSA, we make an ordering assumption on the power codebook entries and derive effective hyperplane based approximations to the channel quantization regions and present a number of low-complexity suboptimal quantized power codebook design algorithms. Unlike existing work on outage minimization for multiple-input multiple-output (MIMO) channels with limited feedback, we do not assume that identical transmission power is used for all channels within each channel quantization region. We also do not resort to a Gaussian approximation for the instantaneous mutual information in general as used in many existing work. Based on our power ordering assumption and hyperplane based approximations, we show that allocating identical power to all channels within a given channel quantization region in the limited feedback scenario is asymptotically optimal only at high average power (or average signal-to-noise ratio (SNR)) for the Rayleigh fading case, whereas for the general Nakagami case, the transmit power allocation for an individual channel within each quantized region is asymptotically proportional to the corresponding Nakagami fading parameter (severity of fading). We also present a novel diversity order result for the outage probability for the Nakagami fading case. Finally, we derive a suitable Gaussian approximation based low-complexity power allocation scheme for a large number of parallel channels, which has important applications in wideband slow-fading orthogonal frequency-division multiplexing (OFDM) systems. Extensive numerical results illustrate that only a few bits of feedback close the gap substantially in outage performance between the limited feedback case and the full instantaneous CSI at the transmitter case.

Instantaneous Mutual Information and Eigen-Channels in MIMO Mobile Rayleigh Fading

In this paper, we study two important metrics in multiple-input multiple-output (MIMO) time-varying Rayleigh flat fading channels. One is the eigen-channel, and the other is the instantaneous mutual information (IMI). Their second-order statistics, such as the correlation coefficient, level crossing rate (LCR), and average fade/outage duration, are investigated, assuming a general nonisotropic scattering environment. Exact closed-form expressions are derived and Monte Carlo simulations are provided to verify the accuracy of the analytical results. For the eigen-channels, we found they tend to be spatio-temporally uncorrelated in large MIMO systems. For the IMI, the results show that its correlation coefficient can be well approximated by the squared amplitude of the correlation coefficient of the channel, under certain conditions. Moreover, we also found the LCR of IMI is much more sensitive to the scattering environment than that of each eigen-channel.

On the Capacity of MIMO Channels with Outdated Channel State Information

A Multiple-input multiple-output (MIMO) time-varying flat fading channel is considered. The transmitter obtained the channel state information (CSI) relying on the reciprocity principle or by the feedback from the receiver. Thus, channel state information at the transmitter (CSIT) is outdated due to the delay between the estimation of the channel and the transmission of the data. In order to achieve the maximum channel capacity, the transmitter linearly precoded the signal before transmission based on the outdated CSIT. Under the assumptions of wide-sense stationary uncorrelated scattering Rayleigh fading channels and Jake’s model, the instantaneous mutual information and ergodic capacity is derived for MIMO channels with outdated CSIT. The information outage probability of MIMO channel with outdated CSIT is also presented.

Power minimization of central wishart MIMO block-fading channels

In this correspondence, a central Wishart multiple input multiple-output (MIMO) K-block fading channel with an information outage probability constraint is studied. Under this block-limited channel condition, we aim to minimize the transmit power required for attaining a given outage probability based on the statistical channel information at the transmitter (SCIT). Using Gaussian approximation to express the probability density function (pdf) of the instantaneous mutual information and by deriving analytically the mean and variance of the mutual information of the MIMO channel, the optimal power allocation can be obtained numerically by a simple one-dimensional sampling method such as dividing rectange (DIRECT).

Asymptotic tradeoff between cross-layer goodput gain and outage diversity in OFDMA systems with slow fading and delayed CSIT

There are two important aspects of cross-layer gains in multiuser OFDMA systems with slow fading channels. They are the system goodput gain as well as the packet diversity gain. The former aspect of cross-layer designs has been well- studied under perfect CSIT conditions and is known as the multi-user diversity gain (MuDiv). In cross-layer OFDMA systems with perfect CSIT, it is well known that the system throughput (ergodic capacity) scales in the order of O(log log Kldquo) due to the MuDiv gain, where K is the number users. However, in slow fading channels with delayed CSIT, there will always be potential packet errors (due to channel outage if the scheduled data rate exceeds the instantaneous mutual information) even if very strong channel coding is applied at the base station. In this case, the cross-layer packet outage diversity is important to protect the packet errors due to channel outage and there is a natural tradeoff between the goodput gain and packet diversity. In this paper, we shall focus on the asymptotic tradeoff analysis between the system goodput gain and the packet outage diversity gain in cross-layer OFDMA systems with delayed CSIT.

Near-Optimal Power Allocation and Multiuser Scheduling with Outage Capacity Constraints Exploiting Only Channel Statistics

In this letter, we consider a delay-constrained wireless communication system where a delay constraint is characterized by the information outage probability in a transmission of K-block Rayleigh flat-fading channels. With multiple users, each given an individual delay requirement, this letter aims to minimize the total transmitted power by optimizing jointly the power distribution and the time sharing of a time-division (TD) multiuser system where the transmitter knows only the channel statistics (CST) but the mobile receivers (or users) have perfect knowledge of the channels. A suboptimal solution which utilizes Gaussian approximation with the mean and variance derived on the instantaneous mutual information and convex optimization, is proposed. Numerical results show that the proposed scheme performs nearly the same as the global optimum.