Expression For Mutual Information Research Articles

Optimization of Covert Communication in Multi-Sensor Asymmetric Noise Systems

This work investigates wireless covert communication in a multi-sensor asymmetric noise scenario. We adopt KL (Kullback–Leibler) divergence as the covertness constraint metric and mutual information as the transmission rate metric. To accurately approximate KL divergence and mutual information in covert communication, we employ the Taylor series expansion technique. Analytical expressions for KL divergence and mutual information in covert communication are derived, and we optimize the amplitude gain and phase angles based on these analytical expressions. Our findings underscore the importance of phase angle selection in covert communication within asymmetric noise systems. We propose an effective method for optimizing the transmission amplitude gain and phase angles in scenarios with asymmetric noise. Numerical results validate the effectiveness and superiority of our proposed method.

Capacity-driven end-to-end learning approach for NOMA with finite-alphabet inputs

Optimal Non-orthogonal multiple access (NOMA) design with practical finite-alphabet inputs instead of ideal Gaussian inputs has been addressed in this paper. With the aid of tight mutual information neural estimation, a capacity-driven end-to-end learning communication framework is proposed to acquire optimal superposition coding and thus approach the capacity region of NOMA with finite-alphabet inputs. For such a purpose, we firstly give a closed-form expression of conditional mutual information to define the capacity region of NOMA with finite-alphabet inputs. A natural tight bound on mutual information is then derived for simplifying the closed-form expression, which is needed to establish a consistent and tight mutual information neural estimator. Lastly, in the end-to-end learning framework, an encoder and a self-attention, which are respectively responsible for constellation learning and power allocation, are guided by the capacity approaching method based on mutual information estimation, while successive interference cancellation (SIC) is designed as a neural unit embedded in the decoder to guarantee decoding performance. Numerical results show that the proposed capacity-driven end-to-end learning scheme achieves the achievable rate region for NOMA, with SER performance comparable to state-of-the-art decoding schemes.

The Double-Sided Information Bottleneck Function.

A double-sided variant of the information bottleneck method is considered. Let be a bivariate source characterized by a joint pmf . The problem is to find two independent channels and (setting the Markovian structure ), that maximize subject to constraints on the relevant mutual information expressions: and . For jointly Gaussian and , we show that Gaussian channels are optimal in the low-SNR regime but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low and are suboptimal for high correlations. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., ) and provide supporting numerical evidence. Furthermore, we present a Blahut–Arimoto type alternating maximization algorithm and demonstrate its performance for a representative setting. This problem is closely related to the domain of biclustering.

Outage Probability and Finite-SNR DMT Analysis for IRS-Aided MIMO Systems: How Large IRSs Need to be?

Intelligent reflecting surfaces (IRSs) are promising enablers for high-capacity wireless communication systems by constructing favorable channels between the transmitter and receiver. However, general, accurate, and tractable outage probability analysis for IRS-aided multiple-input-multiple-output (MIMO) systems is not available in the literature. In this paper, we first characterize the distribution of the mutual information (MI) for IRS-aided MIMO systems by capitalizing on large random matrix theory (RMT). Based on this result, a closed-form approximation for the outage probability is derived and a gradient-based algorithm is proposed to minimize the outage probability with statistical channel state information (CSI). We also investigate the diversity-multiplexing tradeoff (DMT) with finite signal-to-noise ratio (SNR). Based on these theoretical results, we further study the impact of the IRS size on system performance. In the high SNR regime, we provide closed-form expressions for the ergodic mutual information (EMI) and outage probability as a function of the IRS size, which analytically reveal that the benefit of increasing the IRS size saturates quickly. Simulation results validate the accuracy of the theoretical analysis and confirm the increasing cost to improve system performance by deploying larger IRSs. For example, for an IRS-aided MIMO system with 20 antennas at both the transmitter and receiver, we need to double the size of the IRS to increase the throughput from 90% to 95% of its maximum value.

Mutual Information Analysis of Mixed-ADC MIMO Systems Over Rayleigh Channels Based on Random Matrix Theory

The mixed analog-to-digital converters (ADC) architecture is a promising solution to the problem of high energy consumption of multiple-input multiple-output (MIMO) systems. In such a scheme, part of the received signals at the base station are quantized by high-resolution ADCs, while the others are quantized by one-bit ADCs. The signals quantized by the mixed-ADC architecture are correlated and non-identically distributed, which is different from the one-bit quantized MIMO systems. Moreover, unlike the previous works focusing on the achievable rate of mixed-ADC MIMO systems under linear detectors, we derive a closed-form expression of the ergodic mutual information between the transmit signals and the quantized outputs of the mixed-ADC architecture over Rayleigh channels, where the statistical property of the equivalent channel is characterized by random matrix theory. Then, we derive the power scaling law, which shows that reducing the transmit power will not sacrifice the transmission efficiency when the number of receive antennas increases. Furthermore, for any given number of antennas and user transmit power, the optimal number of one-bit ADCs for maximizing the energy efficiency is obtained. Moreover, constrained by a given hardware energy consumption, the theoretical mutual information shows obvious gains over the one of unquantized MIMO systems for low user transmit power.

Low-cost limit of classical communication with restricted quantum measurements

We consider a communication scenario where classical information is encoded in an ensemble of quantum states that admit a power series expansion in a cost parameter and converge to a single zero cost state with vanishing cost. For a given measurement scheme, we derive an approximate expression for mutual information in the leading order of the cost parameter. The general results are applied to selected problems in optical communication, where coherent states of light are used as input symbols and the cost is quantified as the average number of photons per symbol. We show that for an arbitrary individual measurement on phase shift keyed (PSK) symbols, the photon information efficiency is upper bounded by 2 nats of information per photon in the low-cost limit, which coincides with the conventional homodyne detection bound. The presented low-cost approximation facilitates a systematic analysis of few-symbol measurements that exhibit superadditivity of accessible information. For the binary PSK alphabet of coherent states, we present designs for two- and three-symbol measurement schemes based on linear optics, homodyning, and single photon detection that offer respectively 2.49% and 3.40% enhancement relative to individual measurements. We also show how designs for scalable superadditive measurement schemes emerge from the introduced low-cost formalism.

Continuous-variable ramp quantum secret sharing with Gaussian states and operations

We aim to quantify and mitigate quantum-information leakage in continuous-variable quantum secret sharing (CV QSS). Here we introduce a technique for certifying CV ramp quantum secret-sharing (RQSS) schemes in the framework of quantum interactive-proof systems. We devise pseudocodes in order to represent the sequence of steps taken to solve the certification problem. Furthermore, we derive the expression for quantum mutual information between the quantum secret extracted by any multi-player structure and the share held by the referee corresponding to the Tyc–Rowe–Sanders CV QSS scheme. We solve by converting the Tyc–Rowe–Sanders position representation for the state into a Wigner function from which the covariance matrix can be found, then insert the covariance matrix into the standard formula for CR quantum mutual information to obtain quantum mutual information in terms of squeezing. Our quantum mutual information result quantifies the leakage of the RQSS schemes.

Predictive Strategy for Energy Harvesting MIMO Systems With Finite Alphabet Inputs and Limited Feedback

Due to the lack of explicit expression of instantaneous mutual information (IMI), online throughput optimization for energy harvesting (EH) MIMO systems with finite alphabet inputs suffers from heavy computation burden and large feedback overhead. We propose a two-layer predictive strategy with the aid of context information, by constructing a precoding codebook offline at both the transmitter and receiver, to relax the time pressure for the online strategy. Two schemes were provided to accelerate the codebook construction. One is to estimate IMI during optimization by using a reference function. The other is by starting the optimization of each precoder from an adjacent precoder. With the channel distribution information (CDI), the feedback overhead is reduced by minimizing the codebook size. By the methods above, the building speed of the precoding codebook can be up to two orders of magnitude faster than no accelerating strategies. Meanwhile, through topology analysis, we propose a higher layer predictive strategy, which determines the index of allocated energy at each slot within a frame. The higher layer strategy takes full advantage of the constructed precoding codebook and context information when estimating the future throughput. Simulation results demonstrate the effectiveness and practicality of the proposed two-layer strategy.

Mutual Information of Wireless Channels and Block-Jacobi Ergodic Operators

Shannon's mutual information of a random multiple antenna and multipath channel is studied in the general case where the channel impulse response is an ergodic and stationary process which is assumed to be available at the receiver. From this viewpoint, the channel is represented by an ergodic self-adjoint block-Jacobi operator, which is close in many aspects to a block version of a random Schr\"odinger operator. The mutual information is then related to the so-called density of states of this operator. In this paper, it is shown that under the weakest assumptions on the channel, the mutual information can be expressed in terms of a matrix-valued stochastic process coupled with the channel process. This allows numerical approximations of the mutual information in this general setting. Moreover, assuming further that the channel impulse response is a Markov process, a representation for the mutual information offset in the large Signal to Noise Ratio regime is obtained in terms of another related Markov process. This generalizes previous results from Levy~\emph{et.al.}. It is also illustrated how the mutual information expressions that are closely related to those predicted by the random matrix theory can be recovered in the large dimensional regime.

Information Theoretic Multi-Target Feature Selection via Output Space Quantization

A key challenge in information theoretic feature selection is to estimate mutual information expressions that capture three desirable terms—the relevancy of a feature with the output, the redundancy and the complementarity between groups of features. The challenge becomes more pronounced in multi-target problems, where the output space is multi-dimensional. Our work presents an algorithm that captures these three desirable terms and is suitable for the well-known multi-target prediction settings of multi-label/dimensional classification and multivariate regression. We achieve this by combining two ideas—deriving low-order information theoretic approximations for the input space and using quantization algorithms for deriving low-dimensional approximations of the output space. Under the above framework we derive a novel criterion, Group-JMI-Rand, which captures various high-order target interactions. In an extensive experimental study we showed that our suggested criterion achieves competing performance against various other information theoretic feature selection criteria suggested in the literature.

Molecular Communications With Molecular Circuit-Based Transmitters and Receivers.

The performance of a communication link can be improved by maximizing the mutual information between the input and output signals. This paper considers this maximization problem in a molecular communication link where both the transmitter and the receiver are molecular circuit. This general optimization is hard to solve. We simplify the problem by limiting to reactions with linear reaction rates and molecular circuits with a limited number of species. We derive an expression of mutual information and use it for numerical maximization. We show that our parameterized transmitter circuit is able to give mutual information that is close to upper bound obtained in our earlier work.

Mutual Information of Wireless Channels and Block-Jacobi Ergodic Operators

Shannon’s mutual information of a random multiple antenna and multipath time varying channel is studied in the general case where the process constructed from the channel coefficients is an ergodic and stationary process which is assumed to be available at the receiver. From this viewpoint, the channel can also be represented by an ergodic self-adjoint block-Jacobi operator, which is close in many aspects to a block version of a random Schrodinger operator. The mutual information is then related to the so-called density of states of this operator. In this paper, it is shown that under the weakest assumptions on the channel, the mutual information can be expressed in terms of a matrix-valued stochastic process coupled with the channel process. This allows numerical approximations of the mutual information in this general setting. Moreover, assuming further that the channel coefficient process is a Markov process, a representation for the mutual information offset in the large signal to noise ratio regime is obtained in terms of another related Markov process. This generalizes previous results from Levy et al. . It is also illustrated how the mutual information expressions that are closely related to those predicted by the random matrix theory can be recovered in the large dimensional regime.

Information-Theoretic Characterization of MIMO Systems With Multiple Rayleigh Scattering

We present an information-theoretic analysis of a point-to-point multiple-input multiple-output (MIMO) link affected by Rayleigh fading and multiple scattering, under perfect channel state information at the receiver. Unlike previous work addressing this setting, we investigate the random coding error exponent, its associated cutoff rate and the expurgated error exponent, and derive closed-form expressions for them. Moreover, leveraging the average mutual information expression presented by Akemann et al. , we derive another important metric, namely, the sum rate, under linear receive processing and independent stream decoding. In particular, we characterize the performance of the minimum mean squared error receiver in closed form, and that of the zero forcing receiver by resorting to bounding techniques. The bulk of the work relies on results about finite-dimensional random matrix products, a number of which are novel and detailed in the Appendices. The analysis, validated through numerical results, highlights the severe degradation in the performance of linear receivers due to multi-fold scattering. It also unveils the performance trend of multiple scattering MIMO channels as a function of the number of antennas and the number of scattering stages.

Media‐based modulation for secrecy communications

Media-based modulation (MBM) concept is exploited to increase physical layer (PHY) security in the presence of an illegitimate listener. A precoding design is developed to increase the mutual secrecy rate of MBM, and the secrecy mutual information expressions are derived. The provided analyses and numerical results have also shown that MBM is a perfect candidate for PHY security by providing the maximum achievable secrecy mutual information, even in the presence of channel estimation errors.

Information-Theoretic Privacy for Smart Metering Systems with a Rechargeable Battery

Smart-metering systems report electricity usage of a user to the utility provider on almost real-time basis. This could leak private information about the user to the utility provider. In this paper, we investigate the use of a rechargeable battery in order to provide privacy to the user. We assume that the user load sequence is a first-order Markov process, the battery satisfies ideal charge conservation, and that privacy is measured using normalized mutual information (leakage rate) between the user load and the battery output. We study the optimal battery charging policy that minimizes the leakage rate among the class of battery policies that satisfy causality and charge conservation. We propose a series reduction on the original problem and ultimately recast it as a Markov Decision Process (MDP) that can be solved using a dynamic program. In the special case of i.i.d. demand, we explicitly characterize the optimal policy and show that the associated leakage rate can be expressed as a single-letter mutual information expression. In this case, we show that the optimal charging policy admits an intuitive interpretation of preserving a certain invariance property of the state. Interestingly an alternative proof of optimality can be provided that does not rely on the MDP approach, but is based on purely information theoretic reductions.

Statistical Approach for Performance Analysis of Multipath Scattering Environment

In realistic scenarios, multipath scattering environments usually require an approach which can easily model signal behavior. We have considered a Rayleigh multipath scattering channel. The multipath scattering environment captures degradation of signals in fading channels. In this paper, performance of the channel is investigated through the distribution of mutual information in the multipath scattering channel using matrix method. The probability density function for multipath scattering channels is still a developing area of research. The need of the hour is performance evaluation of these communication links in an efficient manner. By using techniques of probability theory, the variance of mutual information is derived for multiple cases based on fading environment. In this paper, we have considered a matrix based multipath scattering model for communication link. We have derived closed-form expression for mutual information in a multipath scattering channel. The matrix method is used to substitute this effect in terms of the Gaussian distribution. This substitution will prove to be very accurate for different cases as analyzed in this paper.

Joint Precoder and Artificial Noise Design for MIMO Wiretap Channels With Finite-Alphabet Inputs Based on the Cut-Off Rate

We consider precoder and artificial noise (AN) design for multi-antenna wiretap channels under the finite-alphabet input assumption. We assume that the transmitter has access to the channel coefficients of the legitimate receiver and knows the statistics of the eavesdropper’s channel. Accordingly, we propose a secrecy rate maximization algorithm using a gradient descent-based optimization of the precoder matrix and an exhaustive search over the power levels allocated to the AN. We also propose algorithms to reduce the complexities of direct ergodic secrecy rate maximization by: 1) maximizing a cut-off rate-based approximation for the ergodic secrecy rate, simplifying the mutual information expression, which lacks a closed-form and 2) diagonalizing the channels toward the legitimate receiver and the eavesdropper, which allows for employing a per-group precoding-based technique. Our numerical results reveal that jointly optimizing the precoder and the AN outperforms the existing solutions in the literature, which rely on the precoder optimization only. We also demonstrate that the proposed low complexity alternatives result in a small loss in performance while offering a significant reduction in computational complexity.

Information-Theoretic Privacy in Smart Metering Systems Using Cascaded Rechargeable Batteries

A rechargeable battery may alleviate the issue of privacy loss in a smart metering system by distorting a household's load profile. However, existing studies involve a single rechargeable battery, whereas in a network scenario, there could be multiple batteries connected together. In this letter, we study the extension where a user's electricity load is input into a network of two rechargeable batteries, connected in series, and operating individually. This battery network attempts to mask the user load from the utility provider. We focus on the case of independent identically distributed load profile and a system of ideal batteries with no conversion loss, and use normalized mutual information (leakage rate) as the privacy metric. We derive upper and lower bounds on the leakage rate in terms of (single-letter) mutual information expressions. On the achievability side, our information-theoretic upper bound captures the novel tension between minimizing the leakage across each individual battery and the effect of their joint interaction. For the lower bound, we show that a system with a single battery, whose storage capacity is the sum of the two individual batteries, can achieve a leakage rate at least as small as our proposed setup. Furthermore, we use simulations to compare achievable leakage of our proposed scheme with several baseline schemes. The achievable leakage rates obtained in this study could help us to elucidate the privacy performance of a network of batteries.

A Framework for Joint Design of Pilot Sequence and Linear Precoder

Most performance measures of pilot-assisted multiple-input multiple-output (MIMO) systems are functions that depend on both the linear precoding filter and the pilot sequence. A framework for the optimization of these two parameters is proposed, based on a matrix-valued generalization of the concept of effective signal-to-noise ratio (SNR) introduced in a famous work by Hassibi and Hochwald. The framework applies to a wide class of utility functions of said effective SNR matrix, most notably a well-known mutual information expression for Gaussian inputs, an upper bound on the minimum mean-square error (MMSE), as well as approximations thereof. The approach consists in decomposing the joint optimization problem into three subproblems: first, we describe how to reformulate the optimization of the linear precoder subject to a fixed pilot sequence as a convex problem. Second, we do likewise for the optimization of the pilot sequence subject to a fixed precoder. Third, we describe how to generate pairs of precoders and pilot sequences that are Pareto optimal in the sense that they attain the Pareto boundary of the set of feasible effective SNR matrices. By combining these three optimization problems into an iteration, we obtain an algorithm which allows to compute jointly optimal pairs of precoders and pilot sequences with respect to some generic utility function of the effective SNR.

Free Deterministic Equivalents for the Analysis of MIMO Multiple Access Channel

In this paper, a free deterministic equivalent is proposed for the capacity analysis of the multi-input multi-output (MIMO) multiple access channel (MAC) with a more general channel model compared to previous works. In particular, a MIMO MAC with one base station (BS) equipped with several distributed antenna sets is considered. Each link between a user and a BS antenna set forms a jointly correlated Rician fading channel. The analysis is based on operator-valued free probability theory, which broadens the range of applicability of free probability techniques tremendously. By replacing independent Gaussian random matrices with operator-valued random variables satisfying certain operator-valued freeness relations, the free deterministic equivalent of the considered channel Gram matrix is obtained. The Shannon transform of the free deterministic equivalent is derived, which provides an approximate expression for the ergodic input-output mutual information of the channel. The sum-rate capacity achieving input covariance matrices are also derived based on the approximate ergodic input-output mutual information. The free deterministic equivalent results are easy to compute, and simulation results show that these approximations are numerically accurate and computationally efficient.