Capacity-achieving Input Distribution Research Articles

Algorithmic Computability and Approximability of Capacity-Achieving Input Distributions

The capacity of a channel can usually be characterized as a maximization of certain entropic quantities. From a practical point of view it is of primary interest to not only compute the capacity value, but also to find the corresponding optimizer, i.e., the capacity-achieving input distribution. This paper addresses the general question of whether or not it is possible to find algorithms that can compute the optimal input distribution depending on the channel. For this purpose, the concept of Turing machines is used which provides the fundamental performance limits of digital computers and therewith fully specifies which tasks are algorithmically feasible in principle. It is shown for discrete memoryless channels that it is impossible to algorithmically compute the capacity-achieving input distribution, where the channel is given as an input to the algorithm (or Turing machine). Finally, it is further shown that it is even impossible to algorithmically approximate these input distributions.

Comments on “Channel Coding Rate in the Finite Blocklength Regime”: On the Quadratic Decaying Property of the Information Rate Function

The quadratic decaying property of the information rate function states that given a fixed conditional distribution $p_{\mathsf{Y}|\mathsf{X}}$, the mutual information between the (finite) discrete random variables $\mathsf{X}$ and $\mathsf{Y}$ decreases at least quadratically in the Euclidean distance as $p_\mathsf{X}$ moves away from the capacity-achieving input distributions. It is a property of the information rate function that is particularly useful in the study of higher order asymptotics and finite blocklength information theory, where it was already implicitly used by Strassen [1] and later, more explicitly, by Polyanskiy-Poor-Verd\'u [2]. However, the proofs outlined in both works contain gaps that are nontrivial to close. This comment provides an alternative, complete proof of this property.

Capacity-Achieving Input Distributions of Additive Vector Gaussian Noise Channels: Even-Moment Constraints and Unbounded or Compact Support.

We investigate the support of a capacity-achieving input to a vector-valued Gaussian noise channel. The input is subjected to a radial even-moment constraint and is either allowed to take any value in Rn or is restricted to a given compact subset of Rn. It is shown that the support of the capacity-achieving distribution is composed of a countable union of submanifolds, each with a dimension of n-1 or less. When the input is restricted to a compact subset of Rn, this union is finite. Finally, the support of the capacity-achieving distribution is shown to have Lebesgue measure 0 and to be nowhere dense in Rn.

New Bounds on Capacity Region of Multiple Access Channels in Visible Light Communications.

In this paper, we propose new inner and outer bounds of the capacity region for multiple access channels in visible light communication (VLC) networks under both peak and average optical power constraints. Specifically, the proposed inner bounds are established by employing the single-user capacity achieving input distribution for each user. The proposed outer bounds are derived by determining single-user capacities for each user and calculating a sum capacity upper bound by relaxing the input constraints. Numerical results show that the proposed new bounds are extremely tight and outperform existing bounds over wide ranges of SNRs.

On the Capacity-Achieving Input of Channels With Phase Quantization

Several information-theoretic studies on channels with output quantization have identified the capacity-achieving input distributions for different fading channels with 1-bit in-phase and quadrature (I/Q) output quantization. However, an exact characterization of the capacity-achieving input distribution for channels with multi-bit phase quantization has not been provided. In this paper, we consider four different channel models with multi-bit phase quantization at the output and identify the optimal input distribution for each channel model. We first consider a complex Gaussian channel with $b$-bit phase-quantized output and prove that the capacity-achieving distribution is a rotated $2^b$-phase shift keying (PSK). The analysis is then extended to multiple fading scenarios. We show that the optimality of rotated $2^b$-PSK continues to hold under noncoherent fast fading Rician channels with $b$-bit phase quantization when line-of-sight (LoS) is present. When channel state information (CSI) is available at the receiver, we identify $\frac{2\pi}{2^b}$-symmetry and constant amplitude as the necessary and sufficient conditions for the ergodic capacity-achieving input distribution; which a $2^b$-PSK satisfies. Finally, an optimum power control scheme is presented which achieves ergodic capacity when CSI is also available at the transmitter.

Capacities and Optimal Input Distributions for Particle-Intensity Channels

This work introduces the particle-intensity channel (PIC) as a new model for molecular communication systems that includes imperfections at both transmitter and receiver and provides a new characterization of the capacity limits as well as properties of the optimal (capacity-achieving) input distributions for such channels. In the PIC, the transmitter encodes information, in symbols of a given duration, based on the probability of particle release, and the receiver detects and decodes the message based on the number of particles detected during the symbol interval. In this channel, the transmitter may be unable to control precisely the probability of particle release, and the receiver may not detect all the particles that arrive. We model this channel using a generalization of the binomial channel and show that the capacity-achieving input distribution for this channel always has mass points at probabilities of particle release of zero and one. To find the capacity-achieving input distributions, we develop a novel and efficient algorithm we call dynamic assignment Blahut-Arimoto (DAB). For diffusive particle transport, we also derive the conditions under which the input with two mass points is capacity-achieving.

The Bit Error Performance and Information Transfer Rate of SPAD Array Optical Receivers

In this paper the photon counting characteristics, the information rate and the bit error performance of single-photon avalanche diode (SPAD) arrays are investigated. It is shown that for sufficiently large arrays, the photocount distribution is well approximated by a Gaussian distribution with dead-time-dependent mean and variance. Because of dead time, the SPAD array channel is subject to counting losses, part of which are due to inter-slot interference (ISI) distortions. Consequently, this channel has memory. The information rate of this channel is assessed. Two auxiliary discrete memoryless channels (DMCs) are proposed which provide upper and lower bounds on the SPAD array information rate. It is shown that in sufficiently large arrays, ISI is negligible and the bounds are tight. Under such conditions, the SPAD array channel is precisely modelled as a memoryless channel. A discrete-time Gaussian channel with input-dependent mean and variance is adopted and the properties of the capacity-achieving input distributions are studied. Using a numerical algorithm, the information rate and the capacity-achieving input distributions, subject to peak and average power constraints are obtained. Furthermore, the bit error performance of a SPAD-based system with on-off keying (OOK) is evaluated for various array sizes, dead times and background count levels.

Achieving Channel Capacity of Visible Light Communication

In this article, we propose an efficient inexact gradient descent method to obtain the channel capacity-achieving input distribution of visible light communication (VLC). Under both a peak and an average optical power constraints, finding the channel capacity of VLC is formulated as a mixed continuous-discrete optimization problem without an analytical expression of the objective function. To solve this challenging problem, we first adopt the numerical integration method to approximate the objective function and its gradient. Here, we prove that the gaps between the original functions and the approximations can be arbitrarily small. Then, based on the approximated functions, we describe the method, and theoretically show that the obtained solution sequence converges to the channel capacity-achieving discrete distribution of VLC. We also provide simulation results to verify the effectiveness and optimality of the proposed method. More importantly, the simulations numerically reveal that on-off keying (OOK) modulation achieves the capacity of VLC channel at low signal-to-noise ratio (SNR), while pulse amplitude modulation (PAM) achieves the capacity at high SNR.

The Capacity Achieving Distribution for the Amplitude Constrained Additive Gaussian Channel: An Upper Bound on the Number of Mass Points

This paper studies an $n$ -dimensional additive Gaussian noise channel with a peak-power-constrained input. It is well known that, in this case, when $n=1$ the capacity-achieving input distribution is discrete with finitely many mass points, and when $n>1$ the capacity-achieving input distribution is supported on finitely many concentric shells. However, due to the previous proof technique, not even a bound on the exact number of mass points/shells was available. This paper provides an alternative proof of the finiteness of the number mass points/shells of the capacity-achieving input distribution while producing the first firm bounds on the number of mass points and shells, paving an alternative way for approaching many such problems. The first main result of this paper is an order tight implicit bound which shows that the number of mass points in the capacity-achieving input distribution is within a factor of two from the number of zeros of the downward shifted capacity-achieving output probability density function. Next, this implicit bound is utilized to provide a first firm upper on the support size of optimal input distribution, an $O(\mathsf {A}^{2})$ upper bound where $\mathsf {A}$ denotes the constraint on the input amplitude. The second main result of this paper generalizes the first one to the case when $n>1$ , showing that, for each and every dimension $n\ge 1$ , the number of shells that the optimal input distribution contains is $O(\mathsf {A}^{2})$ . Finally, the third main result of this paper reconsiders the case $n=1$ with an additional average power constraint, demonstrating a similar $O(\mathsf {A}^{2})$ bound.

Achievable Rate and Capacity Analysis for Ambient Backscatter Communications

In this paper, we analyze the achievable rate for ambient backscatter communications under three different channels: the binary input and binary output (BIBO) channel, the binary input and signal output (BISO) channel, and the binary input and energy output (BIEO) channel. Instead of assuming Gaussian input distribution, the proposed study matches the practical ambient backscatter scenarios, where the input of the tag can only be binary. We derive the closed-form capacity expression as well as the capacity-achieving input distribution for the BIBO channel. To show the influence of the signal-to-noise ratio (SNR) on the capacity, a closed-form tight ceiling is also derived when SNR turns relatively large. For BISO and BIEO channel, we obtain the closed-form mutual information, while the semi-closed-form capacity value can be obtained via one dimensional searching. Simulations are provided to corroborate the theoretical studies. Interestingly, the simulations show that: (i) the detection threshold maximizing the capacity of BIBO channel is the same as the one from the maximum likelihood signal detection; (ii) the maximal of the mutual information of all channels is achieved almost by a uniform input distribution; and (iii) the mutual information of the BIEO channel is larger than that of the BIBO channel, but is smaller than that of the BISO channel.

Finite-Field Matrix Channels for Network Coding

In 2010, Silva et al. studied certain classes of finite-field matrix channels in order to model random linear network coding where exactly $t$ random errors are introduced. In this paper, we consider a generalization of these matrix channels where the number of errors is not required to be constant, indeed the number of errors may follow any distribution. We show that a capacity-achieving input distribution can always be taken to have a very restricted form (the distribution should be uniform given the rank of the input matrix). This result complements, and is inspired by a paper of Nobrega et al. , which establishes a similar result for a class of matrix channels that model network coding with link erasures. Our result shows that the capacity of our channels can be expressed as maximization over probability distributions on the set of possible ranks of input matrices: a set of linear rather than exponential size.

Amplitude Constrained MIMO Channels: Properties of Optimal Input Distributions and Bounds on the Capacity.

In this work, the capacity of multiple-input multiple-output channels that are subject to constraints on the support of the input is studied. The paper consists of two parts. The first part focuses on the general structure of capacity-achieving input distributions. Known results are surveyed and several new results are provided. With regard to the latter, it is shown that the support of a capacity-achieving input distribution is a small set in both a topological and a measure theoretical sense. Moreover, explicit conditions on the channel input space and the channel matrix are found such that the support of a capacity-achieving input distribution is concentrated on the boundary of the input space only. The second part of this paper surveys known bounds on the capacity and provides several novel upper and lower bounds for channels with arbitrary constraints on the support of the channel input symbols. As an immediate practical application, the special case of multiple-input multiple-output channels with amplitude constraints is considered. The bounds are shown to be within a constant gap to the capacity if the channel matrix is invertible and are tight in the high amplitude regime for arbitrary channel matrices. Moreover, in the regime of high amplitudes, it is shown that the capacity scales linearly with the minimum between the number of transmit and receive antennas, similar to the case of average power-constrained inputs.

Secrecy Rate Closed-Form Expressions for the SISO VLC Wiretap Channel With Discrete Input Signaling

We study in this letter the secrecy performance of a degraded single-input single-output visible-light communication wiretap channel. It has been shown in the literature that the secrecy capacity-achieving input distribution for this system is discrete with a finite support set. However, neither the secrecy capacity nor its achieving distribution has been derived in closed-form. In this letter, we derive in closed-form the achievable secrecy rate as a function of the discrete input distribution. We propose a class of discrete input distributions in an effort to enhance the achievable secrecy rate and approach the secrecy capacity. We also derive expressions for the achievable secrecy rate for different scenarios including the one in which the locations of the terminals are randomly located. We provide several examples that demonstrate the accuracy of the derived expressions and show the substantial secrecy rate improvements provided by the proposed scheme over existing ones.

How to Achieve the Capacity of Asymmetric Channels

We survey coding techniques that enable reliable transmission at rates that approach the capacity of an arbitrary discrete memoryless channel. In particular, we take the point of view of modern coding theory and discuss how recent advances in coding for symmetric channels help provide more efficient solutions for the asymmetric case. We consider, in more detail, three basic coding paradigms. The first one is Gallager's scheme that consists of concatenating a linear code with a non-linear mapping so that the input distribution can be appropriately shaped. We explicitly show that both polar codes and spatially coupled codes can be employed in this scenario. Furthermore, we derive a scaling law between the gap to capacity, the cardinality of the input and output alphabets, and the required size of the mapper. The second one is an integrated scheme in which the code is used both for source coding, in order to create codewords distributed according to the capacity-achieving input distribution, and for channel coding, in order to provide error protection. Such a technique has been recently introduced by Honda and Yamamoto in the context of polar codes, and we show how to apply it also to the design of sparse graph codes. The third paradigm is based on an idea of B\"ocherer and Mathar, and separates the two tasks of source coding and channel coding by a chaining construction that binds together several codewords. We present conditions for the source code and the channel code, and we describe how to combine any source code with any channel code that fulfill those conditions, in order to provide capacity-achieving schemes for asymmetric channels. In particular, we show that polar codes, spatially coupled codes, and homophonic codes are suitable as basic building blocks of the proposed coding strategy.

On the Capacity of the AWGN Channel With Additive Radar Interference

This paper investigates the capacity of a communications channel that, in addition to additive white Gaussian noise, also suffers from interference caused by a co-existing radar transmission. The radar interference (of short duty-cycle and of much wider bandwidth than the intended communication signal) is modeled as an additive term whose amplitude is known and constant, but whose phase is independent and identically uniformly distributed at each channel use. The capacity achieving input distribution, under the standard average power constraint, is shown to have independent modulo and phase. The phase is uniformly distributed in $[0,2\pi]$. The modulo is discrete with countably infinitly many mass points, but only finitely many in any bounded interval. From numerical evaluations, a proper-complex Gaussian input is seen to perform quite well for weak radar interference. We also show that for very large radar interference, capacity is equal to $1/2\log (1 + S)$ and a proper-complex Gaussian input achieves it. It is concluded that the presence of the radar interference results in a loss of half of the degrees of freedom compared to an AWGN channel without radar interference.

Low-SNR Asymptotic Capacity of MIMO Optical Intensity Channels with Peak and Average Constraints

The low-SNR asymptotic capacity of the multiple-input multiple-output (MIMO) optical intensity channel is studied under both average and peak intensity constraints. We focus on low SNR, which can be modeled as the scenario where both constraints proportionally vanish, or where the peak constraint is held constant while the average constraint vanishes. A capacity upper bound is derived and is shown to be tight at low SNR under both scenarios. The capacity achieving input distribution at low SNR is shown to be a maximally correlated vector-binary input distribution. Consequently, the low-SNR capacity of the channel is characterized. As a byproduct, it is shown that for a channel with peak intensity constraints only, or with peak intensity constraints and individual (per aperture) average intensity constraints, a simple scheme composed of coded ON–OFF keying, spatial repetition, and maximum-ratio combining is optimal at low SNR.

Capacity Bounds for Discrete-Time, Amplitude-Constrained, Additive White Gaussian Noise Channels

The capacity-achieving input distribution of the discrete-time, additive white Gaussian noise (AWGN) channel with an amplitude constraint is discrete and seems difficult to characterize explicitly. A dual capacity expression is used to derive analytic capacity upper bounds for scalar and vector AWGN channels. The scalar bound improves on McKellips’ bound and is within 0.1 bit of capacity for all signal-to-noise ratios (SNRs). The 2-D bound is within 0.15 bits of capacity provably up to 4.5 dB; numerical evidence suggests a similar gap for all SNRs. As the SNR tends to infinity, these bounds are accurate and match with a volume-based lower bound. For the 2-D complex case, an analytic lower bound is derived by using a concentric constellation and is shown to be within 1 bit of capacity.

The Log-Volume of Optimal Codes for Memoryless Channels, Asymptotically Within a Few Nats

Shannon’s analysis of the fundamental capacity limits for memoryless communication channels has been refined over time. In this paper, the maximum volume $M_ \mathrm {avg}^{*}(n,\epsilon )$ of length- $n$ codes subject to an average decoding error probability $\epsilon $ is shown to satisfy the following tight asymptotic lower and upper bounds as $n \to \infty $ : $\underline {A}_\epsilon + o(1) \le \log M_ \mathrm {avg}^{*}(n,\epsilon ) - [nC - \sqrt {nV_\epsilon } \,Q^{-1}(\epsilon ) + ({1}/{2}) \log n] \le \overline {A}_\epsilon + o(1)$ , where $C$ is the Shannon capacity, $V_\epsilon $ is the $\epsilon $ -channel dispersion, or second-order coding rate, $Q$ is the tail probability of the normal distribution, and the constants $\underline {A}_\epsilon $ and $\overline {A}_\epsilon $ are explicitly identified. This expression holds under mild regularity assumptions on the channel, including nonsingularity. The gap $\overline {A}_\epsilon - \underline {A}_\epsilon $ is one nat for weakly symmetric channels in the Cover–Thomas sense, and typically a few nats for other symmetric channels, for the binary symmetric channel, and for the $Z$ channel. The derivation is based on strong large-deviations analysis and refined central limit asymptotics. A random coding scheme that achieves the lower bound is presented. The codewords are drawn from a capacity-achieving input distribution modified by an $O(1/\sqrt {n})$ correction term.

Capacity of a Simple Intercellular Signal Transduction Channel

We model the ligand-receptor molecular communication channel with a discrete-time Markov model, and show how to obtain the capacity of this channel. We show that the capacity-achieving input distribution is iid; further, unusually for a channel with memory, we show that feedback does not increase the capacity of this channel.

On the capacity of diffusion-based molecular communications with SiNW FET-based receiver.

Molecular communication (MC) is a bio-inspired communication method based on the exchange of molecules for information transfer among nanoscale devices. Although MC has been extensively studied from various aspects, limitations imposed by the physical design of transceiving units have been largely neglected in the literature. Recently, we have proposed a nanobioelectronic MC receiver architecture based on the nanoscale field effect transistor-based biosensor (bioFET) technology, providing noninvasive and sensitive molecular detection at nanoscale while producing electrical signals at the output. In this paper, we derive analytical closed-form expressions for the capacity and capacity-achieving input distribution for a memoryless MC channel with a silicon nanowire (SiNW) FET-based MC receiver. The resulting expressions could be used to optimize the information flow in MC systems equipped with nanobioelectronic receivers.