Capacity-achieving Input Research Articles

Comments on “Channel Coding Rate in the Finite Blocklength Regime”: On the Quadratic Decaying Property of the Information Rate Function

The quadratic decaying property of the information rate function states that given a fixed conditional distribution $p_{\mathsf{Y}|\mathsf{X}}$, the mutual information between the (finite) discrete random variables $\mathsf{X}$ and $\mathsf{Y}$ decreases at least quadratically in the Euclidean distance as $p_\mathsf{X}$ moves away from the capacity-achieving input distributions. It is a property of the information rate function that is particularly useful in the study of higher order asymptotics and finite blocklength information theory, where it was already implicitly used by Strassen [1] and later, more explicitly, by Polyanskiy-Poor-Verd\'u [2]. However, the proofs outlined in both works contain gaps that are nontrivial to close. This comment provides an alternative, complete proof of this property.

Algorithmic Computability and Approximability of Capacity-Achieving Input Distributions

The capacity of a channel can usually be characterized as a maximization of certain entropic quantities. From a practical point of view it is of primary interest to not only compute the capacity value, but also to find the corresponding optimizer, i.e., the capacity-achieving input distribution. This paper addresses the general question of whether or not it is possible to find algorithms that can compute the optimal input distribution depending on the channel. For this purpose, the concept of Turing machines is used which provides the fundamental performance limits of digital computers and therewith fully specifies which tasks are algorithmically feasible in principle. It is shown for discrete memoryless channels that it is impossible to algorithmically compute the capacity-achieving input distribution, where the channel is given as an input to the algorithm (or Turing machine). Finally, it is further shown that it is even impossible to algorithmically approximate these input distributions.

Capacity-Achieving Input Distributions of Additive Vector Gaussian Noise Channels: Even-Moment Constraints and Unbounded or Compact Support.

We investigate the support of a capacity-achieving input to a vector-valued Gaussian noise channel. The input is subjected to a radial even-moment constraint and is either allowed to take any value in Rn or is restricted to a given compact subset of Rn. It is shown that the support of the capacity-achieving distribution is composed of a countable union of submanifolds, each with a dimension of n-1 or less. When the input is restricted to a compact subset of Rn, this union is finite. Finally, the support of the capacity-achieving distribution is shown to have Lebesgue measure 0 and to be nowhere dense in Rn.

On the Capacity-Achieving Input of Channels With Phase Quantization

Several information-theoretic studies on channels with output quantization have identified the capacity-achieving input distributions for different fading channels with 1-bit in-phase and quadrature (I/Q) output quantization. However, an exact characterization of the capacity-achieving input distribution for channels with multi-bit phase quantization has not been provided. In this paper, we consider four different channel models with multi-bit phase quantization at the output and identify the optimal input distribution for each channel model. We first consider a complex Gaussian channel with $b$-bit phase-quantized output and prove that the capacity-achieving distribution is a rotated $2^b$-phase shift keying (PSK). The analysis is then extended to multiple fading scenarios. We show that the optimality of rotated $2^b$-PSK continues to hold under noncoherent fast fading Rician channels with $b$-bit phase quantization when line-of-sight (LoS) is present. When channel state information (CSI) is available at the receiver, we identify $\frac{2\pi}{2^b}$-symmetry and constant amplitude as the necessary and sufficient conditions for the ergodic capacity-achieving input distribution; which a $2^b$-PSK satisfies. Finally, an optimum power control scheme is presented which achieves ergodic capacity when CSI is also available at the transmitter.

On the Capacity-Achieving Input of the Gaussian Channel With Polar Quantization

The polar receiver architecture is a receiver design that captures the envelope and phase information of the signal rather than its in-phase and quadrature components. Several studies have demonstrated the robustness of polar receivers to phase noise and other nonlinearities. Yet, the information-theoretic limits of polar receivers with finite-precision quantizers have not been investigated in the literature. The main contribution of this work is to identify the optimal signaling strategy for the additive white Gaussian noise (AWGN) channel with polar quantization at the output. More precisely, we show that the capacity-achieving modulation scheme has an amplitude phase shift keying (APSK) structure. Using this result, the capacity of the AWGN channel with polar quantization at the output is established by numerically optimizing the probability mass function of the amplitude. The capacity of the polar-quantized AWGN channel with $b_1$-bit phase quantizer and optimized single-bit magnitude quantizer is also presented. Our numerical findings suggest the existence of signal-to-noise ratio (SNR) thresholds, above which the number of amplitude levels of the optimal APSK scheme and their respective probabilities change abruptly. Moreover, the manner in which the capacity-achieving input evolves with increasing SNR depends on the number of phase quantization bits.

Capacities and Optimal Input Distributions for Particle-Intensity Channels

This work introduces the particle-intensity channel (PIC) as a new model for molecular communication systems that includes imperfections at both transmitter and receiver and provides a new characterization of the capacity limits as well as properties of the optimal (capacity-achieving) input distributions for such channels. In the PIC, the transmitter encodes information, in symbols of a given duration, based on the probability of particle release, and the receiver detects and decodes the message based on the number of particles detected during the symbol interval. In this channel, the transmitter may be unable to control precisely the probability of particle release, and the receiver may not detect all the particles that arrive. We model this channel using a generalization of the binomial channel and show that the capacity-achieving input distribution for this channel always has mass points at probabilities of particle release of zero and one. To find the capacity-achieving input distributions, we develop a novel and efficient algorithm we call dynamic assignment Blahut-Arimoto (DAB). For diffusive particle transport, we also derive the conditions under which the input with two mass points is capacity-achieving.

Achieving Channel Capacity of Visible Light Communication

In this article, we propose an efficient inexact gradient descent method to obtain the channel capacity-achieving input distribution of visible light communication (VLC). Under both a peak and an average optical power constraints, finding the channel capacity of VLC is formulated as a mixed continuous-discrete optimization problem without an analytical expression of the objective function. To solve this challenging problem, we first adopt the numerical integration method to approximate the objective function and its gradient. Here, we prove that the gaps between the original functions and the approximations can be arbitrarily small. Then, based on the approximated functions, we describe the method, and theoretically show that the obtained solution sequence converges to the channel capacity-achieving discrete distribution of VLC. We also provide simulation results to verify the effectiveness and optimality of the proposed method. More importantly, the simulations numerically reveal that on-off keying (OOK) modulation achieves the capacity of VLC channel at low signal-to-noise ratio (SNR), while pulse amplitude modulation (PAM) achieves the capacity at high SNR.

The Bit Error Performance and Information Transfer Rate of SPAD Array Optical Receivers

In this paper the photon counting characteristics, the information rate and the bit error performance of single-photon avalanche diode (SPAD) arrays are investigated. It is shown that for sufficiently large arrays, the photocount distribution is well approximated by a Gaussian distribution with dead-time-dependent mean and variance. Because of dead time, the SPAD array channel is subject to counting losses, part of which are due to inter-slot interference (ISI) distortions. Consequently, this channel has memory. The information rate of this channel is assessed. Two auxiliary discrete memoryless channels (DMCs) are proposed which provide upper and lower bounds on the SPAD array information rate. It is shown that in sufficiently large arrays, ISI is negligible and the bounds are tight. Under such conditions, the SPAD array channel is precisely modelled as a memoryless channel. A discrete-time Gaussian channel with input-dependent mean and variance is adopted and the properties of the capacity-achieving input distributions are studied. Using a numerical algorithm, the information rate and the capacity-achieving input distributions, subject to peak and average power constraints are obtained. Furthermore, the bit error performance of a SPAD-based system with on-off keying (OOK) is evaluated for various array sizes, dead times and background count levels.

On the Capacity-Achieving Scheme and Capacity of 1-Bit ADC Gaussian-Mixture Channels

This paper addresses the optimal signaling scheme and capacity of an additive Gaussian mixture (GM) noise channel using 1-bit analog-to-digital converters (ADCs). The consideration of GM noise provides a more realistic baseline for the analysis and design of co-channel interference links and networks. Towards that goal, we first show that the capacityachieving input signal is π/2 circularly symmetric. By examining a necessary and sufficient Kuhn–Tucker condition (KTC) for an input to be optimal, we demonstrate that the maximum number of optimal mass points is four. Our proof relies on Dubin’s theorem and the fact that the KTC coefficient is positive, i.e., the power constraint is active. By combining with the π/2 circularly symmetric property, it is then concluded the optimal input is unique, and it has exactly four mass points forming a square centered at the origin. By further checking the first and second derivatives of the modified KTC, it is then shown that the phase of the optimal mass point located in the first quadrant is π/4. Thus, the capacity-achieving input signal is QPSK. This result helps us obtain the channel capacity in closed-form.

The Capacity Achieving Distribution for the Amplitude Constrained Additive Gaussian Channel: An Upper Bound on the Number of Mass Points

This paper studies an $n$ -dimensional additive Gaussian noise channel with a peak-power-constrained input. It is well known that, in this case, when $n=1$ the capacity-achieving input distribution is discrete with finitely many mass points, and when $n>1$ the capacity-achieving input distribution is supported on finitely many concentric shells. However, due to the previous proof technique, not even a bound on the exact number of mass points/shells was available. This paper provides an alternative proof of the finiteness of the number mass points/shells of the capacity-achieving input distribution while producing the first firm bounds on the number of mass points and shells, paving an alternative way for approaching many such problems. The first main result of this paper is an order tight implicit bound which shows that the number of mass points in the capacity-achieving input distribution is within a factor of two from the number of zeros of the downward shifted capacity-achieving output probability density function. Next, this implicit bound is utilized to provide a first firm upper on the support size of optimal input distribution, an $O(\mathsf {A}^{2})$ upper bound where $\mathsf {A}$ denotes the constraint on the input amplitude. The second main result of this paper generalizes the first one to the case when $n>1$ , showing that, for each and every dimension $n\ge 1$ , the number of shells that the optimal input distribution contains is $O(\mathsf {A}^{2})$ . Finally, the third main result of this paper reconsiders the case $n=1$ with an additional average power constraint, demonstrating a similar $O(\mathsf {A}^{2})$ bound.

Achievable Rate and Capacity Analysis for Ambient Backscatter Communications

In this paper, we analyze the achievable rate for ambient backscatter communications under three different channels: the binary input and binary output (BIBO) channel, the binary input and signal output (BISO) channel, and the binary input and energy output (BIEO) channel. Instead of assuming Gaussian input distribution, the proposed study matches the practical ambient backscatter scenarios, where the input of the tag can only be binary. We derive the closed-form capacity expression as well as the capacity-achieving input distribution for the BIBO channel. To show the influence of the signal-to-noise ratio (SNR) on the capacity, a closed-form tight ceiling is also derived when SNR turns relatively large. For BISO and BIEO channel, we obtain the closed-form mutual information, while the semi-closed-form capacity value can be obtained via one dimensional searching. Simulations are provided to corroborate the theoretical studies. Interestingly, the simulations show that: (i) the detection threshold maximizing the capacity of BIBO channel is the same as the one from the maximum likelihood signal detection; (ii) the maximal of the mutual information of all channels is achieved almost by a uniform input distribution; and (iii) the mutual information of the BIEO channel is larger than that of the BIBO channel, but is smaller than that of the BISO channel.

Gaussian Mixture Noise Channels With Minimum and Peak Amplitude Constraints

Motivated by the idea of “transmitting energy and information simultaneously,” we investigate, in this paper, the impact of constraints on the amount of energy that individual symbols carry, i.e., minimum amplitude constraints. We consider a Gaussian mixture noise channel with both minimum and peak amplitude constraints. First, we prove that the capacity-achieving input has a discrete distribution with a finite number of probability mass points. Then, we further investigate the number and positions of the probability mass points for the capacity-achieving input. Specifically, when the interference is constant and known at both the transmitter and the receiver, it can be totally eliminated so that the channel operates like an AWGN channel. In this case, we give a theorem to determine whether the optimal input is binary. For more general cases, such as non-binary inputs and non-constant interference, we investigate optimal inputs and capacities via numerical computations.

Optimal Signaling Schemes and Capacity of Non-Coherent Rician Fading Channels With Low-Resolution Output Quantization

Low-resolution analog-to-digital converter (ADC) has been considered as a promising solution to save power and cost in communication systems using high bandwidth and/or multiple RF chains. The goal of this work is to address the design of optimal signaling schemes and establish the capacity limit of Rician fading channels with low-resolution output quantization. This fading channel can be used to accurately model a wide range of wireless channels with the line-of-sight (LOS) components, including emerging mm-wave communications. The focus is on non-coherent fast fading channels where neither the transmitter nor the receiver knows the channel state information (CSI). By examining the continuity of the input-output mutual information, the existence of the optimal input signal is first validated. Then, considering the case of 1-bit ADC, we show that the optimal input is $\pi /2$ circularly symmetric. A necessary and sufficient condition for an input signal to be optimal, which is referred to as the Kuhn-Tucker condition (KTC), and Lagrangian optimization problem are then established. By exploiting the novel log-quadratic bounds on the Gaussian $Q$ -function, it is then demonstrated that for a given mass point’s amplitude, the corresponding rotated mass points through the phase of LOS component must form a square grid centered at zero. Furthermore, the amplitude of the mass points in the optimal distribution can take on only one value. As a result, the capacity-achieving input with 1-bit ADC is a rotated quadrature phase-shift keying (QPSK) constellation, and the rotation angle depends on the Rician factor. The characterization of the optimal input has also been extended to the case of multi-bit ADCs. Specifically, it is shown that for a $K$ -bit ADC, the optimal input is discrete having atmost $2^{2K}$ mass points. In both the cases of 1-bit and $K$ -bit ADCs, the channel capacities are established in closed-form.

Finite-Field Matrix Channels for Network Coding

In 2010, Silva et al. studied certain classes of finite-field matrix channels in order to model random linear network coding where exactly $t$ random errors are introduced. In this paper, we consider a generalization of these matrix channels where the number of errors is not required to be constant, indeed the number of errors may follow any distribution. We show that a capacity-achieving input distribution can always be taken to have a very restricted form (the distribution should be uniform given the rank of the input matrix). This result complements, and is inspired by a paper of Nobrega et al. , which establishes a similar result for a class of matrix channels that model network coding with link erasures. Our result shows that the capacity of our channels can be expressed as maximization over probability distributions on the set of possible ranks of input matrices: a set of linear rather than exponential size.

Amplitude Constrained MIMO Channels: Properties of Optimal Input Distributions and Bounds on the Capacity.

In this work, the capacity of multiple-input multiple-output channels that are subject to constraints on the support of the input is studied. The paper consists of two parts. The first part focuses on the general structure of capacity-achieving input distributions. Known results are surveyed and several new results are provided. With regard to the latter, it is shown that the support of a capacity-achieving input distribution is a small set in both a topological and a measure theoretical sense. Moreover, explicit conditions on the channel input space and the channel matrix are found such that the support of a capacity-achieving input distribution is concentrated on the boundary of the input space only. The second part of this paper surveys known bounds on the capacity and provides several novel upper and lower bounds for channels with arbitrary constraints on the support of the channel input symbols. As an immediate practical application, the special case of multiple-input multiple-output channels with amplitude constraints is considered. The bounds are shown to be within a constant gap to the capacity if the channel matrix is invertible and are tight in the high amplitude regime for arbitrary channel matrices. Moreover, in the regime of high amplitudes, it is shown that the capacity scales linearly with the minimum between the number of transmit and receive antennas, similar to the case of average power-constrained inputs.

On the IID Capacity-Achieving Input for Binding Channels With Multiple Ligand Receptors

This paper studies the molecular communication system where the transmitter has limited productivity and the receiver employs ligand-binding receptors. By simplifying the release and propagation process of the information particles, the ligand-binding process is regarded as a binding channel with peak and average constraints and modeled by a finite-state Markov chain. It is proved that the capacity of the constrained independent and identically distributed (IID) binding channel, defined as the IID capacity, is achieved by a discrete input distribution. The sufficient and necessary conditions of an IID input distribution being capacity-achieving is presented. Moreover, an algorithm called modified steepest ascent cutting-plane algorithm is proposed to efficiently compute the IID capacity-achieving distributions. The numerical results show that the IID capacity is a tight lower bound of the capacity for the binding channel.

Secrecy Rate Closed-Form Expressions for the SISO VLC Wiretap Channel With Discrete Input Signaling

We study in this letter the secrecy performance of a degraded single-input single-output visible-light communication wiretap channel. It has been shown in the literature that the secrecy capacity-achieving input distribution for this system is discrete with a finite support set. However, neither the secrecy capacity nor its achieving distribution has been derived in closed-form. In this letter, we derive in closed-form the achievable secrecy rate as a function of the discrete input distribution. We propose a class of discrete input distributions in an effort to enhance the achievable secrecy rate and approach the secrecy capacity. We also derive expressions for the achievable secrecy rate for different scenarios including the one in which the locations of the terminals are randomly located. We provide several examples that demonstrate the accuracy of the derived expressions and show the substantial secrecy rate improvements provided by the proposed scheme over existing ones.

How to Achieve the Capacity of Asymmetric Channels

We survey coding techniques that enable reliable transmission at rates that approach the capacity of an arbitrary discrete memoryless channel. In particular, we take the point of view of modern coding theory and discuss how recent advances in coding for symmetric channels help provide more efficient solutions for the asymmetric case. We consider, in more detail, three basic coding paradigms. The first one is Gallager's scheme that consists of concatenating a linear code with a non-linear mapping so that the input distribution can be appropriately shaped. We explicitly show that both polar codes and spatially coupled codes can be employed in this scenario. Furthermore, we derive a scaling law between the gap to capacity, the cardinality of the input and output alphabets, and the required size of the mapper. The second one is an integrated scheme in which the code is used both for source coding, in order to create codewords distributed according to the capacity-achieving input distribution, and for channel coding, in order to provide error protection. Such a technique has been recently introduced by Honda and Yamamoto in the context of polar codes, and we show how to apply it also to the design of sparse graph codes. The third paradigm is based on an idea of B\"ocherer and Mathar, and separates the two tasks of source coding and channel coding by a chaining construction that binds together several codewords. We present conditions for the source code and the channel code, and we describe how to combine any source code with any channel code that fulfill those conditions, in order to provide capacity-achieving schemes for asymmetric channels. In particular, we show that polar codes, spatially coupled codes, and homophonic codes are suitable as basic building blocks of the proposed coding strategy.

Capacity-Achieving Signals for Point-to-Point and Multiple-Access Channels Under Non-Gaussian Noise and Peak Power Constraint

This paper generalizes and proves the discrete and finite nature of the capacity-achieving signaling schemes for general classes of non-Gaussian point-to-point and multiple-access channels (MACs) under peak power constraints. Specifically, we first investigate the detailed characteristics of capacity-achieving inputs for a single-user channel that is impaired by two types of noise: a Gaussian mixture (GM) noise $Z$ consisting of Gaussian elements with arbitrary means and the interference $U$ with an arbitrary distribution. The only very mild condition imposed on $U$ is that its second moment is finite. To this end, one of the important results is the establishment of the Kuhn–Tucker condition (KTC) on a capacity-achieving input and the proof of analyticity of the KTC using Fubini–Tonelli’s and Morera’s theorems. Using the Bolzano–Weierstrass’s and Identity’s theorems, we then show that a capacity-achieving input is continuous if and only if the KTC function is zero on the entire real line. However, by examining an upper bound on the tail of the output PDF, it is demonstrated that the KTC function must be bounded away from zero. As such, any capacity-achieving input must be discrete with a finite number of mass points. Finally, we exploit $U$ having an arbitrary distribution to show that the optimal input distributions that achieve the sum-capacity of an $M$ -user MAC under GM noise are discrete and finite. We also prove that there exist at least two distinct points that achieve the sum capacity on the rate region.

Capacity Bounds for Discrete-Time, Amplitude-Constrained, Additive White Gaussian Noise Channels

The capacity-achieving input distribution of the discrete-time, additive white Gaussian noise (AWGN) channel with an amplitude constraint is discrete and seems difficult to characterize explicitly. A dual capacity expression is used to derive analytic capacity upper bounds for scalar and vector AWGN channels. The scalar bound improves on McKellips’ bound and is within 0.1 bit of capacity for all signal-to-noise ratios (SNRs). The 2-D bound is within 0.15 bits of capacity provably up to 4.5 dB; numerical evidence suggests a similar gap for all SNRs. As the SNR tends to infinity, these bounds are accurate and match with a volume-based lower bound. For the 2-D complex case, an analytic lower bound is derived by using a concentric constellation and is shown to be within 1 bit of capacity.