Second-order Coding Rate Research Articles

Minimax Converse for Identification via Channels

A minimax converse for the identification via channels is derived. By this converse, a general formula for the identification capacity, which coincides with the transmission capacity, is proved without the assumption of the strong converse property. Furthermore, the optimal second-order coding rate of the identification via channels is characterized when the type I error probability is non-vanishing and the type II error probability is vanishing. Our converse is built upon the so-called partial channel resolvability approach; however, the minimax argument enables us to circumvent a flaw reported in the literature.

Second- and Third-Order Asymptotics of the Continuous-Time Poisson Channel

The paper derives the optimal second-order coding rate for the continuous-time Poisson channel. We also obtain bounds on the third-order coding rate. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output distribution induced by Wyner's discretized channel and the construction of an appropriate $\epsilon$-net of the input probability simplex. While the achievability proof follows the general program to prove the third-order term for non-singular discrete memoryless channels put forth by Polyanskiy, several non-standard techniques -- such as new definitions and bounds on the probabilities of typical sets using logarithmic Sobolev inequalities -- are employed to handle the continuous nature of the channel.

A New Method for Employing Feedback to Improve Coding Performance

We introduce a novel mechanism, called timid/bold coding, by which feedback can be used to improve coding performance. For a certain class of DMCs, called compound-dispersion channels , we show that timid/bold coding allows for an improved second-order coding rate compared with coding without feedback. For DMCs that are not compound dispersion, we show that feedback does not improve the second-order coding rate. Thus we completely determine the class of DMCs for which feedback improves the second-order coding rate. An upper bound on the second-order coding rate is provided for compound-dispersion DMCs. We also show that feedback does not improve the second-order coding rate for very noisy DMCs. The main results are obtained by relating feedback codes to certain controlled diffusions.

Wiretap Channels: Nonasymptotic Fundamental Limits

This paper investigates the maximal secret communication rate over a wiretap channel subject to reliability and secrecy constraints at a given blocklength. New achievability and converse bounds are derived, which are uniformly tighter than existing bounds, and lead to the tightest bounds on the second-order coding rate for discrete memoryless and Gaussian wiretap channels. The exact second-order coding rate is established for semi-deterministic wiretap channels, which characterizes the optimal tradeoff between reliability and secrecy in the finite-blocklength regime. Underlying our achievability bounds are two new privacy amplification results, which not only refine the existing results, but also achieve stronger notions of secrecy.

Union bound for quantum information processing.

In this paper, we prove a quantum union bound that is relevant when performing a sequence of binary-outcome quantum measurements on a quantum state. The quantum union bound proved here involves a tunable parameter that can be optimized, and this tunable parameter plays a similar role to a parameter involved in the Hayashi-Nagaoka inequality (Hayashi & Nagaoka 2003 IEEE Trans. Inf. Theory 49, 1753-1768. (doi:10.1109/TIT.2003.813556)), used often in quantum information theory when analysing the error probability of a square-root measurement. An advantage of the proof delivered here is that it is elementary, relying only on basic properties of projectors, Pythagoras' theorem, and the Cauchy-Schwarz inequality. As a non-trivial application of our quantum union bound, we prove that a sequential decoding strategy for classical communication over a quantum channel achieves a lower bound on the channel's second-order coding rate. This demonstrates the advantage of our quantum union bound in the non-asymptotic regime, in which a communication channel is called a finite number of times. We expect that the bound will find a range of applications in quantum communication theory, quantum algorithms and quantum complexity theory.

Universal Channel Coding for General Output Alphabet

We propose two types of universal codes that are suited to two asymptotic regimes when the output alphabet is possibly continuous. The first class has the property that the error probability decays exponentially fast and we identify an explicit lower bound on the error exponent. The other class attains the epsilon-capacity the channel and we also identify the second-order term in the asymptotic expansion. The proposed encoder is essentially based on the packing lemma of the method of types. For the decoder, we first derive a R\'enyi-relative-entropy version of Clarke and Barron's formula the distance between the true distribution and the Bayesian mixture, which is of independent interest. The universal decoder is stated in terms of this formula and quantities used in the information spectrum method. The methods contained herein allow us to analyze universal codes for channels with continuous and discrete output alphabets in a unified manner, and to analyze their performances in terms of the exponential decay of the error probability and the second-order coding rate.

Applications of position-based coding to classical communication over quantum channels

Recently, a coding technique called position-based coding has been used to establish achievability statements for various kinds of classical communication protocols that use quantum channels. In the present paper, we apply this technique in the entanglement-assisted setting in order to establish lower bounds for error exponents, lower bounds on the second-order coding rate, and one-shot lower bounds. We also demonstrate that position-based coding can be a powerful tool for analyzing other communication settings. In particular, we reduce the quantum simultaneous decoding conjecture for entanglement-assisted or unassisted communication over a quantum multiple access channel to open questions in multiple quantum hypothesis testing. We then determine achievable rate regions for entanglement-assisted or unassisted classical communication over a quantum multiple-access channel, when using a particular quantum simultaneous decoder. The achievable rate regions given in this latter case are generally suboptimal, involving differences of Rényi-two entropies and conditional quantum entropies.

A Tight Upper Bound on the Second-Order Coding Rate of the Parallel Gaussian Channel With Feedback

This paper investigates the asymptotic expansion for the maximum rate of fixed-length codes over a parallel Gaussian channel with feedback under the following setting: a peak power constraint is imposed on every transmitted codeword, and the average error probabilities of decoding the transmitted message are non-vanishing as the blocklength increases. The main contribution of this paper is a self-contained proof of an upper bound on the first- and second-order asymptotics of the parallel Gaussian channel with feedback. The proof techniques involve developing an information spectrum bound followed by using Curtiss’ theorem to show that a sum of dependent random variables associated with the information spectrum bound converges in distribution to a sum of independent random variables, thus facilitating the use of the usual central limit theorem. Combined with existing achievability results, our result implies that the presence of feedback does not improve the first- and second-order asymptotics.

Position-based coding and convex splitting for private communication over quantum channels

The classical-input quantum-output (cq) wiretap channel is a communication model involving a classical sender $X$, a legitimate quantum receiver $B$, and a quantum eavesdropper $E$. The goal of a private communication protocol that uses such a channel is for the sender $X$ to transmit a message in such a way that the legitimate receiver $B$ can decode it reliably, while the eavesdropper $E$ learns essentially nothing about which message was transmitted. The $\varepsilon $-one-shot private capacity of a cq wiretap channel is equal to the maximum number of bits that can be transmitted over the channel, such that the privacy error is no larger than $\varepsilon\in(0,1)$. The present paper provides a lower bound on the $\varepsilon$-one-shot private classical capacity, by exploiting the recently developed techniques of Anshu, Devabathini, Jain, and Warsi, called position-based coding and convex splitting. The lower bound is equal to a difference of the hypothesis testing mutual information between $X$ and $B$ and the "alternate" smooth max-information between $X$ and $E$. The one-shot lower bound then leads to a non-trivial lower bound on the second-order coding rate for private classical communication over a memoryless cq wiretap channel.

Discrete Lossy Gray–Wyner Revisited: Second-Order Asymptotics, Large and Moderate Deviations

In this paper, we revisit the discrete lossy Gray-Wyner problem. In particular, we derive its optimal second-order coding rate region, its error exponent (reliability function) and its moderate deviations constant under mild conditions on the source. To obtain the second-order asymptotics, we extend some ideas from Watanabe's work (2015). In particular, we leverage the properties of an appropriate generalization of the conditional distortion-tilted information density, which was first introduced by Kostina and Verd\'u (2012). The converse part uses a perturbation argument by Gu and Effros (2009) in their strong converse proof of the discrete Gray-Wyner problem. The achievability part uses two novel elements: (i) a generalization of various type covering lemmas; and (ii) the uniform continuity of the conditional rate-distortion function in both the source (joint) distribution and the distortion level. To obtain the error exponent, for the achievability part, we use the same generalized type covering lemma and for the converse, we use the strong converse together with a change-of-measure technique. Finally, to obtain the moderate deviations constant, we apply the moderate deviations theorem to probabilities defined in terms of information spectrum quantities.

Equivocations, Exponents, and Second-Order Coding Rates Under Various Rényi Information Measures

We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under various R\'{e}nyi information measures. Specifically, we consider the effect of applying a hash function on a source and we quantify the level of non-uniformity and dependence of the compressed source from another correlated source when the number of copies of the sources is large. Unlike previous works that use Shannon information measures to quantify randomness, information or uniformity, we define our security measures in terms of a more general class of information measures--the R\'{e}nyi information measures and their Gallager-type counterparts. A special case of these R\'{e}nyi information measure is the class of Shannon information measures. We prove tight asymptotic results for the security measures and their exponential rates of decay. We also prove bounds on the second-order asymptotics and show that these bounds match when the magnitudes of the second-order coding rates are large. We do so by establishing new classes non-asymptotic bounds on the equivocation and evaluating these bounds using various probabilistic limit theorems asymptotically.

Second-Order Region for Gray–Wyner Network

The coding problem over the Gray-Wyner network is studied from the second-order coding rates perspective. A tilted information density for this network is introduced in the spirit of Kostina-Verd\'u, and, under a certain regularity condition, the second-order region is characterized in terms of the variance of this tilted information density and the tangent vector of the first-order region. The second-order region is proved by the type method: the achievability part is proved by the type-covering argument, and the converse part is proved by a refinement of the perturbation approach that was used by Gu-Effros to show the strong converse of the Gray-Wyner network. This is the first instance that the second-order region is characterized for a multi-terminal problem where the characterization of the first-order region involves an auxiliary random variable.

Streaming Data Transmission in the Moderate Deviations and Central Limit Regimes

We consider streaming data transmission over a discrete memoryless channel. A new message is given to the encoder at the beginning of each block and the decoder decodes each message sequentially, after a delay of $T$ blocks. In this streaming setup, we study the fundamental interplay between the rate and error probability in the central limit and moderate deviations regimes and show that: 1) in the moderate deviations regime, the moderate deviations constant improves over the block coding or non-streaming setup by a factor of $T$ and 2) in the central limit regime, the second-order coding rate improves by a factor of approximately $\sqrt {T}$ for a wide range of channel parameters. For both the regimes, we propose coding techniques that incorporate a joint encoding of fresh and previous messages. In particular, for the central limit regime, we propose a coding technique with truncated memory to ensure that a summation of constants, which arises as a result of applications of the central limit theorem, does not diverge in the error analysis. Furthermore, we explore interesting variants of the basic streaming setup in the moderate deviations regime. We first consider a scenario with an erasure option at the decoder, i.e., the decoder can output an erasure symbol instead of a message estimate, and show that both the exponents of the total error and the undetected error probabilities improve by factors of $T$ . Next, by utilizing the erasure option, we show that the exponent of the total error probability can be improved to that of the undetected error probability (in the order sense) at the expense of a variable decoding delay.

The Second-Order Coding Rate of the MIMO Quasi-Static Rayleigh Fading Channel

The second-order coding rate of the multiple-input multiple-output (MIMO) quasi-static Rayleigh fading channel is studied. We tackle this problem via an information-spectrum approach and statistical bounds based on recent random matrix theory techniques. We derive a central limit theorem (CLT) to analyze the information density in the regime where the block length $n$ and the number of transmit and receive antennas $K$ and $N$ , respectively, grow simultaneously large. This result leads to the characterization of closed-form upper and lower bounds on the optimal average error probability when the coding rate is within $ {\mathcal{ O}}(1/\sqrt {nK})$ of the asymptotic capacity.

Second-Order Asymptotics for the Gaussian MAC With Degraded Message Sets

This paper studies the second-order asymptotics of the Gaussian multiple-access channel with degraded message sets. For a fixed average error probability $\varepsilon \in (0,1)$ and an arbitrary point on the boundary of the capacity region, we characterize the speed of convergence of rate pairs that converge to that boundary point for codes that have asymptotic error probability no larger than $\varepsilon$. As a stepping stone to this local notion of second-order asymptotics, we study a global notion, and establish relationships between the two. We provide a numerical example to illustrate how the angle of approach to a boundary point affects the second-order coding rate. This is the first conclusive characterization of the second-order asymptotics of a network information theory problem in which the capacity region is not a polygon.

On the Dispersions of the Gel’fand–Pinsker Channel and Dirty Paper Coding

This paper studies the second-order coding rates for memoryless channels with a state sequence known non-causally at the encoder. In the case of finite alphabets, an achievability result is obtained using constant-composition random coding, and by using a small fraction of the block to transmit the empirical distribution of the state sequence. For error probabilities less than 0.5, it is shown that the second-order rate improves on an existing one based on independent and identically distributed random coding. In the Gaussian case (dirty paper coding) with an almost-sure power constraint, an achievability result is obtained using random coding over the surface of a sphere, and using a small fraction of the block to transmit a quantized description of the state power. It is shown that the second-order asymptotics are identical to the single-user Gaussian channel of the same input power without a state.

A Case Where Interference Does Not Affect the Channel Dispersion

In 1975, Carleial presented a special case of an interference channel in which the interference does not reduce the capacity of the constituent point-to-point Gaussian channels. In this work, we show that if the inequalities in the conditions that Carleial stated are strict, the dispersions are similarly unaffected. More precisely, in this work, we characterize the second-order coding rates of the Gaussian interference channel in the strictly very strong interference regime. In other words, we characterize the speed of convergence of rates of optimal block codes towards a boundary point of the (rectangular) capacity region. These second-order rates are expressed in terms of the average probability of error and variances of some modified information densities which coincide with the dispersion of the (single-user) Gaussian channel. We thus conclude that the dispersions are unaffected by interference in this channel model.

Second-order coding rates for pure-loss bosonic channels

A pure-loss bosonic channel is a simple model for communication over free-space or fiber-optic links. More generally, phase-insensitive bosonic channels model other kinds of noise, such as thermalizing or amplifying processes. Recent work has established the classical capacity of all of these channels, and furthermore, it is now known that a strong converse theorem holds for the classical capacity of these channels under a particular photon number constraint. The goal of the present paper is to initiate the study of second-order coding rates for these channels, by beginning with the simplest one, the pure-loss bosonic channel. In a second-order analysis of communication, one fixes the tolerable error probability and seeks to understand the back-off from capacity for a sufficiently large yet finite number of channel uses. We find a lower bound on the maximum achievable code size for the pure-loss bosonic channel, in terms of the known expression for its capacity and a quantity called channel dispersion. We accomplish this by proving a general "one-shot" coding theorem for channels with classical inputs and pure-state quantum outputs which reside in a separable Hilbert space. The theorem leads to an optimal second-order characterization when the channel output is finite-dimensional, and it remains an open question to determine whether the characterization is optimal for the pure-loss bosonic channel.

Nonasymptotic and Second-Order Achievability Bounds for Coding With Side-Information

We present a novel nonasymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include: 1) the Wyner-Ahlswede-Korner (WAK) problem of almost-lossless source coding with rate-limited side-information; 2) the Wyner–Ziv (WZ) problem of lossy source coding with side-information at the decoder; and 3) the Gel’fand–Pinsker (GP) problem of channel coding with noncausal state information available at the encoder. The bounds are proved using ideas from channel simulation and channel resolvability. Our bounds for all three problems improve on all previous nonasymptotic bounds on the error probability of the WAK, WZ, and GP problems—in particular those derived by Verdu. Using our novel nonasymptotic bounds, we recover the general formulas for the optimal rates of these side-information problems. Finally, we also present achievable second-order coding rates by applying the multidimensional Berry–Esseen theorem to our new nonasymptotic bounds. Numerical results show that the second-order coding rates obtained using our nonasymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds.

Second-Order Rate Region of Constant-Composition Codes for the Multiple-Access Channel

This paper studies the second-order asymptotics of coding rates for the discrete memoryless multiple-access channel (MAC) with a fixed target error probability. Using constant-composition random coding, coded time-sharing, and a variant of Hoeffding's combinatorial central limit theorem, an inner bound on the set of locally achievable second-order coding rates is given for each point on the boundary of the capacity region. It is shown that the inner bound for constant-composition random coding includes that recovered by independent identically distributed random coding, and that the inclusion may be strict. The inner bound is extended to the Gaussian MAC via an increasingly fine quantization of the inputs.