Moderate Deviation Asymptotics Research Articles

Achievable Refined Asymptotics for Successive Refinement Using Gaussian Codebooks

We study the mismatched successive refinement problem where one uses Gaussian codebooks to compress an arbitrary memoryless source with successive minimum Euclidean distance encoding under the quadratic distortion measure. Specifically, we derive achievable refined asymptotics under both the joint excess-distortion probability (JEP) and the separate excess-distortion probabilities (SEP) criteria. For both second-order and moderate deviations asymptotics, we consider two types of codebooks: the spherical codebook where each codeword is drawn independently and uniformly from the surface of a sphere and the i.i.d. Gaussian codebook where each component of each codeword is drawn independently from a Gaussian distribution. We establish the achievable second-order rate-region under JEP and we show that under SEP any memoryless source satisfying mild moment conditions is strongly successively refinable. When specialized to a Gaussian memoryless source (GMS), our results provide an alternative achievability proof with specific code design. We show that under JEP and SEP, the same moderate deviations constant is achievable. For large deviations asymptotics, we only consider the i.i.d. Gaussian codebook since the i.i.d. Gaussian codebook has better performance than the spherical codebook in this regime for the one layer mismatched rate-distortion problem (Zhou, Tan, Motani, TIT, 2019). We derive achievable exponents of both JEP and SEP and specialize our results to a GMS, which appears to be a novel result of independent interest.

Moderate deviation asymptotics of the GI/G/n queue in the Halfin–Whitt regime

Moderate deviation asymptotics of the GI/G/n queue in the Halfin–Whitt regime

Inhomogeneous functionals and approximations of invariant distributions of ergodic diffusions: Central limit theorem and moderate deviation asymptotics

The paper studies asymptotics of inhomogeneous integral functionals of an ergodic diffusion process under the effect of discretization. Convergence to the corresponding functionals of the invariant distribution is shown for suitably chosen discretization steps, and the fluctuations are analyzed through central limit theorem and moderate deviation principle. The results will be particularly useful for understanding accuracy of an Euler discretization based numerical scheme for approximating functionals of invariant distribution of an ergodic diffusion. This is an infinite-time horizon problem, and the accuracy of numerical schemes in this context are comparatively much less studied than the ones used for generating approximate trajectories of diffusions over finite time intervals. The potential applications of these results also extend to other areas including mathematical physics, parameter inference of ergodic diffusions and analysis of multiscale dynamical systems with averaging.

Non-Asymptotic Converse Bounds and Refined Asymptotics for Two Source Coding Problems

In this paper, we revisit two multi-terminal lossy source coding problems: the lossy source coding problem with side information available at the encoder and one of the two decoders, which we term as the Kaspi problem (Kaspi, 1994), and the multiple description coding problem with one semi-deterministic distortion measure, which we refer to as the Fu-Yeung problem (Fu and Yeung, 2002). For the Kaspi problem, we first present the properties of optimal test channels. Subsequently, we generalize the notion of the distortion-tilted information density for the lossy source coding problem to the Kaspi problem and prove a non-asymptotic converse bound using the properties of optimal test channels and the well-defined distortion-tilted information density. Finally, for discrete memoryless sources, we derive refined asymptotics which includes the second-order, large, and moderate deviations asymptotics. In the converse proof of second-order asymptotics, we apply the Berry-Esseen theorem to the derived non-asymptotic converse bound. The achievability proof follows by first proving a type-covering lemma tailored to the Kaspi problem, then properly Taylor expanding the well-defined distortion-tilted information densities and finally applying the Berry-Esseen theorem. We then generalize the methods used in the Kaspi problem to the Fu-Yeung problem. As a result, we obtain the properties of optimal test channels for the minimum sum-rate function, a non-asymptotic converse bound and refined asymptotics for discrete memoryless sources. Since the successive refinement problem is a special case of the Fu-Yeung problem, as a by-product, we obtain a non-asymptotic converse bound for the successive refinement problem, which is a strict generalization of the non-asymptotic converse bound for successively refinable sources (Zhou, Tan, and Motani, 2017).

On Converse Bounds for Classical Communication Over Quantum Channels

We explore several new converse bounds for classical communication over quantum channels in both the one-shot and asymptotic regimes. First, we show that the Matthews-Wehner meta-converse bound for entanglement-assisted classical communication can be achieved by activated, no-signalling assisted codes, suitably generalizing a result for classical channels. Second, we derive a new efficiently computable meta-converse on the amount of classical information unassisted codes can transmit over a single use of a quantum channel. As applications, we provide a finite resource analysis of classical communication over quantum erasure channels, including the second-order and moderate deviation asymptotics. Third, we explore the asymptotic analogue of our new meta-converse, the $\Upsilon$-information of the channel. We show that its regularization is an upper bound on the classical capacity, which is generally tighter than the entanglement-assisted capacity and other known efficiently computable strong converse bounds. For covariant channels we show that the $\Upsilon$-information is a strong converse bound.

Achievable Moderate Deviations Asymptotics for Streaming Compression of Correlated Sources

Motivated by streaming multi-view video coding and wireless sensor networks, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate deviations regime in which the rate pairs of a sequence of codes converge, along a straight line, to various points on the boundary of the Slepian-Wolf region at a speed slower than the inverse square root of the blocklength $n$, while the error probability decays subexponentially fast in $n$. Our main result focuses on directions of approaches to corner points of the Slepian-Wolf region. It states that for each correlated source and all corner points, there exists a non-empty subset of directions of approaches such that the moderate deviations constant (the constant of proportionality for the subexponential decay of the error probability) is enhanced (over the non-streaming case) by at least a factor of $T$, the block delay of decoding source block pairs. We specialize our main result to the setting of streaming lossless source coding and generalize this result to the setting where we have different delay requirements for each of the two source blocks. The proof of our main result involves the use of various analytical tools and amalgamates several ideas from the recent information-theoretic streaming literature. We adapt the so-called truncated memory encoding idea from Draper and Khisti (2011) and Lee, Tan, and Khisti (2016) to ensure that the effect of error accumulation is nullified in the limit of large blocklengths. We also adapt the use of the so-called minimum weighted empirical suffix entropy decoder which was used by Draper, Chang, and Sahai (2014) to derive achievable error exponents for symbolwise streaming Slepian-Wolf coding.

Scaling Exponent and Moderate Deviations Asymptotics of Polar Codes for the AWGN Channel

This paper investigates polar codes for the additive white Gaussian noise (AWGN) channel. The scaling exponent $\mu$ of polar codes for a memoryless channel $q_{Y|X}$ with capacity $I(q_{Y|X})$ characterizes the closest gap between the capacity and non-asymptotic achievable rates in the following way: For a fixed $\varepsilon \in (0, 1)$, the gap between the capacity $I(q_{Y|X})$ and the maximum non-asymptotic rate $R_n^*$ achieved by a length-$n$ polar code with average error probability $\varepsilon$ scales as $n^{-1/\mu}$, i.e., $I(q_{Y|X})-R_n^* = \Theta(n^{-1/\mu})$. It is well known that the scaling exponent $\mu$ for any binary-input memoryless channel (BMC) with $I(q_{Y|X})\in(0,1)$ is bounded above by $4.714$, which was shown by an explicit construction of polar codes. Our main result shows that $4.714$ remains to be a valid upper bound on the scaling exponent for the AWGN channel. Our proof technique involves the following two ideas: (i) The capacity of the AWGN channel can be achieved within a gap of $O(n^{-1/\mu}\sqrt{\log n})$ by using an input alphabet consisting of $n$ constellations and restricting the input distribution to be uniform; (ii) The capacity of a multiple access channel (MAC) with an input alphabet consisting of $n$ constellations can be achieved within a gap of $O(n^{-1/\mu}\log n)$ by using a superposition of $\log n$ binary-input polar codes. In addition, we investigate the performance of polar codes in the moderate deviations regime where both the gap to capacity and the error probability vanish as $n$ grows. An explicit construction of polar codes is proposed to obey a certain tradeoff between the gap to capacity and the decay rate of the error probability for the AWGN channel.

Second-Order and Moderate Deviation Asymptotics for Successive Refinement

We derive the optimal second-order coding region and moderate deviations constant for successive refinement source coding with a joint excess-distortion probability constraint. We consider two scenarios: (i) a discrete memoryless source (DMS) and arbitrary distortion measures at the decoders and (ii) a Gaussian memoryless source (GMS) and quadratic distortion measures at the decoders. For a DMS with arbitrary distortion measures, we prove an achievable second-order coding region, using type covering lemmas by Kanlis and Narayan and by No, Ingber and Weissman. We prove the converse using the perturbation approach by Gu and Effros. When the DMS is successively refinable, the expressions for the second-order coding region and the moderate deviations constant are simplified and easily computable. For this case, we also obtain new insights on the second-order behavior compared to the scenario where separate excess-distortion proabilities are considered. For example, we describe a DMS, for which the optimal second-order region transitions from being characterizable by a bivariate Gaussian to a univariate Gaussian, as the distortion levels are varied. We then consider a GMS with quadratic distortion measures. To prove the direct part, we make use of the sphere covering theorem by Verger-Gaugry, together with appropriately-defined Gaussian type classes. To prove the converse, we generalize Kostina and Verd\'u's one-shot converse bound for point-to-point lossy source coding. We remark that this proof is applicable to general successively refinable sources. In the proofs of the moderate deviations results for both scenarios, we follow a strategy similar to that for the second-order asymptotics and use the moderate deviations principle.

Large deviation properties of constant rate data streams sharing a buffer with long-range dependent traffic in critical loading

We consider a constant rate traffic which shares a buffer with a random cross traffic. A first come first served or priority service discipline is applied at the buffer. After service at the first buffer the constant rate traffic moves to a play-out buffer. Both buffers provide service at constant rate and infinite waiting room. We investigate logarithmic large and moderate deviation asymptotics for the tail probabilities of the steady-state queue length distribution at the play-out buffer for long-range dependent cross traffic in critical loading. We characterize the asymptotic behavior of the cross traffic which leads to a large queue length at the play-out buffer and compare it to the one for renewal cross traffic.

Large deviation properties of constant rate data streams sharing a buffer with long-range dependent traffic in critical loading

We consider a constant rate traffic which shares a buffer with a random cross traffic. A first come first served or priority service discipline is applied at the buffer. After service at the first buffer the constant rate traffic moves to a play-out buffer. Both buffers provide service at constant rate and infinite waiting room. We investigate logarithmic large and moderate deviation asymptotics for the tail probabilities of the steady-state queue length distribution at the play-out buffer for long-range dependent cross traffic in critical loading. We characterize the asymptotic behavior of the cross traffic which leads to a large queue length at the play-out buffer and compare it to the one for renewal cross traffic.

LARGE AND MODERATE DEVIATIONS PRINCIPLES FOR KERNEL ESTIMATION OF A MULTIVARIATE DENSITY AND ITS PARTIAL DERIVATIVES

SummaryThis paper studies the large deviations behaviour of the kernel estimator of a probability density f, by considering the case when the kernel takes negative values. It establishes large and moderate deviations principles for the kernel estimators of the partial derivatives of f. The estimators of the derivatives exhibit a quadratic behaviour for both the large and the moderate deviations scales, whereas for the density estimator there is a classical gap between the large deviations and the moderate deviations asymptotics.

Limit theory for moderate deviations from a unit root

An asymptotic theory is given for autoregressive time series with a root of the form ρn=1+c/kn, which represents moderate deviations from unity when (kn)n∈N is a deterministic sequence increasing to infinity at a rate slower than n, so that kn=o(n) as n→∞. For c<0, the results provide a nkn rate of convergence and asymptotic normality for the first order serial correlation, partially bridging the n and n convergence rates for the stationary (kn=1) and conventional local to unity (kn=n) cases. For c>0, the serial correlation coefficient is shown to have a knρnn convergence rate and a Cauchy limit distribution without assuming Gaussian errors, so an invariance principle applies when ρn>1. This result links moderate deviation asymptotics to earlier results on the explosive autoregression proved under Gaussian errors for kn=1, where the convergence rate of the serial correlation coefficient is (1+c)n and no invariance principle applies.

Stochastic processes in random graphs

We study the asymptotics of large, moderate and normal deviations for the connected components of the sparse random graph by the method of stochastic processes. We obtain the logarithmic asymptotics of large deviations of the joint distribution of the number of connected components, of the sizes of the giant components and of the numbers of the excess edges of the giant components. For the supercritical case, we obtain the asymptotics of normal deviations and the logarithmic asymptotics of large and moderate deviations of the joint distribution of the number of components, of the size of the largest component and of the number of the excess edges of the largest component. For the critical case, we obtain the logarithmic asymptotics of moderate deviations of the joint distribution of the sizes of connected components and of the numbers of the excess edges. Some related asymptotics are also established. The proofs of the large and moderate deviation asymptotics employ methods of idempotent probability theory. As a byproduct of the results, we provide some additional insight into the nature of phase transitions in sparse random graphs.

On Some Degenerate Large Deviation Problems

This paper concerns the issue of obtaining the large deviation principle for solutions of stochastic equations with possibly degenerate coefficients. Specifically, we explore the potential of the methodology that consists in establishing exponential tightness and identifying the action functional via a maxingale problem. In the author's earlier work it has been demonstrated that certain convergence properties of the predictable characteristics of semimartingales ensure both that exponential tightness holds and that every large deviation accumulation point is a solution to a maxingale problem. The focus here is on the uniqueness for the maxingale problem. It is first shown that under certain continuity hypotheses existence and uniqueness of a solution to a maxingale problem of diffusion type are equivalent to Luzin weak existence and uniqueness, respectively, for the associated idempotent Ito equation. Consequently, if the idempotent equation has a unique Luzin weak solution, then the action functional is specified uniquely, so the large deviation principle follows. Two kinds of application are considered. Firstly, we obtain results on the logarithmic asymptotics of moderate deviations for stochastic equations with possibly degenerate diffusion coefficients which, as compared with earlier results, relax the growth conditions on the coefficients, permit certain non-Lipshitz-continuous coefficients, and allow the coefficients to depend on the entire past of the process and to be discontinuous functions of time. The other application concerns multiple-server queues with impatient customers.

Precise Laplace-Type Asymptotics for Moderate Deviations of the Distributions of Sums of Independent Banach-Valued Random Elements

Formulas are deduced allowing one to find precise asymptotics of moderate deviations for the distributions of sums of independent identically distributed Banach-valued random elements. This result is proved by the Laplace method in Banach spaces. This method is an extension of the classical asymptotic Laplace method to the case of integrals with respect to probability measures in infinite-dimensional Banach spaces. By means of the theorem established in the present paper we find asymptotic representations for the probabilities of moderate deviations of statistics of the form $\omega_n^p$, $p\ge 2$.

Moderate Deviations for Queues in Critical Loading

We establish logarithmic asymptotics of moderate deviations for queue-length and waiting-time processes in single server queues and open queueing networks in critical loading. Our results complement earlier diffusion approximation results.