Analytic Information Theory Research Articles

Relative Fisher information of discrete classical orthogonal polynomials

The analytic information theory of discrete distributions was initiated in 1998 by C. Knessl, P. Jacquet and S. Szpankowski who addressed the precise evaluation of the Renyi and Shannon entropies of the Poisson, Pascal (or negative binomial) and binomial distributions. They were able to derive various asymptotic approximations and, at times, lower and upper bounds for these quantities. Here, we extend these investigations in a twofold way. First, we consider a much larger class of distributions, the Rakhmanov distributions , where denote the sequences of discrete hypergeometric-type polynomials that are orthogonal with respect to the weight function ω(x) of Poisson, Pascal, binomial and hypergeometric types, i.e. the polynomials of Charlier, Meixner, Kravchuk and Hahn. Second, we obtain the explicit expressions for the relative Fisher information of these four families of Rakhmanov distributions with respect to their respective weight functions.

Information theory of D-dimensional hydrogenic systems: Application to circular and Rydberg states

The analytic information theory of quantum systems includes the exact determination of their spatial extension or multidimensional spreading in both position and momentum spaces by means of the familiar variance and its generalization, the power and logarithmic moments, and, more appropriately, the Shannon entropy and the Fisher information. These complementary uncertainty measures have a global or local character, respectively, because they are power-like (variance, moments), logarithmic (Shannon) and gradient (Fisher) functionals of the corresponding probability distribution. Here we explicitly discuss all these spreading measures (and their associated uncertainty relations) in both position and momentum for the main prototype in D-dimensional physics, the hydrogenic system, directly in terms of the dimensionality and the hyperquantum numbers which characterize the involved states. Then, we analyze in detail such measures for s-states, circular states (i.e., single-electron states of highest angular momenta allowed within an electronic manifold characterized by a given principal hyperquantum number), and Rydberg states (i.e., states with large radial hyperquantum numbers n). © 2009 Wiley Periodicals, Inc. Int J Quantum Chem, 2010

Fisher information of special functions and second-order differential equations

We investigate a basic question of analytic information theory, namely, the evaluation of the Fisher information and the relative Fisher information with respect to a non-negative function, for the probability distributions obtained by squaring the special functions of mathematical physics which are solutions of second-order differential equations. We obtain explicit expressions for these information-theoretic properties via the expectation values of the coefficients of the differential equation. We illustrate our approach for various nonrelativistic D-dimensional wavefunctions and some special functions of physicomathematical interest. Emphasis is made in the Nikiforov–Uvarov hypergeometric-type functions, which include and generalize the Hermite functions and the Gauss and Kummer hypergeometric functions, among others.

Average Redundancy for Known Sources: Ubiquitous Trees in Source Coding

Analytic information theory aims at studying problems of information theory using analytic techniques of computer science and combinatorics. Following Hadamard's precept, these problems are tackled by complex analysis methods such as generating functions, Mellin transform, Fourier series, saddle point method, analytic poissonization and depoissonization, and singularity analysis. This approach lies at the crossroad of computer science and information theory. In this survey we concentrate on one facet of information theory (i.e., source coding better known as data compression), namely the $\textit{redundancy rate}$ problem. The redundancy rate problem determines by how much the actual code length exceeds the optimal code length. We further restrict our interest to the $\textit{average}$ redundancy for $\textit{known}$ sources, that is, when statistics of information sources are known. We present precise analyses of three types of lossless data compression schemes, namely fixed-to-variable (FV) length codes, variable-to-fixed (VF) length codes, and variable-to-variable (VV) length codes. In particular, we investigate average redundancy of Huffman, Tunstall, and Khodak codes. These codes have succinct representations as $\textit{trees}$, either as coding or parsing trees, and we analyze here some of their parameters (e.g., the average path from the root to a leaf).

Analytic variations on redundancy rates of renewal processes

Csiszar and Shields (1996) proved that the minimax redundancy for a class of (stationary) renewal processes is /spl Theta/(/spl radic/n) where n is the block length. This interesting result provides a nontrivial bound on redundancy for a nonparametric family of processes. The present paper gives a precise estimate of the redundancy rate for such (nonstationary) renewal sources, namely, 2/(log2)/spl radic/((/spl pi//sup 2//6-1)n)+O(log n). This asymptotic expansion is derived by complex-analytic methods that include generating function representations, Mellin transforms, singularity analysis. and saddle-point estimates. This work places itself within the framework of analytic information theory.

Entropy computations via analytic depoissonization

We investigate the basic question of information theory, namely, evaluation of Shannon entropy, and a more general Renyi (1961) entropy, for some discrete distributions (e.g., binomial, negative binomial, etc.). We aim at establishing analytic methods (i.e., those in which complex analysis plays a pivotal role) for such computations which often yield estimates of unparalleled precision. The main analytic tool used here is that of analytic poissonization and depoissonization. We illustrate our approach on the entropy evaluation of the binomial distribution, that is, we prove that for binomial (n, p) distribution Shannon's h/sub n/ becomes h/sub n//spl ap/ 1/2 ln n+ 1/2 +ln/spl radic/(2/spl pi/p(1-p))+/spl Sigma//sub k/spl ges/1/a/sub k/n/sup -k/ where a/sub k/ are explicitly computable constants. Moreover, we argue that analytic methods (e.g., complex asymptotics such as Rice's method and singularity analysis, Mellin transforms, poissonization, and depoissonization) can offer new tools for information theory, especially for studying second-order asymptotics (e.g., redundancy). In fact, there has been a resurgence of interest and a few successful applications of analytic methods to a variety of problems of information theory, therefore, we propose to name such investigations as analytic information theory.