Shannon's Inequality Research Articles

Stability of the logarithmic Sobolev inequality and uncertainty principle for the Tsallis entropy

We consider the stability of the functional inequalities concerning the entropy functional. For the Boltzmann–Shannon entropy, the logarithmic Sobolev inequality holds as a lower bound of the entropy by the Fisher information, and the Heisenberg uncertainty principle follows from combining it with the Shannon inequality. The optimizer for these inequalities is the Gauss function, which is a fundamental solution to the heat equation. In the fields of statistical mechanics and information theory, the Tsallis entropy is known as a one-parameter extension of the Boltzmann–Shannon entropy, and the Wasserstein gradient flow of it corresponds to the quasilinear diffusion equation. We consider the improvement and stability of the optimizer for the logarithmic Sobolev inequality related to the Tsallis entropy. Furthermore, we show the stability results of the uncertainty principle concerning the Tsallis entropy.

Shannon, Sobolev and uncertainty inequalities for the Weinstein transform

ABSTRACT The objective of this paper is to extend some weighted inequalities for the Weinstein transform. Especially, we are interesting in the study of the Beckner logarithmic inequality, the Shannon inequality. We give a sharp version of this inequality. Next, we establish different Sobolev type embedding theorems in our context. Finally, we deal with some uncertainty inequalities and a logarithmic uncertainty inequality for the Weistein transform.

Finite time blow up and concentration phenomena for a solution to drift-diffusion equations in higher dimensions

We show the finite time blow up of a solution to the Cauchy problem of a drift-diffusion equation of a parabolic-elliptic type in higher space dimensions. If the initial data satisfies a certain condition involving the entropy functional, then the corresponding solution to the equation does not exist globally in time and blows up in a finite time for the scaling critical space. Besides there exists a concentration point such that the solution exhibits the concentration in the critical norm. This type of blow up was observed in the scaling critical two dimensions. The proof is based on the profile decomposition and the Shannon inequality in the weighted space.

Shannon's inequality for the Rényi entropy and an application to the uncertainty principle

We consider Shannon's inequality for the Rényi entropy. The Rényi entropy corresponds to a nonlinear diffusion equation, for example, the porous-medium equation. Shannon's inequality implies the positivity of the relative Lyapunov functional for a nonlinear Fokker–Planck type equation. We derive Shannon's inequality for the Rényi entropy and give the other proof of this positivity. Moreover, by combining the Shannon and the logarithmic Sobolev inequalities for the Rényi entropy, we obtain the Heisenberg type uncertainty principle.

Quantifying quantum coherence based on the Tsallis relative operator entropy

Coherence is a fundamental ingredient in quantum physics and a key resource in quantum information processing. The quantification of quantum coherence is of great importance. We present a family of coherence quantifiers based on the Tsallis relative operator entropy. Shannon inequality and its reverse one in Hilbert space operators derived by Furuta [Linear Algebra Appl. 381 (2004) 219] are extended in terms of the parameter of the Tsallis relative operator entropy. These quantifiers are shown to satisfy all the standard criteria for a well-defined measure of coherence and include some existing coherence measures as special cases. Detailed examples are given to show the relations among the measures of quantum coherence.

Some New Karamata Type Inequalities and Their Applications to Some Entropies

Some new inequalities of Karamata type are established with a convex function. The methods of our proof allow us to obtain an extended version of the reverse of Jensen inequality given by PecnaricA and MicAicA. Applying the obtained results, we give reverses for information inequality (Shannon inequality) in different types, namely ratio type and difference type, under some conditions. Also, we provide interesting inequalities for von Neumann entropy and quantum Tsallis entropy which is a parametric extension of von Neumann entropy. The inequality for von Neumann entropy recovers the nonnegativity and gives a refinement for the weaker version of Fannes's inequality for only special cases. Finally, we estimate bounds for the Tsallis relative operator entropy.

Logarithmic Sobolev and Shannon's inequalities and an application to the uncertainty principle

The uncertainty principle of Heisenberg type can be generalized via the Boltzmann entropy functional. After reviewing the $L^p$ generalization of the logarithmic Sobolev inequality by Del Pino-Dolbeault [6], we introduce a generalized version of Shannon's inequality for the Boltzmann entropy functional which may regarded as a counter part of the logarithmic Sobolev inequality. Obtaining best possible constants of both inequalities, we connect both the inequalities to show a generalization of uncertainty principle of the Heisenberg type.

Non-uniform bound and finite time blow up for solutions to a drift–diffusion equation in higher dimensions

We show the non-uniform bound for a solution to the Cauchy problem of a drift–diffusion equation of a parabolic–elliptic type in higher space dimensions. If an initial data satisfies a certain condition involving the entropy functional, then the corresponding solution to the equation does not remain uniformly bounded in a scaling critical space. In other words, the solution grows up at [Formula: see text] in the critical space or blows up in a finite time. Our presenting results correspond to the finite time blowing up result for the two-dimensional case. The proof relies on the logarithmic entropy functional and a generalized version of the Shannon inequality. We also give the sharp constant of the Shannon inequality.

Operator versions of Shannon type inequality

In this paper, we present some refinements and precise estimations of parametric ex- tensions of Shannon inequality and its reverse one given by Furuta in Hilbert space operators. We also demonstrate an extension of operator Shannon type inequality. Mathematics subject classification (2010): Primary 47A63; Secondary 46L05, 46L60. Keywords and phrases: Operator inequality, Operator Shannon type inequality, relative operator en- tropy, generalized relative operator entropy.

Generalizations of operator Shannon inequality based on Tsallis and Rényi relative entropies

Recently, we obtained relations among relative operator entropies of sequences. For example, for relative operator entropy S(A|B), Rényi relative operator entropy It(A|B) and Tsallis relative operator entropy Tt(A|B) of sequences of strictly positive operators A=(A1,…,An) and B=(B1,…,Bn) such that ∑i=1nAi=∑i=1nBi=I,S(A|B)⩽It(A|B)⩽Tt(A|B)⩽0 holds for 0<t<1. This is an extension of operator version of Shannon inequality (briefly, operator Shannon inequality) discussed by Furuta and Yanagi–Kuriyama–Furuichi. In this paper, we shall obtain two generalizations of this inequality by considering generalizations of relative operator entropies of sequences.

Entropy and Følner function in algebras

We introduce the notion of entropic Følner function for algebras and we study its relation with the isoperimetric profile and the lower transcendence degree. Under the assumption of a technical conjecture we use the Shannon inequality to derive a theorem on the lower transcendence degree of domains and division algebras. We finally discuss its relation with some old conjectures of M. Artin, L. Small and J. Zhang.

Bounds for the normalised Jensen functional

New inequalities for the general case of convex functions defined on linear spaces which improve the famous Jensen's inequality are established. Particular instances in the case of normed spaces and for complex and real n-tuples are given. Refinements of Shannon's inequality and the positivity of Kullback-Leibler divergence are obtained.

A simple proof of the entropy-power inequality

This correspondence gives a simple proof of Shannon's entropy-power inequality (EPI) using the relationship between mutual information and minimum mean-square error (MMSE) in Gaussian channels.

Generalized Shannon inequalities based on Tsallis relative operator entropy

Tsallis relative operator entropy is defined and then its properties are given. Shannon inequality and its reverse one in Hilbert space operators derived by Furuta [Linear Algebra Apll. 381 (2004) 219] are extended in terms of the parameter of the Tsallis relative operator entropy. Moreover the generalized Tsallis relative operator entropy is introduced and then several operator inequalities are derived.

Parametric extensions of Shannon inequality and its reverse one in Hilbert space operators

We shall state the following parametric extensions of Shannon inequality and its reverse one in Hilbert space operators. Let p∈[0,1] and also let { A 1, A 2,…, A n } and { B 1, B 2,…, B n } be two sequences of strictly positive operators on a Hilbert space H such that ∑ j=1 n A j ♯ p B j ⩽ I . Then ∑ j=1 nS p+1(A j|B j)⩾ ∑ j=1 n(A j♮ p+1B j)+ I−∑ j=1 nA j♯ pB j × log ∑ j=1 n(A j♮ p+1B j)+ I−∑ j=1 nA j♯ pB j ⩾ log ∑ j=1 n(A j♮ p+1B j)+ I−∑ j=1 nA j♯ pB j ⩾∑ j=1 nS p(A j|B j) ⩾− log ∑ j=1 n(A j♮ p−1B j)+ I−∑ j=1 nA j♯ pB j ⩾− ∑ j=1 n(A j♮ p−1B j)+ I−∑ j=1 nA j♯ pB j × log ∑ j=1 n(A j♮ p−1B j)+ I−∑ j=1 nA j♯ pB j ⩾∑ j=1 nS p−1(A j|B j), where S q(A|B)=A 1 2 A −1 2 BA −1 2 q logA −1 2 BA −1 2 A 1 2 for A>0 , B>0 and any real number q and A♮ qB=A 1 2 A −1 2 BA −1 2 qA 1 2 for A>0 , B>0 and any real number q . In particular, if ∑ j=1 n A j =∑ j=1 n B j = I , then ∑ j=1 nS 2(A j|B j)⩾ ∑ j=1 nB jA j −1B j log ∑ j=1 nB jA j −1B j ⩾ log ∑ j=1 nB jA j −1B j ⩾∑ j=1 nS 1(A j|B j)⩾0 ⩾∑ j=1 nS(A j|B j)⩾− log ∑ j=1 nA jB j −1A j ⩾− ∑ j=1 nA jB j −1A j log ∑ j=1 nA jB j −1A j ⩾∑ j=1 nS −1(A j|B j), where S(A|B)=S 0(A|B)=A 1 2 logA −1 2 BA −1 2 A 1 2 which is the relative operator entropy of A>0 and B>0 . Our results can be considered as parametric extensions of the following celebrated Shannon inequality [Ann. Math. Statistics 22 (1951) 79; Bull. Syst. Tech. J. 27 (1948) 379; Pitman Monographs and Surveys in Pure and Applied Mathematics 97, Addison Wesley Longman, 1998, p. 233] which is very useful and so famous in information theory. Let { a 1, a 2,…, a n } and { b 1, b 2,…, b n } be two probability vectors. Then 0⩾∑ j=1 n a j log b j −∑ j=1 n a j log a j (see inequalities (2.4) of Corollary 2.4).

Comment on Note on generalization of Shannon theorem and inequality

Kwork Sau Fa (1998 J. Phys. A: Math. Gen. 31 8159) has shown an inequality 'S(AB) ≤ S(A) + S(B) + (1 − q)S(A)S(B)' for two interacting systems A and B. A typical example of S(A) is the Tsallis entropy as stated in his paper. However, there exist many counterexamples to the above inequality. The reason leading to the incorrect result is also presented.

Some refinements of Shannon's inequalities

AbstractWe refine Shannon's inequality, in its discrete and integral forms, by presenting upper estimates of the difference between its two sides.

On a Quantum Version of Shannon's Conditional Entropy

In this article we propose a quantum version of Shannon's conditional entropy. Given two density matrices $\rho$ and $\sigma$ on a finite dimensional Hilbert space and with $S(\rho)=-\tr\rho\ln\rho$ being the usual von Neumann entropy, this quantity $S(\rho|\sigma)$ is concave in $\rho$ and satisfies $0\le S(\rho|\sigma)\le S(\rho)$, a quantum analogue of Shannon's famous inequality. Thus we view $S(\rho|\sigma)$ as the entropy of $\rho$ conditioned by $\sigma$.

Note on generalization of Shannon theorem and inequality

Recently, Santos obtained a generalized entropy using four assumptions which stated that an entropy must: (i) be a continuous function of the probabilities ; (ii) be a monotonic increasing function of the number of states W, in the case of equiprobability; (iii) satisfy (where A and B are two independent systems) and (iv) satisfy the relation , where ( and ). Santos showed that the only function which satisfies all of these properties is the generalized Tsallis entropy. In this paper we perform a similar analysis and we obtain a family of entropies which are equivalent to the Tsallis entropy. We also discuss the Shannon inequality in the context of the generalized Tsallis entropy.

On a coding theorem connected with entropy of order α and type β

We prove two generalizations of Shannon's inequality for the case of entropy of order α and type β in a simple way. Noiseless coding theorems are proved using the two generalizations.