Entropy Power Inequality Research Articles

Quantum Rényi entropy functionals for bosonic Gaussian systems

In this study, the quantum Rényi entropy power inequality of order p>1 and power κ is introduced as a quantum analog of the classical Rényi-p entropy power inequality. To derive this inequality, we first exploit the Wehrl-p entropy power inequality on bosonic Gaussian systems via the mixing operation of quantum convolution, which is a generalized beam-splitter operation. This observation directly provides a quantum Rényi-p entropy power inequality over a quasi-probability distribution for D-mode bosonic Gaussian regimes. The proposed inequality is expected to be useful for the nontrivial computing of quantum channel capacities, particularly universal upper bounds on bosonic Gaussian quantum channels, and a Gaussian entanglement witness in the case of Gaussian amplifier via squeezing operations.

Entropy-variance inequalities for discrete log-concave random variables via degree of freedom

We utilize a discrete version of the notion of degree of freedom to prove a sharp min-entropy-variance inequality for integer valued log-concave random variables. More specifically, we show that the geometric distribution minimizes the min-entropy within the class of log-concave probability sequences with fixed variance. As an application, we obtain a discrete Rényi entropy power inequality in the log-concave case, which improves a result of Bobkov, Marsiglietti, and Melbourne (2022).

Joint precoding and power allocation in a FSO channel with signal-dependent noise.

To enhance the throughput of visible light communication (VLC), this paper investigates precoding and power allocation jointly in a free space optical channel with signal-dependent noise (SDN-FSO). With the presence of SDN from a light source, a more realistic multi-user multiple-input single-output (MU-MISO) VLC channel model is introduced. First, based on the entropy power inequality, a capacity bound for the SDN-FSO channel is derived under a MU-MISO scenario. Second, due to the nonnegativity of SDN-FSO input, a zero forcing constraint is introduced to alleviate multi-user interference. Third, from the perspective of resource allocation, min-max and total power budget constraints are taken into consideration. A non-convex joint optimization problem is formulated and divided into two subproblems, i.e.,the precoding subproblem and power allocation subproblem. Moreover, an iterative algorithm based on a concave and convex program and a simplified high signal-to-noise ratio approximation method are iteratively implemented in the precoding subproblem. For the power allocation subproblem, successive convex approximation is employed to ensure its optimality. Simulation results indicate that the proposed joint optimization methods can converge after about two iterations and accomplish a throughput improvement of up to 1∼3n a t s/s/H z as compared with the benchmarks for four different spacing scenarios. When channel state information is interfered with noise, the proposed methods still demonstrate their superior performance with varying degrees of noise.

Covert transmission performance for visible light communications under optical intensity and covert constraints

This paper investigates the fundamental performance limit for an indoor covert visible light communication (VLC) system consisting of a transmitter, a receiver, and a warden. In the system, the input signal of VLC is constrained by nonnegativity, peak optical intensity constraint, average optical intensity constraint and covertness constraint. We derive the optimal distribution of the input signal by using the variational method and the entropy power inequality. The allowable maximum peak optical intensity is obtained by substituting the optimal input signal into the covertness constraint. Then, by substituting the maximum peak optical intensity into the lower bound of the maximum mutual information, we derive an upper bound of the information bits that can be covertly transmitted. The asymptotic analysis shows that the upper bound on the transmittable covert information bits tends to a constant when the number of channel uses tends to infinity. These conclusions are also confirmed by numerical results.

Quantum Rényi-2 entropy power inequalities for bosonic Gaussian operations

We derive quantum Rényi-2 entropy power inequalities for Gaussian operations of the beam-splitting and squeezing type. We first show that known quantum von Neumann entropy power inequalities generalize straightforwardly to quantum Rényi-2 entropy power inequalities for Gaussian states but fail to do so for non-Gaussian states. We then derive quantum Rényi-2 entropy power inequalities that provide lower bounds for the Gaussian operations for any state. The inequality for the squeezing operation is shown to have applications in the generation and detection of quantum entanglement.

Costa’s concavity inequality for dependent variables based on the multivariate Gaussian copula

AbstractAn extension of Shannon’s entropy power inequality when one of the summands is Gaussian was provided by Costa in 1985, known as Costa’s concavity inequality. We consider the additive Gaussian noise channel with a more realistic assumption, i.e. the input and noise components are not independent and their dependence structure follows the well-known multivariate Gaussian copula. Two generalizations for the first- and second-order derivatives of the differential entropy of the output signal for dependent multivariate random variables are derived. It is shown that some previous results in the literature are particular versions of our results. Using these derivatives, concavity of the entropy power, under certain mild conditions, is proved. Finally, special one-dimensional versions of our general results are described which indeed reveal an extension of the one-dimensional case of Costa’s concavity inequality to the dependent case. An illustrative example is also presented.

A Mutual Information Inequality and Some Applications

In this paper we derive an inequality relating linear combinations of mutual information between subsets of mutually independent random variables and an auxiliary random variable. One choice of a family of auxiliary variables leads to a new proof of a Stam-type inequality regarding the Fisher Information of sums of independent random variables. Another choice of a family of auxiliary random variables leads to new results as well as new proofs of results relating to strong data processing constants and maximal correlation between sums of independent random variables. Other results obtained include convexity of Kullback–Leibler divergence over a parameterized path along pairs of binomial and Poisson distributions, as well as a new duality-based argument relating the Stam-type inequality and entropy power inequality.

Unifying the Brascamp-Lieb Inequality and the Entropy Power Inequality

The entropy power inequality (EPI) and the Brascamp-Lieb inequality (BLI) are fundamental inequalities concerning the differential entropies of linear transformations of random vectors. The EPI provides lower bounds for the differential entropy of linear transformations of random vectors with independent components. The BLI, on the other hand, provides upper bounds on the differential entropy of a random vector in terms of the differential entropies of some of its linear transformations. In this paper, we define a family of entropy functionals, which we show are subadditive. We then establish that Gaussians are extremal for these functionals by adapting a proof technique from Geng and Nair (2014). As a consequence, we obtain a new entropy inequality that generalizes both the BLI and EPI. By considering a variety of independence relations among the components of the random vectors appearing in these functionals, we also obtain families of inequalities that lie between the EPI and the BLI.

The Concavity of Generalized Entropy Powers

In this note we introduce a new family of entropy powers which are related to generalized entropies, called Sharma-Mittal entropies, and we prove their concavity along diffusion processes generated by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L^{2}$ </tex-math></inline-formula> -Wasserstein gradient flows of corresponding entropy functionals. This result extends the result of Savaré and Toscani (2014) on the concavity of Rényi entropy powers and reveals a connection to Rényi entropy power inequalities by Bobkov and Marsiglietti (2017).

Lower Bounds on Multivariate Higher Order Derivatives of Differential Entropy.

This paper studies the properties of the derivatives of differential entropy in Costa’s entropy power inequality. For real-valued random variables, Cheng and Geng conjectured that for , , while McKean conjectured a stronger statement, whereby . Here, we study the higher dimensional analogues of these conjectures. In particular, we study the veracity of the following two statements: , where n denotes that is a random vector taking values in , and similarly, . In this paper, we prove some new multivariate cases: . Motivated by our results, we further propose a weaker version of McKean’s conjecture , which is implied by and implies . We prove some multivariate cases of this conjecture under the log-concave condition: and . A systematic procedure to prove is proposed based on symbolic computation and semidefinite programming, and all the new results mentioned above are explicitly and strictly proved using this procedure.

A Generalization of Costa’s Entropy Power Inequality

Aim of this short note is to study Shannon&#x2019;s entropy power along entropic interpolations, thus generalizing Costa&#x2019;s concavity theorem. We shall provide two proofs of independent interest: the former by <inline-formula> <tex-math notation="LaTeX">$\Gamma $ </tex-math></inline-formula>-calculus, hence applicable to more abstract frameworks; the latter with an explicit remainder term, reminiscent of Villani (IEEE Trans. Inf. Theory, 2006), allowing us to characterize the case of equality.

Generalizations of Talagrand Inequality for Sinkhorn Distance Using Entropy Power Inequality.

The distance that compares the difference between two probability distributions plays a fundamental role in statistics and machine learning. Optimal transport (OT) theory provides a theoretical framework to study such distances. Recent advances in OT theory include a generalization of classical OT with an extra entropic constraint or regularization, called entropic OT. Despite its convenience in computation, entropic OT still lacks sufficient theoretical support. In this paper, we show that the quadratic cost in entropic OT can be upper-bounded using entropy power inequality (EPI)-type bounds. First, we prove an HWI-type inequality by making use of the infinitesimal displacement convexity of the OT map. Second, we derive two Talagrand-type inequalities using the saturation of EPI that corresponds to a numerical term in our expressions. These two new inequalities are shown to generalize two previous results obtained by Bolley et al. and Bai et al. Using the new Talagrand-type inequalities, we also show that the geometry observed by Sinkhorn distance is smoothed in the sense of measure concentration. Finally, we corroborate our results with various simulation studies.

Beyond Equal-Power Sparse NOMA: Two User Classes and Closed-Form Bounds on the Achievable Region.

Non-orthogonal multiple access (NOMA) is a promising technology for future beyond-5G wireless networks, whose fundamental information-theoretic limits are yet to be fully explored. Considering regular sparse code-domain NOMA (with a fixed and finite number of orthogonal resources allocated to any designated user and vice versa), this paper extends previous results by the authors to a setting comprising two classes of users with different power constraints. Explicit rigorous closed-form analytical inner and outer bounds on the achievable rate (total class throughput) region in the large-system limit are derived and comparatively investigated in extreme-SNR regimes. The inner bound is based on the conditional vector entropy power inequality (EPI), while the outer bound relies on a recent strengthened version of the EPI. Valuable insights are provided into the potential performance gains of regular sparse NOMA in practically oriented settings, comprising, e.g., a combination of low-complexity devices and broadband users with higher transmit power capabilities, or combinations of cell-edge and cell-center users. The conditions for superior performance over dense code-domain NOMA (taking the form of randomly spread code-division multiple access), as well as a relatively small gap to the ultimate performance limits, are identified. The proposed bounds are also applicable for the analysis of interference networks, e.g., Wyner-type cellular models.

Quantum Wigner entropy

We define the Wigner entropy of a quantum state as the differential Shannon entropy of the Wigner function of the state. This quantity is properly defined only for states that possess a positive Wigner function, which we name Wigner-positive states, but we argue that it is a proper measure of quantum uncertainty in phase space. It is invariant under symplectic transformations (displacements, rotations, and squeezing) and we conjecture that it is lower bounded by $ln\ensuremath{\pi}+1$ within the convex set of Wigner-positive states. It reaches this lower bound for Gaussian pure states, which are natural minimum-uncertainty states. This conjecture bears a resemblance with the Wehrl-Lieb conjecture, and we prove it over the subset of passive states of the harmonic oscillator which are of particular relevance in quantum thermodynamics. Along the way, we present a simple technique to build a broad class of Wigner-positive states exploiting an optical beam splitter and reveal an unexpectedly simple convex decomposition of extremal passive states. The Wigner entropy is anticipated to be a significant physical quantity, for example, in quantum optics where it allows us to establish a Wigner entropy-power inequality. It also opens a way towards stronger entropic uncertainty relations. Finally, we define the Wigner-R\'enyi entropy of Wigner-positive states and conjecture an extended lower bound that is reached for Gaussian pure states.

Reverse Euclidean and Gaussian Isoperimetric Inequalities for Parallel Sets With Applications

The $r$ -parallel set of a set $A \subseteq {\mathbb {R}} ^{d}$ is the set of all points whose distance from $A$ is less than $r$ . In this paper, we show that the surface area of an $r$ -parallel set in ${\mathbb {R}}^{d}$ with volume at most $V$ is upper-bounded by $e^{\Theta (d)}V/r$ , whereas its Gaussian surface area is upper-bounded by $\max (e^{\Theta (d)}, e^{\Theta (d)}/r)$ . We also derive a reverse form of the Brunn-Minkowski inequality for $r$ -parallel sets, and as an aside a reverse entropy power inequality for Gaussian-smoothed random variables. We apply our results to two problems in theoretical machine learning: (1) bounding the computational complexity of learning $r$ -parallel sets under a Gaussian distribution; and (2) bounding the sample complexity of estimating robust risk, which is a notion of risk in the adversarial machine learning literature that is analogous to the Bayes risk in hypothesis testing.

Secret Key Generation From Vector Gaussian Sources With Public and Private Communications

In this paper, we consider the problem of secret key generation with one-way communication through both a rate-limited public channel and a rate-limited secure channels where the public channel is from Alice to Bob and Eve and the secure channel is from Alice to Bob. In this model, we do not pose any constraints on the sources, i.e. Bob is not degraded to or less noisy than Eve. We obtain the optimal secret key rate in this problem, both for the discrete memoryless sources and vector Gaussian sources. The vector Gaussian characterization is derived by suitably applying the enhancement argument, and proving a new extremal inequality. The extremal inequality can be seen as coupling of two extremal inequalities, which are related to the degraded compound MIMO Gaussian broadcast channel, and the vector generalization of Costa's entropy power inequality, accordingly.

Concentration functions and entropy bounds for discrete log-concave distributions

AbstractTwo-sided bounds are explored for concentration functions and Rényi entropies in the class of discrete log-concave probability distributions. They are used to derive certain variants of the entropy power inequalities.

Sharp Moment-Entropy Inequalities and Capacity Bounds for Symmetric Log-Concave Distributions

We show that the uniform distribution minimizes entropy among all one-dimensional symmetric log-concave distributions with fixed variance, as well as various generalizations of this fact to R\'enyi entropies of orders less than 1 and with moment constraints involving $p$-th absolute moments with $p\leq 2$. As consequences, we give new capacity bounds for additive noise channels with symmetric log-concave noises, as well as for timing channels involving positive signal and noise where the noise has a decreasing log-concave density. In particular, we show that the capacity of an additive noise channel with symmetric, log-concave noise under an average power constraint is at most 0.254 bits per channel use greater than the capacity of an additive Gaussian noise channel with the same noise power. Consequences for reverse entropy power inequalities and connections to the slicing problem in convex geometry are also discussed.

An extension of entropy power inequality for dependent random variables

The entropy power inequality (EPI) for convolution of two independent random variables was first proposed by Shannon, C. E., 1948. However, in practice, there are many situations in which the involved random variables are not independent. In this article, considering additive noise channels, it is shown that, under some conditions, EPI holds for the case when the involved random variables are dependent. In order to achieve our main result, meanwhile a lower bound for the Fisher information of the output signal is obtained which is useful on its own. An example is also provided to illustrate our result.

Quantum addition imparts less disorder than mixing and commutes with incoherent channels

We prove a generalized version of a previously conjectured inequality by Zhang et al (Zhang et al 2018 Phys. Lett. A 382, 1516–23) for the quantum addition operation defined by Datta et al (Audenaert et al 2016 J. Math. Phy. 57, 052202) in the context of proving an entropy power inequality for qubits. We also show that the quantum addition operation commutes with an incoherent channel, which may have possible implications for resource theories of coherence in optical settings.