Jensen Divergences Research Articles

A new “E-E” paradigm to construct multi-BPAs based belief Jensen divergence in the evidence theory

The present study solves two problems in the evidence theory. The first problem is the belief entropies have rarely been exploited to induce a belief Jensen divergence family. We provide a solution by proposing two general paradigms, i.e. the “I-P-E” (Information theory-Probability theory-Evidence theory) and “E-E” (Evidence theory-Evidence theory) paradigms. Particularly, the “E-E” paradigm defines the first belief Jensen divergence family that is induced by belief entropies. The second problem is the “E-E” paradigm has rarely been generalized to multiple evidence-based belief Jensen divergences. We settle it by proposing a new Belief Jensen-PGDI (Pignistic Gini Deng Impurity) divergence as a gap filler. To ensure the divergence's feasibility in mathematics, we make two auxiliary works: the Enhanced Pignistic transformation is refined, and a new belief entropy called PGDI is constructed. Several axiomatic properties are also proven for the new divergence, guaranteeing its theoretical rigorousness. We successfully apply this divergence in two evidence fusion challenges, empirically verifying its practicality.

Divergences Induced by the Cumulant and Partition Functions of Exponential Families and Their Deformations Induced by Comparative Convexity.

Exponential families are statistical models which are the workhorses in statistics, information theory, and machine learning, among others. An exponential family can either be normalized subtractively by its cumulant or free energy function, or equivalently normalized divisively by its partition function. Both the cumulant and partition functions are strictly convex and smooth functions inducing corresponding pairs of Bregman and Jensen divergences. It is well known that skewed Bhattacharyya distances between the probability densities of an exponential family amount to skewed Jensen divergences induced by the cumulant function between their corresponding natural parameters, and that in limit cases the sided Kullback-Leibler divergences amount to reverse-sided Bregman divergences. In this work, we first show that the α-divergences between non-normalized densities of an exponential family amount to scaled α-skewed Jensen divergences induced by the partition function. We then show how comparative convexity with respect to a pair of quasi-arithmetical means allows both convex functions and their arguments to be deformed, thereby defining dually flat spaces with corresponding divergences when ordinary convexity is preserved.

Statistical Divergences between Densities of Truncated Exponential Families with Nested Supports: Duo Bregman and Duo Jensen Divergences.

By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence. Inspired by this formula, we define the duo Fenchel–Young divergence and report a majorization condition on its pair of strictly convex generators, which guarantees that this divergence is always non-negative. The duo Fenchel–Young divergence is also equivalent to a duo Bregman divergence. We show how to use these duo divergences by calculating the Kullback–Leibler divergence between densities of truncated exponential families with nested supports, and report a formula for the Kullback–Leibler divergence between truncated normal distributions. Finally, we prove that the skewed Bhattacharyya distances between truncated exponential families amount to equivalent skewed duo Jensen divergences.

Jointly convex quantum Jensen divergences

We investigate the quantum Jensen divergences from the viewpoint of joint convexity. It turns out that the set of the functions which generate jointly convex quantum Jensen divergences on positive matrices coincides with the Matrix Entropy Class which has been introduced by Chen and Tropp quite recently [3].

Generalizing Skew Jensen Divergences and Bregman Divergences With Comparative Convexity

Comparative convexity is a generalization of ordinary convexity based on abstract means instead of arithmetic means. We introduce the generalized skew Jensen divergences and their corresponding Bregman divergences with respect to comparative convexity. To illustrate those novel families of divergences, we consider the convexity induced by quasi-arithmetic means, and report explicit formula for the corresponding Bregman divergences. In particular, we show that those new Bregman divergences are equivalent to conformal ordinary Bregman divergences on monotone embeddings, and further state related results.

Maps on Quantum States Preserving Bregman and Jensen Divergences

We describe the structure of the bijective transformations on the set of density operators which preserve the Bregman $f$-divergence for an arbitrary differentiable strictly convex function $f.$ Furthermore, we determine the preservers of the Jensen $f$-divergence in the case when the generating function $f$ belongs to a recently introduced function class called Matrix Entropy Class.

Maps on positive definite matrices preserving Bregman and Jensen divergences

In this paper we determine those bijective maps of the set of all positive definite n×n complex matrices which preserve a given Bregman divergence corresponding to a differentiable convex function that satisfies certain conditions. We cover the cases of the most important Bregman divergences and present the precise structure of the mentioned transformations. Similar results concerning Jensen divergences and their preservers are also given.

Relative Entropies and Jensen Divergences in the Classical Limit

Metrics and distances in probability spaces have shown to be useful tools for physical purposes. Here we use this idea, with emphasis on Jensen Divergences and relative entropies, to investigate features of the road towards the classical limit. A well-known semiclassical model is used and recourse is made to numerical techniques, via the well-known Bandt and Pompe methodology, to extract probability distributions from the pertinent time-series associated with dynamical data.

Jensen divergence based on Fisher’s information

The measure of Jensen–Fisher divergence between probability distributions is introduced and its theoretical grounds set up. This quantity, in contrast to the remaining Jensen divergences, grasps the fluctuations of the probability distributions because it is controlled by the (local) Fisher information, which is a gradient functional of the distribution. So it is appropriate and informative when studying the similarity of distributions, mainly for those having oscillatory character. The new Jensen–Fisher divergence shares with the Jensen–Shannon divergence the following properties: non-negativity, additivity when applied to an arbitrary number of probability densities, symmetry under exchange of these densities, vanishing under certain conditions and definiteness even when these densities present non-common zeros. Moreover, the Jensen–Fisher divergence is shown to be expressed in terms of the relative Fisher information as the Jensen–Shannon divergence does in terms of the Kullback–Leibler or relative Shannon entropy. Finally, the Jensen–Shannon and Jensen–Fisher divergences are compared for the following three large, non-trivial and qualitatively different families of probability distributions: the sinusoidal, generalized gamma-like and Rakhmanov–Hermite distributions, which are closely related to the quantum-mechanical probability densities of numerous physical systems.

Properties of classical and quantum Jensen-Shannon divergence

Jensen-Shannon divergence (JD) is a symmetrized and smoothed version of the most important divergence measure of information theory, Kullback divergence. As opposed to Kullback divergence it determines in a very direct way a metric; indeed, it is the square of a metric. We consider a family of divergence measures (JD_alpha for alpha>0), the Jensen divergences of order alpha, which generalize JD as JD_1=JD. Using a result of Schoenberg, we prove that JD_alpha is the square of a metric for alpha lies in the interval (0,2], and that the resulting metric space of probability distributions can be isometrically embedded in a real Hilbert space. Quantum Jensen-Shannon divergence (QJD) is a symmetrized and smoothed version of quantum relative entropy and can be extended to a family of quantum Jensen divergences of order alpha (QJD_alpha). We strengthen results by Lamberti et al. by proving that for qubits and pure states, QJD_alpha^1/2 is a metric space which can be isometrically embedded in a real Hilbert space when alpha lies in the interval (0,2]. In analogy with Burbea and Rao's generalization of JD, we also define general QJD by associating a Jensen-type quantity to any weighted family of states. Appropriate interpretations of quantities introduced are discussed and bounds are derived in terms of the total variation and trace distance.