Abstract
A new kind of entropy will be introduced which generalizes both the differential entropy and the cumulative (residual) entropy. The generalization is twofold. First, we simultaneously define the entropy for cumulative distribution functions (cdfs) and survivor functions (sfs), instead of defining it separately for densities, cdfs, or sfs. Secondly, we consider a general “entropy generating function” φ, the same way Burbea et al. (IEEE Trans. Inf. Theory 1982, 28, 489–495) and Liese et al. (Convex Statistical Distances; Teubner-Verlag, 1987) did in the context of φ-divergences. Combining the ideas of φ-entropy and cumulative entropy leads to the new “cumulative paired φ-entropy” ( C P E φ ). This new entropy has already been discussed in at least four scientific disciplines, be it with certain modifications or simplifications. In the fuzzy set theory, for example, cumulative paired φ-entropies were defined for membership functions, whereas in uncertainty and reliability theories some variations of C P E φ were recently considered as measures of information. With a single exception, the discussions in the scientific disciplines appear to be held independently of each other. We consider C P E φ for continuous cdfs and show that C P E φ is rather a measure of dispersion than a measure of information. In the first place, this will be demonstrated by deriving an upper bound which is determined by the standard deviation and by solving the maximum entropy problem under the restriction of a fixed variance. Next, this paper specifically shows that C P E φ satisfies the axioms of a dispersion measure. The corresponding dispersion functional can easily be estimated by an L-estimator, containing all its known asymptotic properties. C P E φ is the basis for several related concepts like mutual φ-information, φ-correlation, and φ-regression, which generalize Gini correlation and Gini regression. In addition, linear rank tests for scale that are based on the new entropy have been developed. We show that almost all known linear rank tests are special cases, and we introduce certain new tests. Moreover, formulas for different distributions and entropy calculations are presented for C P E φ if the cdf is available in a closed form.
Highlights
The φ-entropy Eφ ( F ) = Z φ( f ( x ))dx, (1)R where f is a probability density function and φ is a strictly concave function, was introduced by [1].If we set φ(u) = −u ln u, u ∈ [0, 1], we get Shannon’s differential entropy as the most prominent special case.Shannon et al [2] derived the “entropy power fraction” and showed that there is a close relationship between Shannon entropy and variance
The main difference to the previous discussion of entropies is the fact that the new entropy is defined for distribution functions instead of density functions
This paper shows that this definition has a long tradition in several scientific disciplines like fuzzy set theory, reliability theory, and more recently in uncertainty theory
Summary
R where f is a probability density function and φ is a strictly concave function, was introduced by [1]. A degenerate distribution minimizes the Shannon entropy as well as the variance of a discrete quantitative random variable. We will show that Equation (2) satisfies a popular ordering of scale and attains its maximum if the domain is an interval [ a, b], while a, b occur with a probability of 1/2 This means that Equation (2) behaves like a proper measure of dispersion. To use Equation (2) in order to obtain new related concepts measuring the dependence of random variables (such as mutual φ-information, φ-correlation, and φ-regression). The paper is structured in the same order as these aims After this introduction, in the second section we give a short review of the literature that is concerned with Equation (2) or related measures. In the last section we compute Equation (2) for certain generating functions φ and some selected families of distributions
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
