Abstract

A new kind of entropy will be introduced which generalizes both the differential entropy and the cumulative (residual) entropy. The generalization is twofold. First, we simultaneously define the entropy for cumulative distribution functions (cdfs) and survivor functions (sfs), instead of defining it separately for densities, cdfs, or sfs. Secondly, we consider a general “entropy generating function” φ, the same way Burbea et al. (IEEE Trans. Inf. Theory 1982, 28, 489–495) and Liese et al. (Convex Statistical Distances; Teubner-Verlag, 1987) did in the context of φ-divergences. Combining the ideas of φ-entropy and cumulative entropy leads to the new “cumulative paired φ-entropy” ( C P E φ ). This new entropy has already been discussed in at least four scientific disciplines, be it with certain modifications or simplifications. In the fuzzy set theory, for example, cumulative paired φ-entropies were defined for membership functions, whereas in uncertainty and reliability theories some variations of C P E φ were recently considered as measures of information. With a single exception, the discussions in the scientific disciplines appear to be held independently of each other. We consider C P E φ for continuous cdfs and show that C P E φ is rather a measure of dispersion than a measure of information. In the first place, this will be demonstrated by deriving an upper bound which is determined by the standard deviation and by solving the maximum entropy problem under the restriction of a fixed variance. Next, this paper specifically shows that C P E φ satisfies the axioms of a dispersion measure. The corresponding dispersion functional can easily be estimated by an L-estimator, containing all its known asymptotic properties. C P E φ is the basis for several related concepts like mutual φ-information, φ-correlation, and φ-regression, which generalize Gini correlation and Gini regression. In addition, linear rank tests for scale that are based on the new entropy have been developed. We show that almost all known linear rank tests are special cases, and we introduce certain new tests. Moreover, formulas for different distributions and entropy calculations are presented for C P E φ if the cdf is available in a closed form.

Highlights

  • The φ-entropy Eφ ( F ) = Z φ( f ( x ))dx, (1)R where f is a probability density function and φ is a strictly concave function, was introduced by [1].If we set φ(u) = −u ln u, u ∈ [0, 1], we get Shannon’s differential entropy as the most prominent special case.Shannon et al [2] derived the “entropy power fraction” and showed that there is a close relationship between Shannon entropy and variance

  • The main difference to the previous discussion of entropies is the fact that the new entropy is defined for distribution functions instead of density functions

  • This paper shows that this definition has a long tradition in several scientific disciplines like fuzzy set theory, reliability theory, and more recently in uncertainty theory

Read more

Summary

Introduction

R where f is a probability density function and φ is a strictly concave function, was introduced by [1]. A degenerate distribution minimizes the Shannon entropy as well as the variance of a discrete quantitative random variable. We will show that Equation (2) satisfies a popular ordering of scale and attains its maximum if the domain is an interval [ a, b], while a, b occur with a probability of 1/2 This means that Equation (2) behaves like a proper measure of dispersion. To use Equation (2) in order to obtain new related concepts measuring the dependence of random variables (such as mutual φ-information, φ-correlation, and φ-regression). The paper is structured in the same order as these aims After this introduction, in the second section we give a short review of the literature that is concerned with Equation (2) or related measures. In the last section we compute Equation (2) for certain generating functions φ and some selected families of distributions

State of the Art—An Overview
Fuzzy Set Theory
Generalized Maximum Entropy Principle
Theory of Dispersion
Uncertainty Theory
Reliability Theory
Definition
Advantages of Entropies Based on Cdfs
CPEφ for Grouped Data
Alternative Representations of CPEφ
Deriving an Upper Bound for CPEφ
Stricter Conditions for the Existence of CPEα
Maximum CPEφ Distributions for Given Mean and Variance
Maximum CPEφ Distributions for General Moment Restrictions
Generalized Principle of Maximum Entropy
Basic Properties of CPEφ
CPEφ and Oja’s Axioms for Measures of Scale
CPEφ and Transformations
CPEφ for Sums of Independent Random Variables
Estimation of CPEφ
Related Concepts
Cumulative Paired φ-Divergence
Mutual Cumulative φ-Information
Two-Sample Rank Test on Dispersion
Some Cumulative Paired Entropies for Selected Distribution Functions
9.10. Student-t Distribution
Findings
10. Conclusions
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call