Abstract
As a measure of randomness or uncertainty, the Boltzmann–Shannon entropy H has become one of the most widely used summary measures of a variety of attributes (characteristics) in different disciplines. This paper points out an often overlooked limitation of H: comparisons between differences in H-values are not valid. An alternative entropy H K is introduced as a preferred member of a new family of entropies for which difference comparisons are proved to be valid by satisfying a given value-validity condition. The H K is shown to have the appropriate properties for a randomness (uncertainty) measure, including a close linear relationship to a measurement criterion based on the Euclidean distance between probability distributions. This last point is demonstrated by means of computer generated random distributions. The results are also compared with those of another member of the entropy family. A statistical inference procedure for the entropy H K is formulated.
Highlights
For some probability distribution Pn “ pp1, ..., pn q, with pi ě 0 for i = 1, . . . , n and n ř pi “ 1, the i “1 entropy HpPn q, or H, is defined by: H“ ́n ÿ pi log pi P r0, log ns (1)i “1 where the logarithm is the natural logarithm
Consider that M is a measure of randomness such that its value MpPn q for any probability distribution Pn “ pp1, ..., pn q is bounded as: MpPn0 q ď MpPn q ď MpPn1 q where Pn0 and Pn1 are the degenerate and uniform distributions
As a simple numerical example illustrating the reasoning behind the value-validity arguments in Equations (6)–(13) and the use of Euclidean distances, consider the following probability distributions based on Pnλ in Equation (6):
Summary
Entropy 2016, 18, 159 which reduces to Equation (1), with base-2 logarithm, when α Ñ 1 This entropy family has, for instance, been used as a fractal dimension [11] Such widespread use of H has led to misuse, improper applications, and misleading results due to the fact that, H has a number of desirable properties [14] (Chapter 1), it does suffer from one serious limitation: comparisons between differences in H-values are not valid. The basis for this limitation is explained of this paper.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
