Entropy Evaluation Based on Value Validity

Tarald Kvålseth

doi:10.3390/e16094855

Abstract

Besides its importance in statistical physics and information theory, the Boltzmann-Shannon entropy S has become one of the most widely used and misused summary measures of various attributes (characteristics) in diverse fields of study. It has also been the subject of extensive and perhaps excessive generalizations. This paper introduces the concept and criteria for value validity as a means of determining if an entropy takes on values that reasonably reflect the attribute being measured and that permit different types of comparisons to be made for different probability distributions. While neither S nor its relative entropy equivalent S* meet the value-validity conditions, certain power functions of S and S* do to a considerable extent. No parametric generalization offers any advantage over S in this regard. A measure based on Euclidean distances between probability distributions is introduced as a potential entropy that does comply fully with the value-validity requirements and its statistical inference procedure is discussed.

Highlights

Consider that p1,..., pn, with n p i= 1, are the probabilities of a set of n quantum states accessible to a i =1 system or of a set of n mutually exclusive and exhaustive events of some statistical experiment
Whatever an entropy measure is being used for, it is not uncommon for comparisons to be made between differences in entropy values and for statements or implications to occur about the absolute and relative values of the attributes being measured by means of the entropy
The entropy S3 in Table 1, which is a particular subset of S8 with β = δ = 1 and λ = k / (1− α ), and which was defined for all real α, is strictly Schur-concave only for α ≥ 0

Summary

Introduction

This function has proved to be remarkably versatile and used as a measure of a variety of attributes in various fields of study, ranging from ecology (e.g., [7]) to psychology (e.g., [8]) It has resulted in literally infinitely many alternative entropy formulations and generalizations such as the parameterized families of entropies given in. Whatever an entropy measure is being used for, it is not uncommon for comparisons to be made between differences in entropy values and for statements or implications to occur about the absolute and relative values of the attributes (characteristics) being measured by means of the entropy This can lead to incorrect and misleading results and conclusions unless certain conditions are met as discussed in this paper.

Properties of S

Valid Comparison Conditions

Value-Valid Functions of S and S*

The Case of S

The Case of S*

Assessment of Entropy Families

The Euclidean Entropy

Statistical Inferences

Findings

Concluding Comments