Abstract

A rigorous dynamical systems-based hierarchy is established for the definitions of entropy of Shannon (information), Kolmogorov–Sinai (metric) and Adler, Konheim & McAndrew (topological). In particular, metric entropy, with the imposition of some additional properties, is proven to be a special case of topological entropy and Shannon entropy is shown to be a particular form of metric entropy. This is the first of two papers aimed at establishing a dynamically grounded hierarchy comprising Clausius, Boltzmann, Gibbs, Shannon, metric and topological entropy in which each element is ideally a special case of its successor or some kind of limit thereof.

Highlights

  • Entropy, which can among a variety of other things, be roughly viewed as a measure of uncertainty, has been and remains a fascinating and still not completely understood concept with an amazing range of applications, including quantum and ecological systems [2]

  • Entropy 2019, 21, 938 we prove that metric entropy is a special case of topological entropy if one adds just a few assumptions

  • We shall prove there is a topology on numerous discrete measurable dynamical systems (DMDS) of interest such that the corresponding topological entropy is equal to the K–S

Read more

Summary

Introduction

Entropy, which can among a variety of other things, be roughly viewed as a measure of uncertainty (cf. [1]), has been and remains a fascinating and still not completely understood concept with an amazing range of applications, including quantum and ecological systems [2]. There have been several extensive investigations of various types of entropy including historical accounts of the relevant developments and informative investigations of the linkages among the various forms of entropy such as in [3,4,5,6,7,8,9,10,11,12,13], but they all appear to be somewhat lacking in terms of identification of truly compelling unifying themes for the multifarious definitions Our intention in this and a subsequent paper is to provide a rigorous partial answer to this question by proving that there is a dynamical systems thread connecting all of the following definitions of entropy leading, with the possible addition of some mild assumptions, to the hierarchy. We begin with the topological entropy of a discrete dynamical system on a topological space

Topological Entropy
Kolmogorov–Sinai Metric Entropy
Shannon Entropy
A Relationship among the Entropies
When Topological Entropy Equals Metric Entropy
Constructive-Type Equality
Comparison-Type Equality
Definitions and Additional Notation
The Comparison Theorem
Examples of Topological and Metric Entropy Equivalence
Shannon Entropy as a Special Case of Metric Entropy
A Simple Bernoulli Scheme Example
Concluding Remarks

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.