New Estimations for Shannon and Zipf-Mandelbrot Entropies.

Muhammad Khan,Yu-Ming Chu,Zaid Al-Sahwi

doi:10.3390/e20080608

Muhammad Khan, Yu-Ming Chu + Show 1 more

Open Access

https://doi.org/10.3390/e20080608

Copy DOI

Abstract

The main purpose of this paper is to find new estimations for the Shannon and Zipf–Mandelbrot entropies. We apply some refinements of the Jensen inequality to obtain different bounds for these entropies. Initially, we use a precise convex function in the refinement of the Jensen inequality and then tamper the weight and domain of the function to obtain general bounds for the Shannon entropy (SE). As particular cases of these general bounds, we derive some bounds for the Shannon entropy (SE) which are, in fact, the applications of some other well-known refinements of the Jensen inequality. Finally, we derive different estimations for the Zipf–Mandelbrot entropy (ZME) by using the new bounds of the Shannon entropy for the Zipf–Mandelbrot law (ZML). We also discuss particular cases and the bounds related to two different parametrics of the Zipf–Mandelbrot entropy. At the end of the paper we give some applications in linguistics.

Highlights

As particular cases of these general bounds, we derive some bounds for the Shannon entropy (SE) which are, the applications of some other well-known refinements of the Jensen inequality
We discuss particular cases and the bounds related to two different parametrics of the Zipf–Mandelbrot entropy
The idea of the Shannon entropy [1] plays a key role in information theory, while in some cases, it is denoted as measure of uncertainty

Summary

Introduction

The idea of the Shannon entropy [1] plays a key role in information theory, while in some cases, it is denoted as measure of uncertainty. Kristian Giesen and Jens Suedekum conducted a study on the city measure distributions of single German districts’ reliance on researching the phenomenon [10] They built their study based on the intuition by Gabaix (1999) which states that the Zipf law takes after an irregular development process. For some other results related to the Jensen inequality and the Shannon and Zipf–Mandelbrot entropies, see refs. The main focus of this paper was to associate some refinements of the Jensen inequality to the Shannon and Zipf–Mandelbrot entropies. The idea of this paper can be applied for other results of the Jensen inequality to obtain new estimations for these entropies

Estimations for the Shannon Entropy

Estimations for the Zipf–Mandelbrot Entropy