On an Objective Basis for the Maximum Entropy Principle

David Miller,Hossein Soleimani

doi:10.3390/e17010401

Abstract

In this letter, we elaborate on some of the issues raised by a recent paper by Neapolitan and Jiang concerning the maximum entropy (ME) principle and alternative principles for estimating probabilities consistent with known, measured constraint information. We argue that the ME solution for the “problematic” example introduced by Neapolitan and Jiang has stronger objective basis, rooted in results from information theory, than their alternative proposed solution. We also raise some technical concerns about the Bayesian analysis in their work, which was used to independently support their alternative to the ME solution. The letter concludes by noting some open problems involving maximum entropy statistical inference.

Highlights

In a recent paper, “A note of caution on maximizing entropy” [1], the authors considered the problem of estimating a probability mass function given supplied constraint information
“problematic” the maximum entropy solution for the example of a 3-sided die, where the given constraint information is that the mean die value is two
One can show that the maximum entropy solution, consistent with the given constraints, is the uniform distribution pi = 31, i = 1, 2, 3

Summary

Introduction

“A note of caution on maximizing entropy” [1], the authors considered the problem of estimating a probability mass function given supplied constraint information They identified as “problematic” the maximum entropy solution for the example of a 3-sided die, where the given constraint information is that the mean die value is two. For this example, maximum entropy (ME) solves the following problem: X pME = arg max −. The authors propose an alternative “objectively-based” approach for solving this problem They suppose that the probabilities are random variables, uniformly distributed over their ranges, which are prescribed by the given constraints, i.e., p1 = p3 ∈ [0, 0.5] and p2 ∈ [0, 1]. We identify some open problems in maximum entropy statistical inference

On Objective Bases For Preferring One Probability Assignment Over Another

Encoding Additional Constraints

Open Problems for Maximum Entropy

Conclusions

Reidel