Maximum entropy derived and generalized under idempotent probability to address Bayes-frequentist uncertainty and model revision uncertainty: An information-theoretic semantics for possibility theory

Abstract

Typical statistical methods of data analysis only handle determinate uncertainty, the type of uncertainty that can be modeled under the Bayesian or confidence theories of inference. An example of indeterminate uncertainty is uncertainty about whether the Bayesian theory or the frequentist theory is better suited to the problem at hand. Another example is uncertainty about how to modify a Bayesian model upon learning that its prior is inadequate. Both problems of indeterminate uncertainty have solutions under the proposed framework. The framework is based on an information-theoretic definition of an incoherence function to be minimized. It generalizes the principle of choosing an estimate that minimizes the reverse relative entropy between it and a previous posterior distribution such as a confidence distribution. The simplest form of the incoherence function, called the incoherence distribution, is a min-plus probability distribution, which is equivalent to a possibility distribution rather than a measure-theoretic probability distribution. A simple case of minimizing the incoherence leads to a generalization of minimizing relative entropy and thus of maximizing entropy. The framework of minimum incoherence is applied to problems of Bayesian-confidence uncertainty and to parallel problems of indeterminate uncertainty about model revision.