Maximum Entropy Problem Research Articles

A note on solution of large sparse maximum entropy problems with linear equality constraints

This paper describes a method to solve large sparse maximum entropy problems with linear equality constraints using Newtons and the conjugate gradient method. A numerical example is given to introduce the reader to possible applications of entropy models and this method. Some experience from large scale problems is also reported.

Computer Solution of the Discrete Maximum Entropy Problem

Consider an experiment with a finite number of outcomes. Suppose, for example, we are given the following data on the yield strength of a Bofors steel [8], [9]: Suppose also that we know the means of certain random variables relating to our experiment; in the above example we might calculate the mean and variance of the observed outcomes: ,/ = 35.6, o2 = 4.19. On the basis of this information, how should we choose a probability distribution P to best estimate the unknown probability measure P* underlying our experiment? There is, of course, no set solution to this problem. The maximum entropy method, however, is particularly appealing and has received considerable attention in recent years. Introduced by Shannon [7] in connection with communication theory, entropy was given an information theoretic interpretation first formulated by Jaynes [4] and further developed by Tribus [8]. According to Jaynes, P should be chosen to maximize the entropy E:

Computer Solution of the Discrete Maximum Entropy Problem

Computer Solution of the Discrete Maximum Entropy Problem