Abstract
The principles of maximum entropy (ME) and minimum relative entropy (MRE) are information-theoretic methods for estimating unknown probability distributions based on information about their expected values [Elsasser, 1937; Jaynes, 1957; Kullback, 1959; Csiszar, 1975; Shore and Johnson, 1980]. MRE differs from ME by taking into account an initial estimate of the unknown distribution. Both ME and MRE are used in a variety of successful applications, but widespread use has been hindered by some unresolved issues. One issue is the choice of state space in which to express problems and their solutions. Another issue is the choice and interpretation of initial estimates for MRE. A third issue is the identity of the “correct” expression to use for entropy when ME is applied to spectrum analysis and image enhancement. This paper shows that these issues are interrelated, and presents results that help to resolve them.
Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
