Maximum Likelihood Estimation of the Lagrange Parameters of the Maximum Entropy Distributions

Abstract

The classical maximum entropy (ME) problem consists of determining a probability distribution function from a finite set of expectations µ n = E {o n (x)} of known functions o n (x), 0,..., N. The solution depends on N + 1 Lagrange multipliers which are determined by solving the set of nonlinear equations formed by the N data constraints and the normalization constraint. The problem we address here is different. It consists of estimating these Lagrange multipliers when the available data are the M samples {x 1,..., x M } of the random variable X in place of the expectations µ n . We propose and compare two methods: the maximum likelihood (ML) one and the method of moments (MM). We show also an interesting relation between the classical ME method and the ML method for this problem. Finally, we show the interest of these developpements in determining the prior law of an image in a Bayesian approach to solving the inverse problems of image restoration and reconstruction.