Maximum Entropy Estimation of Density Function Using Order Statistics

Abstract

The main premise of this article is to develop a maximum entropy estimation of an unknown distribution using order statistics. The exact solution using constraints on the order statistic means is derived. The result is an integral of a rational parametric function whose parameters depend on the order statistic constraints. This integral equation can be solved directly for a small number of order statistic means, e.g., 1, 2, or at most 3, via a multidimensional search in the parameter space. As the number of constraints increases, the search for the optimum parameters becomes formidable. To resolve this problem we have proposed an approximate solution to the resultant integral equation based on Bernstein polynomials and estimates of a desired number of quantiles from available samples. The proposed approximation approach does not require any search in the parameter space and is formulated for any number of samples from an unknown distribution. Performance of this approach is evaluated under various practical conditions by using Kullback-Leibler divergence function.