Sample Dependence in the Maximum Entropy Solution to the Generalized Moment Problem

Henryk Gzyl

doi:10.1155/2016/8629049

Abstract

The method of maximum entropy is quite a powerful tool to solve the generalized moment problem, which consists in determining the probability density of a random variableXfrom the knowledge of the expected values of a few functions of the variable. In actual practice, such expected values are determined from empirical samples, leaving open the question of the dependence of the solution upon the sample. It is the purpose of this note to take a few steps towards the analysis of such dependence.

Highlights

Introduction and PreliminariesTo state what the generalized moment problem is about, let (Ω, F, P) be a probability space and let (S, B, m) be a measure space, with m a finite or sigma-finite measure
X stands for a positive random variable and we can compute E[exp(−αkX)] = dk by some Monte Carlo procedure at a finite number of points αk
We recall in a historical survey the notion of entropy of a density, and in the following Journal of Probability and Statistics section we present the basics of the maximum entropy method

Summary

Introduction and Preliminaries

To state what the generalized moment problem is about, let (Ω, F, P) be a probability space and let (S, B, m) be a measure space, with m a finite or sigma-finite measure. The problem that we need to solve amounts to inverting the Laplace transform from such finite collection of values of the transform parameter α. This last problem is of much interest in the banking and insurance industries, where the density is necessary to compute risk premia and regulatory capital of various types, samples may be small, and the estimation of dk reflects that. That the density reconstruction from empirical moments has to depend on the sample seems to be intuitive, but neither the behavior of the maxentropic density as the sample size increases nor the fluctuations of the expected values with the densities fN∗ seem not to have been studied before

The Entropy of a Density

The Standard Maximum Entropy Method

Mathematical Complement

Sample Dependence