On the Optimality of Conditional Expectation as a Bregman Predictor

Abstract

We consider problem of predicting a random variable X from observations, denoted by a random variable Z. It is well known that conditional expectation E[X|Z] is optimal L/sup 2/ predictor (also known as the least-mean-square error predictor) of X, among all (Borel measurable) functions of Z. In this orrespondence, we provide necessary and sufficient conditions for general loss functions under which conditional expectation is unique optimal predictor. We show that E[X|Z] is optimal predictor for all Bregman loss functions (BLFs), of which L/sup 2/ loss function is a special case. Moreover, under mild conditions, we show that BLFs are exhaustive, i.e., if for every random variable X, infimum of E[F(X,y)] over all constants y is attained by expectation E[X], then F is a BLF.