Minimum Expected Relative Entropy Principle

Abstract

Stochastic filtering estimates a time-varying (multivariate) parameter (a hidden variable) from noisy observations. It needs both observation and parameter evolution models. The latter is often missing or makes the estimation too complex. Then, the axiomatic minimum relative entropy (MRE) principle completes the posterior probability density (pd) of the parameter. The MRE principle recommends to modify a prior guess of the constructed pd to the smallest extent enforced by new observations. The MRE principle does not deal with a generic uncertain prior guess. Such uncertainty arises, for instance, when the MRE principle is used recursively. The paper fills this gap. The proposed minimum expected relative entropy (MeRE) principle: (a) makes Bayesian estimation less sensitive to the choice of the prior pd; (b) provides a stabilised parameter tracking with a data-dependent forgetting that copes with abrupt parameter changes; (c) applies in all cases exploiting MRE, for instance, in stochastic modelling.