Bayesian updating and marginal likelihood estimation by cross entropy based importance sampling

Abstract

We introduce a novel simulation-based method for Bayesian analysis to learn model parameters based on data. The method employs importance sampling (IS) to construct a sample-based approximation of the posterior probability density function (PDF) and estimate the marginal likelihood. We propose to build the IS density through an adaptive sampling approach based on the cross entropy (CE) method. The aim is to identify the parameters of a chosen parametric distribution family that minimize its Kullback-Leibler divergence from the target posterior PDF. An adaptive multi-level approach, based on tempering of the likelihood function, is proposed to efficiently solve this CE optimization problem. We investigate the appropriate choice of the parametric distribution, depending on the number of uncertain model parameters and nature of the posterior density. Numerical studies demonstrate the performance of the proposed method.