A New Class of Optimum Importance Sampling Strategies Derived from Statistical Distance Measures

Abstract

Summary Performance analysis of discrete time systems often requires the evaluation of the expected value of a cost function of the system output or equivalently the expected value of a functional of the system input vector. In either case, analytical expressions for the performance in many practical situations are typically unavailable. As an alternative to approximations or bounds, Monte Carlo simulations are often employed as a convergent method for obtaining arbitrarily accurate estimates of the performance. Unfortunately, in many important applications, the required computations can often be prohibitive. Therefore, much recent research has focused on the develop ment of new and efficient forms of the method of Monte Carlo known as Importance Sampling simulations. The fundamental problem in Importance Sampling is to determine the appropriate statistics for the simulation. It is well known that the minimum variance statistics can not be implemented since they require the explicit knowledge of the expectation one is attempting to estimate. Therefore, researchers have worked to determine “good” suboptimal statistics from probability measure constraint classes which exclude the degenerate optimal solution. These so-called “biasing strategies” are typically obtained by minimizing the variance of the Importance Sampling estimator with respect to the biasing statistics over a class of probability measures which exclude the optimal distribution. Recently[3], it was shown by the authors that minimizing the Importance Sampling variance is equivalent to minimizing an Ali-Silvey distance [l] of equivalently an f-divergence [2] between the admissible biasing densities and the well known op timal biasing density. This result has led to a new approach in the design of Importance Sampling strategies where one merely determines the biasing density from an arbitrary constraint class with minimum “distance” to the global optimal distribution to minimize the simulation variance. Extending this previous research, we derive in this work the minimum variance biasing distribution from a constraint class whose controlling parameter is fundamental in the performance analysis of Importance Sampling. In addition, we will show that for the special case of estimating the probability of rare events, the constrained optimal biasing distribution from this class is independent of the unknown parameter and as such, leads to solutions which are both amenable to implementation and yet still optimal with respect to a relevant criterion. To motivate the proposed constraint class, we note that it