Optimizing Adaptive Importance Sampling by Stochastic Approximation

Abstract

We investigate the optimality of adaptive importance sampling by stochastic approximation. To address the well-known difficulty in the implementation of stochastic approximation, that is, the extreme performance sensitivity to the choice of decreasing learning rates, we focus on constant learning rates for a finite computing budget, derived via minimization of an upper bound of the theoretical variance of the empirical mean, not via accelerated convergence of the objective function as in the existing stochastic gradient framework. Strong convexity of the objective function lowers the upper bound of the theoretical variance, while the convexity parameter is not required for implementation. In contrast to existing schemes based on sample average approximation, the proposed approach based on stochastic approximation has the potential to run reasonably robust adaptive importance sampling with significantly less computing cost. We provide numerical results to support the theoretical findings and to illustrate ...