Adaptive importance sampling and control variates

Abstract

We construct and investigate an adaptive variance reduction framework in which both importance sampling and control variates are employed. The three lines (Monte Carlo averaging and two variance reduction parameter search lines) run in parallel on a common sequence of uniform random vectors on the unit hypercube. Given that these two variance reduction techniques are effective often in a complementary way, their combined application is well expected to widen the applicability of adaptive variance reduction. We derive convergence rates of the theoretical estimator variance towards its minimum as a fixed computing budget increases, when stochastic approximation runs with optimal constant learning rates. We derive sufficient conditions for the proposed algorithm to attain the minimal estimator variance in the limit, by stochastic approximation with decreasing learning rates or by sample average approximation, when computing budget is unlimitedly available. Numerical results support our theoretical findings and illustrate the effectiveness of the proposed framework.