Parallel and bootstrapped stochastic approximation

Abstract

Parallelization of stochastic approximation procedures can reduce computation and total observation time of a system. Concerning the number of all observations used by the pure sequential and the suggested parallel method a weak invariance principle implies the asymptotic equivalence of both methods. A loglog invariance principle and a rate of a.s. convergence result describe the pathwise properties. Due to the parallel design asymptotic confidence regions can readily be constructed either by computing the bootstrap distribution or the Gaussian limit distribution determined by the empirical covariance