Stochastic block projection algorithms with extrapolation for convex feasibility problems

Abstract

The stochastic alternating projection (SP) algorithm is a simple but powerful approach for solving convex feasibility problems. At each step, the method projects the current iterate onto a random individual set from the intersection. Hence, it has simple iteration, but, usually, convergences slowly. In this paper, we develop accelerated variants of basic SP method. We achieve acceleration using two ingredients: blocks of sets and adaptive extrapolation. We propose SP-based algorithms based on extrapolated iterations of convex combinations of projections onto block of sets. Our approach is based on several new ideas and tools, including stochastic selection rules for the blocks, stochastic conditioning of feasibility problem, and novel strategies for designing adaptive extrapolated stepsizes. We prove that, under linear regularity of the sets, our stochastic block projection algorithms converge linearly in expectation, with a rate depending on the condition number of the (block) feasibility problem and on the size of the blocks. Otherwise, we prove that our methods converge sublinearly. Our convergence analysis reveals that such algorithms are most effective when a good sampling of the sets into well-conditioned blocks is given. The convergence rates also explain when algorithms combining block projections with adaptive extrapolation work better than their nonextrapolated variants.