An accelerated randomized extended Kaczmarz algorithm

Abstract

The randomized Kaczmarz(RK) is a useful algorithm for solving consistent linear system Ax = b(A ∈ ℝm×n, b ∈ ℝ). It was proved that for inconsistent linear system, with randomized orthogonal projection, the randomized extended Kaczmarz(REK) method converges with an expected exponential rate. We describe an accelerated randomized extended Kaczmarz algorithm(AREK) with Nesterov’s accelerated procedure. The analysis shows that AREK converges better than REK when A is dense and the smallest singular value of ATA is small.