Surrounding the solution of a linear system of equations from all sides

Abstract

Suppose A ∈ R n × n A \in \mathbb {R}^{n \times n} is invertible and we are looking for the solution of A x = b Ax = b . Given an initial guess x 1 ∈ R x_1 \in \mathbb {R} , we show that by reflecting through hyperplanes generated by the rows of A A , we can generate an infinite sequence ( x k ) k = 1 ∞ (x_k)_{k=1}^{\infty } such that all elements have the same distance to the solution x x , i.e. ‖ x k − x ‖ = ‖ x 1 − x ‖ \|x_k - x\| = \|x_1 - x\| . If the hyperplanes are chosen at random, averages over the sequence converge and E ‖ x − 1 m ∑ k = 1 m x k ‖ ≤ 1 + ‖ A ‖ F ‖ A − 1 ‖ m ⋅ ‖ x − x 1 ‖ . \begin{equation*} \mathbb {E} \left \| x - \frac {1}{m} \sum _{k=1}^{m}{ x_k} \right \| \leq \frac {1 + \|A\|_F \|A^{-1}\|}{\sqrt {m}} \cdot \|x-x_1\|. \end{equation*} The bound does not depend on the dimension of the matrix. This introduces a purely geometric way of attacking the problem: are there fast ways of estimating the location of the center of a sphere from knowing many points on the sphere? Our convergence rate (coinciding with that of the Random Kaczmarz method) comes from simple averaging, can one do better?