Minimal polynomial and reduced rank extrapolation methods are related

Abstract

Minimal Polynomial Extrapolation (MPE) and Reduced Rank Extrapolation (RRE) are two polynomial methods used for accelerating the convergence of sequences of vectors {xm}. They are applied successfully in conjunction with fixed-point iterative schemes in the solution of large and sparse systems of linear and nonlinear equations in different disciplines of science and engineering. Both methods produce approximations sk to the limit or antilimit of {xm} that are of the form sk=źi=0kźixi$\boldsymbol {s}_{k}={\sum }^{k}_{i=0}\gamma _{i}\boldsymbol {x}_{i}$ with źi=0kźi=1${\sum }^{k}_{i=0}\gamma _{i}=1$, for some scalars źi. The way the two methods are derived suggests that they might, somehow, be related to each other; this has not been explored so far, however. In this work, we tackle this issue and show that the vectors skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$ and skRRE$\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}$ produced by the two methods are related in more than one way, and independently of the way the xm are generated. One of our results states that RRE stagnates, in the sense that skRRE=skź1RRE$\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}=\boldsymbol {s}_{k-1}^{\textit {{\tiny {RRE}}}}$, if and only if skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$ does not exist. Another result states that, when skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$ exists, there holds μkskRRE=μkź1skź1RRE+źkskMPEwithμk=μkź1+źk,$$\mu_{k}\boldsymbol{s}_{k}^{\textit{{\tiny{RRE}}}}=\mu_{k-1}\boldsymbol{s}_{k-1}^{\textit{{\tiny{RRE}}}}+ \nu_{k}\boldsymbol{s}_{k}^{\textit{{\tiny{MPE}}}}\quad \text{with}\quad \mu_{k}=\mu_{k-1}+\nu_{k}, $$for some positive scalars μk, μkź1, and źk that depend only on skRRE$\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}$, skź1RRE$\boldsymbol {s}_{k-1}^{\textit {{\tiny {RRE}}}}$, and skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$, respectively. Our results are valid when MPE and RRE are defined in any weighted inner product and the norm induced by it. They also contain as special cases the known results pertaining to the connection between the method of Arnoldi and the method of generalized minimal residuals, two important Krylov subspace methods for solving nonsingular linear systems.