Minimal Polynomial Extrapolation Research Articles

Convergence acceleration of iterative sequences for equilibrium chemistry computations

The modeling of thermodynamic equilibria leads to complex nonlinear chemical systems which are often solved with the Newton-Raphson method. But this resolution can lead to a non convergence or an excessive number of iterations due to the very ill-conditioned nature of the problem. In this work, we combine a particular formulation of the equilibrium system called the Positive Continuous Fraction method with two iterative methods, Anderson Acceleration method and Vector extrapolation methods (namely the reduced rank extrapolation and the minimal polynomial extrapolation). The main advantage of this approach is to avoid forming the Jacobian matrix. In addition, a strategy is used to improve the robustness of the Anderson acceleration method which consists in reducing the condition number of matrix of the least squares problem in the implementation of the Anderson acceleration so that the numerical stability can be guaranteed. We compare our numerical results with those obtained with the Newton-Raphson method on the Acid Gallic test and the 1D MoMas benchmark test case and we show the high efficiency of our approach.

Structural sensitivity reanalysis formulations based on the polynomial-type extrapolation methods

This paper presents new structural sensitivity reanalysis formulations based on the polynomial-type extrapolation methods. In these formulations, the displacement vector of the modified structure is expressed in the form of the vector sequences based on the fixed-point iteration method. By using these vector sequences, the minimal polynomial extrapolation (MPE) and the reduced rank extrapolation (RRE) methods calculate the approximate displacement vector of the modified structure by solving reduced linear least-square problems. Based on the definitions of the MPE and RRE methods, two sensitivity reanalysis formulations are derived, in which the first- and second-order sensitivities of the modified structure are obtained by solving a set of the over-determined least-square problems with much smaller size than the complete set of equations of the exact sensitivity analyses. The performance of the proposed sensitivity reanalysis formulations is evaluated by using four structural sensitivity reanalysis problems under multiple modifications in their initial designs. The results obtained from the numerical test problems indicate that the proposed sensitivity reanalysis formulations approximate the first- and second-order sensitivities of the modified structure with a high level of accuracy and they are able to converge to the exact solutions.

A new structural reanalysis approach based on the polynomial-type extrapolation methods

In this study, a new approach based on the polynomial-type extrapolation methods is applied to the approximate structural reanalysis under multiple and large changes in the initial design. In this approach, the sequences of approximate displacement of the modified structure are constructed by using a fixed-point iteration method. These sequences are then further analyzed by two polynomial-type vector extrapolation methods to find the approximate response of the modified structure more accurately, namely, the minimal polynomial extrapolation (MPE) and the reduced rank extrapolation (RRE). Based on a single initial design, the MPE and RRE methods approximate the displacement vector of the modified structure by solving a least-squares problem which is much smaller than the original system of equations of the exact analysis. The accuracy and efficiency of this reanalysis approach is evaluated on three large scale structural reanalysis problems under multiple and large changes in their initial designs. The obtained reanalysis results demonstrate that the MPE and RRE methods not only yield accurate results, but also are computationally efficient.

Shanks Sequence Transformations and Anderson Acceleration

This paper presents a general framework for Shanks transformations of sequences of elements in a vector space. It is shown that Minimal Polynomial Extrapolation (MPE), Modified Minimal Polynomial E...

Minimal polynomial and reduced rank extrapolation methods are related

Minimal Polynomial Extrapolation (MPE) and Reduced Rank Extrapolation (RRE) are two polynomial methods used for accelerating the convergence of sequences of vectors {xm}. They are applied successfully in conjunction with fixed-point iterative schemes in the solution of large and sparse systems of linear and nonlinear equations in different disciplines of science and engineering. Both methods produce approximations sk to the limit or antilimit of {xm} that are of the form sk=źi=0kźixi$\boldsymbol {s}_{k}={\sum }^{k}_{i=0}\gamma _{i}\boldsymbol {x}_{i}$ with źi=0kźi=1${\sum }^{k}_{i=0}\gamma _{i}=1$, for some scalars źi. The way the two methods are derived suggests that they might, somehow, be related to each other; this has not been explored so far, however. In this work, we tackle this issue and show that the vectors skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$ and skRRE$\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}$ produced by the two methods are related in more than one way, and independently of the way the xm are generated. One of our results states that RRE stagnates, in the sense that skRRE=skź1RRE$\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}=\boldsymbol {s}_{k-1}^{\textit {{\tiny {RRE}}}}$, if and only if skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$ does not exist. Another result states that, when skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$ exists, there holds μkskRRE=μkź1skź1RRE+źkskMPEwithμk=μkź1+źk,$$\mu_{k}\boldsymbol{s}_{k}^{\textit{{\tiny{RRE}}}}=\mu_{k-1}\boldsymbol{s}_{k-1}^{\textit{{\tiny{RRE}}}}+ \nu_{k}\boldsymbol{s}_{k}^{\textit{{\tiny{MPE}}}}\quad \text{with}\quad \mu_{k}=\mu_{k-1}+\nu_{k}, $$for some positive scalars μk, μkź1, and źk that depend only on skRRE$\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}$, skź1RRE$\boldsymbol {s}_{k-1}^{\textit {{\tiny {RRE}}}}$, and skMPE$\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}$, respectively. Our results are valid when MPE and RRE are defined in any weighted inner product and the norm induced by it. They also contain as special cases the known results pertaining to the connection between the method of Arnoldi and the method of generalized minimal residuals, two important Krylov subspace methods for solving nonsingular linear systems.

SVD-MPE: An SVD-Based Vector Extrapolation Method of Polynomial Type

An important problem that arises in different areas of science and engineering is that of computing the limits of sequences of vectors , where , N being very large. Such sequences arise, for example, in the solution of systems of linear or nonlinear equations by fixed-point iterative methods, and are simply the required solutions. In most cases of interest, however, these sequences converge to their limits extremely slowly. One practical way to make the sequences converge more quickly is to apply to them vector extrapolation methods. Two types of methods exist in the literature: polynomial type methods and epsilon algorithms. In most applications, the polynomial type methods have proved to be superior convergence accelerators. Three polynomial type methods are known, and these are the minimal polynomial extrapolation (MPE), the reduced rank extrapolation (RRE), and the modified minimal polynomial extrapolation (MMPE). In this work, we develop yet another polynomial type method, which is based on the singular value decomposition, as well as the ideas that lead to MPE. We denote this new method by SVD-MPE. We also design a numerically stable algorithm for its implementation, whose computational cost and storage requirements are minimal. Finally, we illustrate the use of SVD-MPE with numerical examples.

Acceleration of Dominant Transversal Laser Resonator Eigenmode Calculation by Vector Extrapolation Methods

Field tracing provides the flexible and fully vectorial calculation of the dominant transversal resonator mode. Therefore, the eigenvalue problem formulated in a set of coupled, nonlinear operator equations has to be solved. Up to now, only the iterative power method, also known as Fox and Li algorithm, was used for the eigenvector calculation, where serious convergence problems can occur. We accelerate the convergence speed by applying two polynomial-type vector extrapolation methods, namely, the minimal polynomial extrapolation and the reduced rank extrapolation. Numerical examples show that the convergence speed and, consequently, the computational time required for the calculation of the fully vectorial transversal resonator eigenmode, could be reduced by up to 70% compared with the Fox and Li algorithm. Furthermore, it is shown that the calculated eigenmode is in good agreement with the experimental results.

Fast solvers for unsteady thermal fluid structure interaction

SummaryWe consider time‐dependent thermal fluid structure interaction. The respective models are the compressible Navier–Stokes equations and the nonlinear heat equation. A partitioned coupling approach via a Dirichlet–Neumann method and a fixed point iteration is employed. As a reference solver, a previously developed efficient time‐adaptive higher‐order time integration scheme is used.To improve on this, we work on reducing the number of fixed point coupling iterations. Using the idea of extrapolation based on data given from the time integration by deriving such methods for SDIRK2, it is possible to reduce the number of fixed point iterations further by up to a factor of two with linear extrapolation performing better than quadratic. This leads to schemes that can use less than two iterations per time step.Furthermore, widely used vector extrapolation methods for convergence acceleration of the fixed point iteration are tested, namely Aitken relaxation, minimal polynomial extrapolation and reduced rank extrapolation. These have no beneficial effects. Copyright © 2015 John Wiley & Sons, Ltd.

Accelerating numerical simulations of strain-adaptive bone remodeling predictions

Bone modeling and remodeling have been extensively studied in many fields of research. The process of adaptive bone remodeling can be described mathematically and simulated in a computer model, integrated with the finite element method (FEM). The FEM can determine the long-term behavior of bone and the impact on bone biomechanics produced by prosthetic models. As whole bone remodeling simulations are time consuming, the aim of this paper is to combine FE analyses with numerical extrapolation techniques to accelerate bone remodeling simulations. Two vector extrapolation methods, reduce ranked extrapolation (RRE) and minimal polynomial extrapolation (MPE), are used to reduce the simulation time. These extrapolation techniques are illustrated by numerical examples of 2D and 3D models based on a strain-adaptive bone remodeling theory.

Review of two vector extrapolation methods of polynomial type with applications to large-scale problems

AbstractAn important problem that arises in different areas of science and engineering is that of computing limits of sequences of vectors {xn}, where xn∈CN with N very large. Such sequences arise, for example, in the solution of systems of linear or nonlinear equations by fixed-point iterative methods, and limn→∞xn are simply the required solutions. In most cases of interest, these sequences converge to their limits extremely slowly, or even diverge. One practical way to make the sequences {xn} converge more quickly is to apply to them vector extrapolation methods. In this work, we review two polynomial-type vector extrapolation methods that have proved to be very efficient convergence accelerators; namely, the minimal polynomial extrapolation (MPE) and the reduced rank extrapolation (RRE). We discuss their derivation, describe the most accurate and stable algorithms for their implementation along with the effective modes of usage, and present their convergence and stability theory. We also discuss their close connection with the method of Arnoldi and GMRES, two well known Krylov subspace methods for linear systems. Finally, we discuss some of their applications to different large-scale problems, such as solution of large-scale systems of equations, eigenvalue problems, computation of the PageRank of the Google matrix, and summation of vector-valued power series.

Generalized GIPSCAL re-revisited: a fast convergent algorithm with acceleration by the minimal polynomial extrapolation

Generalized GIPSCAL, like DEDICOM, is a model for the analysis of square asymmetric tables. It is a special case of DEDICOM, but unlike DEDICOM, it ensures the nonnegative definiteness (nnd) of the model matrix, thereby allowing a spatial representation of the asymmetric relationships among "objects". A fast convergent algorithm was developed for GIPSCAL with acceleration by the minimal polynomial extrapolation. The proposed algorithm was compared with Trendafilov's algorithm in computational speed. The basic algorithm has been adapted to various extensions of GIPSCAL, including off-diagonal DEDICOM/GIPSCAL, and three-way GIPSCAL.

On the application of the minimum polynomial extrapolation method to incompressible flows with heat transfer

In this paper, the Minimum Polynomial Extrapolation method (MPE) is used to accelerate the convergence of the Characteristic---Based---Split (CBS) scheme for the numerical solution of steady state incompressible flows with heat transfer. The CBS scheme is a fractional step method for the solution of the Navier---Stokes equations while the MPE method is a vector extrapolation method which transforms the original sequence into another sequence converging to the same limit faster then the original one without the explicit knowledge of the sequence generator. The developed algorithm is tested on a two-dimensional benchmark problem (buoyancy---driven convection problem) where the Navier---Stokes equations are coupled with the temperature equation. The obtained results show the feature of the extrapolation procedure to the CBS scheme and the reduction of the computational time of the simulation.

An acceleration method for Ten Berge et al.’s algorithm for orthogonal INDSCAL

INDSCAL (INdividual Differences SCALing) is a useful technique for investigating both common and unique aspects of K similarity data matrices. The model postulates a common stimulus configuration in a low-dimensional Euclidean space, while representing differences among the K data matrices by differential weighting of dimensions by different data sources. Since Carroll and Chang proposed their algorithm for INDSCAL, several issues have been raised: non-symmetric solutions, negative saliency weights, and the degeneracy problem. Orthogonal INDSCAL (O-INDSCAL) which imposes orthogonality constraints on the matrix of stimulus configuration has been proposed to overcome some of these difficulties. Two algorithms have been proposed for O-INDSCAL, one by Ten Berge, Knol, and Kiers, and the other by Trendafilov. In this paper, an acceleration technique called minimal polynomial extrapolation is incorporated in Ten Berge et al.’s algorithm. Simulation studies are conducted to compare the performance of the three algorithms (Ten Berge et al.’s original algorithm, the accelerated algorithm, and Trendafilov’s). Possible extensions of the accelerated algorithm to similar situations are also suggested.

An accelerated algorithm for Navier–Stokes equations

In this paper, the numerical solution of the Navier–Stokes equations by the Characteristic-Based-Split (CBS) scheme is accelerated with the Minimum Polynomial Extrapolation (MPE) method to obtain the steady state solution for evolution incompressible and compressible problems. The CBS is essentially a fractional time-stepping algorithm based on an original finite difference velocity-projection scheme where the convective terms are treated using the idea of the Characteristic-Galerkin method. In this work, the semi-implicit version of the CBS with global time-stepping is used for incompressible problems whereas the fully-explicit version is used for compressible flows. At the other end, the MPE is a vector extrapolation method that transforms the original sequence into another sequence converging to the same limit faster then the original one without the explicit knowledge of the sequence generator. The developed algorithm, tested on two-dimensional benchmark problems, demonstrates the new computational features arising from the introduction of the extrapolation procedure to the CBS scheme. In particular, the results show a remarkable reduction of the computational cost of the simulation.

Fast indirect robust generalized method of moments

The Robust Generalized Methods of Moments (RGMM) and the Indirect Robust GMM (IRGMM) are algorithms for estimating parameter values in statistical models, such as diffusion models for interest rates, in a robust way. The long computation time is one of the main challenges facing these methods. In this paper, we introduce accelerated variants of RGMM and IRGMM. The fixed point iteration in RGMM is accelerated using minimal polynomial extrapolation, and the simulation of pseudo-observations in IRGMM is sped up by using a higher order stochastic Runge–Kutta method. We illustrate the fast performance of these algorithms for an interest rate diffusion model on four datasets.

A solver combining reduced basis and convergence acceleration with applications to non‐linear elasticity

AbstractAn iterative solver is proposed to solve the family of linear equations arising from the numerical computation of non‐linear problems. This solver relies on two quantities coming from previous steps of the computations: the preconditioning matrix is a matrix that has been factorized at an earlier step and previously computed vectors yield a reduced basis. The principle is to define an increment in two sub‐steps. In the first sub‐step, only the projection of the unknown on a reduced subspace is incremented and the projection of the equation on the reduced subspace is satisfied exactly. In the second sub‐step, the full equation is solved approximately with the help of the preconditioner. Last, the convergence of the sequences is accelerated by a well‐known method, the modified minimal polynomial extrapolation. This algorithm assessed by classical benchmarks coming from shell buckling analysis. Finally, its insertion in path following techniques is discussed. This leads to non‐linear solvers with few matrix factorizations and few iterations. Copyright © 2009 John Wiley & Sons, Ltd.

Algorithms for DEDICOM: acceleration, deceleration, or neither?

AbstractTakane's original algorithm for DEDICOM (DEcomposition into DIrectional COMponents) was proposed more than two decades ago. There have been a couple of significant developments since then: Kiers et al.'s modification to ensure monotonic convergence of the algorithm, and Jennrich's recommendation to use the modified algorithm only when Takane's original algorithm violates the monotonicity. In this paper, we argue that neither of these modifications is essential, drawing a close relationship between Takane's algorithm and the simultaneous power method for obtaining dominant eigenvalues and vectors of a symmetric matrix. By ignoring monotonicity, we can develop a much more efficient algorithm by simple modifications of Takane's original algorithm, as demonstrated in this paper. More specifically, we incorporate the minimum polynomial extrapolation (MPE) method to accelerate the convergence of Takane's algorithm, and show that it significantly cuts down the computation time. Copyright © 2009 John Wiley & Sons, Ltd.

Vector extrapolation methods with applications to solution of large systems of equations and to PageRank computations

An important problem that arises in different areas of science and engineering is that of computing the limits of sequences of vectors { x n } , where x n ∈ C N with N very large. Such sequences arise, for example, in the solution of systems of linear or nonlinear equations by fixed-point iterative methods, and lim n → ∞ x n are simply the required solutions. In most cases of interest, however, these sequences converge to their limits extremely slowly. One practical way to make the sequences { x n } converge more quickly is to apply to them vector extrapolation methods. In this work, we review two polynomial-type vector extrapolation methods that have proved to be very efficient convergence accelerators; namely, the minimal polynomial extrapolation (MPE) and the reduced rank extrapolation (RRE). We discuss the derivation of these methods, describe the most accurate and stable algorithms for their implementation along with the effective modes of usage in solving systems of equations, nonlinear as well as linear, and present their convergence and stability theory. We also discuss their close connection with the method of Arnoldi and with GMRES, two well-known Krylov subspace methods for linear systems. We show that they can be used very effectively to obtain the dominant eigenvectors of large sparse matrices when the corresponding eigenvalues are known, and provide the relevant theory as well. One such problem is that of computing the PageRank of the Google matrix, which we discuss in detail. In addition, we show that a recent extrapolation method of Kamvar et al. that was proposed for computing the PageRank is very closely related to MPE. We present a generalization of the method of Kamvar et al. along with a very economical algorithm for this generalization. We also provide the missing convergence theory for it.

Acceleration of Unsteady Incompressible Flow Calculation Using Extrapolation Methods

Extrapolation methods constitute a class of acceleration techniques in which the converged solution is predicted from a series of intermediate solutions generated by the iterative method. Advantages of the extrapolation methods are simplicity and ease of the implementation in existing CFD codes. In this study, the authors apply Minimal Polynomial Extrapolation (MPE) to 2-D laminar and 3-D turbulent problems to examine the performance of MPE in unsteady flow calculation. Also Initial Extrapolation (IE), which provides a good initial guess at the start of each timestep, is tested alongside MPE. Although only six intermediate solutions are used in MPE, it realizes reasonable speed-ups : 2.88 in the 2-D calculation and 1.89 in the 3-D LES. This computational cost is far lower than that in steady calculation reported in a literature. Furthermore, the combination of IE and MPE shows larger acceleration effects than in the case of single application of MPE because the speed-ups by the joint application of the two extrapolation methods reach 4.41 in the 2-D and 2.33 in the 3-D.

Mathematical and numerical connections between polynomial extrapolation and Padé approximants: applications in structural mechanics

AbstractThis paper deals with the convergence acceleration of a sequence of vectors. This acceleration is achieved by using Minimal Polynomial Extrapolation Method (J Comput Appl Math 2000; 122:149–165) or by the vectorial Padé approximants, that have been considered in the framework of high‐order iterative algorithms (Commun Numer Methods Eng 1999; 15:701–708; Comput Methods Appl Mech Eng 2000; 190:1845–1858). It is established that, in some cases, these two procedures are mathematically equivalent. In the practice, such techniques are affected by numerical instabilities. So the numerical robustness of the two methods is discussed, on the basis of a test emanating from non‐linear elastic thin shell analysis. Copyright © 2004 John Wiley & Sons, Ltd.