Convergence Of Fixed-point Iteration Research Articles

Randomized extrapolation for accelerating EM-type fixed-point algorithms

Several extrapolation strategies have been proposed in the literature to accelerate the EM algorithm, with varying degrees of success. One advantage of extrapolation methods is their ease of implementation, as they only require working with the EM iterations and do not need auxiliary quantities, such as gradient and Hessian. In this paper, we introduce a new family of iterative schemes based on vector extrapolation methods. We also construct and numerically test a randomly relaxed version of the scheme. Our results demonstrate that these new strategies can significantly and stably accelerate the convergence of the EM algorithm compared to existing methods. Moreover, these strategies are highly versatile as they can accelerate any linearly convergent fixed point iteration, including EM-type algorithms. Finally, we provide statistical modeling experiments at the end of the paper to demonstrate the applicability and interest of these convergence acceleration schemes, whether applied to the EM algorithm or one of its variants.

Anderson acceleration for seismic inversion

State-of-the-art seismic imaging techniques treat inversion tasks such as full-waveform inversion (FWI) and least-squares reverse time migration (LSRTM) as partial differential equation-constrained optimization problems. Due to the large-scale nature, gradient-based optimization algorithms are preferred in practice to update the model iteratively. Higher-order methods converge in fewer iterations but often require higher computational costs, more line-search steps, and bigger memory storage. A balance among these aspects has to be considered. We have conducted an evaluation using Anderson acceleration (AA), a popular strategy to speed up the convergence of fixed-point iterations, to accelerate the steepest-descent algorithm, which we innovatively treat as a fixed-point iteration. Independent of the unknown parameter dimensionality, the computational cost of implementing the method can be reduced to an extremely low dimensional least-squares problem. The cost can be further reduced by a low-rank update. We determine the theoretical connections and the differences between AA and other well-known optimization methods such as L-BFGS and the restarted generalized minimal residual method and compare their computational cost and memory requirements. Numerical examples of FWI and LSRTM applied to the Marmousi benchmark demonstrate the acceleration effects of AA. Compared with the steepest-descent method, AA can achieve faster convergence and can provide competitive results with some quasi-Newton methods, making it an attractive optimization strategy for seismic inversion.

A block positive-semidefinite splitting preconditioner for generalized saddle point linear systems

In this paper, we propose a new block positive-semidefinite splitting (BPS) preconditioner for a class of generalized saddle point linear systems. The new BPS preconditioner is based on two positive-semidefinite splittings of the generalized saddle point matrix, resulting in an unconditional convergent fixed-point iteration method. Theoretical results show that all eigenvalues of the BPS preconditioned matrix are clustered at only two points as the iteration parameter is close to zero. Two numerical examples arising from the mixed finite element discretization of the linearized Navier–Stokes equation and the meshfree discretization of the piezoelectric structure equation are used to illustrate the effectiveness of the new preconditioner.

A simple algorithm for high order Newton iteration formulae and some new variants

The high order Newton iteration formulas are revisited in this paper. Translating the nonlinear root finding problem into a fixed point iteration involving an unknown general function whose root is searched, a double Taylor series is undertaken regarding the root and the root finding function. Based on the error analysis of the expansion, a simple algorithm is later proposed to construct Newton iteration formulae of any order commencing from the traditional linearly convergent fixed point iteration method and quadratically convergent Newton-Raphson method of frequently at the disposal of the scientific community. It is shown that the well-known variants like the Halley's method or Haouseholder's methods of high order can be reproduced from the general case outlined here. Some further rare single-step classes of any order are shown to be derivable from the presented algorithm. Finally, some new higher order accurate variants are also offered taking into account multi-step compositions which demand less computation of higher derivatives. The efficiency, accuracy and performance of the proposed methods and also their potential advantages over the classical ones are numerically demonstrated and discussed on some well-documented examples from the open literature.

Method of Identification of Dry and Viscous Friction Forces and Construction of Dynamic Deformation Diagrams of Metals in Experiments with Impact Compression

The influence of dry and viscous friction forces on the dynamic deformation of elastovis-coplastic tablet specimens was numerically and experimentally investigated. The main dependencies of their form-changing for metals and alloys have been established. A criterion of the shape changing of tablet specimens is proposed. A new method for identifying of the coefficients of dry friction at contact surfaces, depending on shape changing of the tablet specimens, based on numerical modeling of an axisymmetric dynamic problem and a rapidly convergent fixed-point iteration method was developed. The division of the two-parameter identification problem into two problems of one-parameter parameterization is theoretically justified with a high degree of reliability: the problem of determining of the friction coefficient and the problem of construction of the true diagram of dynamic deformation in this experiment with the friction coefficient found earlier. As a result, the dynamic deformation diagrams with frictional forces and radial inertia consideration are constructed using the iterative method. In known approximation methods of construction of the deformation diagrams with frictional forces and radial inertia consideration, friction coefficients are assumed to be known, whereas methods for their determination in experiments with impact compression are practically unavailable.

A parameterized shift-splitting preconditioner for saddle point problems.

Recently, Chen and Ma [A generalized shift-splitting preconditioner for saddle point problems, Applied Mathematics Letters, 43 (2015) 49-55] introduced a generalized shift-splitting preconditioner for saddle point problems with symmetric positive definite (1,1)-block. In this paper, I establish a parameterized shift-splitting preconditioner for solving the large sparse augmented systems of linear equations. Furthermore, the preconditioner is based on the parameterized shift-splitting of the saddle point matrix, resulting in an unconditional convergent fixed-point iteration, which has the intersection with the generalized shift-splitting preconditioner. In final, one example is provided to confirm the effectiveness.

A variant of the HSS preconditioner for complex symmetric indefinite linear systems

Using the equivalent block two-by-two real linear systems, we establish a new variant of the Hermitian and skew-Hermitian splitting (HSS) preconditioner for a class of complex symmetric indefinite linear systems. The new preconditioner is not only a better approximation to the block two-by-two real coefficient matrix than the well-known HSS preconditioner, but also resulting in an unconditional convergent fixed-point iteration. The quasi-optimal parameter, which minimizes an upper bound of the spectral radius of the iteration matrix, is analyzed. Eigen-properties and an upper bound of the degree of the minimal polynomial of the preconditioned matrix are discussed. Finally, two numerical examples are provided to show the efficiency of the new preconditioner.

Exponential Fourier Collocation Methods for Solving First-Order Differential Equations

In this paper, a novel class of exponential Fourier collocation methods (EFCMs) is presented for solving systems of first-order ordinary differential equations. These so-called exponential Fourier collocation methods are based on the variation-of-constants formula, incorporating a local Fourier expansion of the underlying problem with collocation methods. We discuss in detail the connections of EFCMs with trigonometric Fourier collocation methods (TFCMs), the well-known Hamiltonian Boundary Value Methods (HBVMs), Gauss methods and Radau IIA methods. It turns out that the novel EFCMs are an essential extension of these existing methods. We also analyse the accuracy in preserving the quadratic invariants and the Hamiltonian energy when the underlying system is a Hamiltonian system. Other properties of EFCMs including the order of approximations and the convergence of fixed-point iterations are investigated as well. The analysis given in this paper proves further that EFCMs can achieve arbitrarily high order in a routine manner which allows us to construct higher-order methods for solving systems of first-order ordinary differential equations conveniently. We also derive a practical fourth-order EFCM denoted by EFCM(2,2) as an illustrative example. The numerical experiments using EFCM(2,2) are implemented in comparison with an existing fourth-order HBVM, an energy-preserving collocation method and a fourth-order exponential integrator in the literature. The numerical results demonstrate the remarkable efficiency and robustness of the novel EFCM(2,2).

A class of generalized shift-splitting preconditioners for nonsymmetric saddle point problems

In this paper, a class of generalized shift-splitting preconditioners with two shift parameters are implemented for nonsymmetric saddle point problems with nonsymmetric positive definite (1, 1) block. The generalized shift-splitting (GSS) preconditioner is induced by a generalized shift-splitting of the nonsymmetric saddle point matrix, resulting in an unconditional convergent fixed-point iteration. By removing the shift parameter in the (1, 1) block of the GSS preconditioner, a deteriorated shift-splitting (DSS) preconditioner is presented. Some useful properties of the DSS preconditioned saddle point matrix are studied. Finally, numerical experiments of a model Navier–Stokes problem are presented to show the effectiveness of the proposed preconditioners.

Shift-splitting preconditioners for saddle point problems

In this paper, we first present a shift-splitting preconditioner for saddle point problems. The preconditioner is based on a shift-splitting of the saddle point matrix, resulting in an unconditional convergent fixed-point iteration. Based on the idea of the splitting, we further propose a local shift-splitting preconditioner. Some properties of the local shift-splitting preconditioned matrix are studied. These preconditioners extend those studied by Bai, Yin and Su for solving non-Hermitian positive definite linear systems (Bai et al., 2006). Finally, numerical experiments of a model Stokes problem are presented to show the effectiveness of the proposed preconditioners.

A modified dimensional split preconditioner for generalized saddle point problems

For large and sparse saddle point linear systems arising from 2D linearized Navier–Stokes equations, Benzi and Guo recently studied a dimensional split (DS) preconditioner (Appl. Numer. Math. 61 (2011) 66–76). By further applying it to generalized saddle point problems, in this paper we present a modified dimensional split (MDS) preconditioner. This new preconditioner is based on a splitting of the generalized saddle point matrix, resulting in an unconditional convergent fixed-point iteration. The basic iteration is accelerated by a Krylov subspace method like restarted GMRES. The implementation of the MDS preconditioner is discussed and a similar case is also analyzed. Finally, numerical experiments of a model Navier–Stokes problem are presented to illustrate the effectiveness of the MDS preconditioner.

Higher Order Aitken Extrapolation with Application to Converging and Diverging Gauss-Seidel Iterations

In this paper, Aitken’s extrapolation normally applied to convergent fixed point iteration is extended to extrapolate the solution of a divergent iteration. In addition, higher order Aitken extrapolation is introduced that enables successive decomposition of high Eigen values of the iteration matrix to enable convergence. While extrapolation of a convergent fixed point iteration using a geometric series sum is a known form of Aitken acceleration, it is shown that in this paper, the same formula can be used to estimate the solution of sets of linear equations from diverging Gauss-Seidel iterations. In both convergent and divergent iterations, the ratios of differences among the consecutive values of iteration eventually form a convergent (divergent) series with a factor equal to the largest Eigen value of the iteration matrix. Higher order Aitken extrapolation is shown to eliminate the influence of dominant Eigen values of the iteration matrix in successive order until the iteration is determined by the lowest possible Eigen values. For the convergent part of the Gauss-Seidel iteration, further acceleration is made possible by coupling of the extrapolation technique with the successive over relaxation (SOR) method. Application examples from both convergent and divergent iterations have been provided. Coupling of the extrapolation with the SOR technique is also illustrated for a steady state two dimensional heat flow problem which was solved using MATLAB programming.

Convergence of the Iterative Rational Krylov Algorithm

The iterative rational Krylov algorithm (IRKA) of Gugercin et al. (2008) [8] is an interpolatory model reduction approach to the optimal H2 approximation problem. Even though the method has been illustrated to show rapid convergence in various examples, a proof of convergence has not been provided yet. In this note, we show that in the case of state-space-symmetric systems, IRKA is a locally convergent fixed-point iteration to a local minimum of the underlying H2 approximation problem.

A dimensional split preconditioner for Stokes and linearized Navier–Stokes equations

In this paper we introduce a new preconditioner for linear systems of saddle point type arising from the numerical solution of the Navier–Stokes equations. Our approach is based on a dimensional splitting of the problem along the components of the velocity field, resulting in a convergent fixed-point iteration. The basic iteration is accelerated by a Krylov subspace method like restarted GMRES. The corresponding preconditioner requires at each iteration the solution of a set of discrete scalar elliptic equations, one for each component of the velocity field. Numerical experiments illustrating the convergence behavior for different finite element discretizations of Stokes and Oseen problems are included.

Spectral renormalization method for computing self-localized solutions to nonlinear systems

A new numerical scheme for computing self-localized states--or solitons--of nonlinear waveguides is proposed. The idea behind the method is to transform the underlying equation governing the soliton, such as a nonlinear Schrödinger-type equation, into Fourier space and determine a nonlinear nonlocal integral equation coupled to an algebraic equation. The coupling prevents the numerical scheme from diverging. The nonlinear guided mode is then determined from a convergent fixed point iteration scheme. This spectral renormalization method can find wide applications in nonlinear optics and related fields such as Bose-Einstein condensation and fluid mechanics.

Two Reconfigurable Flight-Control Design Methods: Robust Servomechanism and Control Allocation

Two methods for control system reconfiguration have been investigated. The first method is a robust servomechanism control approach (optimal tracking problem) that is a generalization of the classical proportional-plus-integral control to multiple input-multiple output systems. The second method is a control-allocation approach based on a quadratic programming formulation. A globally convergent fixed-point iteration algorithm has been developed to make onboard implementation of this method feasible. These methods have been applied to reconfigurable entry flight control design for the X-33 vehicle. Examples presented demonstrate simultaneous tracking of angle-of-attack and roll angle commands during failures of the fight body flap actuator. Although simulations demonstrate success of the first method in most cases, the control-allocation method appears to provide uniformly better performance in all cases.