Iterative Methods For Linear Systems Research Articles

Bounded perturbations resilient iterative methods for linear systems and least squares problems: operator-based approaches, analysis, and performance evaluation

Bounded perturbations resilient iterative methods for linear systems and least squares problems: operator-based approaches, analysis, and performance evaluation

Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols

In this work we present a new framework for the analysis and design of randomized gossip algorithms for solving the average consensus problem. We show how classical randomized iterative methods for solving linear systems can be interpreted as gossip algorithms when applied to special systems encoding the underlying network and explain in detail their decentralized nature. Our general framework recovers a comprehensive array of well-known gossip algorithms as special cases, including the pairwise randomized gossip algorithm and path averaging gossip, and allows for the development of provably faster variants. The flexibility of the new approach enables the design of a number of new specific gossip methods. For instance, we propose and analyze novel block and the first provably accelerated randomized gossip protocols, and dual randomized gossip algorithms. From a numerical analysis viewpoint, our work is the first that explores in depth the decentralized nature of randomized iterative methods for linear systems and proposes them as methods for solving the average consensus problem. We evaluate the performance of the proposed gossip protocols by performing extensive experimental testing on typical wireless network topologies.

New iterative methods for dense linear systems

Solving dense linear systems of equations is quite time consuming and requires an efficient parallel implementation on powerful supercomputers. Du, Zheng and Wang presented some new iterative methods for linear systems [Journal of Applied Analysis and Computation, 2011, 1(3): 351-360]. This paper shows that their methods are suitable for solving dense linear system of equations, compared with the classical Jacobi and Gauss-Seidel iterative methods.

Some Hyperbolic Iterative Methods for Linear Systems

The indefinite inner product defined by J=diagj1,…,jn, jk∈−1,+1, arises frequently in some applications, such as the theory of relativity and the research of the polarized light. This indefinite scalar product is referred to as hyperbolic inner product. In this paper, we introduce three indefinite iterative methods: indefinite Arnoldi’s method, indefinite Lanczos method (ILM), and indefinite full orthogonalization method (IFOM). The indefinite Arnoldi’s method is introduced as a process that constructs a J-orthonormal basis for the nondegenerated Krylov subspace. The ILM method is introduced as a special case of the indefinite Arnoldi’s method for J-Hermitian matrices. IFOM is mentioned as a process for solving linear systems of equations with J-Hermitian coefficient matrices. Finally, by providing numerical examples, the FOM, IFOM, and ILM processes have been compared with each other in terms of the required time for solving linear systems and also from the point of the number of iterations.

Accuracy Improvement of the Block Product-type Iterative Methods for Linear Systems with Multiple Right-Hand Sides

Accuracy Improvement of the Block Product-type Iterative Methods for Linear Systems with Multiple Right-Hand Sides

Estimation of physical parameters under location uncertainty using an ensemble2–expectation–maximization algorithm

Estimating the parameters of geophysical dynamic models is an important task in the data assimilation (DA) techniques used to forecast initialization and reanalysis. In the past, most parameter estimation strategies were derived by state augmentation, yielding algorithms that are easy to implement but may exhibit convergence difficulties. The expectation–maximization (EM) algorithm is considered advantageous because it employs two iterative steps to estimate the model state and the model parameter separately. In this work, we propose a novel ensemble formulation of the maximization step in EM that allows a direct optimal estimation of the physical parameters using iterative methods for linear systems. This departs from current EM formulations that are only capable of dealing with additive model error structures. This contribution shows how the EM technique can be used for dynamics identification problems with the model error parameterized as an arbitrary complex form. The proposed technique is used here for the identification of stochastic subgrid terms that account for processes unresolved by a geophysical fluid model. This method, together with the augmented state technique, is evaluated to estimate such subgrid terms through high‐resolution data. As compared to the augmented state technique, our method is shown to yield considerably more accurate parameters. In addition, in terms of prediction capacity, it leads to a smaller generalization error as a result of the overfitting of the trained model on the presented data and eventually better forecasts.

On Optimal Parameter Not Only for the SOR Method

The Jacobi, Gauss-Seidel and SOR methods belong to the class of simple iterative methods for linear systems. Because of the parameter , the SOR method is more effective than the Gauss-Seidel method. Here, a new approach to the simple iterative methods is proposed. A new parameter q can be introduced to every simple iterative method. Then, if a matrix of a system is positive definite and the parameter q is sufficiently large, the method is convergent. The original Jacobi method is convergent only if the matrix is diagonally dominated, while the Jacobi method with the parameter q is convergent for every positive definite matrix. The optimality criterion for the choice of the parameter q is given, and thus, interesting results for the Jacobi, Richardson and Gauss-Seidel methods are obtained. The Gauss-Seidel method with the parameter q, in a sense, is equivalent to the SOR method. From the formula for the optimal value of q results the formula for optimal value of . Up to present, this formula was known only in special cases. Practical useful approximate formula for optimal value  is also given. The influence of the parameter q on the speed of convergence of the simple iterative methods is shown in a numerical example. Numerical experiments confirm: for very large scale systems the speed of convergence of the SOR method with optimal or approximate parameter  is near the same (in some cases better) as the speed of convergence of the conjugate gradients method.

Iterative method for estimating the robust domains of attraction of non-linear systems: Application to cancer chemotherapy model with parametric uncertainties

In this paper, we present an iterative procedure method for estimating the robust domains of attraction of non-linear systems. This method is based on the approximation of the uncertain non-linear system with a parameters-dependent Convex Difference Inclusions (CDI) system and the classical iterative methods for linear systems, which are introduced in this paper. A robust one-step operator computing a sequence of convex sets is derived, and the polyhedral case is discussed. An algorithm summarizing the iterative procedure based on the robust one-step operator is given, which is the theoretical contribution of this paper. This method is applied to cancer chemotherapy model considering parametric uncertainties and it is shown that drastic reduction of the robust domain of attraction of the cancer chemotherapy model has happened and this is caused by the presence of parametric uncertainties. It is also proved that the aggressive chemotherapy is not the effective treatment for all the patients.

Iteration Methods for Linear Systems with Positive Definite Matrix

Two classical first order iteration methods, Richardson iteration and HSS iteration for linear systems with positive definite matrix, are demonstrated. Theoretical analyses and computational results show that the HSS iteration has the advantages of fast convergence speed, high computation efficiency, and without requirement of symmetry.

A convergence analysis of SOR iterative methods for linear systems with weak H-matrices

Abstract It is well known that SOR iterative methods are convergent for linear systems, whose coefficient matrices are strictly or irreducibly diagonally dominant matrices and strong H-matrices (whose comparison matrices are nonsingular M-matrices). However, the same can not be true in case of those iterative methods for linear systems with weak H-matrices (whose comparison matrices are singular M-matrices). This paper proposes some necessary and sufficient conditions such that SOR iterative methods are convergent for linear systems with weak H-matrices. Furthermore, some numerical examples are given to demonstrate the convergence results obtained in this paper.

On the robustness of numerical algorithms for linear systems and signal processing in finite precision arithmetic

SummaryThis paper proposes a model of the perturbations resulting from finite precision arithmetic implementation of numerical methods. Iterative methods for linear systems are represented as control systems, and the numerical errors caused by finite precision are represented as a multiplicative uncertainty. This representation makes it possible to use results from robust control theory to provide stability criteria, which, in turn, imply convergence, under finite precision arithmetic, of algorithms considered. Numerical examples are taken from several fields in order to illustrate the theoretical results. Copyright © 2015 John Wiley & Sons, Ltd.

Convergence on Gauss-Seidel iterative methods for linear systems with general H-matrices

It is well known that as a famous type of iterative methods in numerical linear algebra, Gauss-Seidel iterative methods are convergent for linear systems with strictly or irreducibly diagonally dominant matrices, invertible H−matrices (generalized strictly diagonally dominant matrices) and Hermitian positive definite matrices. But, the same is not necessarily true for linear systems with non-strictly diagonally dominant matrices and general H−matrices. This paper firstly proposes some necessary and sufficient conditions for convergence on Gauss-Seidel iterative methods to establish several new theoretical results on linear systems with nonstrictly diagonally dominant matrices and general H−matrices. Then, the convergence results on preconditioned Gauss-Seidel (PGS) iterative methods for general H−matrices are presented. Finally, some numerical examples are given to demonstrate the results obtained in this paper.

Randomized Iterative Methods for Linear Systems

We develop a novel, fundamental, and surprisingly simple randomized iterative method for solving consistent linear systems. Our method has six different but equivalent interpretations: sketch-and-project, constrain-and-approximate, random intersect, random linear solve, random update, and random fixed point. By varying its two parameters---a positive definite matrix (defining geometry), and a random matrix (sampled in an independent and identically distributed fashion in each iteration)---we recover a comprehensive array of well-known algorithms as special cases, including the randomized Kaczmarz method, randomized Newton method, randomized coordinate descent method, and random Gaussian pursuit. We naturally also obtain variants of all these methods using blocks and importance sampling. However, our method allows for a much wider selection of these two parameters, which leads to a number of new specific methods. We prove exponential convergence of the expected norm of the error in a single theorem, from which existing complexity results for known variants can be obtained. However, we also give an exact formula for the evolution of the expected iterates, which allows us to give lower bounds on the convergence rate.

On parallel multisplitting block iterative methods for linear systems arising in the numerical solution of Euler equations

The paper studies the convergence of some parallel multisplitting block iterative methods for the solution of linear systems arising in the numerical solution of Euler equations. Some sufficient conditions for convergence are proposed. As special cases the convergence of the parallel block generalized AOR (BGAOR), the parallel block AOR (BAOR), the parallel block generalized SOR (BGSOR), the parallel block SOR (BSOR), the extrapolated parallel BAOR and the extrapolated parallel BSOR methods are presented. Furthermore, the convergence of the parallel block iterative methods for linear systems with special block tridiagonal matrices arising in the numerical solution of Euler equations are discussed. Finally, some examples are given to demonstrate the convergence results obtained in this paper.

Thermal Floorplan Base on Conjugate Gradient Solver in HotSpot

In this paper, we introduce an effective iterative method to solve the thermal linear system in HotSpot thermal floorplan, the iterative Conjugate Gradient Method is suitable to solve the traditional sparse matrix linear equations. We define a class of dummy sparse linear systems in iterative thermal floorplan algorithm, the iterative methods for linear system can be extended to apply to other iterative framework algorithm. We apply the conjugate gradient method to solve the thermal model in floorplan of VLSI physical design. The experiments' result shows that thermal floorplan using Conjugate gradient method is effective. The running time of our incremental conjugate gradient thermal solver with Jocabi Precondition is approximate 0.59 comparing with LU decomposition method.

Convergence Analysis of Preconditioned SSOR Iterative Method for Linear Systems

For solving the linear system, different preconditioned iterative methods have been proposed by many authors. In this paper, we present preconditioned SSOR iterative method for solving the linear systems. Meanwhile, we apply the preconditioner to H-matrix and obtain the convergence result. Finally, a numerical example is also given to illustrate our results.

A control Liapunov function approach to generalized and regularized descent methods for zero finding1

This paper revisits a class of recently proposed so-called invariant manifold methods for zero finding of ill-posed problems, showing that they can be profitably viewed as homotopy methods, in which the homotopy parameter is interpreted as a learning parameter. Moreover, it is shown that the choice of this learning parameter can be made in a natural manner from a control Liapunov function approach CLF. From this viewpoint, maintaining manifold invariance is equivalent to ensuring that the CLF satisfies a certain ordinary differential equation, involving the learning parameter, that allows an estimate of rate of convergence. In order to illustrate this approach, algorithms recently proposed using the invariant manifold approach, are rederived, via CLFs, in a unified manner. Adaptive regularization parameters for solving linear algebraic ill-posed problems were also proposed. This paper also shows that the discretizations of the ODEs to solve the zero finding problem, as well as the different adaptive choices of the regularization parameter, yield iterative methods for linear systems, which are also derived using the Liapunov optimizing control LOC method.

A New Iterative Method for Linear Systems from XFEM

The extend finite element method (XFEM) is popular in structural mechanics when dealing with the problem of the cracked domains. XFEM ends up with a linear system. However, XFEM usually leads to nonsymmetric and ill-conditioned stiff matrix. In this paper, we take the linear elastostatics governing equations as the model problem. We propose a new iterative method to solve the linear equations. Here we separate two variablesUand Enr, so that we change the problems into solving the smaller scale equations iteratively. The new program can be easily applied. Finally, numerical examples show that the proposed method is more efficient than common methods; we compare theL2-error and the CPU time in whole process. Furthermore, the new XFEM can be applied and optimized in many other problems.

Geometric interpretation of several classical iterative methods for linear system of equations and diverse relaxation parameter of the SOR method

Two kinds of iterative methods are designed to solve the linear system of equations, we obtain a new interpretation in terms of a geometric concept. Therefore, we have a better insight into the essence of the iterative methods and provide a reference for further study and design. Finally, a new iterative method is designed named as the diverse relaxation parameter of the SOR method which, in particular, demonstrates the geometric characteristics. Many examples prove that the method is quite effective.

A generalization of [formula omitted]-step variants of gradient methods

The s-step methods were proposed by Chronopoulos to gain efficiency in parallel programming of iterative methods for linear systems. They are variants of classical iterative methods based on the construction of a Krylov subspace basis on each iteration. These s-step methods were inferred from algorithms like the Conjugate Gradient, Generalized Conjugate Residual or the Minimal Residual. They converge for all symmetric, nonsymmetric definite and some nonsymmetric indefinite matrices. In this paper, we introduce an s-step variant of a General Orthogonalization Algorithm, that is, a generalization of s-step variants of gradient methods. We prove convergence and obtain error estimates. We also describe an Orthomin variant, together with a convergence theorem. From this we derive the well known s-step methods as particular cases, and some which are newfound to our knowledge. This provides a unified framework to derive and study s-step methods. Some of the methods obtained are convergent for every nonsingular matrix. Finally, we give some numerical results for the new proposed methods, showing that the parallel implementation of these overcomes the original ones.