Generalized Conjugate Gradient Method Research Articles

A generalized conjugate gradient method for eigenvalue problems

A generalized conjugate gradient method is proposed to solve eigenvalue problems. This method is designed bycombining the dumping block inverse power scheme and the subspace projection method. Furthermore, based on theproperties of the proposed method, a series of optimization techniques are developed toimprove the stability, computing efficiency and scalability. We also introducea computing package GCGE (generalized conjugate gradient eigensolver)which is developed based on the proposed method here.Some numerical examples are provided to validate the stability, computing efficiency and scalabilityof the method in this paper. The corresponding computing package can be downloaded from the website <uri xmlns:xlink=http://www.w3.org/1999/xlink xlink:href=http://https://github.com/pase2017/GCGE-1.0>https://github.com/pase2017/GCGE-1.0</uri>

A parallel generalized conjugate gradient method for large scale eigenvalue problems

Based on damping blocked inverse power method, a type of generalized parallel conjugate gradient method is proposed for large scale eigenvalue problems. Techniques for orthogonalization and computing Rayleigh-Ritz problems are introduced to improve the stability, efficiency and scalability. Furthermore, a computing package is built based on the proposed method here. Some numerical tests are provided to validate the stability, efficiency and scalability of the method in this paper. The corresponding computing package can be downloaded from the web site: https://github.com/pase2017/GCGE-1.0 .

Generalized Conjugate Gradient Methods for ℓ1 Regularized Convex Quadratic Programming with Finite Convergence

The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper, we propose some generalized CG (GCG) methods for solving the ℓ1-regularized (possibly not strongly) convex QP that terminate at an optimal solution in a finite number of iterations. At each iteration, our methods first identify a face of an orthant and then either perform an exact line search along the direction of the negative projected minimum-norm subgradient of the objective function or execute a CG subroutine that conducts a sequence of CG iterations until a CG iterate crosses the boundary of this face or an approximate minimizer of over this face or a subface is found. We determine which type of step should be taken by comparing the magnitude of some components of the minimum-norm subgradient of the objective function to that of its rest components. Our analysis on finite convergence of these methods makes use of an error bound result and some key properties of the aforementioned exact line search and the CG subroutine. We also show that the proposed methods are capable of finding an approximate solution of the problem by allowing some inexactness on the execution of the CG subroutine. The overall arithmetic operation cost of our GCG methods for finding an ϵ-optimal solution depends on ϵ in O(log(1/ϵ)), which is superior to the accelerated proximal gradient method (Beck and Teboulle [Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1):183–202], Nesterov [Nesterov Yu (2013) Gradient methods for minimizing composite functions. Math. Program. 140(1):125–161]) that depends on ϵ in [Formula: see text]. In addition, our GCG methods can be extended straightforwardly to solve box-constrained convex QP with finite convergence. Numerical results demonstrate that our methods are very favorable for solving ill-conditioned problems.

Weighted least squares design of 2-D FIR filters using a matrix-based generalized conjugate gradient method

One of the important issues in the weighted least-squares (WLS) design of two-dimensional (2-D) finite impulse response (FIR) filters is the computational complexity of the design algorithms. This paper presents a matrix formulation of the design problem and derives a matrix-based generalized conjugate gradient (GCG) algorithm. By defining a linear operator acting on the coefficient matrix of the filter, the optimality condition of the design problem is expressed as a linear operator equation. By proving the boundedness, self-adjointness and positive-definiteness of the linear operator in the Hilbert space of the coefficient matrix, a GCG algorithm designated for the Hilbert matrix space is used to solve the linear operator equation. The convergence of the proposed algorithm in finite steps is established and its high computational efficiency is demonstrated by design examples and comparison with some existing methods.

06/01615 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations: Chen, G. S. and Sheu, R. D. Progress in Nuclear Energy, 2004, 45, (1), 11–23

06/01615 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations: Chen, G. S. and Sheu, R. D. Progress in Nuclear Energy, 2004, 45, (1), 11–23

05/02020 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations

05/02020 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations

A Class of Nested Iteration Schemes for Linear Systems with a Coefficient Matrix with a Dominant Positive Definite Symmetric Part

We present a class of nested iteration schemes for solving large sparse systems of linear equations with a coefficient matrix with a dominant symmetric positive definite part. These new schemes are actually inner/outer iterations, which employ the classical conjugate gradient method as inner iteration to approximate each outer iterate, while each outer iteration is induced by a convergent and symmetric positive definite splitting of the coefficient matrix. Convergence properties of the new schemes are studied in depth, possible choices of the inner iteration steps are discussed in detail, and numerical examples from the finite-difference discretization of a second-order partial differential equation are used to further examine the effectiveness and robustness of the new schemes over GMRES and its preconditioned variant. Also, we show that the new schemes are, at least, comparable to the variable-step generalized conjugate gradient method and its preconditioned variant.

SIMULATIONS OF THE POISSON TYPE PARTIAL DIFFERENTIAL EQUATIONS WHERE THE NODES ARE BOUND BY THE KNIGHT'S MOVES

International Journal of Computational Engineering ScienceVol. 05, No. 01, pp. 25-39 (2004) No AccessSIMULATIONS OF THE POISSON TYPE PARTIAL DIFFERENTIAL EQUATIONS WHERE THE NODES ARE BOUND BY THE KNIGHT'S MOVESRIO HIROWATI SHARIFFUDIN and ITHNIN ABDUL JALILRIO HIROWATI SHARIFFUDINInstitute of Mathematical Sciences, Faculty of Science, University of Malaya, 50603 Kuala Lumpur, Malaysia Search for more papers by this author and ITHNIN ABDUL JALILDepartment of Physics, Faculty of Science, University of Malaya, 50603 Kuala Lumpur, Malaysia Search for more papers by this author https://doi.org/10.1142/S1465876304002241Cited by:0 PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail AbstractIt is very much desired to have subsystems resulting from finite difference modeling of partial differential equations where the matrices are block diagonal as the work required to solve such subsystems is reduced greatly. The simulation nodes in the red/black order and bound by the usual five-point finite difference molecule have their coefficient matrix with such pattern. In this paper, we attempt to have such subsystems for the coefficient matrix for the nodes bound by the nine-point molecule. This can be accomplished by the knight's move. A comparison of the performances between this model and the usual nine-point model is given. We note that the coefficient matrix favors the acceleration of the basic iterative methods.Keywords:Hamiltonian Circuited SimulationsPoisson's EquationsConjugate Gradient SimulationsKnight's Move Model References F. Harary , Graph Theory ( Addison Wesley , New York , 1971 ) . Google ScholarM. R. Hestenes and E. Steifel, Journal of Research of the Nat. Bur. of Stds. 49, 409 (1952). Crossref, Google ScholarD. F. Shanno, Math. of Operations Research 3919780, 244 (1977). Google Scholar E. M. L. Beale , A Derivation of Conjugate Gradients in Numerical Methods for Monlinear Optimization , ed. F. A. Lootsman ( Academic Press , London , 1972 ) . Google ScholarM. J. D. Powell, Math. Prog. 12, 241 (1977). Crossref, Google ScholarE. Polak and G. Riebiere, Revue Francaise Inform. Rech. Operation 16-R1, 35 (1965). Google ScholarA. G. Buckley, Math. Prog. 15, 200 (1978). Crossref, Google ScholarD. Le, Math. Prog. 32, 41 (1985). Crossref, Google Scholar J. W. Daniel , The conjugate Gradient Method for Linear and Nonlinear Operator Equations doctoral thesis: Stanford University . Google ScholarJ. K. Reid, On the method of Conjugate Gradients for The Solution Of Large Sparse Systems of Linear Equations, Proc. Conf. Large Sets of Linear Equations pp. 231–254. Google Scholar R. Bartels and J. W. Daniel , A Conjugate Approach to Nonlinear Elliptic Boundary Value Problem in Irregular Regions , Proc. Conf. Numerical Solution of Differential Equations ( Dundee , 1974 ) . Google Scholar Axelson, O., 1974. On Preconditioning and Convergence Acceleration in Sparse Matrix Problems. CERN 74-10: European Organization for Nuclear research, Data Handling Division . Google Scholar O'Leary D.P.,1975. Hybrid Conjugate Gradient Algorithms, doctoral thesis: Stanford University Pato Alto, California . Google Scholar R. Chandra , Conjugate Gradient Methods for Partial Differential Equations , Research Report 129 ( Connecticut, Department of Comp. Sc., University of New Haven , 1978 ) . Google ScholarP. Concus and G. H. Golub, SIAM J. Num. Anal. 10, 1103 (1973). Crossref, Google Scholar P. Concus and G. H. Golub , A generalized Conjugate Gradient Method For Nonsymmetric Systems of Linear Equations ( Comp. Sc. Dept., Stanford University , 1976 ) . Crossref, Google ScholarD. J. Evans, J.I.M.A. 4, 295 (1968). Google ScholarJ. A. Meijerink and H. A. van der Vorst, Math. Comp. 31, 148 (1977). Google ScholarR. Hirowati Shariffudin and A. R. Abdullah, International Journal of Computer Mathematics 76, (2001). Google ScholarR. Hirowati Shariffudin and A. R. Abdullah, Applied Mathematics Letters 14, 423 (2001). Google Scholar FiguresReferencesRelatedDetails Recommended Vol. 05, No. 01 Metrics History KeywordsHamiltonian Circuited SimulationsPoisson's EquationsConjugate Gradient SimulationsKnight's Move ModelPDF download

Application of two preconditioned generalizedconjugate gradient methods to three-dimensional neutron and photon transport equations

In this paper the preconditioned generalized conjugate gradient methods are applied to solve the linear system of equations that arise in three-dimensional neutron and photon transport equations. These generalized conjugate gradient methods are the CGS (conjugate gradient square) algorithm and the Bi-CGSTAB (bi-conjugate gradient stabilized) algorithm. Several subroutines are developed from these preconditioned generalized conjugate gradient methods for use in time-independent multi-group three-dimensional neutron and photon transport equations. These subroutines are connected to the computer program TORT. The reason for choosing the preconditioned generalized conjugate gradient methods is that these methods have good residual error control procedures during computation and have good convergence rates. We test a problem having an 8 set of matrix equations with 24255 unknowns each, in a personal computer with an AMD Athlon-XP1600+ central processing unit (CPU) and using Mandrake Linux 6.3 as the operating system. The point-wise incomplete LU factorization (ILU) and modified point-wise incomplete LU factorization (MILU) are the preconditioning techniques used in the test problem. We find that the preconditioned CGS and Bi-CGSTAB methods with the preconditioner ILU are more efficient than with the preconditioner MILU in the test problem. The numerical solution of flux by the preconditioned Bi-CGSTAB and CGS methods produces the same results as those obtained bythe successive over relaxation method (SOR) in the TORT program which is usually used for the calculation of radiation shielding in nuclear power plant and nuclear spent fuel storage.

USE OF GCG METHODS FOR THE EFFICIENT SOLUTION OF MATRIX PROBLEMS ARISING FROM THE FVM FORMULATION OF RADIATIVE TRANSFER

Preconditioned generalized conjugate gradient (GCG) iterative methods are applied to the solution of large, sparse, and unsymmetric linear algebraic equations resulting from the application of the finite-volume method to the problem of radiative heat transfer in an absorbing, emitting, and scattering gray medium, with the boundary surfaces reflecting radiation in both diffuse and specular regimes. The governing radiative transfer equation, which is a complicated integro-differential equation, has been discretized using the S N finite-volume method (FVM). Different variants of GCG methods have been tested on a problem of 2-D radiation in a cylinder, and efficiencies of the methods have been compared. Numerical results indicate that preconditioning suggested in the article dramatically improves the performance of the GCG methods. Results on test problems based on S 8 FVM agree well with exact results reported in the literature.

Numerical experiments to optimize the use of (I)LU preconditioning in the iterative linear solver package LINSOL

In the iterative linear solver package LINSOL several generalized conjugate gradient (CG) methods (or, briefly, CG-type methods) with quite different properties are implemented. With these methods polyalgorithms with automatic method switching are constructed. The “emergency exit” that is taken in the worst case is the ATPRES method (which is very robust, but very slow). In this paper we investigate if (I)LU preconditioning were a better emergency exit and how the drop tolerance for small elements in ILU affects the convergence behavior. The answer will be: it depends.

A framework for generalized conjugate gradient methods—with special emphasis on contributions by Rüdiger Weiss

We discuss a general framework for generalized conjugate gradient methods, that is, iterative methods (for solving linear systems) that are based on generating residuals that are formally orthogonal to each other with respect to some true or formal inner product. This includes methods that generate residuals that are minimal with respect to some norm based on an inner product. A similar, even more general framework was introduced and extensively discussed by Weiss in his work. Weiss also emphasized in his work the relationship between certain pairs of orthogonal and minimal residual methods, where the results of the second method can be generated from those of the first method by applying the minimal residual smoothing process. Examples of such pairs are the conjugate gradient (CG) and the conjugate residual (CR) methods, the full orthogonalization method (FOM) and the generalized minimal residual (GMR es) method, as well as the CGNE and CGNR versions of applying CG to the normal equations.

Numerical experiments with preconditioning by gram matrix approximation for non-linear elliptic equations

The paper analyzes the numerical performances of the Gram matrix approximations preconditioning for solving non-linear elliptic equations. The two test problems are non-linear 2-D elliptic equations which describe: (1) the Plateau problem and (2) the general pseudohomogeneous model of the catalytic chemical reactor. The standard FEM with piecewise linear test and trial functions is used for discretization. The discrete approximations are solved with a double iterative process using the Newton method as outer iteration and the preconditioned generalized conjugate gradient methods (CGS and GMRES) as inner iteration. The Gram matrix approximations consist in replacing the exact solution of the equation with the preconditioner by few iterations of an appropriate iterative scheme. Two iterative algorithms are tested: incomplete Cholesky and multigrid. Numerical experiments indicate that preconditioners improve the convergence properties of the algorithms for both test problems. At the second test problem the numerical performances deteriorate at relatively high values of the Pe numbers.

一般化共役勾配法(GCG法)の有限要素法への適用に関する研究 : その4 特異性判定修正式と平面・立体骨組の悪条件下数値実験

一般化共役勾配法(GCG法)の有限要素法への適用に関する研究 : その4 特異性判定修正式と平面・立体骨組の悪条件下数値実験

Global convergence property ofs-dependent GFR conjugate gradient method

AnS-dependent GFR method and a generalized conjugate gradient method are designed. Two estimated values of upper bounds for the ratio of βk to β k FR are given. The two convergence theorems are proved. The global convergent properties ofs-dependent GFR method are obtained adopting several choice strategies of steplength and some results in recent literature are extended.

Comparison of preconditioned generalized conjugate gradient methods to two-dimensional neutron and photon transport equation

We apply and compare the preconditioned generalized conjugate gradient methods to solve the linear system equation that arises in the two-dimensional neutron and photon transport equation in this paper. Several subroutines are developed on the basis of preconditioned generalized conjugate gradient methods for time-independent, two-dimensional neutron and photon transport equation in the transport theory. These generalized conjugate gradient methods are used: TFQMR (transpose free quasi-minimal residual algorithm) CGS (conjugate gradient square algorithm), Bi-CGSTAB (bi-conjugate gradient stabilized algorithm) and QMRCGSTAB (quasi-minimal residual variant of bi-conjugate gradient stabilized algorithm). These subroutines are connected to computer program DORT. Several problems are tested on a personal computer with Intel Pentium CPU. The reasons to choose the generalized conjugate gradient methods are that the methods have better residual (equivalent to error) control procedures in the computation and have better convergent rate. The pointwise incomplete LU factorization ILU, modified pointwise incomplete LU factorization MILU, block incomplete factorization BILU and modified blockwise incomplete LU factorization MBILU are the preconditioning techniques used in the several testing problems. In Bi-CGSTAB, CGS, TFQMR and QMRCGSTAB method, we find that either CGS or Bi-CGSTAB method combined with preconditioner MBILU is the most efficient algorithm in these methods in the several testing problems. The numerical solution of flux by preconditioned CGS and Bi-CGSTAB methods has the same result as those from Cray computer, obtained by either the point successive relaxation method or the line successive relaxation method combined with Gaussian elimination.

Application of preconditioned transpose-free quasi-minimal residual method for two-group reactor kinetics

Two preconditioned transpose-free quasi-minimal residual methods ( TFQMR) (Freund, SIAM J. Sci. Stat. Comput. 14, 470 1993) and quasi-minimal residual variant of the biconjugate gradient stabilized algorithm ( QMRCGSTAB) (Chan et al., SIAM J. Sci. Stat. Comput. IS. 338 1994) are applied to solve the non-symmetric linear systems of equations which are derived from the time dependent two-dimensional two-energy-group neutron diffusion equations by finite difference approximation. We compare the TFQMR and QMRCGSTAB methods with the other popular method such as the generalized minimal residual method ( GMRES), the conjugate gradient square method ( CGS), and biconjugate gradient stabilized algorithm ( Bi-CGSTAB). In order to accelerate the TFQMR and QMRCGSTAB we use the preconditioning technique. Two of the preconditioners are based on pointwise incomplete factorization: the incomplete factorization ( ILU) and the modified incomplete factorization ( MILU). Another two based on the block tridiagnal structure of the coefficient matrix are blockwise and modified blockwise incomplete factorizations, BILU and MBILU which are suitable for the system of partial differential equations such as two-energy-group neutron diffusion equations. Finally, the last two are the alternating-direction implicit ( ADI) and block successive overrelaxation ( BSOR) preconditioners which are derived from the basic iterative schemes. Comparisons are made by these methods combined with different preconditioners to solve a sequence of time steps reactor transient problems. Numerical results indicate that the preconditioner significantly affects the convergent rate TFQMR and QMRCGSTAB methods in three typical reactor kinetics test problems. Numerical experiments indicate that preconditioned QMRCGSTAB with the preconditioner MBILU requires fewer iterations than other methods in the three typical reactor kinetics test problems. Moreover, numerical results indicate that a good preconditioner can significantly improve the total iteration number (i.e. rate of convergence) of these generalized conjugate gradient methods, TFQMR, QMRCGSTAB, CGS, Bi-CGSTAB and GMRES. For preconditioners and MBILU and BILU, we find that all of the eigenvalues of preconditioned matrix are more clustered about 1 than the eigenvalues of other preconditioners in a typical reactor kinetics test problem. Such eigenvalue distribution is very favorable or the rate of convergence of these generalized conjugate gradient methods.

Induced-polarization modelling using complex electrical conductivities

A numerical technique for induced-polarization (IP) modelling of arbitrary 2-D or 3-D distributions of electrical conductivity is developed. This technique is based on the premise that, due to polarization effects, the conductivity is complex. Complex finite-difference equations are obtained for the complex potential by an elemental volume discretization. The accuracy of the discretization is improved by the application of asymptotic boundary conditions and singularity removal. The resulting linear set of complex equations is symmetric. It is solved by a generalized conjugate gradient method which is speeded up by pre-conditioning techniques. A compact storage scheme fully utilizes the sparsity and the symmetry of the matrix. The complex arithmetic ensures that the resulting values of amplitude and phase angle of apparent resistivity can be directly compared with the measured quantities of a spectral IP survey. Models of a layered half-space and of a buried cube are chosen to demonstrate the good agreement of the apparent resistivity curves with the results from other numerical techniques.

ITERATIVE SOLUTION OF INCOMPRESSIBLE NAVIER-STOKES EQUATIONS ON THE MEIKO COMPUTING SURFACE

The numerical discretization of the equations governing fluid flow results in coupled, quasi-linear and non-symmetric systems. Various approaches exist for resolving the non-linearity and couplings. During each non-linear iteration, nominally linear systems are solved for each of the flow variables. Line relaxation techniques are traditionally employed for solving these systems. However, they could be very expensive for realistic applications and present serious synchronization problems in a distributed memory parallel environment. In this paper the discrete linear systems are solved using the generalized conjugate gradient method of Concus and Golub. The performance of this algorithm is compared with the line Gauss-Seidel algorithm for laminar recirculatory flow in uni- and multiprocessor environments. The uniprocessor performances of these algorithms are also compared with that of a popular iterative solver for non-symmetric systems (the GMRES algorithm).

An engineering approach to generalized conjugate gradient methods and beyond

This is an elementary introductory paper (rather a tutorial) for generalized conjugate gradient (CG) methods. The emphasis is on “understanding” the approach of the now rather sophisticated CG-type methods and beyond. Practical aspects like normalization and smoothing are discussed. Then the development of different types of methods with their characteristic properties is presented. For a model of the Navier-Stokes equations that varies between strong and weak ellipticity, the methods are compared and their typical properties are demonstrated.