Preconditioned Conjugate Gradient Algorithm Research Articles

Application of Diagonally Perturbed Incomplete Factorization Preconditioned Conjugate Gradient Algorithms for Edge Finite-Element Analysis of Helmholtz Equations

The diagonally perturbed incomplete factorization preconditioning scheme is applied to the conjugate gradient (CG) method for solving a large system of linear equations resulting from the use of edge-based finite-element method (FEM). This scheme contains more global information about the coefficient matrix when compared with banded-matrix schemes. The efficient implementation of this preconditioned CG (PCG) algorithm is described in detail for complex coefficient matrix equation. On several electromagnetic problems the PCG approach converges in CPU time, which is 8.6-19.5 times shorter with respect to the CG approach. By comparison with other preconditioned techniques, the results demonstrate that incomplete factorization preconditioning strategy is especially effective for CG iterative method when edge-FEM is applied to solve large-scale time-harmonic electromagnetic field problems

Analysis of a novel preconditioner for a class of p‐level lower rank extracted systems

AbstractThis paper proposes and studies the performance of a preconditioner suitable for solving a class of symmetric positive definite systems, Âx=b, which we call p‐level lower rank extracted systems (p‐level LRES), by the preconditioned conjugate gradient method. The study of these systems is motivated by the numerical approximation of integral equations with convolution kernels defined on arbitrary p‐dimensional domains. This is in contrast to p‐level Toeplitz systems which only apply to rectangular domains. The coefficient matrix, Â, is a principal submatrix of a p‐level Toeplitz matrix, A, and the preconditioner for the preconditioned conjugate gradient algorithm is provided in terms of the inverse of a p‐level circulant matrix constructed from the elements of A. The preconditioner is shown to yield clustering in the spectrum of the preconditioned matrix which leads to a substantial reduction in the computational cost of solving LRE systems. Copyright © 2006 John Wiley & Sons, Ltd.

Fast optimal wavefront reconstruction for multi-conjugate adaptive optics using the Fourier domain preconditioned conjugate gradient algorithm

We present two different implementations of the Fourier domain preconditioned conjugate gradient algorithm (FD-PCG) to efficiently solve the large structured linear systems that arise in optimal volume turbulence estimation, or tomography, for multi-conjugate adaptive optics (MCAO). We describe how to deal with several critical technical issues, including the cone coordinate transformation problem and sensor subaperture grid spacing. We also extend the FD-PCG approach to handle the deformable mirror fitting problem for MCAO.

Integrated surface–groundwater flow modeling: A free-surface overland flow boundary condition in a parallel groundwater flow model

Interactions between surface and groundwater are a key component of the hydrologic budget on the watershed scale. Models that honor these interactions are commonly based on the conductance concept that presumes a distinct interface at the land surface, separating the surface from the subsurface domain. These types of models link the subsurface and surface domains via an exchange flux that depends upon the magnitude and direction of the hydraulic gradient across the interface and a proportionality constant (a measure of the hydraulic connectivity). Because experimental evidence of such a distinct interface is often lacking in field systems, there is a need for a more general coupled modeling approach. A more general coupled model is presented that incorporates a new two-dimensional overland flow simulator into the parallel three-dimensional variably saturated subsurface flow code ParFlow [Ashby SF, Falgout RD. A parallel multigrid preconditioned conjugate gradient algorithm for groundwater flow simulations. Nucl Sci Eng 1996;124(1):145–59; Jones JE, Woodward CS. Newton–Krylov-multigrid solvers for large-scale, highly heterogeneous, variably saturated flow problems. Adv Water Resour 2001;24:763–774]. This new overland flow simulator takes the form of an upper boundary condition and is, thus, fully integrated without relying on the conductance concept. Another important advantage of this approach is the efficient parallelism incorporated into ParFlow, which is exploited by the overland flow simulator. Several verification and simulation examples are presented that focus on the two main processes of runoff production: excess infiltration and saturation. The model is shown to reproduce an analytical solution for overland flow, replicates a laboratory experiment for surface–subsurface flow and compares favorably to other commonly used hydrologic models. The influence of heterogeneity of the shallow subsurface on overland flow is also examined. The results show the propagation of uncertainty due to subsurface heterogeneity to the overland flow predictions and demonstrate the usefulness of our approach. Both the overland flow component and the coupled model are evaluated in a parallel scaling study and show to be efficient.

Analysis of Parallel Preconditioned Conjugate Gradient Algorithms

The conjugate gradient method is an iterative technique used to solve systems of linear equations. The paper analyzes the performance of parallel preconditioned conjugate gradient algorithms. First...

Variable-step preconditioned conjugate gradient method for partial symmetric eigenvalue problems

The iterative methods for partial algebraic symmetric eigenvalue problems are considered for sparse positive definite matrices which arise in approximation of 2D and 3D boundary value problems. The approach is based on subspace iterations, Rayleigh–Ritz method, and the variable-step preconditioned conjugate gradient algorithm, including algebraic multigrid and incomplete factorization. Theorems on the properties of convergence rate are presented. The efficiency of the proposed iterative processes is demonstrated by the results of numerical experiments.

Comparison of different solving strategies in multiple-trait mixed model equations with iterative algorithms

The effects of initial solution and order of observations on the solving of mixed model equations (MME) by traditional iterative algorithms-Gauss-Seidel, successive over-relaxation (SOR), and second-order Jacobi-and by preconditioned conjugate gradient algorithms-scaled conjugate gradient (SCG) and incomplete Cholesky conjugate gradient (ICCG) simulation were investigated by using data in a computer. With SOR, the use of adjusted initial solutions based on phenotype and heritability of the trait and order of the animal within the trait in the MME was effective for fast convergence. Adjustment of initial solutions was not an advantage in preconditioned conjugate gradient algorithms. Of the five algorithms, ICCG required the lowest number of iterations until convergence. However, ICCG needed the largest central processing unit (CPU) time to calculate one round of iteration. The SCG algorithm was an attractive alternative for solving MME because it required the second fewest rounds of iteration and had the shortest CPU time per round of iteration until convergence.

Effects of data structures on the solving of animal model equations by preconditioned conjugate gradient iteration

To predict breeding values in dairy cattle, preconditioned conjugate gradient (PCG) algorithm has become of interest lately as a means of solving large sets of mixed model equations (MME) . Thereby, the effects of data structure on the solving of MME by PCG algorithms were investigated by using data in a computer simulation. A large number of animals did not cause slow convergence if the data structure (e.g. number of generations and mating ratio) was fixed. The numbers of breeding animals and their progeny affected the structure of the A-inverse matrix, but selection methods did not. An increasing number of iterations seemed to be due to an imbalance of nonzero elements in the A-inverse matrix, which resulted in the imbalance of nonzero elements in the MME. The number of iterations until convergence was roughly constant for two-trait animal model when the heritability of a trait was constant regardless of the heritability of another trait, and it was the smallest when the heritabilities of both traits were about 0.5 to 0.6. Increasing absolute values of genetic and environmental correlations resulted in slow convergence. In particular, genetic correlation affected convergence strongly.

Analysis of Parallel Preconditioned Conjugate Gradient Algorithms

The conjugate gradient method is an iterative technique used to solve systems of linear equations. The paper analyzes the performance of parallel preconditioned conjugate gradient algorithms. First, a theoretical model is proposed for estimation of the complexity of PPCG method and a scalability analysis is done for three different data decomposition cases. Computational experiments are done on IBM SP4 computer and some results are presented. It is shown that theoretical predictions agree well with computational results.

Variable-step preconditioned conjugate gradient method for partial symmetric eigenvalue problems

The iterative methods for partial algebraic symmetric eigenvalue problems are considered for sparse positive definite matrices which arise in approximation of 2D and 3D boundary value problems. The approach is based on subspace iterations, Rayleigh–Ritz method, and the variable-step preconditioned conjugate gradient algorithm, including algebraic multigrid and incomplete factorization. Theorems on the properties of convergence rate are presented. The efficiency of the proposed iterative processes is demonstrated by the results of numerical experiments.

On the Complexity of the Preconditioned Conjugate Gradient Algorithm for Solving Toeplitz Systems with a Fisher--Hartwig Singularity

The Toeplitz matrix $T_n$ with generating function $f ( \omega ) = |1 - e ^{-i \omega}|^{-2d} h( \omega )$, where $d \in (-\frac{1}{2}, \frac{1}{2})\setminus \{0\}$ and $h(\omega)$ is positive, continuous on $[-\pi,\pi]$, and differentiable on $[-\pi,\pi]\setminus\{0\}$, has a Fisher--Hartwig singularity [M. E. Fisher and R. E. Hartwig (1968), Adv. Chem. Phys., 32, pp. 190--225]. The complexity of the preconditioned conjugate gradient (PCG) algorithm is known [R. H. Chan and M. Ng (1996), SIAM Rev., 38, pp. 427--482] to be $O(n\log n)$ for Toeplitz systems when $d = 0$. However, the effect on the PCG algorithm of the Fisher--Hartwig singularity in $T_n$ has not been explored in the literature. We show that the complexity of the conjugate gradient (CG) algorithm for solving $T_n x=b$ without any preconditioning grows asymptotically as $n^{1+|d|}\log (n)$. With T. Chan's optimal circulant preconditioner $C_n$ [T. Chan (1988), SIAM J. Sci. Statist. Comput., 9, pp. 766--771], the complexity of the PCG algorithm is $O(n\log^3(n))$.

Efficient iterative solvers for structural dynamics problems

Direct solvers are commonly used in implicit finite element codes for structural dynamics problems. This study explores an alternative approach to solving the resulting linear systems by using preconditioned iterative schemes such as the preconditioned conjugate gradient algorithm and its variants. Preconditioners used in this study include approximate Cholesky factorization, block Jacobi, and the symmetric Gauss–Seidel over-relaxation scheme. The effects of various preconditioners and ordering schemes on the solution time and required storage are investigated. Performance results of these iterative solver are compared against the MA57 direct solver routine of the commercial Harwell Software Library.

Accurate optical flow computation under non-uniform brightness variations

In this paper, we present a very accurate algorithm for computing optical flow with non-uniform brightness variations. The proposed algorithm is based on a generalized dynamic image model (GDIM) in conjunction with a regularization framework to cope with the problem of non-uniform brightness variations. To alleviate flow constraint errors due to image aliasing and noise, we employ a reweighted least-squares method to suppress unreliable flow constraints, thus leading to robust estimation of optical flow. In addition, a dynamic smoothness adjustment scheme is proposed to efficiently suppress the smoothness constraint in the vicinity of the motion and brightness variation discontinuities, thereby preserving motion boundaries. We also employ a constraint refinement scheme, which aims at reducing the approximation errors in the first-order differential flow equation, to refine the optical flow estimation especially for large image motions. To efficiently minimize the resulting energy function for optical flow computation, we utilize an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the large linear system. Experimental results on some synthetic and real image sequences show that the proposed algorithm compares favorably to most existing techniques reported in literature in terms of accuracy in optical flow computation with 100% density.

A globally convergent regularized ordered-subset EM algorithm for list-mode reconstruction

List-mode (LM) acquisition allows collection of data attributes at higher levels of precision than is possible with binned (i.e., histogram-mode) data. Hence, it is particularly attractive for low-count data in emission tomography. An LM likelihood and convergent EM algorithm for LM reconstruction was presented in Parra and Barrett, TMI, v17, 1998. Faster ordered subset (OS) reconstruction algorithms for LM 3-D PET were presented in Reader et al., Phys. Med. Bio., v43, 1998. However, these OS algorithms are not globally convergent and they also do not include regularization using convex priors which can be beneficial in emission tomographic reconstruction. LM-OSEM algorithms incorporating regularization via inter-iteration filtering were presented in Levkovitz et al., TMI, v20, 2001, but these are again not globally convergent. Convergent preconditioned conjugate gradient algorithms for spatio-temporal LM reconstruction incorporating regularization were presented in Nichols, et al., TMI, v21, 2002, but these do not use OS for speedup. In this work, we present a globally convergent and regularized ordered-subset algorithm for LM reconstruction. Our algorithm is derived using an incremental EM approach. We investigated the speedup of our LM OS algorithm (versus a non-OS version) for a SPECT simulation, and found that the speedup was somewhat less than that enjoyed by other OS-type algorithms.

Large-scale reliability-based structural optimization

A robust and efficient methodology is presented for treating large-scale reliability-based structural optimization problems. The optimization is performed with evolution strategies, while the reliability analysis is carried out with the Monte Carlo simulation method incorporating the importance sampling technique to reduce the sample size. Efficient hybrid methods are implemented to solve the reanalysis-type problems that arise in the optimization phase with evolution strategies and in the reliability analysis with Monte Carlo simulations. These hybrid solution methods are based on the preconditioned conjugate gradient algorithm using efficient preconditioning schemes. The numerical tests presented demonstrate the computational advantages of the proposed methods, which become more pronounced for large-scale optimization problems.

A Study on Reconstruction of De‐Aliased Uneven Seismic Data

AbstractSeismic data spatial trace interpolation is one of the most important issues in seismic data processing. In this paper, a novel Fourier transform based algorithm is proposed, which can reconstruct both uneven and aliased seismic data. We formulate band‐limited data reconstruction as a minimum norm least squares inverse problem where an adaptive DFT‐weighted norm regularization term is used. The inverse problem is solved by the preconditioned conjugate gradient algorithm, which makes the solutions stable and convergence quick. Based on the assumption that local seismic data are consisted of finite linear events, from the sampling theorem, aliased events can be attenuated via LS weighting prediction linearly from low frequency. Three application cases are discussed on even gap trace interpolation, uneven gap filling and high frequency trace reconstruction from low frequency data traces constrained by a few high frequency traces. Both synthetic and real data numerical examples show that the proposed method is valid, efficient and applicable. This research is valuable to seismic data regularization and cross well seismic exploration.

Computation of optical flow under non-uniform brightness variations

In this paper, we present a new motion estimation algorithm that provides accurate optical flow computation under non-uniform brightness variations. The proposed algorithm is based on a regularization formulation that minimizes a combination of a modified data constraint energy and a smoothness measure all over the image domain. The data constraint is derived from the conservation of the Laplacian-of-Gaussian (LoG) filtered image function, which alleviates the problem with the traditional brightness constancy assumption under non-uniform illumination variations. The resulting energy minimization is accomplished by an incomplete Cholesky preconditioned conjugate gradient algorithm. Comparisons of experimental results on benchmarking image sequences by using the proposed algorithm and some of the best existing methods are given to demonstrate its superior performance.

A finite element algorithm for reanalysis of structures with added degrees of freedom

This paper is focused on the reanalysis of structures with added degrees of freedom. By suitably partitioning the extended stiffness matrix, we present a preconditioned conjugate gradient algorithm to analyze the modified structure. The algorithm preserves the ease of implementation and improves significantly the quality of results. In particular, the algorithm can adaptively monitor the accuracy of approximate solutions. It is applicable to general finite element systems. The computational cost is significantly reduced. Numerical examples are given to show the effectiveness of the presented reanalysis algorithm.

On the modified conjugate gradient method in cloth simulation

The seminal paper on cloth simulation by Baraff and Witkin [4] presents a modified preconditioned conjugate gradient (MPCG) algorithm for solving certain large, sparse systems of linear equations. These arise when employing implicit time integration methods aimed at achieving large step cloth simulation in the presence of constraints. This paper improves the robustness and efficiency of this MPCG algorithm. We prove convergence. For this, we recast the algorithm into a linear algebra setting, identifying its filtering procedure as an orthogonal projection. This leads not only to a convergence proof but also to a correction in the initiation stage of the original algorithm that improves its efficiency. We give an example to illustrate the performance improvement offered by this correction.

Application of Robust Incomplete Cholesky Factorizations Preconditioned CG Algorithms for FEM Analysis of Millimeter Wave Filters

The Incomplete Cholesky factorizations preconditioning scheme is applied to the conjugate gradient (CG) method for solving a large system of linear equations resulting from finite element method (FEM) analysis of millimeter wave filters. As is well known, the convergence of CG method deteriorates with increasing EM wave number and in millimeter wave band the eigen-values of A are more and more scattered between both the right and the left half-plane. The efficient implementation of this preconditioned CG (PCG) algorithm is described in details for Complex coefficient matrix. With incomplete factorization preconditioning scheme in the conjugate gradient algorithm, this PCG approach can reach convergence in 20 times CPU time shorter than CG for several typical millimeter wave structures.