Generalized Conjugate Gradient Research Articles

Improvement of density inversion efficiency with projected gradient method: Application to FTG data

With the development of instruments for measuring Earth's gravity gradient tensor, full tensor gravity (FTG) data can be obtained validly and accurately. The highly precise performance of FTG data ensures that they can play an important role in many domains. Generally, FTG data can be acquired from airborne platform, and large-scale data can be obtained by long term observations. For this type of data, research methods for large data processing are very important. Three-dimensional data inversion is an effective and significant method for the quantitative interpretation of FTG data, which has received much attention. Large-scale data inversion is inevitable, and may consume large amounts of time with increase in the amount of data and model parameters. The main aim of this study is to improve the calculation efficiency of FTG data inversion compared with general conjugate gradient (CG) method. Accordingly, an improved gradient method with projections onto a convex set (the projected gradient method) was applied to accelerate the recovery rate of the sub-surface rock density, and a corresponding preconditioning factor was introduced to further improve the calculation efficiency of inversion. This new projected gradient method can improve the calculation efficiency of large amounts of FTG data, which is very necessary for practical application of FTG data. The application of synthetic FTG data with Gaussian noise and the real data demonstrated the feasibility and application value of the proposed inversion method.

Generalized RMIL conjugate gradient method under the strong Wolfe line search with application in image processing

RMIL conjugate gradient method originally proposed by Rivaie et al. (2012) has recently gained lots of attention. In this article, we propose a generalized conjugate gradient parameter that contains both RMIL and its variant, that is, RMIL+, as special cases. We show that the search direction generated by the new method is sufficiently descent. Under standard mild conditions, we discuss the convergence analysis of the propose method. We demonstrate the numerical efficiency of the propose method on a set of unconstrained minimization benchmark test problems as well as an image restoration problem. The results of the experiment reveal that the proposed method performs better than its main competitors.

Structure Preprocessing Method for the System of Unclosed Linear Algebraic Equations

The complexity of open linear algebraic equations makes it difficult to obtain analytical solutions, and preprocessing techniques can be applied to coefficient matrices, which has become an effective method to accelerate the convergence of iterative methods. Therefore, it is important to preprocess the structure of open linear algebraic equations to reduce their complexity. Open linear algebraic equations can be divided into symmetric linear equations and asymmetric linear equations. The former is based on 2 × 2. The latter is preprocessed by the improved QMRGCGS method, and the applications of the two methods are analyzed, respectively. The experimental results show that when the step is 500, the pretreatment time of quasi‐minimal residual generalized conjugate gradient square 2 method is 34.23 s, that of conjugate gradient square 2 method is 35.14 s, and that of conjugate gradient square method is 45.20 s, providing a new reference method and idea for solving and preprocessing non‐closed linear algebraic equations.

Minimax Design of 2D FIR Half-Band Filters Using an Efficient Matrix-Based IRLS Algorithm

This paper considers the minimax design of two-dimensional (2D) finite impulse response (FIR) half-band filters. First, the design problem is formulated in a matrix form, where the half-band constraints are expressed as a pair of matrix equations. By matrix transformations, the constrained minimax problem is transformed into an unconstrained one. Then, we propose an efficient iterative reweighted least squares (IRLS) algorithm to solve this problem. The weighted least squares (WLS) subproblems arising from the IRLS algorithm are solved using a generalized conjugate gradient (GCG) algorithm. Moreover, the GCG algorithm is guaranteed to converge in a finite number of iterations. In the proposed algorithm, the design coefficients of filters are solved in their matrix form, leading to a great saving in computations and memory space. Design examples and comparisons with existing methods are provided to demonstrate the effectiveness and efficiency of the proposed algorithm.

Implementing and modifying Broyden class updates for large scale optimization

We consider Broyden class updates for large scale optimization problems in n dimensions, restricting attention to the case when the initial second derivative approximation is the identity matrix. Under this assumption we present an implementation of the Broyden class based on a coordinate transformation on each iteration. It requires only 2nk + O(k^{2}) + O(n) multiplications on the kth iteration and stores nK+ O(K^2) + O(n) numbers, where K is the total number of iterations. We investigate a modification of this algorithm by a scaling approach and show a substantial improvement in performance over the BFGS method. We also study several adaptations of the new implementation to the limited memory situation, presenting algorithms that work with a fixed amount of storage independent of the number of iterations. We show that one such algorithm retains the property of quadratic termination. The practical performance of the new methods is compared with the performance of Nocedal’s (Math Comput 35:773--782, 1980) method, which is considered the benchmark in limited memory algorithms. The tests show that the new algorithms can be significantly more efficient than Nocedal’s method. Finally, we show how a scaling technique can significantly improve both Nocedal’s method and the new generalized conjugate gradient algorithm.

A generalized conjugate gradient method for eigenvalue problems

A generalized conjugate gradient method is proposed to solve eigenvalue problems. This method is designed bycombining the dumping block inverse power scheme and the subspace projection method. Furthermore, based on theproperties of the proposed method, a series of optimization techniques are developed toimprove the stability, computing efficiency and scalability. We also introducea computing package GCGE (generalized conjugate gradient eigensolver)which is developed based on the proposed method here.Some numerical examples are provided to validate the stability, computing efficiency and scalabilityof the method in this paper. The corresponding computing package can be downloaded from the website <uri xmlns:xlink=http://www.w3.org/1999/xlink xlink:href=http://https://github.com/pase2017/GCGE-1.0>https://github.com/pase2017/GCGE-1.0</uri>

Inner product free iterative solution and elimination methods for linear systems of a three-by-three block matrix form

Large scale systems of algebraic equations are frequently solved by iterative solution methods, such as the conjugate gradient method for symmetric or a generalized conjugate gradient or generalized minimum residual method for nonsymmetric linear systems. In practice, to get an acceptable elapsed computing time when solving large scale problems, one shall use parallel computer platforms. However, such methods involve orthogonalization of search vectors which requires computation of many inner products and, hence, needs global communication of data, which will be costly in computer times. In this paper, we propose various inner product free methods, such as the Chebyshev acceleration method. We study the solution of linear systems arising from optimal control problems for PDEs, such as the edge element discretization of the time-periodic eddy current optimal control problem. Following a discretize-then-optimize scheme, the resulting linear system is of a three-by-three block matrix form. Various solution methods based on an approximate Schur complement and inner product free iterative solution methods for this linear system are analyzed and compared with an earlier used method for two-by-two block matrices with square blocks. The convergence properties and implementation details of the proposed methods are analyzed to show their effectiveness and practicality. Both serial and parallel numerical experiments are presented to further investigate the performance of the proposed methods compared with some other existing methods.

A parallel generalized conjugate gradient method for large scale eigenvalue problems

Based on damping blocked inverse power method, a type of generalized parallel conjugate gradient method is proposed for large scale eigenvalue problems. Techniques for orthogonalization and computing Rayleigh-Ritz problems are introduced to improve the stability, efficiency and scalability. Furthermore, a computing package is built based on the proposed method here. Some numerical tests are provided to validate the stability, efficiency and scalability of the method in this paper. The corresponding computing package can be downloaded from the web site: https://github.com/pase2017/GCGE-1.0 .

Conjugate gradient variants for {ell}_{p}-regularized image reconstruction in low-field MRI

We consider the MRI physics in a low-field MRI scanner, in which permanent magnets are used to generate a magnetic field in the millitesla range. A model describing the relationship between measured signal and image is derived, resulting in an ill-posed inverse problem. In order to solve it, a regularization penalty is added to the least-squares minimization problem. We generalize the conjugate gradient minimal error (CGME) algorithm to the weighted and regularized least-squares problem. Analysis of the convergence of generalized CGME (GCGME) and the classical generalized conjugate gradient least squares (GCGLS) shows that GCGME can be expected to converge faster for ill-conditioned regularization matrices. The {ell}_{p}-regularized problem is solved using iterative reweighted least squares for p=1 and p=frac{1}{2}, with both cases leading to an increasingly ill-conditioned regularization matrix. Numerical results show that GCGME needs a significantly lower number of iterations to converge than GCGLS.

Parallel Processing Transport Model MT3DMS by Using OpenMP.

Solute transport modeling resolves advection, dispersion, and chemical reactions in groundwater systems with its accuracy depending on the resolution of domain at all scales, thus the computational efficiency of a simulator becomes a bottleneck for the wide application of numerical simulations. However, the traditional serial numerical simulators have reached their limits for the prohibitive computational time and memory requirement in solving large-scale problems. These limitations have greatly hindered the wide application of groundwater solute transport modeling. Thus, the development of an efficient method for handling large-scale groundwater solute transport simulation is urgently required. In this study, we developed and assessed a parallelized MT3DMS (Modular Three-Dimensional Multispecies Transport Model) by using OpenMP (Open specifications for Multi-Processing) to accelerate the solute transport simulation process. The parallelization was achieved by adding OpenMP compile directives (i.e., defining various types of parallel regions) into the most time-consuming packages, including the Advection package (ADV), Dispersion package (DSP), and Generalized Conjugate Gradient Solver package (GCG). This allows parallel processing on shared-memory multiprocessors, i.e., both the memory requirement and computing efforts are automatically distributed among all processors. Moreover, we discussed two different parallelization strategies for handling numerical models with either many layers or few layers. The performance of parallelized MT3DMS was assessed by two benchmark numerical models with different model domain sizes via a workstation with two quad-core processors. Results showed that the running time of parallelized MT3DMS can be 4.15 times faster than that using sequential MT3DMS. The effects of using different preconditioners (procedures that transform a given problem into a form that is more suitable for numerical solving methods) in the GCG package were additionally evaluated. The modified strategy for handling numerical models with few layers also achieved satisfactory results with running time two times faster than that via sequential simulation. Thus, the proposed parallelization allows high-resolution groundwater transport simulation with higher efficiency for large-scale or multimillion-cell simulation problems.

Generalized Conjugate Gradient Methods for ℓ1 Regularized Convex Quadratic Programming with Finite Convergence

The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper, we propose some generalized CG (GCG) methods for solving the ℓ1-regularized (possibly not strongly) convex QP that terminate at an optimal solution in a finite number of iterations. At each iteration, our methods first identify a face of an orthant and then either perform an exact line search along the direction of the negative projected minimum-norm subgradient of the objective function or execute a CG subroutine that conducts a sequence of CG iterations until a CG iterate crosses the boundary of this face or an approximate minimizer of over this face or a subface is found. We determine which type of step should be taken by comparing the magnitude of some components of the minimum-norm subgradient of the objective function to that of its rest components. Our analysis on finite convergence of these methods makes use of an error bound result and some key properties of the aforementioned exact line search and the CG subroutine. We also show that the proposed methods are capable of finding an approximate solution of the problem by allowing some inexactness on the execution of the CG subroutine. The overall arithmetic operation cost of our GCG methods for finding an ϵ-optimal solution depends on ϵ in O(log(1/ϵ)), which is superior to the accelerated proximal gradient method (Beck and Teboulle [Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1):183–202], Nesterov [Nesterov Yu (2013) Gradient methods for minimizing composite functions. Math. Program. 140(1):125–161]) that depends on ϵ in [Formula: see text]. In addition, our GCG methods can be extended straightforwardly to solve box-constrained convex QP with finite convergence. Numerical results demonstrate that our methods are very favorable for solving ill-conditioned problems.

Weighted least squares design of 2-D FIR filters using a matrix-based generalized conjugate gradient method

One of the important issues in the weighted least-squares (WLS) design of two-dimensional (2-D) finite impulse response (FIR) filters is the computational complexity of the design algorithms. This paper presents a matrix formulation of the design problem and derives a matrix-based generalized conjugate gradient (GCG) algorithm. By defining a linear operator acting on the coefficient matrix of the filter, the optimality condition of the design problem is expressed as a linear operator equation. By proving the boundedness, self-adjointness and positive-definiteness of the linear operator in the Hilbert space of the coefficient matrix, a GCG algorithm designated for the Hilbert matrix space is used to solve the linear operator equation. The convergence of the proposed algorithm in finite steps is established and its high computational efficiency is demonstrated by design examples and comparison with some existing methods.

MT3DMSP – A parallelized version of the MT3DMS code

A parallelized version of the 3-D multi-species transport model MT3DMS was developed and tested. Specifically, the open multiprocessing (OpenMP) was utilized for communication between the processors. MT3DMS emulates the solute transport by dividing the calculation into the flow and transport steps. In this article, a new preconditioner, derived from Symmetric Successive Over Relaxation (SSOR) was added into the generalized conjugate gradient solver. This preconditioner is well suited and appropriate for the parallel architecture. A case study in the test field at TU Bergakademie Freiberg was used to produce the results and analyze the code performance. It was observed that most of running time would be required for the advection, dispersion. As a result, the parallel version decreases significantly running time of solute transport modeling. In addition, this work provides a first attempt to demonstrate the capability and versatility of MT3DMS5P to simulate the solute transport in fractured gneiss rock.

On hypermonogenic functions

We research a function theory in higher dimensions based on the hyperbolic metric . The complex numbers are extended by the Clifford algebra Cl 0,n generated by the anti-commutating elements e i satisfying . In 1992, H. Leutwiler noticed that the power function (x 0 + x 1 e 1 + ··· + x n e n ) m is the generalized conjugate gradient of the functions . In the complex field (n = 1) this function h is harmonic in the usual sense, but in the higher dimensional case it is harmonic with respect to the Laplace–Beltrami operator with respect to the Riemannian hyperbolic metric . He started to study these type of functions, called H-solutions, that include positive and negative powers and elementary functions. Their total Clifford algebra valued generalizations, called hypermonogenic functions, are defined by H. Leutwiler and the first author in 2000. The integral formula has been proved by the first author. In this article, we present a simple way to find hyperbolic harmonic functions depending on the hyperbolic distance. We use these functions to determine a better presentation of the kernel, that is surprisingly the shifted Euclidean Cauchy kernel. We prove a power series expansion of hypermonogenic functions and present a version of the Maximum Modulus theorem.

A New hybrid generalized CG- method for non-linear functions

In this paper a new extended generalized conjugate gradient algorithm is proposed for unconstrained optimization, which is considered as anew inverse hyperbolic model .In order to improve the rate of convergence of the new technique, a new hybrid technique between the standard F/R CG-method and Sloboda CG-method using quadratic and non-quadratic models is proposed by using exact and inexact line searches. This method is more efficient and robust when applied on number of well-known nonlinear test function.

The FETI family of domain decomposition methods for inequality-constrained quadratic programming: Application to contact problems with conforming and nonconforming interfaces

Two domain decomposition methods with Lagrange multipliers for solving iteratively quadratic programming problems with inequality constraints are presented. These methods are based on the FETI and FETI-DP substructuring algorithms. In the case of linear constraints, they do not perform any Newton-like iteration. Instead, they solve a constrained problem by an active set strategy and a generalized conjugate gradient based descent method equipped with controls to guarantee convergence monotonicity. Both methods possess the desirable feature of minimizing numerical oscillations during the iterative solution process. Performance results and comparisons are reported for several numerical simulations that suggest that both methods are numerically scalable with respect to both the problem size and the number of subdomains. Their parallel scalability is also illustrated on a Linux cluster for a complex 1.4 million degree of freedom multibody problem with frictionless contact and nonconforming discrete interfaces.

On the performance of parallel approximate inverse preconditioning using Java multithreading techniques

In this paper a parallel shared memory Java multithreaded design and implementation of the explicit approximate inverse preconditioning is presented for solving efficiently arrow-type linear systems on symmetric multiprocessor systems. A new parallel algorithm for computing a class of optimized approximate inverse matrix is introduced. The performance on a symmetric multiprocessor system, using Java multithreading, is investigated by solving characteristic arrow-type linear systems and numerical results are given, considering the parallel performance of the construction of the optimized approximate inverse and the explicit preconditioned generalized conjugate gradient square scheme.

06/01615 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations: Chen, G. S. and Sheu, R. D. Progress in Nuclear Energy, 2004, 45, (1), 11–23

06/01615 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations: Chen, G. S. and Sheu, R. D. Progress in Nuclear Energy, 2004, 45, (1), 11–23

Modified the CG-Algorithm for Unconstrained Non-Linear Optimization by Using Oren’s Update

In this paper we have modified a new extended generalized conjugate gradient steps with self-scaling variable metric updates for unconstrained optimization. The new proposed algorithm is based on the inexact line searches and it is examined by using different non-linear test functions with various dimensions.

05/02020 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations

05/02020 Application of two preconditioned generalized conjugate gradient methods to three-dimensional neutron and photon transport equations