Three-term Conjugate Gradient Methods Research Articles

Two efficient three-term conjugate gradient methods for impulse noise removal from medical images

Two efficient three-term conjugate gradient methods for impulse noise removal from medical images

Global convergence of three-term conjugate gradient methods on general functions under a new inexact line search strategy

This article deals with the global convergence of 3-Term Conjugate Gradient (3TCG) methods on general functions under a novel Inexact Line Search (ILS) strategy. The proposed algorithm satisfies the sufficient descent condition, and its global convergence can be easily proved. To verify the efficiency of the authors' theoretical approach, some numerical experiments and comparisons are presented. These results show that the suggested method is sufficiently efficient and comparable to new similar strategies for solving unconstrained optimization problems.

Solving Unconstrained Optimization Problems with Some Three-term Conjugate Gradient Methods

In this paper, based on the efficient Conjugate Descent ({\tt CD}) method, two generalized {\tt CD}algorithms are proposed to solve the unconstrained optimization problems.These methods are three-term conjugate gradient methods which the generateddirections by using the conjugate gradient parameters and independent of the line searchsatisfy in the sufficient descent condition. Furthermore, under the strong Wolfe line search,the global convergence of the proposed methods are proved. Also, the preliminary numericalresults on the {\tt CUTEst} collection are presented to show effectiveness of our methods.

Scaled Three-Term Conjugate Gradient Methods for Solving Monotone Equations with Application

In this paper, we derived a modified conjugate gradient (CG) parameter by adopting the Birgin and Marti´nez strategy using the descent three-term CG direction and the Newton direction. The proposed CG parameter is applied and suggests a robust algorithm for solving constrained monotone equations with an application to image restoration problems. The global convergence of this algorithm is established using some proper assumptions. Lastly, the numerical comparison with some existing algorithms shows that the proposed algorithm is a robust approach for solving large-scale systems of monotone equations. Additionally, the proposed CG parameter can be used to solve the symmetric system of nonlinear equations as well as other relevant classes of nonlinear equations.

A new family of hybrid three-term conjugate gradient methods with applications in image restoration

A new family of hybrid three-term conjugate gradient methods with applications in image restoration

A Descent Four-Term Conjugate Gradient Method with Global Convergence Properties for Large-Scale Unconstrained Optimisation Problems

The conjugate gradient method is a useful method to solve large-scale unconstrained optimisation problems and to be used in some applications in several fields such as engineering, medical science, image restorations, neural network, and many others. The main benefit of the conjugate gradient method is not using the second derivative or its approximation, such as Newton’s method or its approximation. Moreover, the algorithm of the conjugate gradient method is simple and easy to apply. This study proposes a new modified conjugate gradient method that contains four terms depending on popular two- and three-term conjugate gradient methods. The new algorithm satisfies the descent condition. In addition, the new CG algorithm possesses the convergence property. In the numerical results part, we compare the new algorithm with famous methods such as CG-Descent. We conclude from numerical results that the new algorithm is more efficient than other popular CG methods such as CG-Descent 6.8 in terms of number of function evaluations, number of gradient evaluations, number of iterations, and CPU time.

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

A New Hybrid Three-Term Conjugate Gradient Algorithm for Large-Scale Unconstrained Problems

Three-term conjugate gradient methods have attracted much attention for large-scale unconstrained problems in recent years, since they have attractive practical factors such as simple computation, low memory requirement, better descent property and strong global convergence property. In this paper, a hybrid three-term conjugate gradient algorithm is proposed and it owns a sufficient descent property, independent of any line search technique. Under some mild conditions, the proposed method is globally convergent for uniformly convex objective functions. Meanwhile, by using the modified secant equation, the proposed method is also global convergence without convexity assumption on the objective function. Numerical results also indicate that the proposed algorithm is more efficient and reliable than the other methods for the testing problems.

Some generalized three-term conjugate gradient methods based on CD approach for unconstrained optimization problems

Abstract In this paper, based on the efficient Conjugate Descent (CD) method, two generalized CD algorithms are proposed to solve the unconstrained optimization problems. These methods are three-term conjugate gradient methods which the generated directions by using the conjugate gradient parameters and independent of the line search satisfy in the sufficient descent condition. Furthermore, under the strong Wolfe line search, the global convergence of the proposed methods are proved. Also, the preliminary numerical results on the CUTEst collection are presented to show effectiveness of our methods.

Global convergence of a new sufficient descent spectral three-term conjugate gradient class for large-scale optimization

ABSTRACT To solve a large-scale unconstrained optimization problem, in this paper we propose a class of spectral three-term conjugate gradient methods. We indicate that the proposed class, in fact, generates sufficient descent directions and also fulfill Dai–Liao conjugacy condition. We prove the global convergence of the presented class for either uniformly convex or general smooth functions under some suitable conditions, in detail. Finally, in a set of numerical experiments which contains eight conjugate gradient methods and 260 standard problems, we illustrate the efficiency and effectiveness of our class.

A spectral three-term Hestenes–Stiefel conjugate gradient method

In this paper, according to some suitable features of three-term conjugate gradient methods and excellent theoretical properties of the quasi-Newton methods, a new spectral three-term conjugate gradient is proposed. A modified secant condition is used to compute a suitable spectral parameter. The new search direction ensures the sufficient descent condition without any line search. It is established that the new scheme possesses the global convergence, under the strong Wolfe conditions. Preliminary numerical experiments show the efficiency of the new method, dealing with unconstrained optimization problems.

Least-squares-based three-term conjugate gradient methods

In this paper, we first propose a new three-term conjugate gradient (CG) method, which is based on the least-squares technique, to determine the CG parameter, named LSTT. And then, we present two improved variants of the LSTT CG method, aiming to obtain the global convergence property for general nonlinear functions. The least-squares technique used here well combines the advantages of two existing efficient CG methods. The search directions produced by the proposed three methods are sufficient descent directions independent of any line search procedure. Moreover, with the Wolfe–Powell line search, LSTT is proved to be globally convergent for uniformly convex functions, and the two improved variants are globally convergent for general nonlinear functions. Preliminary numerical results are reported to illustrate that our methods are efficient and have advantages over two famous three-term CG methods.

Some three-term conjugate gradient methods with the new direction structure

In this paper, a three-term conjugate gradient method with the new direction structure is proposed for solving large-scale unconstrained optimization problems, which generates a sufficient descent direction in per-iteration by the aid of some inexact line search conditions. Under suitable assumptions, the proposed method is globally convergent for nonconvex smooth problems. We further generalize the new direction structure to other traditional methods and obtain some algorithms with the same structure as the proposed method. Preliminary numerical comparisons show that the proposed methods are effective and promising for the test problems.

Two families of scaled three-term conjugate gradient methods with sufficient descent property for nonconvex optimization

In this paper, we present two families of modified three-term conjugate gradient methods for solving unconstrained large-scale smooth optimization problems. We show that our new families satisfy the Dai-Liao conjugacy condition and the sufficient descent condition under any line search technique which guarantees the positiveness of ${y_{k}^{T}} s_{k}$. For uniformly convex functions, we indicate that our families are globally convergent under weak-Wolfe-Powell line search technique and standard conditions on the objective function. We also establish a weaker global convergence theorem for general smooth functions under similar assumptions. Our numerical experiments for 260 standard problems and seven other recently developed conjugate gradient methods illustrate that the members of our families are numerically efficient and effective.

A new class of three-term conjugate Gradient methods for solving unconstrained minimization problems

A new class of three-term conjugate Gradient methods for solving unconstrained minimization problems

A New Modified Three-Term Hestenes–Stiefel Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence

This paper describes a modified three-term Hestenes–Stiefel (HS) method. The original HS method is the earliest conjugate gradient method. Although the HS method achieves global convergence using an exact line search, this is not guaranteed in the case of an inexact line search. In addition, the HS method does not usually satisfy the descent property. Our modified three-term conjugate gradient method possesses a sufficient descent property regardless of the type of line search and guarantees global convergence using the inexact Wolfe–Powell line search. The numerical efficiency of the modified three-term HS method is checked using 75 standard test functions. It is known that three-term conjugate gradient methods are numerically more efficient than two-term conjugate gradient methods. Importantly, this paper quantifies how much better the three-term performance is compared with two-term methods. Thus, in the numerical results, we compare our new modification with an efficient two-term conjugate gradient method. We also compare our modification with a state-of-the-art three-term HS method. Finally, we conclude that our proposed modification is globally convergent and numerically efficient.

A one-parameter class of three-term conjugate gradient methods with an adaptive parameter choice

ABSTRACT In this paper, a one-parameter class of three-term conjugate gradient methods is proposed for solving large-scale unconstrained optimization problem. Based on eigenvalue study, an adaptive choice for the embedded parameter and the corresponding three-term conjugate gradient method is obtained. In addition, the convergence properties of the obtained method are studied for both convex and general function. Numerical experiments show that the proposed method has efficient and robust performance.

Another Improved Three Term PRP-CG Method with Global Convergent Properties for Solving Unconstrained Optimization Problems

Recently, several three-term conjugate gradient (CG) methods for solving unconstrained optimization problems have been proposed. Most of the methods centred on improving the convergence of the classical PRP-CG method while retaining its excellent numerical performance. Generally the PRP method is not convergent, because it failed to satisfy the sufficient descent property, especially, under modified Armijo line search or Wolfe line search method. In this paper, we propose an efficient three-term conjugate gradient method by utilizing the modified PRP formula which satisfies both the sufficient descent and the global convergence properties under the Wolfe line search. In particular, a new conjugate parameter is constructed. This parameter retains the numerator of PRP method and constructs an acceleration model in the denominator. The new denominator is designed to enhance the reduction in the number of iteration, CPU time as well as the convergence. Numerical results generated using various standard unconstrained optimization problem shows that the proposed method is promising and demonstrates a better numerical performance in comparison with other well-known CG methods.

A modified Hestenes–Stiefel conjugate gradient method with an optimal property

In this paper, based on the numerical efficiency of Hestenes–Stiefel (HS) method, a new modified HS algorithm is proposed for unconstrained optimization. The new direction independent of the line search satisfies in the sufficient descent condition. Motivated by theoretical and numerical features of three-term conjugate gradient (CG) methods proposed by Narushima et al., similar to Dai and Kou approach, the new direction is computed by minimizing the distance between the CG direction and the direction of the three-term CG methods proposed by Narushima et al. Under some mild conditions, we establish global convergence of the new method for general functions when the standard Wolfe line search is used. Numerical experiments on some test problems from the CUTEst collection are given to show the efficiency of the proposed method.

Some three-term conjugate gradient methods with the inexact line search condition

The three-term conjugate gradient methods solving large-scale optimization problems are favored by many researchers because of their nice descent and convergent properties. In this paper, we extend some new conjugate gradient methods, and construct some three-term conjugate gradient methods. An remarkable property of the proposed methods is that the search direction always satisfies the sufficient descent condition without any line search. Under the standard Wolfe line search, the global convergence properties of the proposed methods are proved merely by assuming that the objective function is Lipschitz continuous. Preliminary numerical results and comparisons show that the proposed methods are efficient and promising.