Hestenes-Stiefel Conjugate Gradient Research Articles

A class of derivative free three-term descent Hestenes-Stiefel conjugate gradient algorithms for constrained nonlinear problems

This paper presents a novel method for solving large-scale nonlinear monotone equations with convex constraints. The method builds upon the conjugate gradient approach and incorporates a search direction of three terms, complemented by a modified line search. The proposed method generates a search direction that guarantees sufficient descent property, irrespective of the line search technique used. We establish the global convergence of the method under mild conditions and conduct numerical experiments to compare it with existing algorithms from the literature. The results demonstrate the method’s effectiveness and robustness, highlighting its potential for practical applications.

A class of spectral three-term descent Hestenes-Stiefel conjugate gradient algorithms for large-scale unconstrained optimization and image restoration problems

Conjugate gradient methods are much effective and widely used for large-scale unconstrained optimization problems by their simple computation, low memory requirement and strong global convergence property. Spectral gradient methods are also effective for large-scale problems. In this paper, a class of new descent spectral three-term conjugate gradient algorithms are proposed which automatically have the sufficient descent property and satisfy the Dai-Liao conjugate condition. Under the Wolfe line search technique and some standard conditions, the proposed methods are globally convergent for strongly convex functions and general nonlinear functions with the help of the modified secant equations. In numerical part, 732 problems with dimensions varying from 1500 to 150000 and three image restoration problems with three noise levels are considered. Numerical results indicate that the proposed algorithms are more efficient, reliable and robust than the other methods for the testing problems.

New DY-HS hybrid conjugate gradient algorithm for solving optimization problem of unsteady partial differential equations with convection term

This paper studies an optimization problem for the unsteady partial differential equations (PDEs) with convection term, widely used in continuous casting process. Considering the change of casting speed, a dynamic optimization method based on new DY-HS hybrid conjugate gradient algorithm (DY-HSHCGA) is proposed. In the DY-HSHCGA, the Dai–Yuan and the Hestenes–Stiefel conjugate gradient algorithms are convex combined, and a new conjugate parameter θk is obtained through the condition of quasi-Newton direction. Moreover, Lipschitz continuity of the gradient of cost function, as an important conditions for convergence, is analyzed in this paper. On the basis on this condition, the global convergence of DY-HSHCGA is proved. Finally, the effectiveness of DY-HSHCGA is verified by some instances from the steel plant. Comparing with other algorithms DY-HSHCGA obviously accelerates the convergence rate and reduces the number of iteration. The optimizer based on the DY-HSHCGA shows a more stable results.

A family of the modified three-term Hestenes–Stiefel conjugate gradient method with sufficient descent and conjugacy conditions

A family of the modified three-term Hestenes–Stiefel conjugate gradient method with sufficient descent and conjugacy conditions

A descent family of the spectral Hestenes–Stiefel method by considering the quasi-Newton method

The prominent computational features of the Hestenes–Stiefel parameter as one of the fundamental members of conjugate gradient methods have attracted the attention of many researchers. Yet, as a weak stop, it lacks global convergence for general functions. To overcome this defect, a family of spectral version of Hestenes–Stiefel conjugate gradient methods is introduced. To compute the spectral parameter, in the account of worthy properties of quasi-Newton methods, we minimize the distance between the search direction matrix of the spectral conjugate gradient method and the BFGS (Broyden–Fletcher–Goldfarb–Shanno) update. To achieve sufficient descent property, the search direction is projected in the orthogonal subspace to the gradient of the objective function. The convergence analysis of the proposed method is carried out under standard assumptions for general functions. Finally, the practical merits of the suggested method are investigated by numerical experiments on a set of CUTEr test functions using the Dolan–Moré performance profile. The results show the computational efficiency of the proposed method.

Performance evaluation of a conjugate gradient method considering step length computation techniques in geometry fitting of coordinate measuring machine data

This paper focuses on performance evaluation of Hestenes–Stiefel conjugate gradient (CG) method along with two well-known line search conditions, which are weak Wolfe conditions (WWC) and strong Wolfe conditions (SWC), and a new step length computation algorithm in geometry fitting of five primitive geometries (i.e., circle, square, equilateral triangle, ellipse and rectangle). A coordinate measuring machine (CMM) is used to acquire 2D profiles of these geometries. Total number of function evaluations, when the CG method-step length computation technique combination being used successfully completes the geometry fitting, are considered to be performance criteria of the combination. It has been seen that the step length computation techniques for the line search crucially affect the success and performance of the Hestenes–Stiefel CG method in the geometry fitting. Therefore, they require a special care for optimal computational efficiency in addition to the line search direction.

A Convex Combination between Two Different Search Directions of Conjugate Gradient Method and Application in Image Restoration

The conjugate gradient is a useful tool in solving large- and small-scale unconstrained optimization problems. In addition, the conjugate gradient method can be applied in many fields, such as engineering, medical research, and computer science. In this paper, a convex combination of two different search directions is proposed. The new combination satisfies the sufficient descent condition and the convergence analysis. Moreover, a new conjugate gradient formula is proposed. The new formula satisfies the convergence properties with the descent property related to Hestenes–Stiefel conjugate gradient formula. The numerical results show that the new search direction outperforms both two search directions, making it convex between them. The numerical result includes the number of iterations, function evaluations, and central processing unit time. Finally, we present some examples about image restoration as an application of the proposed conjugate gradient method.

A Linear Hybridization of Dai-Yuan and Hestenes-Stiefel Conjugate Gradient Method for Unconstrained Optimization

A Linear Hybridization of Dai-Yuan and Hestenes-Stiefel Conjugate Gradient Method for Unconstrained Optimization

Some modified Hestenes-Stiefel conjugate gradient algorithms with application in image restoration

It is efficient to use the Hestenes–Stiefe (HS) conjugate gradient algorithm in solving large-scale complex smooth optimization problems because of its simplicity and low calculation requirements. Additionally, this algorithm has been used to simultaneously solve large-scale nonsmooth problems, nonlinear equations, and practical application. In this paper, the authors propose some modified HS conjugate gradient algorithms that not only address large-scale nonlinear equations and nonsmooth convex problems but also have the following properties. i) The algorithms portray a decreasing trait and trust-region property without any additional condition. ii) It combines the steepest descent algorithm with the conjugate gradient algorithm. iii) They employ the global convergence theory and iv) can be successfully used to solve nonlinear optimization problems and image restoration.

A Family of Derivative-Free Conjugate Gradient Methods for Constrained Nonlinear Equations and Image Restoration

In this paper, a derivative-free conjugate gradient method for solving nonlinear equations with convex constraints is proposed. The proposed method can be viewed as an extension of the three-term modified Polak-Ribiere-Polyak method (TTPRP) and the three-term Hestenes-Stiefel conjugate gradient method (TTHS) using the projection technique of Solodov and Svaiter [Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, 1998, 355-369]. The proposed method adopts the adaptive line search scheme proposed by Ou and Li [Journal of Applied Mathematics and Computing 56.1-2 (2018): 195-216] which reduces the computational cost of the method. Under the assumption that the underlying operator is Lipschitz continuous and satisfies a weaker condition of monotonicity, the global convergence of the proposed method is established. Furthermore, the proposed method is extended to solve image restoration problem arising in compressive sensing. Numerical results are presented to demonstrate the effectiveness of the proposed method.

On the extension of the Hager–Zhang conjugate gradient method for vector optimization

The extension of the Hager–Zhang (HZ) nonlinear conjugate gradient method for vector optimization is discussed in the present research. In the scalar minimization case, this method generates descent directions whenever, for example, the line search satisfies the standard Wolfe conditions. We first show that, in general, the direct extension of the HZ method for vector optimization does not yield descent (in the vector sense) even when an exact line search is employed. By using a sufficiently accurate line search, we then propose a self-adjusting HZ method which possesses the descent property. The proposed HZ method with suitable parameters reduces to the classical one in the scalar minimization case. Global convergence of the new scheme is proved without regular restarts and any convex assumption. Finally, numerical experiments illustrating the practical behavior of the approach are presented, and comparisons with the Hestenes–Stiefel conjugate gradient and the steepest descent methods are discussed.

Two new spectral conjugate gradient algorithms based on Hestenes–Stiefel

The spectral conjugate gradient algorithm, which is a variant of conjugate gradient method, is one of the effective methods for solving unconstrained optimization problems. In this paper, based on Hestenes–Stiefel method, two new spectral conjugate gradient algorithms (Descend Hestenes-Stiefel (DHS) and Wang-Hestenes-Stiefel (WHS)) are proposed. Under Wolfe line search and mild assumptions on objective function, the two algorithms possess sufficient descent property without any other conditions and are always globally convergent. Numerical results turn out the new algorithms outperform Hestenes–Stiefel conjugate gradient method.

Some new three-term Hestenes–Stiefel conjugate gradient methods with affine combination

In this paper, a modified Hestenes–Stiefel conjugate gradient method for unconstrained problems is developed, which can achieves the twin goals of generating sufficient descent direction at each iteration as well as being close to the Newton direction. In our methods, the hybridization parameter can also be obtained based on other kinds of conjugacy conditions. Under mild condition, we establish their global convergence for general objective functions. Numerical experimentation with the new method indicates that it efficiently solves the test problems and therefore is promising.

A Modified Nonmonotone Hestenes–Stiefel Type Conjugate Gradient Methods for Large-Scale Unconstrained Problems

ABSTRACTIn this article, by slightly modifying the search direction of the nonmonotone Hestenes–Stiefel method, a variant Hestenes–Stiefel conjugate gradient method is proposed that satisfies the sufficient descent condition independent of any line search. This algorithm also possesses information about the gradient value and the function value. We establish the global convergence of our methods without the assumption that the steplength is bounded away from zero. Numerical results illustrate that our method can efficiently solve the test problems, and therefore is promising.

An efficient modification of the Hestenes-Stiefel nonlinear conjugate gradient method with restart property

The conjugate gradient (CG) method is one of the most popular methods to solve nonlinear unconstrained optimization problems. The Hestenes-Stiefel (HS) CG formula is considered one of the most efficient methods developed in this century. In addition, the HS coefficient is related to the conjugacy condition regardless of the line search method used. However, the HS parameter may not satisfy the global convergence properties of the CG method with the Wolfe-Powell line search if the descent condition is not satisfied. In this paper, we use the original HS CG formula with a mild condition to construct a CG method with restart using the negative gradient. The convergence and descent properties with the strong Wolfe-Powell (SWP) and weak Wolfe-Powell (WWP) line searches are established. Using this condition, we guarantee that the HS formula is non-negative, its value is restricted, and the number of restarts is not too high. Numerical computations with the SWP line search and some standard optimization problems demonstrate the robustness and efficiency of the new version of the CG parameter in comparison with the latest and classical CG formulas. An example is used to describe the benefit of using different initial points to obtain different solutions for multimodal optimization functions.

A modified Hestenes–Stiefel conjugate gradient method with sufficient descent condition and conjugacy condition

In this paper, by taking a little modification to the Hestenes–Stiefel method, we propose a new way to construct descent directions satisfying the sufficient descent condition. Also, an adaptive conjugacy condition and a intrinsic self-restarting mechanism are revealed, a dynamical adjustment can be regarded as the inheritance and development of properties of standard Hestenes–Stiefel method. Furthermore, we establish global convergence for general nonconvex objective function under mild condition. Numerical results show that our presented methods can be efficient for solving large-scale test problems and therefore is promising.

A hybridization of the Hestenes–Stiefel and Dai–Yuan conjugate gradient methods based on a least-squares approach

Following Andrei's approach of combining the conjugate gradient parameters convexly, a hybridization of the Hestenes–Stiefel (HS) and Dai–Yuan conjugate gradient (CG) methods is proposed. The hybridization parameter is computed by solving the least-squares problem of minimizing the distance between search directions of the hybrid method and a three-term conjugate gradient method proposed by Zhang et al. which possesses the sufficient descent property. Also, Powell's non-negative restriction of the HS CG parameter is employed in the hybrid method. A brief global convergence analysis is made without convexity assumption on the objective function. Comparative testing results are reported; they demonstrate efficiency of the proposed hybrid CG method in the sense of the Dolan–Moré performance profile.

Some descent three-term conjugate gradient methods and their global convergence

In this paper, we propose a three-term conjugate gradient method which can produce sufficient descent condition, that is, . This property is independent of any line search used. When an exact line search is used, this method reduces to the standard Hestenes–Stiefel conjugate gradient method. We also introduce two variants of the proposed method which still preserve the sufficient descent property, and prove that these two methods converge globally with standard Wolfe line search even if the minimization function is nonconvex. We also report some numerical experiment to show the efficiency of the proposed methods.

An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization

Recently, we propose a nonlinear conjugate gradient method, which produces a descent search direction at every iteration and converges globally provided that the line search satisfies the weak Wolfe conditions. In this paper, we will study methods related to the new nonlinear conjugate gradient method. Specifically, if the size of the scalar β k with respect to the one in the new method belongs to some interval, then the corresponding methods are proved to be globally convergent; otherwise, we are able to construct a convex quadratic example showing that the methods need not converge. Numerical experiments are made for two combinations of the new method and the Hestenes–Stiefel conjugate gradient method. The initial results show that, one of the hybrid methods is especially efficient for the given test problems.

Convergence Conditions for Ascent Methods

Convergence Conditions for Ascent Methods