An efficient modification of the Hestenes-Stiefel nonlinear conjugate gradient method with restart property

Zabidin Salleh,Ahmad Alhawarat

doi:10.1186/s13660-016-1049-5

Zabidin Salleh, Ahmad Alhawarat

Open Access

https://doi.org/10.1186/s13660-016-1049-5

Copy DOI

Abstract

The conjugate gradient (CG) method is one of the most popular methods to solve nonlinear unconstrained optimization problems. The Hestenes-Stiefel (HS) CG formula is considered one of the most efficient methods developed in this century. In addition, the HS coefficient is related to the conjugacy condition regardless of the line search method used. However, the HS parameter may not satisfy the global convergence properties of the CG method with the Wolfe-Powell line search if the descent condition is not satisfied. In this paper, we use the original HS CG formula with a mild condition to construct a CG method with restart using the negative gradient. The convergence and descent properties with the strong Wolfe-Powell (SWP) and weak Wolfe-Powell (WWP) line searches are established. Using this condition, we guarantee that the HS formula is non-negative, its value is restricted, and the number of restarts is not too high. Numerical computations with the SWP line search and some standard optimization problems demonstrate the robustness and efficiency of the new version of the CG parameter in comparison with the latest and classical CG formulas. An example is used to describe the benefit of using different initial points to obtain different solutions for multimodal optimization functions.

Highlights

Consider the following form for the unconstrained optimization problem: min f (x), x ∈ Rn, ( )where f : Rn → R is a smooth nonlinear function
In numerical computations and convergence analyses, the three main conjugate gradient (CG) formulas are different if we use nonquadratic functions
The Hestenes-Stiefel (HS) CG formula is considered one of the most efficient methods developed in this century

Summary

Introduction

Consider the following form for the unconstrained optimization problem: min f (x), x ∈ Rn,. Dai and Liao [ ] proposed the following novel conjugacy condition for an inexact line search: dkT gk – dkT gk– = –tαk– gkT dk– , t >. Because the PRP and HS formulas cannot satisfy the descent property when the SWP or WWP line searches are used, Gilbert and Nocedal [ ] use Powell’s [ ] suggestion to solve the convergence problem of the PRP method as follows: βkPRP+ = max , βkPRP , βkHS+ = max , βkHS. The CG formulas in the WYL family are clearly positive and satisfy the global convergence with descent properties. This family does not inherit the restart property. To learn about many versions of CG parameters related to classical CG methods and their convergence properties, we refer the reader to [ , ]

Motivation and the new modification

Global convergence properties for the βkZA method

Conclusion