Dai-Yuan Conjugate Gradient Research Articles

The spectral form of the Dai-Yuan conjugate gradient algorithm

Conjugate Gradient (CG) methods comprise a class of unconstrained optimization algorithms which are characterized by low memory requirements and strong local and global convergence properties. Most of CG methods do not always generate a descent search directions, so the descent or sufficient descent condition is usually assumed in the analysis and implementations. By assuming a descent and pure conjugacy conditions a new version of spectral Dai-Yuan (DY) non-linear conjugate gradient method introduced in this article. Descent property for the suggested method is proved and numerical tests and comparisons with other methods for large-scale unconstrained problems are given.

Convergence of the descent Dai–Yuan conjugate gradient method for unconstrained optimization

The conjugate gradient method is one of the most useful and the earliest-discovered techniques for solving large-scale nonlinear optimization problems. Many variants of this method have been proposed, and some are widely used in practice. In this article, we study the descent Dai–Yuan conjugate gradient method which guarantees the sufficient descent condition for any line search. With exact line search, the introduced conjugate gradient method reduces to the Dai–Yuan conjugate gradient method. Finally, a global convergence result is established when the line search fulfils the Goldstein conditions.

A hybrid conjugate gradient method based on a quadratic relaxation of the Dai–Yuan hybrid conjugate gradient parameter

To take advantage of the attractive features of the Hestenes–Stiefel and Dai–Yuan conjugate gradient (CG) methods, we suggest a hybridization of these methods using a quadratic relaxation of a hybrid CG parameter proposed by Dai and Yuan. In the proposed method, the hybridization parameter is computed based on a conjugacy condition. Under proper conditions, we show that our method is globally convergent for uniformly convex functions. We give a numerical comparison of the implementations of our method and two efficient hybrid CG methods proposed by Dai and Yuan using a set of unconstrained optimization test problems from the CUTEr collection. Numerical results show the efficiency of the proposed hybrid CG method in the sense of the performance profile introduced by Dolan and Moré.

Two effective hybrid conjugate gradient algorithms based on modified BFGS updates

Based on two modified secant equations proposed by Yuan, and Li and Fukushima, we extend the approach proposed by Andrei, and introduce two hybrid conjugate gradient methods for unconstrained optimization problems. Our methods are hybridizations of Hestenes-Stiefel and Dai-Yuan conjugate gradient methods. Under proper conditions, we show that one of the proposed algorithms is globally convergent for uniformly convex functions and the other is globally convergent for general functions. To enhance the performance of the line search procedure, we propose a new approach for computing the initial value of the steplength for initiating the line search procedure. We give a comparison of the implementations of our algorithms with two efficiently representative hybrid conjugate gradient methods proposed by Andrei using unconstrained optimization test problems from the CUTEr collection. Numerical results show that, in the sense of the performance profile introduced by Dolan and More, the proposed hybrid algorithms are competitive, and in some cases more efficient.

New accelerated conjugate gradient algorithms as a modification of Dai–Yuan’s computational scheme for unconstrained optimization

New accelerated nonlinear conjugate gradient algorithms which are mainly modifications of Dai and Yuan’s for unconstrained optimization are proposed. Using the exact line search, the algorithm reduces to the Dai and Yuan conjugate gradient computational scheme. For inexact line search the algorithm satisfies the sufficient descent condition. Since the step lengths in conjugate gradient algorithms may differ from 1 by two orders of magnitude and tend to vary in a very unpredictable manner, the algorithms are equipped with an acceleration scheme able to improve the efficiency of the algorithms. Computational results for a set consisting of 750 unconstrained optimization test problems show that these new conjugate gradient algorithms substantially outperform the Dai–Yuan conjugate gradient algorithm and its hybrid variants, Hestenes–Stiefel, Polak–Ribière–Polyak, CONMIN conjugate gradient algorithms, limited quasi-Newton algorithm LBFGS and compare favorably with CG_DESCENT. In the frame of this numerical study the accelerated scaled memoryless BFGS preconditioned conjugate gradient ASCALCG algorithm proved to be more robust.

Hybrid Conjugate Gradient Algorithm for Unconstrained Optimization

In this paper a new hybrid conjugate gradient algorithm is proposed and analyzed. The parameter βk is computed as a convex combination of the Polak-Ribiere-Polyak and the Dai-Yuan conjugate gradient algorithms, i.e. βkN=(1−θk)βkPRP+θkβkDY. The parameter θk in the convex combination is computed in such a way that the conjugacy condition is satisfied, independently of the line search. The line search uses the standard Wolfe conditions. The algorithm generates descent directions and when the iterates jam the directions satisfy the sufficient descent condition. Numerical comparisons with conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that this hybrid computational scheme outperforms the known hybrid conjugate gradient algorithms.

Another hybrid conjugate gradient algorithm for unconstrained optimization

Another hybrid conjugate gradient algorithm is subject to analysis. The parameter β k is computed as a convex combination of \( \beta ^{{HS}}_{k} \) (Hestenes-Stiefel) and \( \beta ^{{DY}}_{k} \) (Dai-Yuan) algorithms, i.e. \( \beta ^{C}_{k} = {\left( {1 - \theta _{k} } \right)}\beta ^{{HS}}_{k} + \theta _{k} \beta ^{{DY}}_{k} \). The parameter θ k in the convex combination is computed in such a way so that the direction corresponding to the conjugate gradient algorithm to be the Newton direction and the pair (s k , y k ) to satisfy the quasi-Newton equation \( \nabla ^{2} f{\left( {x_{{k + 1}} } \right)}s_{k} = y_{k} \), where \( s_{k} = x_{{k + 1}} - x_{k} \) and \( y_{k} = g_{{k + 1}} - g_{k} \). The algorithm uses the standard Wolfe line search conditions. Numerical comparisons with conjugate gradient algorithms show that this hybrid computational scheme outperforms the Hestenes-Stiefel and the Dai-Yuan conjugate gradient algorithms as well as the hybrid conjugate gradient algorithms of Dai and Yuan. A set of 750 unconstrained optimization problems are used, some of them from the CUTE library.

Global Convergence of the Dai-Yuan Conjugate Gradient Method with Perturbations

In this paper, the authors propose a class of Dai-Yuan (abbr. DY) conjugate gradient methods with linesearch in the presence of perturbations on general function and uniformly convex function respectively. Their iterate formula is xk+1 = xk + αk (sk + ωk), where the main direction sk is obtained by DY conjugate gradient method, ωk is perturbation term, and stepsize αk is determined by linesearch which does not tend to zero in the limit necessarily. The authors prove the global convergence of these methods under mild conditions. Preliminary computational experience is also reported.

A Dai–Yuan conjugate gradient algorithm with sufficient descent and conjugacy conditions for unconstrained optimization

A modification of the Dai–Yuan conjugate gradient algorithm is proposed. Using exact line search, the algorithm reduces to the original version of the Dai and Yuan computational scheme. For inexact line search the algorithm satisfies both sufficient descent and conjugacy conditions. A global convergence result is proved when the Wolfe line search conditions are used. Computational results, for a set consisting of 750 unconstrained optimization test problems, show that this new conjugate gradient algorithm substantially outperforms the Dai–Yuan conjugate gradient algorithm and comes close to its hybrid variants.