A Modified Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

Zabidin Salleh,Ahmad Alhawarat,Ghaliah Alhamzi,Ibitsam Masmali

doi:10.3390/a14080227

Abstract

The conjugate gradient method is one of the most popular methods to solve large-scale unconstrained optimization problems since it does not require the second derivative, such as Newton’s method or approximations. Moreover, the conjugate gradient method can be applied in many fields such as neural networks, image restoration, etc. Many complicated methods are proposed to solve these optimization functions in two or three terms. In this paper, we propose a simple, easy, efficient, and robust conjugate gradient method. The new method is constructed based on the Liu and Storey method to overcome the convergence problem and descent property. The new modified method satisfies the convergence properties and the sufficient descent condition under some assumptions. The numerical results show that the new method outperforms famous CG methods such as CG-Descent 5.3, Liu and Storey, and Dai and Liao. The numerical results include the number of iterations and CPU time.

Highlights

The CG method generates a sequence of iterates xk as follows: Where xk is the current iteration and αk > 0 is a step size obtained from a line search xk+1 = xk + αk dk, k = 1, 2, ..., with regard to jurisdictional claims in published maps and institutional affiliations
We proposed a modification of the Liu and Storey CG method that satisfies the following main challenges: 1
The modified method was restarted based on the suggestion presented by [12] instead of using the steepest descent, improving the efficiency of the proposed method

Summary

Introduction

The optimization problem that we want to solve takes the following form: Frank Werner. Publisher’s Note: MDPI stays neutral min f ( x ) ,x ∈ Rn , where f : Rn → R is a continuous and differentiable function, and its gradient (∇ f ( x )) is available. The CG method generates a sequence of iterates xk (vector) as follows: Where xk is the current iteration and αk > 0 is a step size obtained from a line search xk+1 = xk + αk dk , k = 1, 2, ..., with regard to jurisdictional claims in published maps and institutional affiliations. This article is an open access article (1). Such as exact or inexact line search. The search direction dk in the CG method requests only the first derivative of the optimization function, and it is defined as follows:.

Methods

Results

Conclusion