An adaptive conjugate gradient algorithm for large-scale unconstrained optimization

Abstract

An adaptive conjugate gradient algorithm is presented. The search direction is computed as the sum of the negative gradient and a vector determined by minimizing the quadratic approximation of objective function at the current point. Using a special approximation of the inverse Hessian of the objective function, which depends by a positive parameter, we get the search direction which satisfies both the sufficient descent condition and Dai–Liao’s conjugacy condition. The parameter in the search direction is determined in an adaptive manner by minimizing the largest eigenvalue of the matrix defining it in order to cluster all the eigenvalues. The global convergence of the algorithm is proved for uniformly convex functions. Using a set of 800 unconstrained optimization test problems we prove that our algorithm is significantly more efficient and more robust than CG-DESCENT algorithm. By solving five applications from the MINPACK-2 test problem collection, with 106 variables, we show that the suggested adaptive conjugate gradient algorithm is top performer versus CG-DESCENT.