Spectral Conjugate Gradient Research Articles

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Correction: Two sufficient descent spectral conjugate gradient algorithms for unconstrained optimization with application

Correction: Two sufficient descent spectral conjugate gradient algorithms for unconstrained optimization with application

A new spectral conjugate subgradient method with application in computed tomography image reconstruction

A new spectral conjugate subgradient method is presented to solve nonsmooth unconstrained optimization problems. The method combines the spectral conjugate gradient method for smooth problems with the spectral subgradient method for nonsmooth problems. We study the effect of two different choices of line search, as well as three formulas for determining the conjugate directions. In addition to numerical experiments with standard nonsmooth test problems, we also apply the method to several image reconstruction problems in computed tomography, using total variation regularization. Performance profiles are used to compare the performance of the algorithm using different line search strategies and conjugate directions to that of the original spectral subgradient method. Our results show that the spectral conjugate subgradient algorithm outperforms the original spectral subgradient method, and that the use of the Polak–Ribière formula for conjugate directions provides the best and most robust performance.

The spectral conjugate gradient method in variational adjoint assimilation for model terrain correction III: convergence analysis

Abstract In the former parts of this paper, the spectral conjugate gradient (SCG) algorithm was proposed to identify the optimal bottom terrain used for predicting the meteorological fields. In this part, the convergence analysis of the SCG method is presented to verify its validity when solving a typical ill-posed inverse problem. The scaled cost function, the scaled gradient norm, the linear convergence rate, and the Hessian approximation condition number are evaluated for convergence analysis. The SCG method reduces the cost functional value and the gradient norm sufficiently. It always performs linear rates of convergence with a quickly reduced condition number of the Hessian matrix. Sometimes it provides very fast linear convergence rates. The SCG code with optimal step size and Andrei’s equation is effective on ill-posed test problems.

A family of spectral conjugate gradient methods with strong convergence and its applications in image restoration and machine learning

In this paper, we propose a family of spectral conjugate gradient methods for solving unconstrained optimization problems. Specifically, we provide two classes of bounded spectral parameters to be chosen, design a new truncation scheme of the non-negative conjugate parameter and set a restart procedure in our search direction. Independently of the specific spectral parameter, conjugate parameter and line search criterion, we prove that our proposed family satisfies the sufficient descent condition. We also prove its strong convergence under mild assumptions and the weak Wolfe line search. Numerical comparisons with other methods demonstrate the outstanding performances of our algorithm for solving medium–large-scale unconstrained optimization, image restoration and machine learning problems.

Two efficient spectral hybrid CG methods based on memoryless BFGS direction and Dai–Liao conjugacy condition

The spectral conjugate gradient (CG) method is one of the effective methods for solving unconstrained optimization problems. In this work, we introduce a composite hybrid CG parameter which is a convex combination of two new adaptive hybrid CG parameters. To derive the optimal choice of the combination coefficient in our proposed composite CG parameter, two approaches to calculating it are introduced. One is to minimize the distance between the hybrid CG direction and the self-scaling memoryless BFGS direction and the other is to apply the Dai–Liao conjugacy condition. Further, to make the search direction have better theoretical performance, two effective spectral hybrid CG methods are generated. Our proposed methods ensure the sufficient descent property regardless of the line search. And the global convergence results for general non-convex functions are established under some fundamental assumptions and Wolfe line search. Numerical experiments on solving unconstrained optimization problems illustrate the effectiveness of our proposed methods.

Two sufficient descent spectral conjugate gradient algorithms for unconstrained optimization with application

Two sufficient descent spectral conjugate gradient algorithms for unconstrained optimization with application

A Solution Method for Nonlinear Monotone Equations via Hybrid Spectral Conjugate Gradient and Signal Recovery Problems

A Solution Method for Nonlinear Monotone Equations via Hybrid Spectral Conjugate Gradient and Signal Recovery Problems

An improved spectral conjugate gradient projection method for monotone nonlinear equations with application

An improved spectral conjugate gradient projection method for monotone nonlinear equations with application

A revised Liu–Storey conjugate gradient parameter for unconstrained optimization problems with applications

ABSTRACT Conjugate Gradient (CG) methods are renowned for their efficiency and low memory requirements when solving optimization problems. However, certain formulations of CG methods that switch between two or more CG parameters often overlook specific values that could have been integrated. Moreover, the demonstration of the sufficient descent property in these methods usually relies on a strategy that assumes the possible exclusion of certain function values. To alleviate this assumption, this article introduces a structured Liu–Storey spectral CG method. This method extends the formulation of the spectral Fletcher conjugate descent method, enabling it to maintain fast convergence and inherit a good restart property. Therefore, the method ensures that the sufficient descent property holds without additional requirements and converges globally via some standard assumptions. Additionally, the method proves useful in solving robotic and image reconstruction models. Finally, it demonstrates robustness when compared to some standard algorithms.

A hybrid complex spectral conjugate gradient learning algorithm for complex-valued data processing

Complex-valued neural networks (CVNNs) have become a powerful modelling tool for complex-valued data processing. Because most of the critical points of CVNNs are saddle points, the gradient-based learning algorithms for CVNNs enjoy more chances to reach the global minima while suffering from slow convergence. To this end, we propose a hybrid complex spectral conjugate gradient learning algorithm for fast training CVNNs in this paper. The proposed algorithm combines the scaled negative gradient with a Barzilai–Borwein stepsize and an optimized conjugate term to define a new training direction, thus providing an accurate approximation of the second-order curvature of the objective function. The complex Wolfe conditions are employed to adaptively determine the optimal training stepsize. Under mild conditions, the descent property of the training direction and the convergence of the proposed algorithm are theoretically established. Simulation results on a number of benchmark complex-valued data processing problems demonstrate the efficiency of the proposed algorithm.

An accelerated relaxed-inertial strategy based CGP algorithm with restart technique for constrained nonlinear pseudo-monotone equations to image de-blurring problems

In this paper, an accelerated spectral conjugate gradient projection algorithm is proposed for solving the constrained nonlinear pseudo-monotone equations. We set a restart procedure related to the conjugate parameter in the search direction of the spectral conjugate gradient method. We use an accelerated gradient descent scheme to generate the spectral parameter. Finally, we adopt the relaxed-inertial strategy to accelerated algorithm iteration process. Global convergence is proved under mild conditions without the Lipschitz continuity assumption. Numerical comparisons with other methods show the performances of our algorithm for solving large-scale equations. Our applications are on regularized decentralized logistic regression and image de-blurring problems.

An inertial spectral conjugate gradient projection method for constrained nonlinear pseudo-monotone equations

An inertial spectral conjugate gradient projection method for constrained nonlinear pseudo-monotone equations

A new structured spectral conjugate gradient method for nonlinear least squares problems

A new structured spectral conjugate gradient method for nonlinear least squares problems

A Structured Fletcher-Revees Spectral Conjugate Gradient Method for Unconstrained Optimization with Application in Robotic Model

A Structured Fletcher-Revees Spectral Conjugate Gradient Method for Unconstrained Optimization with Application in Robotic Model

Optimization of unconstrained problems using a developed algorithm of spectral conjugate gradient method calculation

This paper presents a numerical investigation of the spectral conjugate directions formulation for optimizing unconstrained problems. A novel modified algorithm is proposed based on the conjugate gradient coefficient method. The algorithm employs the Wolfe inexact line search conditions to determine the optimum step length at each iteration and selects the appropriate conjugate gradient coefficient accordingly. The algorithm is evaluated through several numerical experiments using various unconstrained functions. The results indicate that the algorithm is highly stable, regardless of the starting point, and has better convergence rates and efficiency compared to classical methods in certain cases. Overall, this research provides a promising approach to solving unconstrained optimization problems.

Spectral conjugate gradient methods for vector optimization problems

Spectral conjugate gradient methods for vector optimization problems

The global convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery.

In 2012, Rivaie et al. introduced RMIL conjugate gradient (CG) method which is globally convergent under the exact line search. Later, Dai (2016) pointed out abnormality in the convergence result and thus, imposed certain restricted RMIL CG parameter as a remedy. In this paper, we suggest an efficient RMIL spectral CG method. The remarkable feature of this method is that, the convergence result is free from additional condition usually imposed on RMIL. Subsequently, the search direction is sufficiently descent independent of any line search technique. Thus, numerical experiments on some set of benchmark problems indicate that the method is promising and efficient. Furthermore, the efficiency of the proposed method is demonstrated on applications arising from arm robotic model and image restoration problems.

A class of spectral conjugate gradient methods for Riemannian optimization

Spectral conjugate gradient (SCG) methods are combinations of spectral gradient method and conjugate gradient (CG) methods, which have been well studied in Euclidean space. In this paper, we aim to extend this class of methods to solve optimization problems on Riemannian manifolds. Firstly, we present a Riemannian version of the spectral parameter, which guarantees that the search direction always satisfies the sufficient descent property without the help of any line search strategy. Secondly, we introduce a generic algorithmic framework for the Riemannian SCG methods, in which the selection of the CG parameter is very flexible. Under the Riemannian Wolfe conditions, the global convergence of the proposed algorithmic framework is established whenever the absolute value of the CG parameter is no more than the Riemannian Fletcher–Reeves CG parameter. Finally, some preliminary numerical results are reported and compared with several classical Riemannian CG methods, which show that our new methods are efficient.

New spectral LS conjugate gradient method for nonlinear unconstrained optimization

ABSTRACT In this work, we propose a novel algorithm to perform spectral conjugate gradient descent for an unconstrained, nonlinear optimization problem. First, we theoretically prove that the proposed method satisfies the sufficient descent condition, the conjugacy condition, and the global convergence theorem. The experimental setup uses Powell’s conjugacy condition coupled with a cubic polynomial line search using strong Wolfe conditions to ensure quick convergence. The experimental results demonstrate that the proposed method shows superior performance in terms of the number of iterations to convergence and the number of function evaluations when compared to traditional methods such as Liu and Storey (LS) and Conjugate Descent (CD).