Spectral Conjugate Gradient Research Articles

A new spectral conjugate gradient method for large-scale unconstrained optimization

The spectral conjugate gradient methods, with simple construction and nice numerical performance, are a kind of effective methods for solving large-scale unconstrained optimization problems. In this paper, based on quasi-Newton direction and quasi-Newton condition, and motivated by the idea of spectral conjugate gradient method as well as Dai-Kou's selecting technique for conjugate parameter [SIAM J. Optim. 23 (2013), pp. 296–320], a new approach for generating spectral parameters is presented, where a new double-truncating technique, which can ensure both the sufficient descent property of the search directions and the bounded property of the sequence of spectral parameters, is introduced. Then a new associated spectral conjugate gradient method for large-scale unconstrained optimization is proposed. Under either the strong Wolfe line search or the generalized Wolfe line search, the proposed method is always globally convergent. Finally, a large number of comparison numerical experiments on large-scale instances from one thousand to two million variables are reported. The numerical results show that the proposed method is more promising.

Retrieving the variable coefficient for a nonlinear convection–diffusion problem with spectral conjugate gradient method

In the framework of variational adjoint system, the spectral conjugate gradient (SCG) method, coupled with adjoint method, is adopted in this study as the functional minimization tool to retrieve the variable coefficient for the nonlinear convection–diffusion equation. The main steps focus on the determination of the search step size and the search direction. The former is deduced and then achieved numerically by solving the given sensitivity equation. The latter is formulated by the spectral gradient. The effectiveness of this solution strategy is evaluated by comparing it with L-BFGS iterative descent algorithm using different data types (dense and sparse, noise-free and noisy data). The effects of different forms of initial guess distribution of variable coefficient, as well as various levels of noise, on the retrieval results are also covered. The results indicate that (1) the performance of the SCG is superior to that of the L-BFGS in both the anti-noise ability within a suitable noise range and the enhancement of numerical accuracy. This is mainly due to the fact: first, the SCG adopts optimal step length calculated via the sensitivity problem at each iteration, and second, the SCG itself belongs to the iterative regularisation class; (2) it is demonstrated in our numerical test examples that the effects on the retrieval results caused by the changes of different initial guesses for the variable coefficient are much less in the SCG than in the L-BFGS.

An efficient adaptive scaling parameter for the spectral conjugate gradient method

This paper deals with the choice of the scaling parameter in the spectral conjugate gradient (SCG) method proposed by Birgin and Martinez (in Appl Math Optim 43:117–128, 2001). Theoretical analyses show that the scaling parameter selection not only influences the numerical stability, but also plays an important role in ensuring the descent property of the SCG method. Based on these analyses, an adaptive scaling parameter is proposed to overcome the drawback of the original choice. Global convergence of the SCG method with our new parameter is established for both convex and nonconvex objective functions. Numerical results on CUTEr problems indicate that the proposed scaling parameter is very efficient and promising.

A Spectral Dai-Yuan-Type Conjugate Gradient Method for Unconstrained Optimization

A new spectral conjugate gradient method (SDYCG) is presented for solving unconstrained optimization problems in this paper. Our method provides a new expression of spectral parameter. This formula ensures that the sufficient descent condition holds. The search direction in the SDYCG can be viewed as a combination of the spectral gradient and the Dai-Yuan conjugate gradient. The global convergence of the SDYCG is also obtained. Numerical results show that the SDYCG may be capable of solving large-scale nonlinear unconstrained optimization problems.

Global convergence of a new spectral conjugate gradient by using Strong Wolfe line search

Unconstrained optimization problems can be solved by using few popular methods such as Conjugate Gradient (CG) method, Steepest Descent (SD) Method and Broyden-Fletcher-Goldfarb-Shanno (BFGS) method. The simplest solving method is by using SD method but nowadays CG method is used worldwide due to its convergence analysis. A few of unconstrained optimization problems with several different variables are used to prove the global convergence result of new spectral conjugate gradient to be compared with five most common k  proposed by the early researches by using inexact line search.

A Globally Convergence Spectral Conjugate Gradient Method for Solving Unconstrained Optimization Problems

In this paper, a modified spectral conjugate gradient method for solving unconstrained optimization problems is studied, which has sufficient descent direction and global convergence with an inexact line searches. The Fletcher-Reeves restarting criterion was employed to the standard and new versions and gave dramatic savings in the computational time. The Numerical results show that the proposed method is effective by comparing it with the FR-method.

Two modified scaled nonlinear conjugate gradient methods

Following the scaled conjugate gradient methods proposed by Andrei, we hybridize the memoryless BFGS preconditioned conjugate gradient method suggested by Shanno and the spectral conjugate gradient method suggested by Birgin and Martínez based on a modified secant equation suggested by Yuan, and propose two modified scaled conjugate gradient methods. The interesting features of our methods are applying the function values in addition to the gradient values and satisfying the sufficient descent condition for the generated search directions which leads to the global convergence for uniformly convex functions. Numerical comparisons between the implementations of one of our methods which generates descent search directions for general functions and an efficient scaled conjugate gradient method proposed by Andrei are made on a set of unconstrained optimization test problems from the CUTEr collection, using the performance profile introduced by Dolan and Moré.

A modified spectral conjugate gradient projection algorithm for total variation image restoration

In this study, a modified spectral conjugate gradient projection method is presented to solve total variation image restoration, which is transferred into the nonlinear constrained optimization with the closed constrained set. The global convergence of the proposed scheme is analyzed. In the end, some numerical results illustrate the efficiency of this method.

A modified scaled memoryless BFGS preconditioned conjugate gradient method for unconstrained optimization

In order to propose a scaled conjugate gradient method, the memoryless BFGS preconditioned conjugate gradient method suggested by Shanno and the spectral conjugate gradient method suggested by Birgin and Martinez are hybridized following Andrei’s approach. Since the proposed method is designed based on a revised form of a modified secant equation suggested by Zhang et al., one of its interesting features is applying the available function values in addition to the gradient values. It is shown that, for the uniformly convex objective functions, search directions of the method fulfill the sufficient descent condition which leads to the global convergence. Numerical comparisons of the implementations of the method and an efficient scaled conjugate gradient method proposed by Andrei, made on a set of unconstrained optimization test problems of the CUTEr collection, show the efficiency of the proposed modified scaled conjugate gradient method in the sense of the performance profile introduced by Dolan and More.

An Extended Spectral Conjugate Gradient Method for Unconstrained Optimization Problems

In this paper, an extended spectral conjugate gradient method is proposed for solving unconstrained optimization problems, where the search direction is a linear combination of the gradient vector at current iteration and the search direction at the previous iteration. Instead of specifying a fixed expression to compute each combination coefficient in the existent m ethods, only suitable conditions are presented for the combination coefficients such that the values of coefficients are chosen freely in a range. Under some mild assumptions, with step lengths satisfying the Armijo condition, global convergence is establi shed for the developed algorithm. It is shown that some existent methods are the special cases of the presented method in this paper.

A Self-Adjusting Spectral Conjugate Gradient Method for Large-Scale Unconstrained Optimization

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

Global convergence of two spectral conjugate gradient methods

Two new nonlinear spectral conjugate gradient methods for solving unconstrained optimization problems are proposed. One is based on the Hestenes and Stiefel (HS) method and the spectral conjugate gradient method. The other is based on a mixed spectral HS-CD conjugate gradient method, which combines the advantages of the spectral conjugate gradient method, the HS method, and the CD method. The directions generated by the methods are descent directions for the objective function. Under mild conditions, we prove that the spectral conjugate gradient methods with an Armijo-type line search are globally convergent. Numerical results show the proposed methods are promising.

An Improved Spectral Conjugate Gradient Algorithm for Nonconvex Unconstrained Optimization Problems

In this paper, an improved spectral conjugate gradient algorithm is developed for solving nonconvex unconstrained optimization problems. Different from the existent methods, the spectral and conjugate parameters are chosen such that the obtained search direction is always sufficiently descent as well as being close to the quasi-Newton direction. With these suitable choices, the additional assumption in the method proposed by Andrei on the boundedness of the spectral parameter is removed. Under some mild conditions, global convergence is established. Numerical experiments are employed to demonstrate the efficiency of the algorithm for solving large-scale benchmark test problems, particularly in comparison with the existent state-of-the-art algorithms available in the literature.

A new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization

Conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration, convergence properties and their low memory requirements. In this work, we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search. Moreover, an attractive property of our proposed methods is that they achieve a high-order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by Babaie-Kafaki et al. [S. Babaie-Kafaki, R. Ghanbari, N. Mahdavi-Amiri, Two new conjugate gradient methods based on modified secant equations, Journal of Computational and Applied Mathematics 234 (2010) 1374–1386]. Further, a global convergence result for general functions is established provided that the line search satisfies the Wolfe conditions. Our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness.

A Modified Spectral Conjugate Gradient Methods with the New Line Search Condition

In this Paper, Combining the Nonmonotone and Monotone Line Search ,a Spectral Conjugate Gradient Methods Are Used in this Paper. by the Concept of Spectral, we Construct a Class of Spectral Conjugate Gradient with Different form . we Also Give the Algorithm for Conjugate Gradient Method. the Numerical Result Show the Efficiency of the Method.

AN IMPROVED SPECTRAL CONJUGATE GRADIENT NEURAL NETWORK TRAINING ALGORITHM

Conjugate gradient methods constitute excellent neural network training methods which are characterized by their simplicity and their very low memory requirements. In this paper, we propose a new spectral conjugate gradient method which guarantees sufficient descent using any line search, avoiding thereby the usually inefficient restarts. Moreover, we establish the global convergence of our proposed method under some assumptions. Experimental results provide evidence that our proposed method is preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness.

Global Convergence of a Spectral Conjugate Gradient Method for Unconstrained Optimization

A new nonlinear spectral conjugate descent method for solving unconstrained optimization problems is proposed on the basis of the CD method and the spectral conjugate gradient method. For any line search, the new method satisfies the sufficient descent condition . Moreover, we prove that the new method is globally convergent under the strong Wolfe line search. The numerical results show that the new method is more effective for the given test problems from the CUTE test problem library (Bongartz et al., 1995) in contrast to the famous CD method, FR method, and PRP method.

A Mixed Spectral CD‐DY Conjugate Gradient Method

A mixed spectral CD‐DY conjugate descent method for solving unconstrained optimization problems is proposed, which combines the advantages of the spectral conjugate gradient method, the CD method, and the DY method. Under the Wolfe line search, the proposed method can generate a descent direction in each iteration, and the global convergence property can be also guaranteed. Numerical results show that the new method is efficient and stationary compared to the CD (Fletcher 1987) method, the DY (Dai and Yuan 1999) method, and the SFR (Du and Chen 2008) method; so it can be widely used in scientific computation.

A Spectral Conjugate Gradient Method with Inexact line searches

In this paper, a new Spectral Conjugate Gradient (SCG) method for solving unconstrained optimization problems; based on inexact line searches is investigated. The search directions of the new CG method are always sufficient descent. The global convergence property of the proposed method has been proved. Finally, we have presented some numerical results to examine the efficiency of the proposed method.

Regularized image denoising based on spectral gradient optimization

Image restoration methods, such as denoising, deblurring, inpainting, etc, are often based on the minimization of an appropriately defined energy function. We consider energy functions for image denoising which combine a quadratic data-fidelity term and a regularization term, where the properties of the latter are determined by a used potential function. Many potential functions are suggested for different purposes in the literature. We compare the denoising performance achieved by ten different potential functions. Several methods for efficient minimization of regularized energy functions exist. Most are only applicable to particular choices of potential functions, however. To enable a comparison of all the observed potential functions, we propose to minimize the objective function using a spectral gradient approach; spectral gradient methods put very weak restrictions on the used potential function. We present and evaluate the performance of one spectral conjugate gradient and one cyclic spectral gradient algorithm, and conclude from experiments that both are well suited for the task. We compare the performance with three total variation-based state-of-the-art methods for image denoising. From the empirical evaluation, we conclude that denoising using the Huber potential (for images degraded by higher levels of noise; signal-to-noise ratio below 10 dB) and the Geman and McClure potential (for less noisy images), in combination with the spectral conjugate gradient minimization algorithm, shows the overall best performance.