Family Of Conjugate Gradient Methods Research Articles

Efficient one-parameter family of conjugate gradient methods

We present an efficient one-parameter family of conjugate gradient methods for unconstrained optimization problems. These methods are defined using a combination of the Polak-Ribiere-Polyak method and the Rivaie-Mustafa-Ismail-Leong method. We prove the global convergence based on the Wolfe line search for nonlinear objective functions. Finally, we give some numerical experiments, proving the efficiency of the proposed approach.

A new family of conjugate gradient methods to solve unconstrained optimization problems

A conjugate optimal coefficient is a crucial characteristic of conjugate gradient algorithms. The idea of accelerating the conjugate gradient by utilizing the conjugacy condition information and quadratic model. The gradient of the objective function is used to specify the search directions for traditional techniques. This work provides nonlinear conjugate gradient algorithms that primarily consider objective function information. One of the ways is as efficient as or more efficient than the conventional methods, according to numerical examples.

A globally convergent projection method for a system of nonlinear monotone equations

ABSTRACT In this paper, a new conjugate gradient-based projection method for solving a system of nonlinear monotone equations is proposed. The method can be viewed as an extension of a family of conjugate gradient methods for unconstrained optimization by Li et al. [A new family of conjugate gradient methods for unconstrained optimization, J. Appl. Math. Comput. 58 (2018), pp. 219–234]. The proposed method is derivative-free which makes it suitable for large-scale nonlinear monotone equations. We show that the method satisfies the descent condition independent of line searches and that the method is globally convergent. Numerical results indicate that the proposed method is efficient.

Convergence of a two-parameter family of conjugate gradient methods with a fixed formula of stepsize

We prove the global convergence of a two-parameter family of conjugate gradient methods that use a new and different formula of stepsize from Wu \cite% {14}. Numerical results are presented to confirm the effectiveness of the proposed stepsizes by comparing with the stepsizes suggested by Sun and his colleagues \cite% {2, 12}.\\

A new conjugate gradient method with an efficient memory structure

A new family of conjugate gradient methods for large-scale unconstrained optimization problems is described. It is based on minimizing a penalty function, and uses a limited memory structure to exploit the useful information provided by the iterations. Our penalty function combines the good properties of the linear conjugate gradient method using some penalty parameters. We propose a suitable penalty parameter by solving an auxiliary linear optimization problem and show that the proposed parameter is promising. The global convergence of the new method is investigated under mild assumptions. Numerical results show that the new method is efficient and confirm the effectiveness of the memory structure.

An optimal parameter choice for the Dai–Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix

Based on a singular value analysis conducted on the Dai–Liao conjugate gradient method, it is shown that when the gradient approximately lies in the direction of the maximum magnification by the search direction matrix, the method may get into some computational errors and also, the convergence may occur hardly. Hence, we obtain a formula for computing the Dai–Liao parameter which makes the direction of the maximum magnification by the search direction matrix to be orthogonal to the gradient. We briefly discuss global convergence of the corresponding Dai–Liao method with and without convexity assumption on the objective function. Numerical experiments on a set of test problems of the CUTEr collection show practical effectiveness of the suggested adaptive choice of the Dai–Liao parameter in the sense of the Dolan–More performance profile.

A new family of conjugate gradient methods for unconstrained optimization

A new family of conjugate gradient methods is proposed by minimizing the distance between two certain directions. It is a subfamily of Dai–Liao family, which consists of Hager–Zhang family and Dai–Kou method. The direction of the proposed method is an approximation to that of the memoryless Broyden–Fletcher–Goldfarb–Shanno method. With the suitable intervals of parameters, the direction of the proposed method possesses the sufficient descent property independent of the line search. Under mild assumptions, we analyze the global convergence of the method for strongly convex functions and general functions where the stepsize is obtained by the standard Wolfe rules. Numerical results indicate that the proposed method is a promising method which outperforms CGOPT and CG_DESCENT on a set of unconstrained optimization testing problems.

A family of conjugate gradient methods for large-scale nonlinear equations

In this paper, we present a family of conjugate gradient projection methods for solving large-scale nonlinear equations. At each iteration, it needs low storage and the subproblem can be easily solved. Compared with the existing solution methods for solving the problem, its global convergence is established without the restriction of the Lipschitz continuity on the underlying mapping. Preliminary numerical results are reported to show the efficiency of the proposed method.

Some nonlinear conjugate gradient methods based on spectral scaling secant equations

Three conjugate gradient methods based on the spectral equations are proposed. One is a conjugate gradient method based on the spectral scaling secant equation proposed by Cheng and Li (J Optim Thoery Appl 146:305–319, 2010), which gives the most efficient Dai–Kou conjugate gradient method with sufficient descent in Dai and Kou (SIAM J Optim 23:296–320, 2013) family of conjugate gradient methods. The other is a special case in Dai and Liao (Appl Math Optim 43:87–101, 2001) family of conjugate gradient methods, which adopts a special parameter, an approximation to the spectral of the Hessian of the objective function. Another is a sufficient descent conjugate gradient method, which is based on the second one and can be viewed as a three-term conjugate gradient method based on a spectral scaling secant equation. Convergence properties of the three methods are discussed, and numerical results imply that conjugate gradient methods based on the spectral secant equations perform well, especially when the sufficient descent condition is satisfied.

On Hager and Zhang’s conjugate gradient method with guaranteed descent

In this paper, we point out that Hager–Zhang family of conjugate gradient methods are based on the spectral scaling secant equation, are really members of Dai–Liao type of gradient methods with common constant parameters. We propose a possible efficient parameter for Hager–Zhang’s conjugate gradient method with guaranteed descent (CG_DESCENT) and present a conjugate gradient method CG_DESCENT1. The CG_DESCENT1 method preserves the same global convergence properties as CG_DESCENT method of Hager and Zhang, and is numerically tested compared with CG_DESCENT method and other conjugate gradient methods. Considering the difference of the search directions approximation to the search direction of the memoryless BFGS method, we give the reason why the proposed method is more efficient than CG_DESCENT method under some conditions, and further find an optimal conjugate gradient method and an approximation optimal conjugate method in Hager–Zhang family of conjugate gradient methods.

A new two-parameter family of nonlinear conjugate gradient methods

Conjugate gradient methods are an important class of methods for unconstrained optimization, especially for large-scale problems. Recently, they have been much studied. In this paper, we propose a new two-parameter family of conjugate gradient methods for unconstrained optimization. The two-parameter family of methods not only includes the already existing three practical nonlinear conjugate gradient methods, but has other family of conjugate gradient methods as subfamily. The two-parameter family of methods with the Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for the two-parameter family of methods. The numerical results show that this method is efficient for the given test problems. In addition, the methods related to this family are uniformly discussed.

A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search

In this paper, we seek the conjugate gradient direction closest to the direction of the scaled memoryless BFGS method and propose a family of conjugate gradient methods for unconstrained optimization. An improved Wolfe line search is also proposed, which can avoid a numerical drawback of the original Wolfe line search and guarantee the global convergence of the conjugate gradient method under mild conditions. To accelerate the algorithm, we introduce adaptive restarts along negative gradients based on the extent to which the function approximates some quadratic function during previous iterations. Numerical experiments with the CUTEr collection show that the proposed algorithm is promising.

Słownik Otolaryngologów Polskich

We study the global convergence of a two-parameter family of conjugate gradient methods in which the line search procedure is replaced by a fixed formula of stepsize. This character is of significance if the line search is expensive in a particular application. In addition to the convergence results, we present computational results for various conjugate gradient methods without line search including those discussed by Sun and Zhang (Ann. Oper. Res. 103 (2001) 161–173).

A new family of conjugate gradient methods

In this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems. A new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions. In particular, Polak–Ribiére–Polyak and Liu–Storey conjugate gradient methods are special cases of the new class of conjugate gradient methods. By estimating the local Lipschitz constant of the derivative of objective functions, we can find an adequate step size and substantially decrease the function evaluations at each iteration. Numerical results show that these new conjugate gradient methods are effective in minimizing large-scale non-convex non-quadratic functions.

GLOBAL CONVERGENCE OF SHORTEST-RESIDUAL FAMILY OF CONJUGATE GRADIENT METHODS WITHOUT LINE SEARCH

The shortest-residual family of conjugate gradient methods was first proposed by Hestenes and was studied by Pytlak, and Dai and Yuan. Recently, a no-line-search scheme in conjugate gradient methods was given by Sun and Zhang, and Chen and Sun. In this paper, we show the global convergence of two shortest-residual conjugate gradient methods (FRSR and PRPSR) without line search. In addition, computational results are presented to show that the methods with line search have similar numerical behavior to the methods without line search.

Global convergence of a two-parameter family of conjugate gradient methods without line search

We study the global convergence of a two-parameter family of conjugate gradient methods in which the line search procedure is replaced by a fixed formula of stepsize. This character is of significance if the line search is expensive in a particular application. In addition to the convergence results, we present computational results for various conjugate gradient methods without line search including those discussed by Sun and Zhang (Ann. Oper. Res. 103 (2001) 161–173).

A three-parameter family of nonlinear conjugate gradient methods

In this paper, we propose a three-parameter family of conjugate gradient methods for unconstrained optimization. The three-parameter family of methods not only includes the already existing six practical nonlinear conjugate gradient methods, but subsumes some other families of nonlinear conjugate gradient methods as its subfamilies. With Powell's restart criterion, the three-parameter family of methods with the strong Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for the three-parameter family of methods. This paper can also be regarded as a brief review on nonlinear conjugate gradient methods.

Calculating resolution and covariance matrices for seismic tomography with the LSQR method

Tn seismic tomography, the LSQR algorithm is commonly used for solving the inverse problem. LSQR belongs to the family of conjugate gradient methods, so the generalized inverse is not solved explic ...