Three-term Conjugate Gradient Research Articles

A three-term conjugate gradient descent method with some applications

The stationary point of optimization problems can be obtained via conjugate gradient (CG) methods without the second derivative. Many researchers have used this method to solve applications in various fields, such as neural networks and image restoration. In this study, we construct a three-term CG method that fulfills convergence analysis and a descent property. Next, in the second term, we employ a Hestenses-Stiefel CG formula with some restrictions to be positive. The third term includes a negative gradient used as a search direction multiplied by an accelerating expression. We also provide some numerical results collected using a strong Wolfe line search with different sigma values over 166 optimization functions from the CUTEr library. The result shows the proposed approach is far more efficient than alternative prevalent CG methods regarding central processing unit (CPU) time, number of iterations, number of function evaluations, and gradient evaluations. Moreover, we present some applications for the proposed three-term search direction in image restoration, and we compare the results with well-known CG methods with respect to the number of iterations, CPU time, as well as root-mean-square error (RMSE). Finally, we present three applications in regression analysis, image restoration, and electrical engineering.

A New Hybrid Three-Term HS-DY Conjugate Gradient In Solving Unconstrained Optimization Problems

Conjugate Gradient (CG) method is an interesting tool to solve optimization problems in many fields, such design, economics, physics and engineering. Until now, many CG methods have been developed to improve computational performance and have applied in the real-world problems. Combining two CG parameters with distinct denominators may result in non-optimal outcomes and congestion.In this paper, a new hybrid three-term CG method is proposed for solving unconstrained optimization problems. The hybrid threeterm search direction combines Hestenes-Stiefel (HS) and Dai-Yuan (DY) CG parameters which standardized by using a spectral to determine the suitable conjugate parameter choice and it satisfies the sufficient descent condition. Additionally, the global convergence was proved under standard Wolfe conditions and some suitable assumptions. Furthermore, the numerical experiments showed the proposed method is most robust and superior efficiency compared to some existing methods.

A NEW THREE -TERM CONJUGATE GRADIENT ALGORITHM FOR SOLVING MINIMIZATION PROBLEMS

The method of optimization is used to determine the most precise value for certain functions within a certain domain; it is mostly studied and employed in the fields of mathematics, computer science, and physics. This work presents a novel three-term conjugate gradient (CG) approach for unconstrained optimization problems. Both the descending criteria and the sufficient descent criterion were met by the new approach. The novel method that has been proposed has been evaluated for global convergence. The outcomes of numerical trials on a few well-known test functions demonstrated how highly successful our new modified method is, depending on the number of iterations (NOI) and the number of functions to be evaluated (NOF).

A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection

<abstract><p>The conjugate gradient (CG) method is an optimization method, which, in its application, has a fast convergence. Until now, many CG methods have been developed to improve computational performance and have been applied to real-world problems. In this paper, a new hybrid three-term CG method is proposed for solving unconstrained optimization problems. The search direction is a three-term hybrid form of the Hestenes-Stiefel (HS) and the Polak-Ribiére-Polyak (PRP) CG coefficients, and it satisfies the sufficient descent condition. In addition, the global convergence properties of the proposed method will also be proved under the weak Wolfe line search. By using several test functions, numerical results show that the proposed method is most efficient compared to some of the existing methods. In addition, the proposed method is used in practical application problems for image restoration and portfolio selection.</p></abstract>

Scaled Three-Term Conjugate Gradient Methods for Solving Monotone Equations with Application

In this paper, we derived a modified conjugate gradient (CG) parameter by adopting the Birgin and Marti´nez strategy using the descent three-term CG direction and the Newton direction. The proposed CG parameter is applied and suggests a robust algorithm for solving constrained monotone equations with an application to image restoration problems. The global convergence of this algorithm is established using some proper assumptions. Lastly, the numerical comparison with some existing algorithms shows that the proposed algorithm is a robust approach for solving large-scale systems of monotone equations. Additionally, the proposed CG parameter can be used to solve the symmetric system of nonlinear equations as well as other relevant classes of nonlinear equations.

A hybrid approach for finding approximate solutions to constrained nonlinear monotone operator equations with applications

In this article, a hybrid approach technique incorporated with three-term conjugate gradient (CG) method is proposed to solve constrained nonlinear monotone operator equations. The search direction is defined such that it is close to the one obtained by the memoryless Broyden-Fletcher-Goldferb-Shanno (BFGS) method. Independent of the line search, the search direction possesses the sufficient descent and trust region properties. Furthermore, the sequence of iterates generated converge globally under some appropriate assumptions. In addition, numerical experiments are carried out to test the efficiency of the proposed method in contrast with existing methods. Finally, the applicability of the proposed method in compressive sensing is shown.

Three-term conjugate gradient method for X-ray luminescence computed tomography.

X-ray luminescence computed tomography (XLCT) has become an emerging hybrid molecular imaging technology with high detection sensitivity and low cost. However, the inverse problem of reconstruction has severe ill-posed consequences. The original regularization algorithm needs to take much time to solve the problem. To reduce the cost of time, a three-term conjugate gradient (TTCG) algorithm is proposed for XLCT. Useful truncation information is added to the descent direction to find the optimal solution quickly in our proposed algorithm. Both numerical simulation experiments and real experiments are carried out to verify the performance of the algorithm. Experimental results show that the presented algorithm can effectively speed up the reconstruction process.

The Performance Analysis of a New Modification of Conjugate Gradient Parameter for Unconstrained Optimization Models

Conjugate Gradient (CG) method is the most prominent iterative mathematical technique that can be useful for the optimization of both linear and non-linear systems due to its simplicity, low memory requirement, computational cost, and global convergence properties. However, some of the classical CG methods have some drawbacks which include weak global convergence, poor numerical performance both in terms of number of iterations and the CPU time. To overcome these drawbacks, researchers proposed new variants of the CG parameters with efficient numerical results and nice convergence properties. Some of the variants of the CG method include the scale CG method, hybrid CG method, spectral CG method, three-term CG method, and many more. The hybrid conjugate gradient (CG) algorithm is among the efficient variant in the class of the conjugate gradient methods mentioned above. Some interesting features of the hybrid modifications include inherenting the nice convergence properties and efficient numerical performance of the existing CG methods. In this paper, we proposed a new hybrid CG algorithm that inherits the features of the Rivaie et al. (RMIL*) and Dai (RMIL+) conjugate gradient methods. The proposed algorithm generates a descent direction under the strong Wolfe line search conditions. Preliminary results on some benchmark problems show that the proposed method efficient and promising.

An efficient three-term conjugate gradient-type algorithm for monotone nonlinear equations

In this article, we proposed two Conjugate Gradient (CG) parameters using the modified Dai–Liao condition and the descent three-term CG search direction. Both parameters are incorporated with the projection technique for solving large-scale monotone nonlinear equations. Using the Lipschitz and monotone assumptions, the global convergence of methods has been proved. Finally, numerical results are provided to illustrate the robustness of the proposed methods.

New Three-Term Conjugate Gradient Method with Exact Line Search

Conjugate Gradient (CG) methods have an important role in solving largescale unconstrained optimization problems. Nowadays, the Three-Term CG method hasbecome a research trend of the CG methods. However, the existing Three-Term CGmethods could only be used with the inexact line search. When the exact line searchis applied, this Three-Term CG method will be reduced to the standard CG method.Hence in this paper, a new Three-Term CG method that could be used with the exactline search is proposed. This new Three-Term CG method satisfies the descent conditionusing the exact line search. Performance profile based on numerical results show thatthis proposed method outperforms the well-known classical CG method and some relatedhybrid methods. In addition, the proposed method is also robust in term of number ofiterations and CPU time.

A Modified Three-Term Type CD Conjugate Gradient Algorithm for Unconstrained Optimization Problems

Conjugate gradient methods are well-known methods which are widely applied in many practical fields. CD conjugate gradient method is one of the classical types. In this paper, a modified three-term type CD conjugate gradient algorithm is proposed. Some good features are presented as follows: (i) A modified three-term type CD conjugate gradient formula is presented. (ii) The given algorithm possesses sufficient descent property and trust region property. (iii) The algorithm has global convergence with the modified weak Wolfe–Powell (MWWP) line search technique and projection technique for general function. The new algorithm has made great progress in numerical experiments. It shows that the modified three-term type CD conjugate gradient method is more competitive than the classical CD conjugate gradient method.

The convergence properties of some descent conjugate gradient algorithms for optimization models

The three-term conjugate gradient (CG) algorithms are among the efficient variants of CG method for convex and nonconvex functions. This is because most three-term algorithms are constructed using the classical CG method whose numerical performance has been tested and convergence proved. In this paper, we present a modification of RMIL$+$ CG method proposed by Dai [Z. Dai, Appl. Math. Comput., \(\bf 267\) (2016), 297--300] based on the convergence analysis of RMIL (2012) CG method. Interestingly, the modified method possesses sufficient descent condition and the global convergence prove was established using exact minimization condition. We further extended the results of the modified RMIL$+$ to construct a three-term CG algorithm and also show that the method satisfies the sufficient descent condition under the strong Wolfe line search. Preliminary numerical results are reported based on known benchmark problems which show that the proposed methods are efficient and promising compare to other CG methods.

Least-squares-based three-term conjugate gradient methods

In this paper, we first propose a new three-term conjugate gradient (CG) method, which is based on the least-squares technique, to determine the CG parameter, named LSTT. And then, we present two improved variants of the LSTT CG method, aiming to obtain the global convergence property for general nonlinear functions. The least-squares technique used here well combines the advantages of two existing efficient CG methods. The search directions produced by the proposed three methods are sufficient descent directions independent of any line search procedure. Moreover, with the Wolfe–Powell line search, LSTT is proved to be globally convergent for uniformly convex functions, and the two improved variants are globally convergent for general nonlinear functions. Preliminary numerical results are reported to illustrate that our methods are efficient and have advantages over two famous three-term CG methods.

A new class of conjugate gradient methods for unconstrained smooth optimization and absolute value equations

In this paper, we introduce a new three-term conjugate gradient (NTTCG) method to solve unconstrained smooth optimization problems. NTTCG is based on conjugate gradient methods proposed by Dai and Yuan (SIAM J Optim 10:177–182, 1999) and Polak and Ribiere (Rev Francaise Inform Rech Oper 3(16):35–43, 1969). The descent property of the direction generated by NTTCG in each iteration is established. Under some standard assumptions, the global convergence results of the new methods are investigated. The extension of this algorithm is proposed to solve absolute value equations (AVE), called three-term conjugate subgradient (NTTCS) method. Numerical experiments are reported for unconstrained CUTEst problems and AVE.

A modified Hestenes–Stiefel conjugate gradient method with an optimal property

In this paper, based on the numerical efficiency of Hestenes–Stiefel (HS) method, a new modified HS algorithm is proposed for unconstrained optimization. The new direction independent of the line search satisfies in the sufficient descent condition. Motivated by theoretical and numerical features of three-term conjugate gradient (CG) methods proposed by Narushima et al., similar to Dai and Kou approach, the new direction is computed by minimizing the distance between the CG direction and the direction of the three-term CG methods proposed by Narushima et al. Under some mild conditions, we establish global convergence of the new method for general functions when the standard Wolfe line search is used. Numerical experiments on some test problems from the CUTEst collection are given to show the efficiency of the proposed method.

A modified three-term PRP conjugate gradient algorithm for optimization models

The nonlinear conjugate gradient (CG) algorithm is a very effective method for optimization, especially for large-scale problems, because of its low memory requirement and simplicity. Zhang et al. (IMA J. Numer. Anal. 26:629-649, 2006) firstly propose a three-term CG algorithm based on the well known Polak-Ribière-Polyak (PRP) formula for unconstrained optimization, where their method has the sufficient descent property without any line search technique. They proved the global convergence of the Armijo line search but this fails for the Wolfe line search technique. Inspired by their method, we will make a further study and give a modified three-term PRP CG algorithm. The presented method possesses the following features: (1) The sufficient descent property also holds without any line search technique; (2) the trust region property of the search direction is automatically satisfied; (3) the steplengh is bounded from below; (4) the global convergence will be established under the Wolfe line search. Numerical results show that the new algorithm is more effective than that of the normal method.

A New Modified Three-Term Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence

A new modified three-term conjugate gradient (CG) method is shown for solving the large scale optimization problems. The idea relates to the famous Polak-Ribière-Polyak (PRP) formula. As the numerator of PRP plays a vital role in numerical result and not having the jamming issue, PRP method is not globally convergent. So, for the new three-term CG method, the idea is to use the PRP numerator and combine it with any good CG formula’s denominator that performs well. The new modification of three-term CG method possesses the sufficient descent condition independent of any line search. The novelty is that by using the Wolfe Powell line search the new modification possesses global convergence properties with convex and nonconvex functions. Numerical computation with the Wolfe Powell line search by using the standard test function of optimization shows the efficiency and robustness of the new modification.

Two optimal Dai–Liao conjugate gradient methods

Two adaptive choices for the parameter of Dai–Liao conjugate gradient (CG) method are suggested. One of which is obtained by minimizing the distance between search directions of Dai–Liao method and a three-term CG method proposed by Zhang et al. and the other one is obtained by minimizing Frobenius condition number of the search direction matrix. Global convergence analyses are made briefly. Numerical results are reported; they demonstrate effectiveness of the suggested adaptive choices.

New three-term conjugate gradient method with guaranteed global convergence

The conjugate gradient method is one of the most effective methods to solve the unconstrained optimization problems. In this paper, we develop a new three-term conjugate gradient (TTCG) method by applying the Powell symmetrical technique to the Hestenes–Stiefel method. The proposed method satisfies both the sufficient descent property and the conjugacy condition , which do not rely on any line search. Under the standard Wolfe line search, the global convergence of the proposed method is also established. The numerical results also show that the proposed method is very effective and interesting by comparing with other TTCG methods using a classical set of test problems.

Two Modified Three-Term Type Conjugate Gradient Methods and Their Global Convergence for Unconstrained Optimization

Two modified three-term type conjugate gradient algorithms which satisfy both the descent condition and the Dai-Liao type conjugacy condition are presented for unconstrained optimization. The first algorithm is a modification of the Hager and Zhang type algorithm in such a way that the search direction is descent and satisfies Dai-Liao’s type conjugacy condition. The second simple three-term type conjugate gradient method can generate sufficient decent directions at every iteration; moreover, this property is independent of the steplength line search. Also, the algorithms could be considered as a modification of the MBFGS method, but with differentzk. Under some mild conditions, the given methods are global convergence, which is independent of the Wolfe line search for general functions. The numerical experiments show that the proposed methods are very robust and efficient.