Hybrid Conjugate Gradient Method Research Articles

A Convex combination of improved Fletcher-Reeves and Rivaie-Mustafa-Ismail-Leong conjugate gradient methods for unconstrained optimization problems and applications

Conjugate gradient methods are probably the most used methods in solving large scale unconstrained optimization problems. They have become popular because of their simplicity and low memory requirements. In this paper, we propose a hybrid conjugate gradient method based on the improved Fletcher-Reeves (IFR) and Rivaie-Mustafa-Ismail-Leong+ (RMIL+) methods and establish its global convergence under the strong Wolfe line search conditions. The new conjugate gradient direction satisfies the sufficient descent condition. The method is then compared to other methods in the literature and numerical experiments show that it is competent when solving large scale unconstrained optimization problems. Furthermore, the method is applied to solve a problem in portfolio selection.

A novel hybrid conjugate gradient method for multiobjective optimization problems

In this paper, we propose a hybrid nonlinear conjugate gradient method with the strong Wolfe line search step-size rule to find the Pareto critical point of a multiobjective optimization problem. The novelty of our multiobjective conjugate gradient method (in short, MCGM) lies in that it uses a hybrid parameter β k CD − DY which is combined the CD parameter β k CD with the DY parameter β k DY introduced by Lucambio Pérez and Prudente. The global convergence of the proposed hybrid nonlinear MCGM is established under mild assumptions. Numerical experiments demonstrate the effectiveness of the proposed method.

Another hybrid conjugate gradient method as a convex combination of WYL and CD methods

Another hybrid conjugate gradient method as a convex combination of WYL and CD methods

Two efficient spectral hybrid CG methods based on memoryless BFGS direction and Dai–Liao conjugacy condition

The spectral conjugate gradient (CG) method is one of the effective methods for solving unconstrained optimization problems. In this work, we introduce a composite hybrid CG parameter which is a convex combination of two new adaptive hybrid CG parameters. To derive the optimal choice of the combination coefficient in our proposed composite CG parameter, two approaches to calculating it are introduced. One is to minimize the distance between the hybrid CG direction and the self-scaling memoryless BFGS direction and the other is to apply the Dai–Liao conjugacy condition. Further, to make the search direction have better theoretical performance, two effective spectral hybrid CG methods are generated. Our proposed methods ensure the sufficient descent property regardless of the line search. And the global convergence results for general non-convex functions are established under some fundamental assumptions and Wolfe line search. Numerical experiments on solving unconstrained optimization problems illustrate the effectiveness of our proposed methods.

A sufficient descent hybrid conjugate gradient method without line search consideration and application

PurposePrevious RMIL versions of the conjugate gradient method proposed in literature exhibit sufficient descent with Wolfe line search conditions, yet their global convergence depends on certain restrictions. To alleviate these assumptions, a hybrid conjugate gradient method is proposed based on the conjugacy condition.Design/methodology/approachThe conjugate gradient (CG) method strategically alternates between RMIL and KMD CG methods by using a convex combination of the two schemes, mitigating their respective weaknesses. The theoretical analysis of the hybrid method, conducted without line search consideration, demonstrates its sufficient descent property. This theoretical understanding of sufficient descent enables the removal of restrictions previously imposed on versions of the RMIL CG method for global convergence result.FindingsNumerical experiments conducted using a hybrid strategy that combines the RMIL and KMD CG methods demonstrate superior performance compared to each method used individually and even outperform some recent versions of the RMIL method. Furthermore, when applied to solve an image reconstruction model, the method exhibits reliable results.Originality/valueThe strategy used to demonstrate the sufficient descent property and convergence result of RMIL CG without line search consideration through hybrid techniques has not been previously explored in literature. Additionally, the two CG schemes involved in the combination exhibit similar sufficient descent structures based on the assumption regarding the norm of the search direction.

A family of accelerated hybrid conjugate gradient method for unconstrained optimization and image restoration

A family of accelerated hybrid conjugate gradient method for unconstrained optimization and image restoration

A New Hybrid Three-Term LS-CD Conjugate Gradient In Solving Unconstrained Optimization Problems

The Conjugate Gradient (CG) method is renowned for its rapid convergence in optimization applications. Over the years, several modifications to CG methods have emerged to improve computational efficiency and tackle practical challenges. This paper presents a new three-term hybrid CG method for solving unconstrained optimization problems. This algorithm utilizes a search direction that combines Liu-Storey (LS) and Conjugate Descent (CD) CG coefficients and standardizes it using a spectral which acts as a scheme for the choices of the conjugate parameters. This resultant direction closely approximates the memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton direction, known for its bounded nature and compliance with the sufficient descent condition. The paper establishes the global convergence under standard Wolfe conditions and some appropriate assumptions. Additionally, the numerical experiments were conducted to emphasize the robustness and superior efficiency of this hybrid algorithm in comparison to existing approaches.

A new family of hybrid conjugate gradient method for unconstrained optimization and its application to regression analysis

We know many conjugate gradient algorithms (CG) for solving unconstrained optimization problems. In this paper, based on the three famous Liu–Storey (LS), Fletcher–Reeves (FR) and Polak–Ribiére–Polyak (PRP) conjugate gradient methods, a new hybrid CG method is proposed. Furthermore, the search direction satisfies the sufficient descent condition independent of the line search. Likewise, we prove, under the strong Wolfe line search, the global convergence of the new method. In this respect, numerical experiments are performed and reported, which show that the proposed method is efficient and promising. In virtue of this, the application of the proposed method for solving regression models of COVID-19 is provided.

Global convergence of hybrid conjugate gradient method and its application to nonparametric estimation

Global convergence of hybrid conjugate gradient method and its application to nonparametric estimation

A Sufficient Descent Dai-Liao Type Conjugate Gradient Update Parameter

In recent years, conjugate gradient methods have gained popularity as efficient iterative techniques for unconstrained optimization problems without the need for matrix storage. Based on the Dai-Laio conjugacy condition, this article presents a new hybrid conjugate gradient method that combines features of the Dai-Yuan and Dai-Laio methods. The proposed method addresses the numerical instability and slow convergence of the Dai-Yuan method as well as the potential poor performance of the Dai-Laio method in highly non-linear optimization problems. The hybrid method solves optimization problems with faster convergence rates and greater stability by combining the advantages of both methods. The resulting algorithm is shown to be more effective and reliable, and theoretical analysis reveals that it has sufficient descent properties. The proposed method's competitive performance is shown through a number of benchmark tests and comparisons with other approaches, indicating that it has the potential to be an effective approach for complex, unconstrained optimization.

A family of hybrid conjugate gradient method with restart procedure for unconstrained optimizations and image restorations

Conjugate gradient method is one of the most effective methods for solving large-scale optimization problems. Based on the CD conjugate parameter and an improved PRP conjugate parameter, a hybrid conjugate parameter with a single-parameter is designed by using two hybrid techniques, and then a restart procedure is set in its search direction to improve its descent property and computational efficiency. Accordingly, a family of hybrid conjugate gradient method with restart procedure is established, which is sufficient descent at each iteration without depending on any selection of line search criterions. Under usual assumptions and using the weak Wolfe line search criterion to generate the steplengths, the global convergence of the proposed family is proved. Finally, choosing a specific algorithm from this family to solve large-scale unconstrained optimization problems and image restorations, all the numerical results show that the proposed algorithm is effective.

A modified hybrid conjugate gradient method for solving unconstrained non-linear optimization problems has been developed

This paper presents a new hybrid conjugate gradient method, combining the Conjugate Descent (CD) and Al-Bayati & Al-Assady (BA) methods, for solving unconstrained optimization problems. We provide a convergence analysis of the proposed method and demonstrate its effectiveness through numerical examples.

Two families of hybrid conjugate gradient methods with restart procedures and their applications

In this paper, two families of hybrid conjugate gradient methods with restart procedures are proposed. Their hybrid conjugate parameters are yielded by projection or convex combination of the classical parameters. Moreover, their restart procedures are given uniformly, which are determined by the proposed hybrid conjugate parameters. The search directions of the presented families satisfy the sufficient descent condition. Under usual assumption and the weak Wolfe line search, the proposed families are proved to be globally convergent. Finally, choosing a specific parameter for each family to solve large-scale unconstrained optimization problems, convex constrained nonlinear monotone equations and image restoration problems. All the numerical results are reported and analysed, which show that the proposed families of hybrid conjugate gradient methods are promising.

Using a new hybrid conjugate gradient method with descent property

The conjugate gradient method was an efficient technique for solving optimization problems. In this paper, we propose a new efficient (CG) coefficient bk, is computed as a convex combination of Salleh and Alhawarat algorithm and CG-Descent algorithm. We prove the sufficient descent condition and the global convergence of the proposed method. It is established that the ak satisfies the strong Wolfe line search conditions. The numerical results indicate that our method is robust and competitive.

Study on MPPT Algorithm Based on an Efficient Hybrid Conjugate Gradient Method in a Photovoltaic System

An MPPT (Maximum Power Point Tracking) plays a vital role in a photovoltaic(PV) power generation system. Applying the optimum MPPT is a significant way to achieve an efficient PV system. The MPPT methods’ efficiency counts on the two MPPT factors, including the perturbation period and the perturbation amplitude. Nevertheless, realizing MPPT in the photovoltaic array is very hard when the natural conditions change, especially for the temperature and light intensity. This paper proposed a new method based on Conjugate Gradient (CG) to address the problem of hard reasonable considering both dynamic and steady-state performance in some existing MPPT algorithms. Meanwhile, the presented algorithm can effectively adjust the output of PV cells by controlling the boost DC/DC converter. Finally, the experimental and simulation results show the system can quickly get the maximum power point and have performance.

Solving Unconstrained Optimization Problems by a New Conjugate Gradient Method with Sufficient Descent Property

There have been some conjugate gradient methods with strong convergence but numerical instability and conversely‎. Improving these methods is an interesting idea to produce new methods with both strong convergence and‎‏ ‎numerical stability‎. ‎In this paper‎, ‎a new hybrid conjugate gradient method is introduced based on the Fletcher ‎formula (CD) with strong convergence and the Liu and Storey formula (LS) with good numerical results‎. ‎New directions satisfy the sufficient descent property‎, ‎independent of line search‎. ‎Under some mild assumptions‎, ‎the global convergence of new hybrid method is proved‎. ‎Numerical results on unconstrained CUTEst test problems show that the new algorithm is ‎very robust and efficient‎.

A first order reliability method based on hybrid conjugate approach with adaptive Barzilai–Borwein steps

In the first order reliability method (FORM), the Hasofer–Lind and Rackwitz–Flessler (HL–RF) algorithm sometimes encounters numerical instability problems due to the highly nonlinear limit state function (LSF). In this paper, an improved HL–RF algorithm introducing the hybrid conjugate gradient method with adaptive Barzilai–Borwein step sizes is developed to enhance the robustness and efficiency of the original HL–RF method. The proposed algorithm is composed of two stages, the steepest descent method is performed in the first stage to move to the vicinity of the most probable failure point (MPFP) and provide a good initial location for the second stage. Along with the hybrid conjugate search direction defined by the descent direction of the non-differential merit function, the second stage quantizes the adaptive Barzilai–Borwein step sizes to accelerate the process of locating the final MPFP under the nonmonotone line search rule. Eight illustrative examples with nonlinear LSFs are analyzed in detail to validate the performance of the proposed algorithm compared with other first order reliability methods. The results indicate that the proposed algorithm is not only computationally efficient but also robust in terms of convergence, especially for those problems with super nonlinear LSFs and with nonlinear LSFs involving high-frequency noise terms.

Covid-19 data modelling using hybrid conjugate gradient method

Optimization problem is a type of mathematical problem with several uses in real life. As one of the commonly applied methods for solving optimization problem, the conjugate gradient (CG) method has undergone numerous modifications in order to improve its efficiency. The hybrid CG is one of the best CG methods which can also be applied for data fitting [1]. Nowadays, Corona Virus Disease (Covid-19) has been extensively studied by many researchers worldwide due to the rising number of infected cases and deaths resulting from contracting the virus. In 2020, the Malaysian government enforced the first ‘Movement Control Order (MCO)’ in order to significantly lower the infection rate. However, the number of cases rose again at the end of year 2020, leading to the necessity to implement the second MCO in January 2021. In this study, a new hybrid conjugate gradient method is introduced to analyse the Covid-19 cases in Malaysia before MCO 2.0. This new hybrid method yields a better numerical result compared to classical and other hybrid CG methods.

An efficient hybrid conjugate gradient method for unconstrained optimization

In this paper, we propose a hybrid conjugate gradient method for unconstrained optimization, obtained by a convex combination of the LS and KMD conjugate gradient parameters. A favourite property of the proposed method is that the search direction satisfies the Dai–Liao conjugacy condition and the quasi-Newton direction. In addition, this property does not depend on the line search. Under a modified strong Wolfe line search, we establish the global convergence of the method. Numerical comparison using a set of 109 unconstrained optimization test problems from the CUTEst library show that the proposed method outperforms the Liu–Storey and Hager–Zhang conjugate gradient methods.

New Hybrid Conjugate Gradient Method as a Convex Combination of Dai–Liao and Wei–Yao–Liu

In this research, a new method of hybrid conjugated gradient methods was developed. This method is based mainly on the hybridization of Dia-Laio and Wei-Yao-Liu algorithms, by using convex fitting and conjugate condition of line Uncontrolled search. The resulting algorithm fulfills the condition of sufficient proportions and has universal convergence under certain assumptions. The numerical results indicated the efficiency of this method in solving nonlinear test functions in the given unconstrained optimization.