Classical Conjugate Gradient Method Research Articles

Integration of Modified Classical Conjugate Gradient Methods for Unconstrained Optimization

The integration of modified classical conjugate gradient methods (CGMs) for unconstrained optimization represents a crucial and evolving area of research within the field of optimization algorithms. Over time, numerous studies have put forth diverse modifications and novel approaches to enhance the effectiveness of classical CGMs. These modifications aim to address specific challenges and improve the overall performance of optimization algorithms in unconstrained scenarios. In order to tackle unconstrained optimization challenges and improve our understanding of their synergies, this ongoing study aims to unify different modified classical CGMs. Conventional CGMs have proven effective for optimization tasks, and a range of different approaches have been produced by carefully modifying these techniques. The main goal of this paper is to combine these modified versions, with particular attention to those that have similar numerators. The integration process involves systematically merging the advantageous aspects of these modified methods to develop not only innovative but also more resilient approaches to unconstrained optimization problems. The ultimate goal of this unification effort is to capitalize on the strengths inherent in different approaches to create a cohesive framework that significantly improves overall optimization performance. To thoroughly assess the efficacy of the integrated methods, a series of comprehensive performance tests are conducted. These tests include a meticulous comparison of outcomes with those of classical CGMs, providing valuable insights into the relative strengths and weaknesses of the modified approaches across diverse optimization scenarios. The evaluation criteria encompass convergence rates, solution accuracy, and computational efficiency. The conclusive outcome demonstrates that the unified approaches consistently outperform individual methods across all three crucial evaluation criteria.

Pre-Numerical Tests for a New Conjugate Gradient Method

The performance of a new beta in conjugate gradient method is generally measured on its CPU time and the number of iterations using large scale problems without concerning the accuracy of the solution attained. The method is claimed to be the best when less CPU time is used in comparison with the other classical beta. However, the accuracy of the solution gained compared to the exact solution of the function has never been tested as part of the performance measurement. Our previous research has already proven that our new beta perform well in both speed and number of iterations used. Therefore, in this research, we will accommodate this with alternative numerical test on the new beta in assessing the accuracy of the gained solution. This is to further ensure the efficiency of the new beta. Assessment was done with four other classical conjugate gradient methods on fourteen small scale problems. Results attained were promising to be tested for more tests in real functional problems.

Development of a New Numerical Conjugate Gradient Technique for Image Processing

We present a new iterative conjugate gradient technique for image processing. The technique is based on a new derivation of the conjugacy coefficient and develops a variant of the classical Fletcher-Reeves conjugate gradient method. The derivation exploits a quadratic function model. The new method is intended to minimize the presence of noise by utilizing the adaptive median filter (AMF) to reduce salt-and-pepper noise, while the adaptive center-weighted median filter (ACWMF) is used to reduce random-valued noise. The theoretical convergence properties of the method are proven and then tested on a basic set of images using MATLAB. The results show that the proposed algorithm is more efficient than the classical Fletcher-Reeves (FR) method, as measured by the signal-to-noise ratio (PSNR). The number of iterations and the number of function evaluations are also lower for the proposed method. The favorable performance of the new algorithm provides promise for deriving similar techniques that enhance the speed and efficiency of image-processing libraries.

A survey on the Dai–Liao family of nonlinear conjugate gradient methods

At the beginning of this century, which is characterized by huge flows of emerging data, Dai and Liao proposed a pervasive conjugacy condition that triggered the interest of many optimization scholars. Recognized as a sophisticated conjugate gradient (CG) algorithm after about two decades, here we share our visions and thoughts on the method in the framework of a review study. In this regard, we first discuss the modified Dai–Liao methods based on the modified secant equations given in the literature, mostly with the aim of applying the objective function values in addition to the gradient information. Then, several adaptive, sort of optimal choices for the parameter of the method are studied. Especially, we devote a part of our study to the modified versions of the Hager–Zhang and Dai–Kou CG algorithms, being well-known members of the Dai–Liao class of CG methods. Extensions of the classical CG methods based on the Dai–Liao approach are also reviewed. Finally, we discuss the optimization models of practical disciplines that have been addressed by the Dai–Liao approach, including the nonlinear systems of equations, image restoration and compressed sensing.

An Efficient Hybrid Conjugate Gradient Algorithm for Solving Intuitionistic Fuzzy Nonlinear Equations

This paper presents an iterative algorithm for solving intuitionistic fuzzy nonlinear equations (IFNEs). The proposed method is based on the classical conjugate gradient (CG) search direction. An interesting feature of the new algorithm is that it considers problems based on the special triangular intuitionistic fuzzy number. For this purpose, intuitionistic fuzzy quantities are transformed into membership and non-membership parametric forms, and a line search procedure is employed to compute the step length. Preliminary results from numerical experiments are presented to demonstrate the performance of the method. It is observed that the proposed hybrid CG method is highly effective and promising. KEYWORDS Hybrid CG, intuitionistic fuzzy nonlinear equation, parametric form, step length, inexact line search

A class of improved conjugate gradient methods for nonconvex unconstrained optimization

AbstractIn this paper, based on a new class of conjugate gradient methods which are proposed by Rivaie, Dai and Omer et al. we propose a class of improved conjugate gradient methods for nonconvex unconstrained optimization. Different from the above methods, our methods possess the following properties: (i) the search direction always satisfies the sufficient descent condition independent of any line search; (ii) these approaches are globally convergent with the standard Wolfe line search or standard Armijo line search without any convexity assumption. Moreover, our numerical results also demonstrated the efficiencies of the proposed methods.

Nonlinear Conjugate Gradient Coefficients with Exact and Strong Wolfe Line Searches Techniques

Nonlinear conjugate gradient (CG) methods are very important for solving unconstrained optimization problems. These methods have been subjected to extensive researches in terms of enhancing them. Exact and strong Wolfe line search techniques are usually used in practice for the analysis and implementation of conjugate gradient methods. For better results, several studies have been carried out to modify classical CG methods. The method of Fletcher and Reeves (FR) is one of the most well-known CG methods. It has strong convergence properties, but it gives poor numerical results in practice. The main goal of this paper is to enhance this method in terms of numerical performance via a convexity type of modification on its coefficient β k . We ensure that with this modification, the method is still achieving the sufficient descent condition and global convergence via both exact and strong Wolfe line searches. The numerical results show that this modified FR is more robust and effective.

The Convergent Properties of a New Parameter for Unconstrained Optimization

Because of its simplicity, low memory requirement, low computational cost, and global convergence properties, the Conjugate Gradient (CG) method is the most popular iterative mathematical technique for optimizing both linear and nonlinear systems. Some classical CG methods, however, have drawbacks such as poor global convergence and numerical performance in terms of iterations and function evaluations. To address these shortcomings, researchers proposed new CG parameter variants with efficient numerical results and good convergence properties. We present a new conjugate gradient formula based on the memoryless self-scale DFP quasi-Newton (QN) method in this paper. The proposed new formula fulfills the sufficient descent property and the global convergent condition with any proposed line research. When the exact line search is used, the proposed formula is reduced to the classical HS formula. Finally, we conclude that our proposed method is effective.

Assessment and mitigation of DC bias risk in power grids at provincial boundaries

With the wide application of high voltage/ultra-high voltage (HV/UHV) DC transmission technology, the impact of DC grounding electrode location selection on the surrounding power grid has become increasingly prominent, especially the problem of DC bias hazard caused by DC grounding electrodes at provincial grid boundaries needs to be solved urgently. This paper studies the assessment and prevention of trans-regional DC bias risk, and proposes an inversion method of earth resistivity model based on the measured data of neutral current in the provincial boundary area. Firstly, the DC bias risk of provincial boundary power grid is simulated and calculated, and the influence of reasonable selection of earth model on the accuracy of risk assessment results is explained. Based on the classical Fletcher-Reeves conjugate gradient method and the measured neutral current data, a reduced-order inversion method for the earth resistivity parameters of the provincial boundary power grid is proposed. On this basis, a set of DC bias risk control scheme is formulated for the actual project of Gannan power grid. Finally, the feasibility of the scheme is verified by simulation and measured data.

Multiparameter viscoelastic full-waveform inversion of shallow-seismic surface waves with a pre-conditioned truncated Newton method

SUMMARY 2-D full-waveform inversion (FWI) of shallow-seismic Rayleigh waves has become a powerful method for reconstructing viscoelastic multiparameter models of shallow subsurface with high resolution. The multiparameter reconstruction in FWI is challenging due to the potential presence of cross-talk between different parameters and the unbalanced sensitivity of Rayleigh-wave data with respect to different parameter classes. Accounting for the inverse Hessian using truncated Newton methods based on second-order adjoint methods provides an effective tool to mitigate cross-talk caused by the coupling between different parameters. In this study, we apply a pre-conditioned truncated Newton (PTN) method to shallow-seismic FWI to simultaneously invert for multiparameter near-surface models (P- and S-wave velocities, attenuation of P and S waves, and density). We first investigate scattered wavefields caused by these parameters to evaluate the coupling between them. Then we investigate the performance of the PTN method on shallow-seismic FWI of Rayleigh wave for reconstructing all five parameters simultaneously. The application to spatially correlated and uncorrelated models demonstrates that the PTN method helps to mitigate the cross-talk and improves the resolution of the multiparameter reconstructions, especially for the weak parameters with small sensitivity such as attenuation and density parameters. The attenuation of Pwaves cannot be inverted reliably due to its negligible sensitivity on the Rayleigh wave. The comparison with the classical pre-conditioned conjugate gradient method highlights the improved performance of the PTN method and thus the benefit of accounting for the information included in the Hessian.

The Performance Analysis of a New Modification of Conjugate Gradient Parameter for Unconstrained Optimization Models

Conjugate Gradient (CG) method is the most prominent iterative mathematical technique that can be useful for the optimization of both linear and non-linear systems due to its simplicity, low memory requirement, computational cost, and global convergence properties. However, some of the classical CG methods have some drawbacks which include weak global convergence, poor numerical performance both in terms of number of iterations and the CPU time. To overcome these drawbacks, researchers proposed new variants of the CG parameters with efficient numerical results and nice convergence properties. Some of the variants of the CG method include the scale CG method, hybrid CG method, spectral CG method, three-term CG method, and many more. The hybrid conjugate gradient (CG) algorithm is among the efficient variant in the class of the conjugate gradient methods mentioned above. Some interesting features of the hybrid modifications include inherenting the nice convergence properties and efficient numerical performance of the existing CG methods. In this paper, we proposed a new hybrid CG algorithm that inherits the features of the Rivaie et al. (RMIL*) and Dai (RMIL+) conjugate gradient methods. The proposed algorithm generates a descent direction under the strong Wolfe line search conditions. Preliminary results on some benchmark problems show that the proposed method efficient and promising.

New Three-Term Conjugate Gradient Method with Exact Line Search

Conjugate Gradient (CG) methods have an important role in solving largescale unconstrained optimization problems. Nowadays, the Three-Term CG method hasbecome a research trend of the CG methods. However, the existing Three-Term CGmethods could only be used with the inexact line search. When the exact line searchis applied, this Three-Term CG method will be reduced to the standard CG method.Hence in this paper, a new Three-Term CG method that could be used with the exactline search is proposed. This new Three-Term CG method satisfies the descent conditionusing the exact line search. Performance profile based on numerical results show thatthis proposed method outperforms the well-known classical CG method and some relatedhybrid methods. In addition, the proposed method is also robust in term of number ofiterations and CPU time.

A Modified Three-Term Type CD Conjugate Gradient Algorithm for Unconstrained Optimization Problems

Conjugate gradient methods are well-known methods which are widely applied in many practical fields. CD conjugate gradient method is one of the classical types. In this paper, a modified three-term type CD conjugate gradient algorithm is proposed. Some good features are presented as follows: (i) A modified three-term type CD conjugate gradient formula is presented. (ii) The given algorithm possesses sufficient descent property and trust region property. (iii) The algorithm has global convergence with the modified weak Wolfe–Powell (MWWP) line search technique and projection technique for general function. The new algorithm has made great progress in numerical experiments. It shows that the modified three-term type CD conjugate gradient method is more competitive than the classical CD conjugate gradient method.

A new subspace minimization conjugate gradient method based on modified secant equation for unconstrained optimization

In this paper, a new subspace minimization conjugate gradient method based on modified secant equation is proposed and analyzed. For a classical subspace minimization conjugate gradient method, the search direction is derived by minimizing an approximate quadratic model of objective function in a two-dimensional subspace. Generally, the approximate Hessian matrix in the above quadratic model is required to satisfy the standard secant equation, while we consider an approximate Hessian matrix which satisfies the modified secant equation. We give some rules such that if these rules are satisfied, we choose the standard secant equation, otherwise we choose the modified one. We can prove that the proposed directions satisfy the sufficient descent property under some extra conditions. We also present a modified nonmonotone Wolfe line search and establish the global convergence of the proposed method for general nonlinear functions under mild assumptions. Numerical comparisons are given with famous CG_DESCENT (5.3) (Hager and Zhang in SIAM J Optim 16(1):170–192, 2005) and SMCG_BB (Liu and Liu in J Optim Theory Appl 180(3):879–906, 2019), and show that the proposed algorithm is very promising.

The convergence properties of some descent conjugate gradient algorithms for optimization models

The three-term conjugate gradient (CG) algorithms are among the efficient variants of CG method for convex and nonconvex functions. This is because most three-term algorithms are constructed using the classical CG method whose numerical performance has been tested and convergence proved. In this paper, we present a modification of RMIL$+$ CG method proposed by Dai [Z. Dai, Appl. Math. Comput., \(\bf 267\) (2016), 297--300] based on the convergence analysis of RMIL (2012) CG method. Interestingly, the modified method possesses sufficient descent condition and the global convergence prove was established using exact minimization condition. We further extended the results of the modified RMIL$+$ to construct a three-term CG algorithm and also show that the method satisfies the sufficient descent condition under the strong Wolfe line search. Preliminary numerical results are reported based on known benchmark problems which show that the proposed methods are efficient and promising compare to other CG methods.

Параллельное решение систем линейных уравнений на гибридной архитектуре CPU+GPU

The article discusses the parallel implementation of solving systems of linear algebraic equations on computational nodes containing a central processing unit (CPU) and graphic accelerators (GPU). The performance of parallel algorithms for the classical conjugate gradient method schemes when using the CPU and GPU together is significantly limited by the synchronization points. The article investigates the pipeline version of the conjugate gradient method with one synchronization point, the possibility of asynchronous calculations, load balancing between the CPU and GPU when solving the large linear systems. Numerical experiments were carried out on test matrices and computational nodes of different performance of a heterogeneous cluster, which allowed us to estimate the contribution of communication costs. The algorithms are implemented with the joint use of technologies: MPI, OpenMP and CUDA. The proposed algorithms, in addition to reducing the execution time, allow solving large linear systems, for which there are not enough memory resources of one GPU or a computing node. At the same time, block algorithm with the pipelining decreases the total execution time by reducing synchronization points and aggregating some messages in one.

Adaptive Beamformers for High-Speed Mobile Communication

In this work, we propose a fast conjugate gradient method (CGM) for beamforming, after thoroughly analyzing the performances of the least mean square (LMS), the recursive least square (RLS), and the sample matrix inversion (SMI) adaptive beamforming algorithms. Various experiments are carried out to analyze the performances of each beamformer in detail. The proposed conjugate gradient method does not use the Eigen spread of the signal correlation matrix as in the case of the LMS and the RLS methods. It computes antenna array weights orthogonally for each iteration. Hence the convergence rate and the null depths of the proposed method are much better than the LMS, the SMI the RLS and the classical CGM. Also, the simulation results confirm that this method has a speed improvement of about 60% over the classical conjugate gradient method. This aspect significantly reduces the processor burden and saves a lot of power during the beamforming process. Hence the proposed method is superior compared to the LMS, the RLS, the SMI, and classical CGM and most suitable for high-speed mobile communication.

A COMPARATIVE STUDY OF SOME MODIFICATIONS OF CG METHODS UNDER EXACT LINE SEARCH

Conjugate Gradient (CG) method is a technique used in solving nonlinear unconstrained optimization problems. In this paper, we analysed the performance of two modifications and compared the results with the classical conjugate gradient methods of. These proposed methods possesse global convergence properties for general functions using exact line search. Numerical experiments show that the two modifications are more efficient for the test problems compared to classical CG coefficients.

A delayed weighted gradient method for strictly convex quadratic minimization

In this paper is developed an accelerated version of the steepest descent method by a two-step iteration. The new algorithm uses information with delay to define the iterations. Specifically, in the first step, a prediction of the new test point is calculated by using the gradient method with the exact minimal gradient steplength and then, a correction is computed by a weighted sum between the prediction and the predecessor iterate of the current point. A convergence result is provided. In order to compare the efficiency and effectiveness of the proposal, with similar methods existing in the literature, numerical experiments are performed. The numerical comparison of the new algorithm with the classical conjugate gradient method shows that our method is a good alternative to solve large-scale problems.

A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems

Nonlinear systems present a quite complicate problem. As the number of dimensions increases, it becomes more difficult to find the solution of the problem. In this paper, a modified conjugate gradient method is designed that has a sufficient descent property and trust region property. It is interesting that the formula for search direction makes full use of the property of convex combination between the deepest descent algorithm and the classical LS conjugate gradient (CG) method. The global convergence property is established under certain conditions, and the numerical results show that the modified method is effective for normal nonlinear equations and image restoration problems.