Descent Property Research Articles

Improved Fletcher-Reeves Methods Based on New Scaling Techniques

This paper introduces a scaling parameter to the Fletcher-Reeves (FR) nonlinear conjugate gradient method. The main aim is to improve its theoretical and numerical properties when applied with inexact line searches to unconstrained optimization problems. We show that the sufficient descent and global convergence properties of Al-Baali for the FR method with a fairly accurate line search are maintained. We also consider the possibility of extending this result to less accurate line search for appropriate values of the scaling parameter. The reported numerical results show that several values for the proposed scaling parameter improve the performance of the FR method significantly.

New scaled algorithm for non-linear conjugate gradients in unconstrained optimization

<p>A new scaled conjugate gradient (SCG) method is proposed throughout this paper, the SCG technique may be a special important generalization conjugate gradient (CG) method, and it is an efficient numerical method for solving nonlinear large scale unconstrained optimization. As a result, we proposed the new SCG method with a strong Wolfe condition (SWC) line search is proposed. The proposed technique's descent property, as well as its global convergence property, are satisfied without the use of any line searches under some suitable assumptions. The proposed technique's efficiency and feasibility are backed up by numerical experiments comparing them to traditional CG techniques.</p>

A Descent Four-Term of Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

The conjugate gradient (CG) method is a useful tool for obtaining the optimum point for unconstrained optimization problems since it does not require a second derivative or its approximations. Moreover, the conjugate gradient method can be applied in many fields such as machine learning, deep learning, neural network, and many others. This paper constructs a four-term conjugate gradient method that satisfies the descent property and convergence properties to obtain the stationary point. The new modification was constructed based on Liu and Storey's conjugate gradient method, two-term conjugate gradient method, and three-term conjugate gradient method. To analyze the efficiency and robustness, we used more than 150 optimization functions from the CUTEst library with different dimensions and shapes. The numerical results show that the new modification outperforms the recent conjugate gradient methods such as CG-Descent, Dai and Liao, and others in terms of number of functions evaluations, number of gradient evaluations, number of iterations, and CPU time.

Local Stability of McKean-Vlasov Equations Arising from Heterogeneous Gibbs Systems Using Limit of Relative Entropies.

A family of heterogeneous mean-field systems with jumps is analyzed. These systems are constructed as a Gibbs measure on block graphs. When the total number of particles goes to infinity, the law of large numbers is shown to hold in a multi-class context, resulting in the weak convergence of the empirical vector towards the solution of a McKean–Vlasov system of equations. We then investigate the local stability of the limiting McKean–Vlasov system through the construction of a local Lyapunov function. We first compute the limit of adequately scaled relative entropy functions associated with the explicit stationary distribution of the N-particles system. Using a Laplace principle for empirical vectors, we show that the limit takes an explicit form. Then we demonstrate that this limit satisfies a descent property, which, combined with some mild assumptions shows that it is indeed a local Lyapunov function.

An efficient hybrid conjugate gradient method with sufficient descent property for unconstrained optimization

In order to take advantage of the strong theoretical properties of the FR method and computational efficiency of the method, we present a new hybrid conjugate gradient method based on the convex combination of these methods. In our method, the search directions satisfy the sufficient descent condition independent of any line search. Under some standard assumptions, we established global convergence property of our proposed method for general functions. Numerical comparisons on some test problems from the CUTEst library illustrate the efficiency and robustness of our proposed method in practice.

Numerical visualization of drop and opening process for parachute-payload system adopting fluid\u2013solid coupling simulation

In order to study the fluid–solid coupling dynamic characteristics of parachute-payload system during drop process and analyze the unsteady aerodynamic characteristics under finite mass opening conditions, an adaptive moving fluid mesh method is developed on the basis of the existing arbitrary Lagrangian–Eulerian (ALE) fluid–solid coupling method. The calculation results of open force and drop velocity on the C-9 parachute demonstrate the effectiveness of this method. On this basis, the effect of canopies with three different permeability on parachute-payload system motion characteristic including opening property, steady descent property and stability is studied. Comparative analysis is conducted for structures and characteristics of vortex with different canopy materials, and interference mechanism of unsteady flow for parachute-payload system in unsteady oscillation is revealed. The results show that the adaptive moving fluid mesh method can effectively eliminate restrictions of existing simulation methods for parachute-payload system and significantly reduce calculation time. For the lightweight parachute, permeability has significant effect on kinetic characteristic of parachute-payload system. Canopy with large permeability has small opening load and structural stress in opening stage. After opening, there are mainly small vortexes distributed evenly behind the canopy with good stability. However, canopy with small permeability has obvious breath behavior and oscillation in opening stage. The main vortexes periodically shed off after opening. With the change of permeability from small to large, Parachute-payload system eventually presents three steady descent modes: conical descent, gliding descent and stable vertical descent.Graphical abstract

An efficient projection-based algorithm without Lipschitz continuity for large-scale nonlinear pseudo-monotone equations

In this paper, an efficient algorithm is proposed for solving convex constrained equations. The method has the following characteristics: (i) its search direction satisfies the sufficient descent property which is independent of any line search conditions; (ii) the equations only satisfy pseudo-monotone property; (iii) its global convergence is proved without the Lipschitz continuity. We compare our numerical results for test problems with those of other derivative-free projection methods.

A Modified Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

The conjugate gradient method is one of the most popular methods to solve large-scale unconstrained optimization problems since it does not require the second derivative, such as Newton’s method or approximations. Moreover, the conjugate gradient method can be applied in many fields such as neural networks, image restoration, etc. Many complicated methods are proposed to solve these optimization functions in two or three terms. In this paper, we propose a simple, easy, efficient, and robust conjugate gradient method. The new method is constructed based on the Liu and Storey method to overcome the convergence problem and descent property. The new modified method satisfies the convergence properties and the sufficient descent condition under some assumptions. The numerical results show that the new method outperforms famous CG methods such as CG-Descent 5.3, Liu and Storey, and Dai and Liao. The numerical results include the number of iterations and CPU time.

New hybrid three-term spectral-conjugate gradient method for finding solutions of nonlinear monotone operator equations with applications

In this paper, we present a new hybrid spectral-conjugate gradient (SCG) algorithm for finding approximate solutions to nonlinear monotone operator equations. The hybrid conjugate gradient parameter has the Polak–Ribière–Polyak (PRP), Dai–Yuan (DY), Hestenes–Stiefel (HS) and Fletcher–Reeves (FR) as special cases. Moreover, the spectral parameter is selected such that the search direction has the descent property. Also, the search directions are bounded and the sequence of iterates generated by the new hybrid algorithm converge globally. Furthermore, numerical experiments were conducted on some benchmark nonlinear monotone operator equations to assess the efficiency of the proposed algorithm. Finally, the algorithm is shown to have the ability to recover disturbed signals.

A Convex Combination between Two Different Search Directions of Conjugate Gradient Method and Application in Image Restoration

The conjugate gradient is a useful tool in solving large- and small-scale unconstrained optimization problems. In addition, the conjugate gradient method can be applied in many fields, such as engineering, medical research, and computer science. In this paper, a convex combination of two different search directions is proposed. The new combination satisfies the sufficient descent condition and the convergence analysis. Moreover, a new conjugate gradient formula is proposed. The new formula satisfies the convergence properties with the descent property related to Hestenes–Stiefel conjugate gradient formula. The numerical results show that the new search direction outperforms both two search directions, making it convex between them. The numerical result includes the number of iterations, function evaluations, and central processing unit time. Finally, we present some examples about image restoration as an application of the proposed conjugate gradient method.

A robust algorithm for global optimization problems

In this paper, a global optimization algorithm namely Kerk and Rohanin’s Trusted Region is used to find the global minimizers by employing an interval technique; with it, the algorithm can find the region where a minimizer is located and will not get trapped in a local one. It is able to find the convex part within the non-convex feasible region. This algorithm has descent property and global convergence. The numerical results have shown the algorithm has an outstanding capability in locating global minimizers.

An efficient hybrid conjugate gradient method with descent properties under strong Wolfe line search

The hybrid conjugate gradient parameters are among the efficient variants of conjugate gradient (CG) methods for solving large-scale unconstrained optimization problems. This is due to their nice convergence properties and low memory requirements. In this paper, we present a new hybrid conjugate gradient method based on famous CG algorithms for large-scale unconstrained optimization. The proposed hybrid CG method can generate a descent search direction at each iteration provided the strong Wolfe line search is employed. Numerical results have been presented which show that the proposed method is efficient and promising.

A new parameter in three-term conjugate gradient algorithms for unconstrained optimization

<span>In this study, we develop a different parameter of three term conjugate gradient kind, this scheme depends principally on pure conjugacy condition (PCC), Whereas, the conjugacy condition (PCC) is an important condition in unconstrained non-linear optimization in general and in conjugate gradient methods in particular. The proposed method becomes converged, and satisfy conditions descent property by assuming some hypothesis, The numerical results display the effectiveness of the new method for solving test unconstrained non-linear optimization problems compared to other conjugate gradient algorithms such as Fletcher and Revees (FR) algorithm and three term Fletcher and Revees (TTFR) algorithm. and as shown in Table (1) from where in a number of iterations and evaluation of function and in Figures (1), (2) and (3) from where in A comparison of the number of iterations, A comparison of the number of times a function is calculated and A comparison of the time taken to perform the functions.</span>

Good Characteristics of The New Spectral Conjugate Gradient Method for Unconstrained Optimization

The spectral conjugate gradient (SCG) method is an effective method to solve large-scale nonlinear unconstrained optimization problems. In this work, we propose a new SCG method in which performance is numerically analyzed. We established the descent property and global convergence conditions based on assumptions through the strongWolfe-Powell line search. Numerical results were performed using benchmark functions widely used in many conventional functions to evaluate the efficiency of the proposed method.Subject Classification: 90C30, 90C06, 65K05, 65K10.

A new hybrid conjugate gradient method of unconstrained optimization methods

In this paper, we present a new hybrid method to solve a nonlinear unconstrained optimization problem by using conjugate gradient, which is a convex combination of Liu–Storey (LS) conjugate gradient method and Hager–Zhang (HZ) conjugate gradient method. This method possesses the sufficient descent property with Strong Wolfe line search and the global convergence with the strong Wolfe line search. In the end of this paper, we illustrate our method by giving some numerical examples.

A New Hybrid Three-Term Conjugate Gradient Algorithm for Large-Scale Unconstrained Problems

Three-term conjugate gradient methods have attracted much attention for large-scale unconstrained problems in recent years, since they have attractive practical factors such as simple computation, low memory requirement, better descent property and strong global convergence property. In this paper, a hybrid three-term conjugate gradient algorithm is proposed and it owns a sufficient descent property, independent of any line search technique. Under some mild conditions, the proposed method is globally convergent for uniformly convex objective functions. Meanwhile, by using the modified secant equation, the proposed method is also global convergence without convexity assumption on the objective function. Numerical results also indicate that the proposed algorithm is more efficient and reliable than the other methods for the testing problems.

Two-versions of descent conjugate gradient methods for large-scale unconstrained optimization

<p>The conjugate gradient methods are noted to be exceedingly valuable for solving large-scale unconstrained optimization problems since it needn't the storage of matrices. Mostly the parameter conjugate is the focus for conjugate gradient methods. The current paper proposes new methods of parameter of conjugate gradient type to solve problems of large-scale unconstrained optimization. A Hessian approximation in a diagonal matrix form on the basis of second and third-order Taylor series expansion was employed in this study. The sufficient descent property for the proposed algorithm are proved. The new method was converged globally. This new algorithm is found to be competitive to the algorithm of fletcher-reeves (FR) in a number of numerical experiments.</p>

Global Convergence Condition for a New Spectral Conjugate Gradient Method for Large-Scale Optimization

The spectral conjugate gradient (SCG) method is an effective method to solve large-scale nonlinear unconstrained optimization problems. In this work, a new spectral conjugate gradient method is proposed with a strong Wolfe-Powell line search (SWP). The idea of the new one is using the βBZA formula which is proposed by Baluch and et al., with suitable parameter φ denoted by (SCGBZA). Under the usual assumptions, the descent properties and overall global convergence of the proposed method (SCGBZA) are proved. The proposed method is numerically proven to be effective.

A fast algorithm for group square-root Lasso based group-sparse regression

Group square-root Lasso (GSRL) is a promising tool for group-sparse regression since the hyperparameter is independent of noise level. Recent works also reveal its connections to some statistically sound and hyperparameter-free methods, e.g., group-sparse iterative covariance-based estimation (GSPICE). However, the non-smoothness of the data-fitting term leads to the difficulty in solving the optimization problem of GSRL, and available solvers usually suffer either slow convergence or restrictions on the dictionary. In this paper, we propose a class of efficient solvers for GSRL in a block coordinate descent manner, including group-wise cyclic minimization (GCM) for group-wise orthonormal dictionary and generalized GCM (G-GCM) for general dictionary. Both strict descent property and global convergence are proved. To cope with signal processing applications, the complex-valued multiple measurement vectors (MMV) case is considered. The proposed algorithm can also be used for the fast implementation of methods with theoretical equivalence to GSRL, e.g., GSPICE. Significant superiority in computational efficiency is verified by simulation results.

An Efficient Modified AZPRP Conjugate Gradient Method for Large-Scale Unconstrained Optimization Problem

To find a solution of unconstrained optimization problems, we normally use a conjugate gradient (CG) method since it does not cost memory or storage of second derivative like Newton’s method or Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. Recently, a new modification of Polak and Ribiere method was proposed with new restart condition to give a so-call AZPRP method. In this paper, we propose a new modification of AZPRP CG method to solve large-scale unconstrained optimization problems based on a modification of restart condition. The new parameter satisfies the descent property and the global convergence analysis with the strong Wolfe-Powell line search. The numerical results prove that the new CG method is strongly aggressive compared with CG_Descent method. The comparisons are made under a set of more than 140 standard functions from the CUTEst library. The comparison includes number of iterations and CPU time.