Descent Condition Research Articles

On the Barzilai–Borwein gradient methods with structured secant equation for nonlinear least squares problems

ABSTRACT We propose structured spectral gradient algorithms for solving nonlinear least squares problems based on a modified structured secant equation. The idea was to integrate more details of the Hessian of the objective function into the standardized spectral parameters with the goal of improving numerical efficiency. We safeguard the structured spectral parameters to avoid negative curvature search direction. The sequence of the search direction generated by the respective proposed algorithm satisfies the sufficient descent condition. Using a nonmonotone line search strategy, we establish the global convergence of the proposed algorithms under some standard assumptions. Numerical experiments on some benchmark test problems show that the proposed algorithms are efficient and outperform some existing competitors.

Application of Spectral PRP Conjugate Gradient Parameter for Unconstrained Optimization Problems

Conjugate Gradient (CG) method have been utilised to solve nonlinear unconstrained optimization problems because of less storage locations and fewer computational cost in dealing with large-scale problems. In this paper, we present a real life application of spectral PRP Conjugate Gradient method in regression analysis, the proposed method is suitably deriving from the CG search direction without secant condition. Some benchmark functions with several variables have been use to prove the global convergence properties and satisfies sufficient descent condition. The numerical results are certifying by exact line search techniques; the method outperform the prominent least square method

A New conjugate gradient method for unconstrained optimization problems with descent property

In this paper, we propose a new conjugate gradient method for solving nonlinear unconstrained optimization. The new method consists of three parts, the first part of them is the parameter of Hestenes-Stiefel (HS). The proposed method is satisfying the descent condition, sufficient descent condition and conjugacy condition. We give some numerical results to show the efficiency of the suggested method.

A variant of hybrid conjugate gradient methods based on the convex combination for optimization

<p><span>On some studies a conjugate parameter plays an important role for the conjugate gradient methods. In this paper, a variant of hybrid is provided in the search direction based on the convex combination. This search direction ensures that the descent condition holds. The global convergence of the variant of hybrid is also obtained. Our strong evidence is a numerical analysis showing that the proposed variant of hybrid method is efficient than the Hestenes and Stiefel method. </span></p>

A Modified Descent Spectral Conjugate Gradient Method for Unconstrained Optimization

Recently, Peyghami et al. (Optim Meth Soft 43:1–28, 2015) proposed a modified secant equation, which applied a new updating Yabe and Takano’s rule as an adaptive version of the parameter of conjugate gradient. Here, using this modified secant equation, we propose a new modified descent spectral nonlinear conjugate gradient method. The proposed method includes two major features; the higher-order accuracy in approximating the second-order curvature information of the objective function and the sufficient descent condition. Global convergence of the proposed method is proved for uniformly convex functions and general functions. Numerical experiments are done on a set of test functions of the CUTEr collection. The results are compared with some well-known methods.

New subspace minimization conjugate gradient methods based on regularization model for unconstrained optimization

In this paper, two new subspace minimization conjugate gradient methods based on p-regularization models are proposed, where a special scaled norm in p-regularization model is analyzed. Different choices of special scaled norm lead to different solutions to the p-regularized subproblem. Based on the analyses of the solutions in a two-dimensional subspace, we derive new directions satisfying the sufficient descent condition. With a modified nonmonotone line search, we establish the global convergence of the proposed methods under mild assumptions. R-linear convergence of the proposed methods is also analyzed. Numerical results show that, for the CUTEr library, the proposed methods are superior to four conjugate gradient methods, which were proposed by Hager and Zhang (SIAM J. Optim. 16(1):170–192, 2005), Dai and Kou (SIAM J. Optim. 23(1):296–320, 2013), Liu and Liu (J. Optim. Theory. Appl. 180(3):879–906, 2019) and Li et al. (Comput. Appl. Math. 38(1):2019), respectively.

GAIT CHARACTERISTICS OF ELDERLY WOMEN IN OVERGROUND, ASCENT AND DESCENT WALKWAY CONDITIONS

Gait assessment is important for identification of potential faller among the elderly populations. Slope walking is associated with fall risk factor and elderly women have higher fall rate compared with elderly men. Therefore, this study investigated gait characteristics of elderly women in overground and slope walkway conditions. Thirty healthy elderly women (15 younger-elderly women and 15 older-elderly women) walked along the linear walkway including three walking conditions (overground, ascent and descent conditions). Temporal gait variables and normalized peak vertical GRF (ground reaction force) variables were derived from commercial motion analysis software. Repeated-measures analysis of variance (ANOVA) was evaluated to compare mean differences of the three conditions and mean difference between younger and older elderly women. All gait characteristics were significantly different from the slope walking conditions ([Formula: see text]). Elderly women walked with longer loading response and mid stance phase during descent walking. Also, ascent walking induced a longer terminal stance phase. Interactions of age and walkway conditions were also significant in vertical GRF, where older-elderly women were greater than younger-elderly women in ascent walkway condition ([Formula: see text]) and in descent walkway condition ([Formula: see text]). These findings suggest that specific-walkway condition should be considered for fall prevention and clinical interventions in elderly women.

Conjugate Gradient ‎Method: A Developed Version to Resolve Unconstrained Optimization Problems

One of the important methods that are widely utilized to resolve ‎unconstrained ‎optimization problems is the Conjugate Gradient (CG) method. This paper aims to propose a new version of the CG method on the basis of Weak Wolfe-Powell (WWP) line search. The assumption is bounded below optimization problems with the Lipschitz continuous gradient. The new parameter obtains global convergence properties when the WWP line search is used. The descent condition is established without using any line search. The performance of the proposed CG method is tested by obtaining some unconstrained optimization problems from the CUTEst library. Testing results show that the proposed version of the CG method outperforms CG-DESCENT version 5.3 in terms of CPU time, function evaluations, gradient evaluations and number of iterations.

A modified nonlinear Polak–Ribière–Polyak conjugate gradient method with sufficient descent property

In this paper, a small and necessary revision on an assumption condition of Aminifard and Babaie-Kafaki (Calcolo, 2019. https://doi.org/10.1007/s10092-019-0312-9 ) is made. By a little modification, a new conjugate gradient method is proposed, in which the search directions satisfy the sufficient descent condition with the strong Wolfe line search. The main difference between two algorithms is that the proposed method is globally convergent without boundedness assumption on the steplength. Comparative numerical results demonstrating efficiency of the proposed method are reported.

A new spectral conjugate gradient method with descent condition and global convergence property for unconstrained optimization

The Spectral conjugate gradient method is an efficient method for solving large-scale unconstrained optimization problems. In this paper, we propose a new spectral conjugate gradient method in which performance is analyzed numerically. We establish the descent condition and global convergence property under some assumptions and the strong Wolfe line search. Numerical experiments to evaluate the method’s efficiency are conducted using 98 problems with various dimensions and initial points. The numerical results based on the number of iterations and central processing unit time show that the new method has a high performance computational.

Enhanced Dai–Liao conjugate gradient methods for systems of monotone nonlinear equations

In this paper, we propose two conjugate gradient methods for solving large-scale monotone nonlinear equations. The methods are developed by combining the hyperplane projection method by Solodov and Svaiter (Reformulation: nonsmooth, piecewise smooth, semismooth and smoothing methods. Springer, pp 355–369, 1998) and two modified search directions of the famous Dai and Liao (Appl Math Optim 43(1): 87–101, 2001) method. It is shown that the proposed schemes satisfy the sufficient descent condition. The global convergence of the methods are established under mild conditions, and computational experiments on some benchmark test problems show that the methods are promising.

The convergence properties of some descent conjugate gradient algorithms for optimization models

The three-term conjugate gradient (CG) algorithms are among the efficient variants of CG method for convex and nonconvex functions. This is because most three-term algorithms are constructed using the classical CG method whose numerical performance has been tested and convergence proved. In this paper, we present a modification of RMIL$+$ CG method proposed by Dai [Z. Dai, Appl. Math. Comput., \(\bf 267\) (2016), 297--300] based on the convergence analysis of RMIL (2012) CG method. Interestingly, the modified method possesses sufficient descent condition and the global convergence prove was established using exact minimization condition. We further extended the results of the modified RMIL$+$ to construct a three-term CG algorithm and also show that the method satisfies the sufficient descent condition under the strong Wolfe line search. Preliminary numerical results are reported based on known benchmark problems which show that the proposed methods are efficient and promising compare to other CG methods.

A Hessian-Free Gradient Flow (HFGF) method for the optimisation of deep learning neural networks

This paper presents a novel optimisation method, termed Hessian-free Gradient Flow, for the optimisation of deep neural networks. The algorithm entails the design characteristics of the Truncated Newton, Conjugate Gradient and Gradient Flow method. It employs a finite difference approximation scheme to make the algorithm Hessian-free and makes use of Armijo conditions to determine the descent condition. The method is first tested on standard testing functions with a high optimisation model dimensionality. Performance on the testing functions has demonstrated the potential of the algorithm to be applied to large-scale optimisation problems. The algorithm is then tested on classification and regression tasks using real-world datasets. Comparable performance to conventional optimisers has been obtained in both cases.

A new projection technique for developing a Liu-Storey method to solve nonlinear systems of monotone equations

The projection technique is a very important method and efficient for solving unconstraint optimization and nonlinear equations. In this study, we developed a Liu-Story (LS) algorithm for solving monotone equations of nonlinear systems. The new algorithm satisfies the sufficient descent condition and it’s a suitable method of large scale equations for its limited memory. We established a global convergence of suggest method under the mild conditions. Numerical results proved that the new algorithm works well and promising.

A globally convergent projection method for a system of nonlinear monotone equations

ABSTRACT In this paper, a new conjugate gradient-based projection method for solving a system of nonlinear monotone equations is proposed. The method can be viewed as an extension of a family of conjugate gradient methods for unconstrained optimization by Li et al. [A new family of conjugate gradient methods for unconstrained optimization, J. Appl. Math. Comput. 58 (2018), pp. 219–234]. The proposed method is derivative-free which makes it suitable for large-scale nonlinear monotone equations. We show that the method satisfies the descent condition independent of line searches and that the method is globally convergent. Numerical results indicate that the proposed method is efficient.

A Barzilai-Borwein gradient projection method for sparse signal and blurred image restoration

We present a Barzilai-Borwein gradient method using the hyperplane projection technique of Solodov and Svaiter (1998) for solving the non-smooth nonlinear monotone equation arising from the reformulation of the ℓ1-norm regularized problem. The proposed method is an extension of the modified method by Liu and Duan (J. Inequal. Appl. 2015(1), 8, 2015) for solving signal and image restoration problems. The method is derivative-free and its search direction satisfies the sufficient descent condition. Numerical experiments presented show that the proposed method can recover sparse signals in fewer iterations and less CPU time and can reconstruct blurred images with higher quality compared to similar methods in the literature.

New conjugate gradient algorithms based on self-scaling memoryless Broyden–Fletcher–Goldfarb–Shanno method

Three new procedures for computation the scaling parameter in the self-scaling memoryless Broyden–Fletcher–Goldfarb–Shanno search direction with a parameter are presented. The first two are based on clustering the eigenvalues of the self-scaling memoryless Broyden–Fletcher–Goldfarb–Shanno iteration matrix with a parameter by using the determinant or the trace of this matrix. The third one is based on minimizing the measure function of Byrd and Nocedal (SIAM J Numer Anal 26:727–739, 1989). For all these three algorithms the sufficient descent condition is established. The stepsize is computed using the standard Wolfe line search. Under the standard Wolfe line search the global convergence of these algorithms is established. By using 80 unconstrained optimization test problems, with different structures and complexities, it is shown that the performances of the self-scaling memoryless algorithms based on the determinant or on the trace of the iteration matrix or on minimizing the measure function are better than those of CG_DESCENT (version 1.4) with Wolfe line search (Hager and Zhang in SIAM J Optim 16:170–192, 2005), the self-scaling memoryless BFGS algorithms with scaling parameter proposed by Oren and Spedicato (Math Program 10:70–90, 1976) and by Oren and Luenberger (Manag Sci 20:845–862, 1974), LBFGS by Liu and Nocedal (Math Program 45:503–528, 1989) and the standard BFGS. The self-scaling memoryless algorithm based on minimizing the measure function of Byrd and Nocedal is slightly top performer versus the same algorithms based on the determinant or on the trace of the iteration matrix.

A Projection Hestenes–Stiefel Method with Spectral Parameter for Nonlinear Monotone Equations and Signal Processing

A number of practical problems in science and engineering can be converted into a system of nonlinear equations and therefore, it is imperative to develop efficient methods for solving such equations. Due to their nice convergence properties and low storage requirements, conjugate gradient methods are considered among the most efficient for solving large-scale nonlinear equations. In this paper, a modified conjugate gradient method is proposed based on a projection technique and a suitable line search strategy. The proposed method is matrix-free and its sequence of search directions satisfies sufficient descent condition. Under the assumption that the underlying function is monotone and Lipschitzian continuous, the global convergence of the proposed method is established. The method is applied to solve some benchmark monotone nonlinear equations and also extended to solve ℓ 1 -norm regularized problems to reconstruct a sparse signal in compressive sensing. Numerical comparison with some existing methods shows that the proposed method is competitive, efficient and promising.

The new spectral conjugate gradient method for large-scale unconstrained optimisation

The spectral conjugate gradient methods are very interesting and have been proved to be effective for strictly convex quadratic minimisation. In this paper, a new spectral conjugate gradient method is proposed to solve large-scale unconstrained optimisation problems. Motivated by the advantages of approximate optimal stepsize strategy used in the gradient method, we design a new scheme for the choices of the spectral and conjugate parameters. Furthermore, the new search direction satisfies the spectral property and sufficient descent condition. Under some suitable assumptions, the global convergence of the developed method is established. Numerical comparisons show better behaviour of the proposed method with respect to some existing methods for a set of 130 test problems.

A new modification nonlinear conjugate gradient method with strong wolf-powell line search

<span>The conjugate gradient method has played a special role in solving large-scale unconstrained Optimization problems. In this paper, we propose a new family of CG coefficients that possess sufficient descent conditions and global convergence properties this CG method is similar to (Wei et al) [7]. Global convergence result is established under Strong Wolf-Powell line search. Numerical results to find the optimum solution of some test functions show the new proposed formula has the best result in CPU time and the number of iterations, and the number of gradient evaluations when it comparing with FR, PRP, DY, and WYL </span>