Nonmonotone Line Search Technique Research Articles

A modified nonmonotone BFGS algorithm for unconstrained optimization

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size alpha_{k} to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method.

An adaptive nonmonotone global Barzilai–Borwein gradient method for unconstrained optimization

The Barzilai–Borwein (BB) gradient method has received many studies due to its simplicity and numerical efficiency. By incorporating a nonmonotone line search, Raydan (SIAM J Optim. 1997;7:26–33) has successfully extended the BB gradient method for solving general unconstrained optimization problems so that it is competitive with conjugate gradient methods. However, the numerical results reported by Raydan are poor for very ill-conditioned problems because the effect of the degree of nonmonotonicity may be noticeable. In this paper, we focus more on the nonmonotone line search technique used in the global Barzilai–Borwein (GBB) gradient method. We improve the performance of the GBB gradient method by proposing an adaptive nonmonotone line search based on the morphology of the objective function. We also prove the global convergence and the R-linear convergence rate of the proposed method under reasonable assumptions. Finally, we give some numerical experiments made on a set of unconstrained optimization test problems of the CUTEr collection. The results show the efficiency of the proposed method in the sense of the performance profile introduced (Math Program. 2002;91:201–213) by Dolan and Moré.

A quasi-Newton algorithm for large-scale nonlinear equations

In this paper, the algorithm for large-scale nonlinear equations is designed by the following steps: (i) a conjugate gradient (CG) algorithm is designed as a sub-algorithm to obtain the initial points of the main algorithm, where the sub-algorithm’s initial point does not have any restrictions; (ii) a quasi-Newton algorithm with the initial points given by sub-algorithm is defined as main algorithm, where a new nonmonotone line search technique is presented to get the step length alpha_{k}. The given nonmonotone line search technique can avoid computing the Jacobian matrix. The global convergence and the 1+q-order convergent rate of the main algorithm are established under suitable conditions. Numerical results show that the proposed method is competitive with a similar method for large-scale problems.

A new cubic convergent method for solving a system of nonlinear equations

ABSTRACTWe present a new cubic convergent method for solving a system of nonlinear equations. The new method can be viewed as a modified Chebyshev's method in which the difference of Jacobian matrixes replaces three order tensor. Therefore, the new method reduces the storage and computational cost. The new method possesses the local cubic convergence as well as Chebyshev's method. A rule is deduced to ensure the descent property of the search direction, and a nonmonotone line search technique is used to guarantee the global convergence. Numerical results indicate that the new method is competitive and efficient for some classical test problems.

A nonmonotone PRP conjugate gradient method for solving square and under-determined systems of equations

In this paper, we propose a new derivative-free preconditioned conjugate gradient method in order for solving large-scale square and under-determined nonlinear systems of equations. The proposed method is also equipped with a relaxed nonmonotone line search technique. Under some suitable assumptions, the global convergence property is established. Numerical results on some square and under-determined test systems show the efficiency and effectiveness of the new method in practice. An application of the new method for solving nonlinear integro-differential equations is also provided.

An approach based on dwindling filter method for positive definite generalized eigenvalue problem

In this paper, we present a new spectral residual method for solving large-scale positive definite generalized eigenvalue problems. The proposed algorithm is equipped with a dwindling multidimensional nonmonotone filter, in which the envelope dwindles as the step length decreases. We have also employed a relaxed nonmonotone line search technique in the structure of the algorithm which allows it to enjoy the nonmonotonicity from scratch. Under some mild and standard assumptions, the global convergence property of the proposed algorithm is established. An implementation of the new algorithm on some test problems shows the efficiency and effectiveness of the proposed algorithm in practice.

A nonmonotone Jacobian smoothing inexact Newton method for NCP

In this paper we propose Jacobian smoothing inexact Newton method for nonlinear complementarity problems (NCP) with derivative-free nonmonotone line search. This nonmonotone line search technique ensures globalization and is a combination of Grippo-Lampariello-Lucidi (GLL) and Li-Fukushima (LF) strategies, with the aim to take into account their advantages. The method is based on very well known Fischer-Burmeister reformulation of NCP and its smoothing Kanzow's approximation. The mixed Newton equation, which combines the semismooth function with the Jacobian of its smooth operator, is solved approximately in every iteration, so the method belongs to the class of Jacobian smoothing inexact Newton methods. The inexact search direction is not in general a descent direction and this is the reason why nonmonotone scheme is used for globalization. Global convergence and local superlinear convergence of method are proved. Numerical performances are also analyzed and point out that high level of nonmonotonicity of this line search rule enables robust and efficient method.

A new smoothing and regularization Newton method for the symmetric cone complementarity problem

Based on a regularized Chen–Harker–Kanzow–Smale (CHKS) smoothing function, we propose a new smoothing and regularization Newton method for solving the symmetric cone complementarity problem. By using the theory of Euclidean Jordan algebras, we establish the global and local quadratic convergence of the method on certain assumptions. The proposed method uses a nonmonotone line search technique which includes the usual monotone line search as a special case. In addition, our method treats both the smoothing parameter $$\mu $$ and the regularization parameter $$\varepsilon $$ as independent variables. Preliminary numerical results are reported which indicate that the proposed method is effective.

An efficient interior-point algorithm with new non-monotone line search filter method for nonlinear constrained programming

ABSTRACTAn efficient primal-dual interior-point algorithm using a new non-monotone line search filter method is presented for nonlinear constrained programming, which is widely applied in engineering optimization. The new non-monotone line search technique is introduced to lead to relaxed step acceptance conditions and improved convergence performance. It can also avoid the choice of the upper bound on the memory, which brings obvious disadvantages to traditional techniques. Under mild assumptions, the global convergence of the new non-monotone line search filter method is analysed, and fast local convergence is ensured by second order corrections. The proposed algorithm is applied to the classical alkylation process optimization problem and the results illustrate its effectiveness. Some comprehensive comparisons to existing methods are also presented.

A nonmonotone inexact Newton method for unconstrained optimization

It is well known that the Newton method has a second order rate of convergence and that it is widely used to solve optimization problems and nonlinear equations which arise from computational science, engineering analysis and other applications. However, two big disadvantages hinder its application: high computational cost for large scale problems and poor global performance in some complicated and difficult problems. Some inexact Newton methods have emerged over time. Among them, the Newton preconditioned conjugate gradient method is the most efficient and popular approach to overcome the first shortcoming while keeping rapid convergence. In this paper, we have improved the global performance of the inexact Newton method by developing a nonmonotone line search technique. We have also proved the global convergence of the proposed method under some conditions. Numerical experiments on a set of standard test problems are reported. They have shown that the proposed algorithm is promising.

An ODE-like nonmonotone method for nonsmooth convex optimization

Based on the Moreau–Yosida regularization and a modified nonmonotone line search technique, this paper presents an implementable ordinary differentiable equation-like method for solving a possibly nondifferentiable convex minimization problem by converting the original objective function to a once continuously differentiable function. The proposed method makes use of approximate function and gradient values of the Moreau–Yosida regularization instead of the corresponding exact values. Under some reasonable conditions, the proposed method is shown to be globally and superlinearly convergent. Some preliminary numerical results are also reported to show the efficiency of the proposed method.

Nonmonotone smoothing inexact Newton method for the nonlinear complementarity problem

Smoothing Newton methods have been successfully applied to solve the nonlinear complementarity problem (NCP). In this paper, we first study some properties of the generalized Fischer–Burmeister smoothing function. Based on this function, we then design a smoothing inexact Newton method for the NCP. At each iteration, a system of linear equations is solved only inexactly. Moreover, our method uses a nonmonotone line search technique which is much simpler than existing nonmonotone line searches used in smoothing Newton methods. Under suitable assumptions, we prove that the proposed method is globally and locally superlinearly convergent. Some numerical results are also reported.

A modified PRP conjugate gradient algorithm with nonmonotone line search for nonsmooth convex optimization problems

It is well-known that nonlinear conjugate gradient (CG) methods are preferred to solve large-scale smooth optimization problems due to their simplicity and low storage. However, the CG methods for nonsmooth optimization have not been studied. In this paper, a modified Polak–Ribiere–Polyak CG algorithm which combines with a nonmonotone line search technique is proposed for nonsmooth convex minimization. The search direction of the given method not only possesses the sufficiently descent property but also belongs to a trust region. Moreover, the search direction has not only the gradients information but also the functions information. The global convergence of the presented algorithm is established under suitable conditions. Numerical results show that the given method is competitive to other three methods.

A corrected Levenberg–Marquardt algorithm with a nonmonotone line search for the system of nonlinear equations

In this paper, we propose a corrected Levenberg–Marquardt method for the system of nonlinear equations, in which not only an L-M step and a corrected step are computed at every iteration but also a nonmonotone line search to find a new iteration point will be performed if a trial step is not accepted. To ensure the global convergence of the new method, a new nonmonotone line search technique is introduced for the merit function. The cubic convergence of the new method is proved under the local error bound condition which is weaker than nonsingularity. Some numerical results are reported, which shows that the algorithm is quite effective.

A new nonmonotone filter Barzilai–Borwein method for solving unconstrained optimization problems

In this paper, a finite filter is used in the structure of the Barzilai–Browein (BB) gradient method in order to propose a new modified BB algorithm for solving large-scale unconstrained optimization problems. Our algorithm is equipped with a relaxed nonmonotone line search technique which allows the algorithm to enjoy the nonmonotonicity properties from scratch. Under some suitable conditions, the global convergence property of the new proposed algorithm is established. Numerical results on some test problems in CUTEr library show the efficiency and effectiveness of the new algorithm in practice too.

A Nonmonotone Weighting Self-Adaptive Trust Region Algorithm for Unconstrained Nonconvex Optimization

A new trust region method is presented, which combines nonmonotone line search technique, a self-adaptive update rule for the trust region radius, and the weighting technique for the ratio between the actual reduction and the predicted reduction. Under reasonable assumptions, the global convergence of the method is established for unconstrained nonconvex optimization. Numerical results show that the new method is efficient and robust for solving unconstrained optimization problems.

A New Nonmonotone Trust Region Algorithm for Solving Unconstrained Optimization Problems

Based on the nonmonotone line search technique proposed by Gu and Mo (Appl. Math. Comput. 55, (2008) pp. 2158-2172), a new nonmonotone trust region algorithm is proposed for solving unconstrained optimization problems in this paper. The new algorithm

A new nonmonotone line search technique for unconstrained optimization

In this paper, a new nonmonotone line search rule is proposed,which is verified to be an improved version of the nonmonotone line search technique proposed by Zhang and Hager. Unlike the Zhang and Hager's method, our nonmonotone line search is proved to own a nice property similar to the standard Armijo line search. In virtue of such a property, global convergence is established for the developed algorithm, where the search direction is supposed to satisfy some mild conditions and the stepsize is chosen by the new line search rule. R-linear convergence of the developed algorithm is proved for strongly convex objective functions. The developed algorithm is used to solve the test problems available in the CUTEr, the numerical results demonstrate that the new line search strategy outperforms the other similar ones.

A line search SQP method without a penalty or a filter

This paper describes a new algorithm for nonlinear programming problems with inequality constraints. The proposed approach first solves a sequence of quadratic programming subproblems via the line search technique and to induce the global convergence, it establishes a new step acceptance mechanism that is neither a penalty function nor a filter. A nonmonotone line search technique from the unconstraint optimization is applied to accelerate the algorithm. Under some reasonable assumptions, the method can be proved to be globally convergent to a KT point. Preliminary numerical results are presented that show the potential efficiency of the new approach.

A modified Polak–Ribière–Polyak conjugate gradient algorithm for large-scale optimization problems

Mathematical programming is a rich and well-advanced area in operations research. However, there are still many challenging problems in mathematical programming, and the large-scale optimization problem is one of them. In this article, a modified Polak–Ribière–Polyak conjugate gradient algorithm that incorporates a non-monotone line search technique is presented. This method possesses not only gradient value information but also function value information. Moreover, the sufficient descent condition holds without any line search. Under suitable conditions, the global convergence is established for non-convex functions. Numerical results show that the proposed method is competitive with other conjugate gradient methods for large-scale optimization problems.