Nonmonotone Technique Research Articles

A fast algorithm for nonsmooth penalized clustering

Abstract As a novel framework of clustering analysis, penalized clustering is able to learn the number of clusters automatically, and therefore has aroused widespread interest recently. To address the computational difficulties arising from the nonsmoothness of the penalty, a simple iterative algorithm based on smoothing trust region (STR) can be used. However, since STR only needs first-order information of the model, it might exhibit slow convergence rate sometimes. To accelerate STR and further improve the efficiency of penalized clustering, we propose a nonmonotone smoothing trust region (NSTR) algorithm, in which nonmonotone technique and the Barzilai and Borwein (BB) method are utilized together. We also prove that the new algorithm is globally convergent and estimate its worst case computational complexity. Experimental results on both simulated and real-life data sets validate the effectiveness and efficiency of the proposed method.

An accelerated nonmonotone trust region method with adaptive trust region for unconstrained optimization

Trust region method is a robust method for optimization problems. In this paper, we propose a novel adaptive nonmonotone technique based on trust region methods for solving unconstrained optimization. In order to accelerate the convergence of trust region methods, an adaptive trust region is generated according to the Hessian of the iterate point. Both the nonmonotone techniques and this adaptive strategies can improve the trust region methods in the sense of convergence. We prove that the proposed method is locally superlinear convergence under some standard assumptions. Numerical results show that the new method is effective and has a high speed of convergence in practice.

A modified scaled memoryless BFGS preconditioned conjugate gradient algorithm for nonsmooth convex optimization

This paper presents a nonmonotone scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving nonsmooth convex optimization problems, which combines the idea of scaled memoryless BFGS preconditioned conjugate gradient method with the nonmonotone technique and the Moreau-Yosida regularization. The proposed method makes use of approximate function and gradient values of the Moreau-Yosida regularization instead of the corresponding exact values. Under mild conditions, the global convergence of the proposed method is established. Preliminary numerical results and related comparisons show that the proposed method can be applied to solve large scale nonsmooth convex optimization problems.

A nonmonotone scaled conjugate gradient algorithm for large-scale unconstrained optimization

ABSTRACTThis paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden–Fletcher–Goldfarb–Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation.

Nonmonotone Self-adaptive Levenberg–Marquardt Approach for Solving Systems of Nonlinear Equations

ABSTRACTThe well-known Levenberg–Marquardt method is used extensively to solve systems of nonlinear equations. An extension of the Levenberg–Marquardt method based on new nonmonotone technique is described. To decrease the total number of iterations, this method allows the sequence of objective function values to be nonmonotone, especially in the case where the objective function is ill-conditioned. Moreover, the parameter of Levenberg–Marquardt is produced according to the new nonmonotone strategy to use the advantages of the faster convergence of the Gauss–Newton method whenever iterates are near the optimizer, and the robustness of the steepest descent method in the case in which iterates are far away from the optimizer. The global and quadratic convergence of the proposed method is established. The results of numerical experiments are reported.

A New Restarting Adaptive Trust-Region Method for Unconstrained Optimization

In this paper, we present a new adaptive trust-region method for solving nonlinear unconstrained optimization problems. More precisely, a trust-region radius based on a nonmonotone technique uses an approximation of Hessian which is adaptively chosen. We produce a suitable trust-region radius; preserve the global convergence under classical assumptions to the first-order critical points; improve the practical performance of the new algorithm compared to other exiting variants. Moreover, the quadratic convergence rate is established under suitable conditions. Computational results on the CUTEst test collection of unconstrained problems are presented to show the effectiveness of the proposed algorithm compared with some exiting methods.

A Nonmonotone Projection Method for Constrained System of Nonlinear Equations

This paper deals with the nonmonotone projection algorithm for constrained nonlinear equations. For some starting points, the previous projection algorithms for the problem may encounter slow convergence which is related to the monotone behavior of the iterative sequence as well as the iterative direction. To circumvent this situation, we adopt the nonmonotone technique introduced by Dang to develop a nonmonotone projection algorithm. After constructing the nonmonotone projection algorithm, we show its convergence under some suitable condition. Preliminary numerical experiment is reported at the end of this paper, from which we can see that the algorithm we propose converges more quickly than that of the usual projection algorithm for some starting points.

Stable equilibrium configuration of two bar truss by an efficient nonmonotone global Barzilai–Borwein gradient method in a fuzzy environment

The stable equilibrium configuration of structures is a main goal in structural optimization. This goal may be achieved through minimizing the potential energy function. In the real world, sometimes, the input data and parameters of structural engineering design problems may be considered as fuzzy numbers which lead us to develop structural optimization methods in a fuzzy environment. In this regard, the present paper is intended to propose a fuzzy optimization scheme according to the nonmonotone globalization technique, the Barzilai–Borwein (BB) gradient method and the generalized Hukuhara differentiability (gH-differentiability). In fact, using the best benefits of BB-like methods i.e., simplicity, efficiency and low memory requirements, a modified global Barzilai–Borwein (GBB) gradient method is proposed for obtaining a non-dominated solution of the unconstrained fuzzy optimization related to the two bar asymmetric shallow truss in a fuzzy environment. The global convergence to first-order stationary points is also proved and the R-linear convergence rate is established under suitable assumptions. Furthermore, some numerical examples are given to illustrate the main results.

A memory gradient method based on the nonmonotone technique

In this paper, we present a new nonmonotone memory gradient algorithm for unconstrained optimization problems. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under mild assumptions, the global and local convergence results of the proposed algorithm are established respectively. Numerical results are also reported to show that the proposed method is suitable to solve large-scale optimization problems and is more stable than other similar methods in practical computation.

A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques

In recent years, cubic regularization algorithms for unconstrained optimization have been defined as alternatives to trust-region and line search schemes. These regularization techniques are based on the strategy of computing an (approximate) global minimizer of a cubic overestimator of the objective function. In this work we focus on the adaptive regularization algorithm using cubics (ARC) proposed in Cartis et al. [Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results, Mathematical Programming A 127 (2011), pp. 245–295]. Our purpose is to design a modified version of ARC in order to improve the computational efficiency preserving global convergence properties. The basic idea is to suitably combine a Goldstein-type line search and a nonmonotone accepting criterion with the aim of advantageously exploiting the possible good descent properties of the trial step computed as (approximate) minimizer of the cubic model. Global convergence properties of the proposed nonmonotone ARC algorithm are proved. Numerical experiments are performed and the obtained results clearly show satisfactory performance of the new algorithm when compared to the basic ARC algorithm.

Two globally convergent nonmonotone trust-region methods for unconstrained optimization

This paper addresses some trust-region methods equipped with nonmonotone strategies for solving nonlinear unconstrained optimization problems. More specifically, the importance of using nonmonotone techniques in nonlinear optimization is motivated, then two new nonmonotone terms are proposed, and their combinations into the traditional trust-region framework are studied. The global convergence to first- and second-order stationary points and local superlinear and quadratic convergence rates for both algorithms are established. Numerical experiments on the CUTEst test collection of unconstrained problems and some highly nonlinear test functions are reported, where a comparison among state-of-the-art nonmonotone trust-region methods shows the efficiency of the proposed nonmonotone schemes.

An Adaptive Trust Region Method Based on Simple Conic Models

A new adaptive trust region algorithm with simple conic models is proposed. By use of the simple conic model, the new method needs less memory capacitance and computational complexity. The nonmonotone and adaptive techniques are introduced to improve the efficiency of the proposed algorithm. The convergence results of the method are proved under certain conditions. Numerical tests show that the new algorithm is efficient and robust.

A relaxed nonmonotone adaptive trust region method for solving unconstrained optimization problems

In this paper, we present a new relaxed nonmonotone trust region method with adaptive radius for solving unconstrained optimization problems. The proposed method combines a relaxed nonmonotone technique with a modified version of the adaptive trust region strategy proposed by Shi and Guo (J Comput Appl Math 213:509---520, 2008). Under some suitable and standard assumptions, we establish the global convergence property as well as the superlinear convergence rate for the new method. Numerical results on some test problems show the efficiency and effectiveness of the new proposed method in practice.

AN IMPROVED ADAPTIVE TRUST-REGION METHOD FOR UNCONSTRAINED OPTIMIZATION

In this study, we propose a trust-region-based procedure to solve unconstrained optimization problems that take advantage of the nonmonotone technique to introduce an efficient adaptive radius strategy. In our approach, the adaptive technique leads to decreasing the total number of iterations, while utilizing the structure of nonmonotone formula helps us to handle large-scale problems. The new algorithm preserves the global convergence and has quadratic convergence under suitable conditions. Preliminary numerical experiments on standard test problems indicate the efficiency and robustness of the proposed approach for solving unconstrained optimization problems.

Global and local convergence of a nonmonotone SQP method for constrained nonlinear optimization

In this paper, we propose a robust sequential quadratic programming (SQP) method for nonlinear programming without using any explicit penalty function and filter. The method embeds the modified QP subproblem proposed by Burke and Han (Math Program 43:277---303, 1989) for the search direction, which overcomes the common difficulty in the traditional SQP methods, namely the inconsistency of the quadratic programming subproblems. A non-monotonic technique is employed further in a framework in which the trial point is accepted whenever there is a sufficient relaxed reduction of the objective function or the constraint violation function. A forcing sequence possibly tending to zero is introduced to control the constraint violation dynamically, which is able to prevent the constraint violation from over-relaxing and plays a crucial role in global convergence and the local fast convergence as well. We prove that the method converges globally without the Mangasarian---Fromovitz constraint qualification (MFCQ). In particular, we show that any feasible limit point that satisfies the relaxed constant positive linear dependence constraint qualification is also a Karush---Kuhn---Tucker point. Under the strict MFCQ and the second order sufficient condition, furthermore, we establish the superlinear convergence. Preliminary numerical results show the efficiency of our method.

An efficient adaptive trust-region method for systems of nonlinear equations

We present an adaptive trust-region algorithm to solve systems of nonlinear equations. Using the nonmonotone technique of Grippo, Lampariello and Lucidi, we introduce a new adaptive radius to decrease the total number of iterations and function evaluations. In contrast with the pervious methods, the new adaptive radius ensures that the size of radius is not too large or too small. We show that the sequence generated by the proposed adaptive radius is decreasing, so it prevents the production of too large radius as possible. Furthermore, it is shown that this sequence is reduced slowly, so it prevents the production of the intensely small radius. The global and quadratic convergence of the proposed approach are proved. Preliminary numerical results of our algorithm are also reported which indicate the promising behaviour of the new procedure to solve systems of nonlinear equations.

A new trust region method for solving least-square transformation of system of equalities and inequalities

In this paper, a new nonmonotone trust region method with adaptive radius is proposed for solving system of equalities and inequalities. This method combines a new nonmonotone technique with a new adaptive strategy based on the Shi and Guo’s adaptive technique in (J Comput Appl Math 213:509–520, 2008), which makes full use of the current point information. Under some standard assumptions, the global convergence property as well as the superlinear convergence rate are established for the new method. Numerical results on some nonlinear systems of equalities and inequalities indicate the efficiency and robustness of the proposed method in practice.

Smoothing inexact Newton methods for NCP with various nonmonotone techniques

AbstractVarious iterative methods for solving nonlinear complementarity problems (NCP) are developed in recent years. In this paper we propose Jacobian smoothing inexact Newton methods for NCP with different nonmonotone strategies. The methods are based on semismooth equation reformulation of NCP by Fischer‐Burmeister function. Nonmonotone line‐search techniques are used for globalization procedure. Numerical performance of algorithms are compared. (© 2013 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

A new nonmonotone trust region method for unconstrained optimization equipped by an efficient adaptive radius

In this paper, we present a new nonmonotone trust region method with adaptive radius which is equipped by a nonmonotone line search technique for solving unconstrained optimization problems. The proposed method combines a modified version of the Li and Fukushima's nonmonotone technique in D.H. Li and M. Fukushima [A derivative-free line search and global convergence of Broyden like method for nonlinear equations, Optim. Methods Softw.13 (2000), pp. 181–201] for solving nonlinear systems with a new variant of Shi and Guo's adaptive strategy in Z.J. Shi and J.H. Guo [A new trust region methods for unconstrained optimization, J. Comput. Appl. Math. 213 (2008), pp. 509–520] for updating the trust region radius. The method performs a nonmonotone Armijo-type line search whenever the trial step is rejected. Under some standard assumptions, we provide the global convergence property as well as the superlinear and quadratic convergence rates for the new method. Numerical results show the efficiency and effectiveness of the new proposed method in practice.

A BFGS trust-region method with a new nonmonotone technique for nonlinear equations

In this paper, we consider a trust-region method for solving nonlinear equations which employs a new nonmonotone technique. A strong nonmonotone strategy and a weaker nonmonotone strategy can be obtained by choosing the parameter adaptively. Thus, the disadvantages of the traditional nonmonotone strategy can be avoided. It does not need to compute the Jacobian matrix at every iteration, so that the workload and time are decreased. Theoretical analysis indicates that the new algorithm preserves the global convergence under classical assumptions. Moreover, superlinear and quadratic convergence are established under suitable conditions. Numerical experiments show the efficiency and effectiveness of the proposed method for solving nonlinear equations.