Nonmonotone Technique Research Articles

A nonmonotonic hybrid algorithm for min-max problem

In this paper nonlinear min-max problems are discussed. Instead of the traditional nonmonotone approach presented by Grippo et al., we use the nonmonotonic technique proposed by Zhang and Hager (SIAM J. Optim. 14(4):1043–1056, 2004), to propose a new nonmonotonic algorithm for min-max problem. Moreover, using hybrid technique can make best use of the advantages of both trust region methods and line search methods. In addition, the new approach can reduce the possibility of solving two quadratic subproblems furthest, and circumvent the difficulties of the Maratos effect occurred in the nonsmooth optimization. Under reasonable conditions, the global convergence and the rate of superlinear convergence are established. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

A nonmonotone approximate sequence algorithm for unconstrained nonlinear optimization

A new nonmonotone algorithm is proposed and analyzed for unconstrained nonlinear optimization. The nonmonotone techniques applied in this algorithm are based on the estimate sequence proposed by Nesterov (Introductory Lectures on Convex Optimization: A Basic Course, 2004) for convex optimization. Under proper assumptions, global convergence of this algorithm is established for minimizing general nonlinear objective function with Lipschitz continuous derivatives. For convex objective function, this algorithm maintains the optimal convergence rate of convex optimization. In numerical experiments, this algorithm is specified by employing safe-guarded nonlinear conjugate gradient search directions. Numerical results show the nonmonotone algorithm performs significantly better than the corresponding monotone algorithm for solving the unconstrained optimization problems in the CUTEr (Bongartz et al. in ACM Trans. Math. Softw. 21:123---160, 1995) library.

An inexact line search approach using modified nonmonotone strategy for unconstrained optimization

This paper concerns with a new nonmonotone strategy and its application to the line search approach for unconstrained optimization. It has been believed that nonmonotone techniques can improve the possibility of finding the global optimum and increase the convergence rate of the algorithms. We first introduce a new nonmonotone strategy which includes a convex combination of the maximum function value of some preceding successful iterates and the current function value. We then incorporate the proposed nonmonotone strategy into an inexact Armijo-type line search approach to construct a more relaxed line search procedure. The global convergence to first-order stationary points is subsequently proved and the R-linear convergence rate are established under suitable assumptions. Preliminary numerical results finally show the efficiency and the robustness of the proposed approach for solving unconstrained nonlinear optimization problems.

An adaptive nonmonotone trust-region method with curvilinear search for minimax problem

In this paper we propose an adaptive nonmonotone algorithm for minimax problem. Unlike traditional nonmonotone method, the nonmonotone technique applied to our method is based on the nonmonotone technique proposed by Zhang and Hager [H.C. Zhang, W.W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14(4)(2004) 1043–1056] instead of that presented by Grippo et al. [L. Grippo, F. Lampariello, S. Lucidi, A nonmonotone line search technique for Newton’s method, SIAM J. Numer. Anal. 23(4)(1986) 707–716]. Meanwhile, by using adaptive technique, it can adaptively perform the nonmonotone trust-region step or nonmonotone curvilinear search step when the solution of subproblems is unacceptable. Global and superlinear convergences of the method are obtained under suitable conditions. Preliminary numerical results are reported to show the effectiveness of the proposed algorithm.

A modified ODE-based algorithm for unconstrained optimization problems

This paper presents a modified ODE-based algorithm for unconstrained optimization problems. It combines the idea of IMPBOT algorithm with nonmonotone and subspace techniques. The main feature of this method is that at each iteration, a lower dimensional system of linear equations is solved to obtain a trial step. Under some standard assumptions, the method is proven to be globally convergent. Numerical results show the efficiency of this proposed method in practical computation.

Nonmonotone retrospective conic trust region method for unconstrained optimization

We propose a retrospective conic trust region method for unconstrained optimization. It can be regarded as an extension of the retrospective trust region method based on a quadratic model which was first proposed by Bastin et al (2010). Nonmonotone technique is added to accelerate the speed of the algorithm. Under some mild conditions, the sequencegenerated by our algorithm converges to a stationary point. Numerical tests on a set of standard testing problems confirm the efficiency of our new method.

Parallel Variable Distribution Algorithm for Constrained Optimization with Nonmonotone Technique

A modified parallel variable distribution (PVD) algorithm for solving large-scale constrained optimization problems is developed, which modifies quadratic subproblemQPlat each iteration instead of theQPl0of the SQP-type PVD algorithm proposed by C. A. Sagastizábal and M. V. Solodov in 2002. The algorithm can circumvent the difficulties associated with the possible inconsistency ofQPl0subproblem of the original SQP method. Moreover, we introduce a nonmonotone technique instead of the penalty function to carry out the line search procedure with more flexibly. Under appropriate conditions, the global convergence of the method is established. In the final part, parallel numerical experiments are implemented on CUDA based on GPU (Graphics Processing unit).

A non-monotone retrospective trust-region method for unconstrained optimization

In this paper, a new non-monotone trust-region algorithm is proposed for solving unconstrained nonlinear optimization problems. We modify the retrospective ratio which is introduced by Bastin et al. [Math. Program., Ser. A (2010) 123: 395-418] to form a convex combination ratio for updating the trust-region radius. Then we combine the non-monotone technique with this new framework of trust-region algorithm. The new algorithm is shown to be globally convergent to a first-order criticalpoint. Numerical experiments on CUTEr problems indicate that it is competitive with both the original retrospective trust-region algorithm and the classical trust-region algorithms.

A Nonmonotone Line Search Slackness Technique for Unconstrained Optimization

This paper mainly aims to study a new nonmonotone line search slackness technique for unconstrained optimization problems and show that it possesses the global convergence without needing condition of convexity. We establish the corresponding algorithm and illustrate its effectiveness by virtue of some numerical tests. Simulation results indicate that the proposed method is very effective for non-convex functions.

An ODE-based nonmonotone method for unconstrained optimization problems

This paper proposes an ODE-based nonmonotone method for unconstrained optimization problems, which combines the idea of IMPBOT with the nonmonotone technique. The main characteristic of this method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified L-BFGS two loop recursion that requires only vector inner products, thus reducing the matrix computation and storage. Then a modified nonmonotone line search is performed to generate next iterative point instead of resolving the linear system. Under some reasonable assumptions, the method is proven to be globally and superlinearly convergent. Numerical results show the efficiency of this proposed method in practical computation.

Trust-region method for box-constrained semismooth equations and its applications to complementary problems

In this paper, we propose a new trust-region algorithm for bound-constrained semismooth systems of equations. Trust-region subproblem is defined by minimizing a quadratic function subject only to a rectangular constraint. By employing a new active set and nonmonotone techniques, solution of the equations can be found effective. Global and local convergence results of the proposed algorithm are established under reasonable conditions. The algorithm is applied and tested on complementary problems and the experiments show that our method is efficient.

A new nonmonotone adaptive trust region method based on simple quadratic models

Based on simple quadratic models of the trust region subproblem, we combine the trust region method with the nonmonotone and adaptive techniques to propose a new nonmonotone adaptive trust region algorithm for unconstrained optimization. Unlike traditional trust region method, our trust region subproblem is very simple by using a new scale approximation of the minimizing function’s Hessian. The new method needs less memory capacitance and computational complexity. The convergence results of the method are proved under certain conditions. Numerical results show that the new method is effective and attractive for large scale unconstrained problems.

A filter algorithm for nonlinear systems of equalities and inequalities

In this paper, we propose a filter algorithm for nonlinear systems of equalities and inequalities. In the algorithm, the systems of nonlinear equalities and inequalities are transformed into a nonlinear programming problem. The new algorithm uses the non-monotone technique in conjunction with filter idea and a global line search strategy that ensures convergence from any starting point. Under mild conditions, global convergence and local superlinear convergence rate are obtained. Numerical results demonstrate that the new algorithm is effective.

A new modified nonmonotone adaptive trust region method for unconstrained optimization

In this paper, we present an adaptive trust region method for solving unconstrained optimization problems which combines nonmonotone technique with a new update rule for the trust region radius. At each iteration, our method can adjust the trust region radius of related subproblem. We construct a new ratio to adjust the next trust region radius which is different from the ratio in the traditional trust region methods. The global and superlinear convergence results of the method are established under reasonable assumptions. Numerical results show that the new method is efficient for unconstrained optimization problems.

An inexact secant algorithm for large scale nonlinear systems of equalities and inequalities

In this paper, an inexact secant algorithm in association with nonmonotone technique and filter is proposed for solving the large scale nonlinear systems of equalities and inequalities. The systems are transformed into a continuous constrained optimization solved by inexact secant algorithm. Global convergence of the proposed algorithm is established under the reasonable conditions. Numerical results validate the effectiveness of our approach.

Nonmonotone BFGS-trained recurrent neural networks for temporal sequence processing

In this paper we propose a nonmonotone approach to recurrent neural networks training for temporal sequence processing applications. This approach allows learning performance to deteriorate in some iterations, nevertheless the network’s performance is improved over time. A self-scaling BFGS is equipped with an adaptive nonmonotone technique that employs approximations of the Lipschitz constant and is tested on a set of sequence processing problems. Simulation results show that the proposed algorithm outperforms the BFGS as well as other methods previously applied to these sequences, providing an effective modification that is capable of training recurrent networks of various architectures.

A nonmonotone adaptive trust region method for unconstrained optimization based on conic model

In this paper, we present a nonmonotone adaptive trust region method for unconstrained optimization based on conic model. The new method combines nonmonotone technique and a new way to determine trust region radius at each iteration. The local and global convergence properties are proved under reasonable assumptions. Numerical experiments show that our algorithm is effective.

Nonmonotone trust region algorithm for unconstrained optimization problems

In this paper, a nonmonotone trust region algorithm for unconstrained optimization problems is presented. In the algorithm, a kind of nonmonotone technique, which is evidently different from Grippo, Lampariello and Lucidi’s approach, is used. Under mild conditions, global and local convergence results of the algorithm are established. Preliminary numerical results show that the new algorithm is efficient.

Global Convergence of a Nonmonotone Trust Region Algorithm with Memory for Unconstrained Optimization

In this paper, we consider a trust region algorithm for unconstrained optimization problems. Unlike the traditional memoryless trust region methods, our trust region model includes memory of the past iteration, which makes the algorithm less myopic in the sense that its behavior is not completely dominated by the local nature of the objective function, but rather by a more global view. The global convergence is established by using a nonmonotone technique. The numerical tests are also given to show the efficiency of our proposed method.

A non-monotone line search multidimensional filter-SQP method for general nonlinear programming

In this paper, we propose a non-monotone line search multidimensional filter-SQP method for general nonlinear programming based on the Wachter---Biegler methods for nonlinear equality constrained programming. Under mild conditions, the global convergence of the new method is proved. Furthermore, with the non-monotone technique and second order correction step, it is shown that the proposed method does not suffer from the Maratos effect, so that fast local convergence to second order sufficient local solutions is achieved. Numerical results show that the new approach is efficient.