Nonmonotonic Criterion Research Articles

An affine scaling interior point backtracking algorithm for nonlinear constrained optimisation

AbstractIn this paper we propose a new affine scaling interior trust region algorithm with a nonmonotonic backtracking technique for nonlinear equality constrained optimisation with nonnegative constraints on the variables. In order to deal with large problems, the general full trust region subproblem is decomposed into a pair of trust region subproblems in horizontal and vertical subspaces. The horizontal trust region subproblem in the algorithm is defined by minimising a quadratic function subject only to an ellipsoidal constraint in a null tangential subspace and the vertical trust region subproblem is defined by the least squares subproblem subject only to an ellipsoidal constraint. By adopting Fletcher's penalty function as the merit function, combining a trust region strategy and a nonmonotone line search, the mixing technique will switch to a backtracking step generated by the two trust region subproblems to obtain an acceptable step. The global convergence of the proposed algorithm is proved while maintaining a fast local superlinear convergence rate, which is established under some reasonable conditions. A nonmonotonic criterion is used to speed up the convergence progress in some highly nonlinear cases.

An interior affine scaling projective algorithm for nonlinear equality and linear inequality constrained optimization

In this paper, we propose a new nonmonotonic interior point backtracking strategy to modify the reduced projective affine scaling trust region algorithm for solving optimization subject to nonlinear equality and linear inequality constraints. The general full trust region subproblem for solving the nonlinear equality and linear inequality constrained optimization is decomposed to a pair of trust region subproblems in horizontal and vertical subspaces of linearize equality constraints and extended affine scaling equality constraints. The horizontal subproblem in the proposed algorithm is defined by minimizing a quadratic projective reduced Hessian function subject only to an ellipsoidal trust region constraint in a null subspace of the tangential space, while the vertical subproblem is also defined by the least squares subproblem subject only to an ellipsoidal trust region constraint. By introducing the Fletcher's penalty function as the merit function, trust region strategy with interior point backtracking technique will switch to strictly feasible interior point step generated by a component direction of the two trust region subproblems. The global convergence of the proposed algorithm while maintaining fast local convergence rate of the proposed algorithm are established under some reasonable conditions. A nonmonotonic criterion should bring about speeding up the convergence progress in some high nonlinear function conditioned cases.

A new affine scaling interior point algorithm for nonlinear optimization subject to linear equality and inequality constraints

In this paper we propose a new interior affine scaling region algorithm with nonmonotonic interior point backtracking technique for nonlinear optimization subject to linear equality and inequality constraints. The trust region subproblem in the proposed algorithm is defined by minimizing a quadratic function subject only to an affine scaling ellipsoidal constraint in a null subspace of the extended equality constraints. Using both trust region strategy and line search technique, the affine scaling trust region subproblem at each iteration generates backtracking interior step to obtain a new accepted step. The global convergence and fast local convergence rate of the proposed algorithm are established under some reasonable conditions. A nonmonotonic criterion should bring about speeding up the convergence progress in some ill-conditioned cases.

Nonmonotonic back-tracking trust region interior point algorithm for linear constrained optimization

In this paper, we modify the trust region interior point algorithm proposed by Bonnans and Pola in (SIAM J. Optim. 7(3) (1997) 717) for linear constrained optimization. A mixed strategy using both trust region and line-search techniques is adopted which switches to back-tracking steps when a trial step produced by the trust region subproblem may be unacceptable. The global convergence and local convergence rate of the improved algorithm are established under some reasonable conditions. A nonmonotonic criterion is used to speed up the convergence progress in some ill-conditioned cases. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

Deterministic nonmonotone strategies for effective training of multilayer perceptrons

We present deterministic nonmonotone learning strategies for multilayer perceptrons (MLPs), i.e., deterministic training algorithms in which error function values are allowed to increase at some epochs. To this end, we argue that the current error function value must satisfy a nonmonotone criterion with respect to the maximum error function value of the M previous epochs, and we propose a subprocedure to dynamically compute M. The nonmonotone strategy can be incorporated in any batch training algorithm and provides fast, stable, and reliable learning. Experimental results in different classes of problems show that this approach improves the convergence speed and success percentage of first-order training algorithms and alleviates the need for fine-tuning problem-depended heuristic parameters.

Nonmonotonic projected algorithm with both trust region and line search for constrained optimization

In this paper we combine a reduced Hessian method with a mixed strategy using both trust region and line search techniques for constrained optimization. The adopted strategy switches to back tracking steps when a trial step produced by the trust region subproblem is unacceptable. By using Fletcher's penalty function as a merit function, the resulting algorithm possesses global convergence while maintaining a superlinear local convergence rate under some reasonable conditions. A nonmonotonic criterion is suggested which does not require the merit function to reduce its value after every iteration.