Nonmonotone Algorithm Research Articles

Feature selection using Information Gain and decision information in neighborhood decision system

Feature selection is a significant preprocessing technique for data mining, which can promote the accuracy of data classification and shrink feature space by eliminating redundant features. Since traditional feature selection algorithms have high time complexity and low classification accuracy, an effective algorithm using Information Gain and decision information is designed. The algorithm introduces Information Gain for performing preliminary dimensionality reduction on high dimensional datasets, and then the decision information is regarded as an evaluation function of features to select features with important information. First, the concept of joint information granule is defined, and neighborhood information entropy measures are proposed based on the joint information granule. In addition, the relationship between these measures is studied, which is helpful to study the uncertainty in data. Second, a nonmonotonic algorithm using the decision information in the neighborhood information entropy measures is proposed to overcome the shortcoming of algorithms based on monotonic evaluation functions, thereby improving the accuracy of data classification. Third, to reduce the time cost of the designed algorithm for high dimensional datasets, Information Gain is introduced to preliminarily eliminate irrelevant features in high dimensional datasets. Finally, the ablation and comparison experiments on twelve public datasets demonstrate the low time cost and high classification accuracy of our algorithm, respectively.

A unified convergence framework for nonmonotone inexact decomposition methods

In this work we propose a general framework that provides a unified convergence analysis for nonmonotone decomposition algorithms. The main motivation to embed nonmonotone strategies within a decomposition approach lies in the fact that enforcing the reduction of the objective function could be unnecessarily expensive, taking into account that groups of variables are individually updated. We define different search directions and line searches satisfying the conditions required by the presented nonmonotone decomposition framework to obtain global convergence. We employ a set of large-scale network equilibrium problems as a computational example to show the advantages of a nonmonotone algorithm over its monotone counterpart. In conclusion, a new smart implementation for decomposition methods has been derived to solve numerical issues on large-scale partially separable functions.

A non-monotone pattern search approach for systems of nonlinear equations

ABSTRACTIn this paper, a new pattern search is proposed to solve the systems of nonlinear equations. We introduce a new non-monotone strategy which includes a convex combination of the maximum function of some preceding successful iterates and the current function. First, we produce a stronger non-monotone strategy in relation to the generated strategy by Gasparo et al. [Nonmonotone algorithms for pattern search methods, Numer. Algorithms 28 (2001), pp. 171–186] whenever iterates are far away from the optimizer. Second, when iterates are near the optimizer, we produce a weaker non-monotone strategy with respect to the generated strategy by Ahookhosh and Amini [An efficient nonmonotone trust-region method for unconstrained optimization, Numer. Algorithms 59 (2012), pp. 523–540]. Third, whenever iterates are neither near the optimizer nor far away from it, we produce a medium non-monotone strategy which will be laid between the generated strategy by Gasparo et al. [Nonmonotone algorithms for pattern search methods, Numer. Algorithms 28 (2001), pp. 171–186] and Ahookhosh and Amini [An efficient nonmonotone trust-region method for unconstrained optimization, Numer. Algorithms 59 (2012), pp. 523–540]. Reported are numerical results of the proposed algorithm for which the global convergence is established.

A modified nonmonotone BFGS algorithm for unconstrained optimization

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size alpha_{k} to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method.

On the worst-case evaluation complexity of non-monotone line search algorithms

A general class of non-monotone line search algorithms has been proposed by Sachs and Sachs (Control Cybern 40:1059–1075, 2011) for smooth unconstrained optimization, generalizing various non-monotone step size rules such as the modified Armijo rule of Zhang and Hager (SIAM J Optim 14:1043–1056, 2004). In this paper, the worst-case complexity of this class of non-monotone algorithms is studied. The analysis is carried out in the context of non-convex, convex and strongly convex objectives with Lipschitz continuous gradients. Despite de nonmonotonicity in the decrease of function values, the complexity bounds obtained agree in order with the bounds already established for monotone algorithms.

Strong convergence properties of a modified nonmonotone smoothing algorithm for the SCCP

The smoothing algorithms have been successfully applied to solve the symmetric cone complementarity problem (denoted by SCCP), which in general have the global and local superlinear/quadratic convergence if the solution set of the SCCP is nonempty and bounded. Huang, Hu and Han [Science in China Series A: Mathematics, 52: 833–848, 2009] presented a nonmonotone smoothing algorithm for solving the SCCP, whose global convergence is established by just requiring that the solution set of the SCCP is nonempty. In this paper, we propose a new nonmonotone smoothing algorithm for solving the SCCP by modifying the version of Huang-Hu-Han’s algorithm. We prove that the modified nonmonotone smoothing algorithm not only is globally convergent but also has local superlinear/quadratical convergence if the solution set of the SCCP is nonempty. This convergence result is stronger than those obtained by most smoothing-type algorithms. Finally, some numerical results are reported.

A Nonmonotone Projection Method for Constrained System of Nonlinear Equations

This paper deals with the nonmonotone projection algorithm for constrained nonlinear equations. For some starting points, the previous projection algorithms for the problem may encounter slow convergence which is related to the monotone behavior of the iterative sequence as well as the iterative direction. To circumvent this situation, we adopt the nonmonotone technique introduced by Dang to develop a nonmonotone projection algorithm. After constructing the nonmonotone projection algorithm, we show its convergence under some suitable condition. Preliminary numerical experiment is reported at the end of this paper, from which we can see that the algorithm we propose converges more quickly than that of the usual projection algorithm for some starting points.

Non-monotone algorithm for minimization on arbitrary domains with applications to large-scale orthogonal Procrustes problem

This paper concerns a non-monotone algorithm for minimizing differentiable functions on closed sets. A general numerical scheme is proposed which combines a regularization/trust-region framework with a non-monotone strategy. Global convergence to stationary points is proved under usual assumptions. Numerical experiments for a particular version of the general algorithm are reported. In addition, a promising numerical scheme for medium/large-scale orthogonal Procrustes problem is also proposed and numerically illustrated.

A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques

In recent years, cubic regularization algorithms for unconstrained optimization have been defined as alternatives to trust-region and line search schemes. These regularization techniques are based on the strategy of computing an (approximate) global minimizer of a cubic overestimator of the objective function. In this work we focus on the adaptive regularization algorithm using cubics (ARC) proposed in Cartis et al. [Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results, Mathematical Programming A 127 (2011), pp. 245–295]. Our purpose is to design a modified version of ARC in order to improve the computational efficiency preserving global convergence properties. The basic idea is to suitably combine a Goldstein-type line search and a nonmonotone accepting criterion with the aim of advantageously exploiting the possible good descent properties of the trial step computed as (approximate) minimizer of the cubic model. Global convergence properties of the proposed nonmonotone ARC algorithm are proved. Numerical experiments are performed and the obtained results clearly show satisfactory performance of the new algorithm when compared to the basic ARC algorithm.

A Non-Monotone Line Search Combination Technique for Unconstrained Optimization

We propose a non-monotone line search combination rule for unconstrained optimization problems, the corresponding non-monotone search algorithm is established and its global convergence can be proved. Finally, we use some numerical experiments to illustrate the new combination of non-monotone search algorithm’s effectiveness.

A class of derivative-free nonmonotone optimization algorithms employing coordinate rotations and gradient approximations

In this paper we study a class of derivative-free unconstrained minimization algorithms employing nonmonotone inexact linesearch techniques along a set of suitable search directions. In particular, we define globally convergent nonmonotone versions of some well-known derivative-free methods and we propose a new linesearch-based nonmonotone algorithm, where search directions are constructed by combining coordinate rotations with simplex gradients. Through extensive numerical experimentation, we show that the proposed algorithm is highly competitive in comparison with some of the most efficient direct search methods and model based methods on a large set of test problems.

A hybrid of adjustable trust-region and nonmonotone algorithms for unconstrained optimization

This study devotes to incorporating a nonmonotone strategy with an automatically adjusted trust-region radius to propose a more efficient hybrid of trust-region approaches for unconstrained optimization. The primary objective of the paper is to introduce a more relaxed trust-region approach based on a novel extension in trust-region ratio and radius. The next aim is to employ stronger nonmonotone strategies, i.e. bigger trust-region ratios, far from the optimizer and weaker nonmonotone strategies, i.e. smaller trust-region ratios, close to the optimizer. The global convergence to first-order stationary points as well as the local superlinear and quadratic convergence rates are also proved under some reasonable conditions. Some preliminary numerical results and comparisons are also reported.

A nonmonotonic hybrid algorithm for min-max problem

In this paper nonlinear min-max problems are discussed. Instead of the traditional nonmonotone approach presented by Grippo et al., we use the nonmonotonic technique proposed by Zhang and Hager (SIAM J. Optim. 14(4):1043–1056, 2004), to propose a new nonmonotonic algorithm for min-max problem. Moreover, using hybrid technique can make best use of the advantages of both trust region methods and line search methods. In addition, the new approach can reduce the possibility of solving two quadratic subproblems furthest, and circumvent the difficulties of the Maratos effect occurred in the nonsmooth optimization. Under reasonable conditions, the global convergence and the rate of superlinear convergence are established. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

A Nonmonotone Algorithm of Moving Asymptotes for Solving Unconstrained Optimization Problems

A Nonmonotone Algorithm of Moving Asymptotes for Solving Unconstrained Optimization Problems

A nonmonotone approximate sequence algorithm for unconstrained nonlinear optimization

A new nonmonotone algorithm is proposed and analyzed for unconstrained nonlinear optimization. The nonmonotone techniques applied in this algorithm are based on the estimate sequence proposed by Nesterov (Introductory Lectures on Convex Optimization: A Basic Course, 2004) for convex optimization. Under proper assumptions, global convergence of this algorithm is established for minimizing general nonlinear objective function with Lipschitz continuous derivatives. For convex objective function, this algorithm maintains the optimal convergence rate of convex optimization. In numerical experiments, this algorithm is specified by employing safe-guarded nonlinear conjugate gradient search directions. Numerical results show the nonmonotone algorithm performs significantly better than the corresponding monotone algorithm for solving the unconstrained optimization problems in the CUTEr (Bongartz et al. in ACM Trans. Math. Softw. 21:123---160, 1995) library.

An adaptive nonmonotone trust-region method with curvilinear search for minimax problem

In this paper we propose an adaptive nonmonotone algorithm for minimax problem. Unlike traditional nonmonotone method, the nonmonotone technique applied to our method is based on the nonmonotone technique proposed by Zhang and Hager [H.C. Zhang, W.W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14(4)(2004) 1043–1056] instead of that presented by Grippo et al. [L. Grippo, F. Lampariello, S. Lucidi, A nonmonotone line search technique for Newton’s method, SIAM J. Numer. Anal. 23(4)(1986) 707–716]. Meanwhile, by using adaptive technique, it can adaptively perform the nonmonotone trust-region step or nonmonotone curvilinear search step when the solution of subproblems is unacceptable. Global and superlinear convergences of the method are obtained under suitable conditions. Preliminary numerical results are reported to show the effectiveness of the proposed algorithm.

On sequential and parallel non-monotone derivative-free algorithms for box constrained optimization

This paper proposes a space decomposition scheme for non-monotone (NM) derivative-free parallel and sequential algorithms for solving the box-constrained optimization problem (BCOP). Convergence to Karush Kuhn Tucker points is proved under the same conditions for NM and monotone algorithms for solving unconstrained and BCOPs. The parallel algorithm has two unique features: all processors exchange information on discrete quasi-minimal points, and are able to sample function values on the whole set of directions that conform to non-negative spanning sets for each decomposed subspace. The parallel algorithm has a high degree of fault tolerance: convergence prevails even if only one processor remains running. Preliminary results are encouraging for solving small- and medium-sized problems.

Monotone and nonmonotone trust-region-based algorithms for large scale unconstrained optimization problems

Two trust regions algorithms for unconstrained nonlinear optimization problems are presented: a monotone and a nonmonotone one. Both of them solve the trust region subproblem by the spectral projected gradient (SPG) method proposed by Birgin, Martinez and Raydan (in SIAM J. Optim. 10(4):1196---1211, 2000). SPG is a nonmonotone projected gradient algorithm for solving large-scale convex-constrained optimization problems. It combines the classical projected gradient method with the spectral gradient choice of steplength and a nonmonotone line search strategy. The simplicity (only requires matrix-vector products, and one projection per iteration) and rapid convergence of this scheme fits nicely with globalization techniques based on the trust region philosophy, for large-scale problems. In the nonmonotone algorithm the trial step is evaluated by acceptance via a rule which can be considered a generalization of the well known fraction of Cauchy decrease condition and a generalization of the nonmonotone line search proposed by Grippo, Lampariello and Lucidi (in SIAM J. Numer. Anal. 23:707---716, 1986). Convergence properties and extensive numerical results are presented. Our results establish the robustness and efficiency of the new algorithms.

Nonmonotone algorithm for minimization on closed sets with applications to minimization on Stiefel manifolds

A nonmonotone Levenberg–Marquardt-based algorithm is proposed for minimization problems on closed domains. By preserving the feasible set’s geometry throughout the process, the method generates a feasible sequence converging to a stationary point independently of the initial guess. As an application, a specific algorithm is derived for minimization on Stiefel manifolds and numerical results involving a weighted orthogonal Procrustes problem are reported.

A nonmonotone PSB algorithm for solving unconstrained optimization

PSB (Powell-Symmetric-Broyden) algorithm is a very important algorithm and has been extensively used in trust region methods. However, there are few studies on the line search type PSB algorithm. The primary reason is that the direction generated by this class of algorithms is not necessarily a descent direction of the objective function. In this paper, by combining a nonmonotone line search technique with the PSB method, we propose a nonmonotone PSB algorithm for solving unconstrained optimization. Under proper conditions, we establish global convergence and superlinear convergence of the proposed algorithm. At the same time we verify the efficiency of the proposed algorithm by some numerical experiments.