Methods For Unconstrained Optimization Problems Research Articles

A Simulated Annealing-Based Barzilai–Borwein Gradient Method for Unconstrained Optimization Problems

In this paper, we propose a simulated annealing-based Barzilai–Borwein (SABB) gradient method for unconstrained optimization problems. The SABB method accepts the Barzilai–Borwein (BB) step by a simulated annealing rule. If the BB step cannot be accepted, the Armijo line search is used. The global convergence of the SABB method is established under some mild conditions. Numerical experiments indicate that, compared to some existing BB methods using nonmonotone line search technique, the SABB method performs well with high efficiency.

Two-phase quasi-Newton method for unconstrained optimization problem

In this paper, a two-phase quasi-Newton scheme is proposed for solving an unconstrained optimization problem. The global convergence property of the scheme is provided under mild assumptions. The superlinear rate of the scheme is also proved in the vicinity of the solution. The advantages of the proposed scheme over the traditional scheme are justified with numerical table and graphical illustrations.

Using a new regularized factorization method for unconstrained optimization problems

AbstractIn this paper, a new regularized factorization method is presented for solving unconstrained minimization problems in which the Hessian matrix may not be a positive definite or may be close to singular. With employing the modified quadrant interlocking factorization (QIF), an efficient algorithm is developed to find a suitable method. Moreover, under suitable conditions, the global convergence of the proposed method is established. Some interesting examples are solved by the proposed method and the improved Cholesky factorization method. The results show that the proposed method, in the most cases, the algorithm has a faster convergence than the improved Cholesky factorization method.

A Stochastic Trust Region Method for Unconstrained Optimization Problems

In this paper, a stochastic trust region method is proposed to solve unconstrained minimization problems with stochastic objectives. Particularly, this method can be used to deal with nonconvex problems. At each iteration, we construct a quadratic model of the objective function. In the model, stochastic gradient is used to take the place of deterministic gradient for both the determination of descent directions and the approximation of the Hessians of the objective function. The behavior and the convergence properties of the proposed method are discussed under some reasonable conditions. Some preliminary numerical results show that our method is potentially efficient.

A family of quasi-Newton methods for unconstrained optimization problems

ABSTRACTIn this paper, we present a class of approximating matrices as a function of a scalar parameter that includes the Davidon-Fletcher-Powell and Broyden-Fletcher-Goldfarb-Shanno methods as special cases. A powerful iterative descent method for finding a local minimum of a function of several variables is described. The new method maintains the positive definiteness of the approximating matrices. For a region in which the function depends quadratically on the variables, no more than n iterations are required, where n is the number of variables. A set of computational results that verifies the superiority of the new method are presented.

A new modified BFGS method for unconstrained optimization problems

Using chain rule, we propose a modified secant equation to get a more accurate approximation of the second curvature of the objective function. Then, based on this modified secant equation we present a new BFGS method for solving unconstrained optimization problems. The proposed method makes use of both gradient and function values, and utilizes information from two most recent steps, while the usual secant relation uses only the latest step information. Under appropriate conditions, we show that the proposed method is globally convergent without convexity assumption on the objective function. Comparative numerical results show computational efficiency of the proposed method in the sense of the Dolan–More performance profiles.

A modifications of conjugate gradient method for unconstrained optimization problems

The Conjugate Gradient (CG) methods play an important role in solving large-scale unconstrained optimization problems. Several studies have been recently devoted to improving and modifying these methods in relation to efficiency and robustness. In this paper, a new parameter of CG method has been proposed. The new parameter possesses global convergence properties under the Strong Wolfe-Powell (SWP) line search. The numerical results show that the proposed formula is more efficient and robust compared with Polak-Rribiere Ployak (PRP), Fletcher-Reeves (FR) and Wei, Yao, and Liu (WYL) parameters.

Global convergence analysis of a new hybrid conjugate gradient method for unconstrained optimization problems

In this paper, we propose a new hybrid conjugate gradient method for unconstrained optimization problems. The proposed method comprises of beta (DY), beta (WHY), beta (RAMI) and beta (New). The beta (New) was constructed purposely for this proposed hybrid method.The method possesses sufficient descent property irrespective of the line search. Under Strong Wolfe-Powell line search, we proved that the method is globally convergent. Numerical experimentation shows the effectiveness and robustness of the proposed method when compare with some hybrid as well as some modified conjugate gradient methods.

On the convergence of s-dependent GFR conjugate gradient method for unconstrained optimization

In this paper, the authors present an s-dependent conjugate gradient method for unconstrained optimization problem and make two different kinds of estimations of upper bounds of β k with respect to $$\beta_{k}^{\mathrm{FR}}$$ which are called dependent ratio. The global convergence of s-dependent GFR conjugate gradient method using several step-size rules is obtained.

Two New Dai–Liao-Type Conjugate Gradient Methods for Unconstrained Optimization Problems

In this paper, we present two new Dai–Liao-type conjugate gradient methods for unconstrained optimization problems. Their convergence under the strong Wolfe line search conditions is analysed for uniformly convex objective functions and general objective functions, respectively. Numerical experiments show that our methods can outperform some existing Dai–Liao-type methods by using Dolan and More’s performance profile.

A modified quadratic hybridization of Polak-Ribiere-Polyak and Fletcher-Reeves conjugate gradient method for unconstrained optimization problems

This article presents a modified quadratic hybridization of the Polak–Ribiere–Polyak and Fletcher–Reeves conjugate gradient method for solving unconstrained optimization problems. Global convergence, with the strong Wolfe line search conditions, of the proposed quadratic hybrid conjugate gradient method is established. We also report some numerical results to show the competitiveness of the new hybrid method.

A matrix-free implementation of Riemannian Newton’s method on the Stiefel manifold

Newton’s method for unconstrained optimization problems on the Euclidean space can be generalized to that on Riemannian manifolds. The truncated singular value problem is one particular problem defined on the product of two Stiefel manifolds, and an algorithm of the Riemannian Newton’s method for this problem has been designed. However, this algorithm is not easy to implement in its original form because the Newton equation is expressed by a system of matrix equations which is difficult to solve directly. In the present paper, we propose an effective implementation of the Newton algorithm. A matrix-free Krylov subspace method is used to solve a symmetric linear system into which the Newton equation is rewritten. The presented approach can be used on other problems as well. Numerical experiments demonstrate that the proposed method is effective for the above optimization problem.

Global Convergence of Curve Search Methods for Unconstrained Optimization

In this paper we propose a new family of curve search methods for unconstrained optimization problems, which are based on searching a new iterate along a curve through the current iterate at each iteration, while line search methods are based on finding a new iterate on a line starting from the current iterate at each iteration. The global convergence and linear convergence rate of these curve search methods are investigated under some mild conditions. Numerical results show that some curve search methods are stable and effective in solving some large scale minimization problems.

A New Nonmonotone Adaptive Retrospective Trust Region Method for Unconstrained Optimization Problems

In this paper, we propose a new nonmonotone adaptive retrospective Trust Region (TR) method for solving unconstrained optimization problems. Inspired by the retrospective ratio proposed in Bastin et al. (Math Program Ser A 123:395---418, 2010), a new nonmonotone TR ratio is introduced based on a convex combination of the nonmonotone classical and retrospective ratios. Due to the value of the new ratio, the TR radius is updated adaptively by a variant of the rule as proposed in Shi and Guo (J Comput Appl Math 213:509---520, 2008). Global convergence property of the new algorithm, as well as its superlinear convergence rate, is established under some standard assumptions. Numerical results on some test problems show the efficiency and effectiveness of the new method in practice, too.

A Barzilai and Borwein scaling conjugate gradient method for unconstrained optimization problems

In this paper, we combine the conjugate gradient method with the Barzilai and Borwein gradient method, and propose a Barzilai and Borwein scaling conjugate gradient method for nonlinear unconstrained optimization problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions, and has an advantage of less computational efforts. Moreover, the descent direction property and the global convergence are established when the line search fulfills the Wolfe conditions. The limited numerical experiments and comparisons show that the proposed algorithm is potentially efficient.

Convergence analysis of a new conjugate gradient method for unconstrained optimization

In this paper, the convergence analysis of a proposed new conjugate gradient method for unconstrained optimization problems was considered. This method inherits an important property of Polak-RibierePolyak (PRP). Under the exact line search condition, we established the descent condition of the method as well as the global convergence of the method. Numerical results show that our formula is eective by comparing with some existing formulas. Mathematics Subject Classication: 49M37, 13P25, 65N12, 90C52

Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization

In this paper, we propose a hybrid ODE-based quasi-Newton (QN) method for unconstrained optimization problems, which combines the idea of low-order implicit Runge–Kutta (RK) techniques for gradient systems with the QN type updates of the Jacobian matrix such as the symmetric rank-one (SR1) update. The main idea of this approach is to associate a QN matrix to approximate numerically the Jacobian matrix in the gradient system. Fundamentally this is a gradient system based on the first order optimality conditions of the optimization problem. To further extend the methods for solving large-scale problems, a feature incorporated to the proposed methods is that a limited memory setting for matrix–vector multiplications is used, thus avoiding the computational and storage issues when computing Jacobian information. Under suitable assumptions, global convergence of the proposed method is proved. Practical insights on the effectiveness of these approaches on a set of test functions are given by a numerical comparison with that of the limited memory BFGS algorithm (L-BFGS) and conjugate gradient algorithm (CG).

An Inexact PRP Conjugate Gradient Method for Symmetric Nonlinear Equations

In this article, without computing exact gradient and Jacobian, we proposed a derivative-free Polak-Ribière-Polyak (PRP) method for solving nonlinear equations whose Jacobian is symmetric. This method is a generalization of the classical PRP method for unconstrained optimization problems. By utilizing the symmetric structure of the system sufficiently, we prove global convergence of the proposed method with some backtracking type line search under suitable assumptions. Moreover, we extend the proposed method to nonsmooth equations by adopting the smoothing technique. We also report some numerical results to show its efficiency.

A Novel Self-Adaptive Trust Region Algorithm for Unconstrained Optimization

A new self-adaptive rule of trust region radius is introduced, which is given by a piecewise function on the ratio between the actual and predicted reductions of the objective function. A self-adaptive trust region method for unconstrained optimization problems is presented. The convergence properties of the method are established under reasonable assumptions. Preliminary numerical results show that the new method is significant and robust for solving unconstrained optimization problems.

A Nonmonotone Adaptive Trust Region Method Based on Conic Model for Unconstrained Optimization

We propose a nonmonotone adaptive trust region method for unconstrained optimization problems which combines a conic model and a new update rule for adjusting the trust region radius. Unlike the traditional adaptive trust region methods, the subproblem of the new method is the conic minimization subproblem. Moreover, at each iteration, we use the last and the current iterative information to define a suitable initial trust region radius. The global and superlinear convergence properties of the proposed method are established under reasonable conditions. Numerical results show that the new method is efficient and attractive for unconstrained optimization problems.