Methods For Unconstrained Optimization Problems Research Articles

An improved preconditioned conjugate gradient method for unconstrained optimization problem with application in Robot arm control

AbstractThis work suggests improved conjugate gradient methods for enhancing the efficiency and robustness of the classical conjugate gradient methods. The study modifies the diagonal of the inverse Hessian approximation of the Broyden–Fletcher–Goldfarb–Shanno (BFGS) quasi‐Newton update in order to build a preconditioner for nonlinear conjugate gradient (NCG) methods applied to large‐scale unconstrained optimization problems. Damping techniques were embedded into the algorithm to impose the positive definiteness of the diagonal approximation. This made the methods easy to use and presented a viable way to increase the effectiveness of unconstrained optimization techniques. Experimental findings from a collection of benchmark problems demonstrate the efficiency and robustness of the proposed method when compared to five other existing NCG algorithms. Moreover, the successful application of the algorithm to manipulate robotic planar motion control systems with 3 degrees of freedom has been demonstrated, highlighting the practicality of the proposed approach.

A Convex combination of improved Fletcher-Reeves and Rivaie-Mustafa-Ismail-Leong conjugate gradient methods for unconstrained optimization problems and applications

Conjugate gradient methods are probably the most used methods in solving large scale unconstrained optimization problems. They have become popular because of their simplicity and low memory requirements. In this paper, we propose a hybrid conjugate gradient method based on the improved Fletcher-Reeves (IFR) and Rivaie-Mustafa-Ismail-Leong+ (RMIL+) methods and establish its global convergence under the strong Wolfe line search conditions. The new conjugate gradient direction satisfies the sufficient descent condition. The method is then compared to other methods in the literature and numerical experiments show that it is competent when solving large scale unconstrained optimization problems. Furthermore, the method is applied to solve a problem in portfolio selection.

Efficient one-parameter family of conjugate gradient methods

We present an efficient one-parameter family of conjugate gradient methods for unconstrained optimization problems. These methods are defined using a combination of the Polak-Ribiere-Polyak method and the Rivaie-Mustafa-Ismail-Leong method. We prove the global convergence based on the Wolfe line search for nonlinear objective functions. Finally, we give some numerical experiments, proving the efficiency of the proposed approach.

Some New Descent Nonlinear Conjugate Gradient Methods for Unconstrained Optimization Problems with Global Convergence

In this paper, we develop some three term nonlinear conjugate gradient methods based on the Hestenes–Stiefel (HS), the Polak–Ribière–Polyak (PRP) and the Liu–Storey (LS) methods. The proposed algorithms always generate sufficient descent directions which satisfy [Formula: see text]. When the Wolfe or the Armijo line search is used, we establish the global convergence of the proposed methods in a concise way. Moreover, the linear convergence rate of the methods is discussed as well. The extensive numerical results show the efficiency of the proposed methods.

Feasible Newton methods for symmetric tensor Z-eigenvalue problems

Finding a Z-eigenpair of a symmetric tensor is equivalent to finding a Karush–Kuhn–Tucker point of a sphere constrained minimization problem. Based on this equivalency, in this paper, we first propose a class of iterative methods to get a Z-eigenpair of a symmetric tensor. Each method can generate a sequence of feasible points such that the sequence of function evaluations is decreasing. These methods can be regarded as extensions of the descent methods for unconstrained optimization problems. We pay particular attention to the Newton method. We show that under appropriate conditions, the Newton method is globally and quadratically convergent. Moreover, after finitely many iterations, the unit steplength will always be accepted. We also propose a nonlinear equations-based Newton method and establish its global and quadratic convergence. In the end, we do several numerical experiments to test the proposed Newton methods. The results show that both Newton methods are very efficient.

An accelerated Sobolev gradient method for unconstrained optimization problems based on variable inner products

We propose an accelerated version of the classical gradient method for unconstrained optimization problems defined on a Sobolev space H with Hilbert structure. Motivated by empirical results available in the literature demonstrating improved convergence of Sobolev gradient methods with suitably chosen weights, we develop a rigorous and constructive approach allowing one to identify the optimal gradient gk=g(λk) among gradients g(λ) parameterized by a weight function λ belonging to a finite-dimensional space of weights, which defines the inner product 〈⋅,⋅〉λ in the space H. At the kth iteration of the method, where an approximation uk∈H to the minimizer is given, an optimal weight λk is found as a solution of a nonlinear minimization problem in the space of weights R+N. The weight λk defines the optimal gradient gk equal to the projection of the Newton step hk onto a certain finite-dimensional subspace Tk, in the sense that Pk(σgk−hk)=0, where Pk is the projection operator onto Tk and σ a fixed step size. This property ensures that thus constructed gradient method attains a quadratic convergence in a certain sense for error components in Tk, in addition to the linear convergence typical of the classical gradient method. A numerical implementation of the new approach is also proposed. Computational results based on two model problems confirm the theoretically established convergence properties, demonstrating that the proposed approach outperforms the standard steepest-descent method based on Sobolev gradients and compares favorably to the Newton–Krylov method.

Two spectral conjugate gradient methods for unconstrained optimization problems

Two spectral conjugate gradient methods for unconstrained optimization problems

A Globally Convergent Hybrid FR-PRP Conjugate Gradient Method for Unconstrained Optimization Problems

In this paper, a new conjugate gradient (CG) parameter is proposed through the convex combination of the Fletcher-Reeves (FR) and Polak-Ribiére-Polyak (PRP) CG update parameters such that the conjugacy condition of Dai-Liao is satisfied. The computational efficiency of the PRP method and the convergence profile of the FR method motivated the choice of these two CG methods. The corresponding CG algorithm satisfies the sufficient descent property and was shown to be globally convergent under the strong Wolfe line search procedure. Numerical tests on selected benchmark test functions show that the algorithm is efficient and very competitive in comparison with some existing classical methods.

Sufficient Descent Condition of the Polak-Ribière-Polyak (PRP) Conjugate Gradient Method without Line Search

Nonlinear conjugate gradient methods for unconstrained optimization problems are used in many aspects of theoretical and applied sciences. They are iterative methods, so at any iteration a step length is computed using a method called line search. In most cases, the sufficient descent condition plays an important role to prove the global convergence of a conjugate gradient method. Due to its outperformance in practical computation, the Polak-Ribière-Polyak (PRP) conjugate gradient method is widely used for solving nonlinear unconstrained optimization problems. However, the sufficient descent condition of PRP has not established without line search yet. In this study, we established the sufficient descent condition without line search based on the conditions 0 < bkPRP £ x bkFR and 1 < bkPRP £mbkFR, where 0 < x < 1 and m > 1. As a result, we found that under certain conditions, the sufficient descent condition is satisfied when the PRP implemented without line search

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

A Nonmonotone Trust Region Method for Unconstrained Optimization Problems on Riemannian Manifolds

We propose a nonmonotone trust region method for unconstrained optimization problems on Riemannian manifolds. Global convergence to the first-order stationary points is proved under some reasonable conditions. We also establish local R-linear, super-linear and quadratic convergence rates. Preliminary experiments show that the algorithm is efficient.

A New Nonmonotone Trust Region Barzilai-Borwein Method for Unconstrained Optimization Problems

A New Nonmonotone Trust Region Barzilai-Borwein Method for Unconstrained Optimization Problems

A New conjugate gradient method for unconstrained optimization problems with descent property

In this paper, we propose a new conjugate gradient method for solving nonlinear unconstrained optimization. The new method consists of three parts, the first part of them is the parameter of Hestenes-Stiefel (HS). The proposed method is satisfying the descent condition, sufficient descent condition and conjugacy condition. We give some numerical results to show the efficiency of the suggested method.

A new self-scaling variable metric (DFP) method for unconstrained optimization problems

A new self-scaling variable metric (DFP) method for unconstrained optimization problems

Using nonlinear functions to approximate a new quasi-Newton method for unconstrained optimization problems

In order to get a higher order accuracy of approximating the Hessian matrix of the objective function, we use the chain rule and propose two modified secant equations. An interesting property of the proposed methods is that these utilize information from two most recent steps where the usual secant equation uses only the latest step information. The other point of interest to one of the proposed methods is that it makes use of both gradient and function value information. We show that the modified BFGS methods based on the new secant equations are globally convergent. The presented experimental results illustrate that the proposed methods are efficient.

An improvement of adaptive cubic regularization method for unconstrained optimization problems

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

Two modified DY conjugate gradient methods for unconstrained optimization problems

In this paper, we study the unconstrained optimization problems, and the two modified DY conjugate gradient methods (DDY1 method and DDY2 method) are proposed under the DY conjugate gradient method. By using the standard Wolfe line search, we prove the global convergence of the two methods. The search direction of DDY1 method is descent with the standard Wolfe line search. The search direction generated by the DDY2 method is sufficient descent, in which the property does not depend on any line search. Preliminary numerical results show that the two methods are effective.

Derivative Free Conjugate Gradient Method via Broyden’s Update for solving symmetric systems of nonlinear equations

The applications of mathematics in many areas of computing, scientific and engineering research mostly give rise to a systems of nonlinear equations. Various iterative methods have been developed to solve such equations, this includes Newton method, Quasi-Newton’s etc. Over the years, there has been significant theoretical study on quasi-Newton methods for solving such systems, but unfortunately the methods suffers setback. To overcome such problems, a Derivative free Method for Solving Symmetric Systems of Nonlinear Equations Using Broyden’s Update is presented. The modification is achieved by simply approximating the inverse Hessian matrix to with (δk and I represents acceleration parameter and an identity matrix respectively) without computing any derivative. The method uses the symmetric structure of the system sufficiently and the generalized classical Broyden’s update method for unconstrained optimization problems. The squared norm merit function is used, both the direction and the line search technique are derivative-free, this attractive feature of the proposed method makes it to have a very low storage requirement thereby solving large scale problems successfully. In an effort to solve nonlinear problems of the form F(x) = 0, 0, x ∈ Rn different initial starting points were used on a set of benchmark test problems, the output is based on number of iterations and CPU time. A comparison between the proposed method and the classical methods were made and found that the proposed method is efficient, robust and outperformed the existing method.

An Alternative Modified Conjugate Gradient Coefficient for Solving Nonlinear System of Equations

In mathematical term, the method of solving models and finding the best alternatives is known as optimization. Conjugate gradient (CG) method is an evolution of computational method in solving optimization problems. In this article, an alternative modified conjugate gradient coefficient for solving large-scale nonlinear system of equations is presented. The method is an improved version of the Rivaie et el conjugate gradient method for unconstrained optimization problems. The new CG is tested on a set of test functions under exact line search. The approach is easy to implement due to its derivative-free nature and has been proven to be effective in solving real-life application. Under some mild assumptions, the global convergence of the proposed method is established. The new CG coefficient also retains the sufficient descent condition. The performance of the new method is compared to the well-known previous PRP CG methods based on number of iterations and CPU time. Numerical results using some benchmark problems show that the proposed method is promising and has the best efficiency amongst all the methods tested.

Newton-2EGSOR Method for Unconstrained Optimization Problems with a Block Diagonal Hessian

We find that finding the unconstrained optimizer for large-scale problems with the Newton method can be very expensive. Therefore, in this paper, we developed a new efficient method for solving large-scale unconstrained optimization problems with block diagonal Hessian matrices to reduce the cost of computing Newton’s direction. This method is a combination of the Newton method and the 2-point explicit group successive-over relaxation (2EGSOR) block iterative method. To calculate the performance of the developed method, we used a combination of the Newton method with Gauss–Seidel (Newton-GS) point iteration and the Newton method with successive-over relaxation (Newton-SOR) point iteration as reference methods. The numerical experiment has proven that the developed algorithms generate results that are more efficient compared to the reference methods with less execution time and fewer iterations. Through 90 test cases, we observe that the speedup ratio for our proposed method is up to 659.87 times faster compared to the Newton-SOR method and up to 1963.57 times more rapidly than the Newton-GS method.