Nonlinear Unconstrained Optimization Problems Research Articles

A novel stepsize for gradient descent method

We propose a novel adaptive stepsize for the gradient descent scheme to solve unconstrained nonlinear optimization problems. With the convex and smooth objective satisfying locally Lipschitz gradient we obtain the complexity O(1k) of f(xk)−f⁎ at most. By using the idea of the new stepsize, we propose another new algorithm based on the projected gradient for solving a class of nonconvex optimization problems over a closed convex set. The computational experiments show the efficiency of the new method.

Two new conjugate gradient methods in unconstrained optimization problems

Zheng Y. and Zheng B. in [14] modified Dai-Liao conjugate gradient method to come up with two new Dai-Liao-type conjugate gradient methods. These methods were shown to have satisfied descent condition taken into consideration the strong Wolfe line search. Convergence for objective functions were also guarantied. In this work, two new conjugate gradient methods are introduced in line with the work of Zheng Y. and Zheng B. [14] by changing the first term in AyO-CG method [3] to solve unconstrained non-linear optimization problems. Descent properties of these methods are shown and guarantied. Convergence analyses of these methods in line with strong Wolfe conditions showed that they are globally convergent. Comparison based on Dolan More performance profile of the numerical strength of these methods with the two modified Dai-Laio type methods proved that our methods compare favorably well with them

A modified hybrid conjugate gradient method for solving unconstrained non-linear optimization problems has been developed

This paper presents a new hybrid conjugate gradient method, combining the Conjugate Descent (CD) and Al-Bayati & Al-Assady (BA) methods, for solving unconstrained optimization problems. We provide a convergence analysis of the proposed method and demonstrate its effectiveness through numerical examples.

A new hybrid conjugate gradient algorithm for unconstrained optimization

It is well known that conjugate gradient methods are useful for solving large-scale unconstrained nonlinear optimization problems. In this paper, we consider combining the best features of two conjugate gradient methods. In particular, we give a new conjugate gradient method, based on the hybridization of the useful DY (Dai-Yuan), and HZ (Hager-Zhang) methods. The hybrid parameters are chosen such that the proposed method satisfies the conjugacy and sufficient descent conditions. It is shown that the new method maintains the global convergence property of the above two methods. The numerical results are described for a set of standard test problems. It is shown that the performance of the proposed method is better than that of the DY and HZ methods in most cases.

A Modified q-BFGS Algorithm for Unconstrained Optimization

This paper presents a modification of the q-BFGS method for nonlinear unconstrained optimization problems. For this modification, we use a simple symmetric positive definite matrix and propose a new q-quasi-Newton equation, which is close to the ordinary q-quasi-Newton equation in the limiting case. This method uses only first order q-derivatives to build an approximate q-Hessian over a number of iterations. The q-Armijo-Wolfe line search condition is used to calculate step length, which guarantees that the objective function value is decreasing. This modified q-BFGS method preserves the global convergence properties of the q-BFGS method, without the convexity assumption on the objective function. Numerical results on some test problems are presented, which show that an improvement has been achieved. Moreover, we depict the numerical results through the performance profiles.

A harmonic framework for stepsize selection in gradient methods

We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This not only provides an elegant and flexible framework to parametrize and reinterpret existing stepsize schemes, but it also gives inspiration for new flexible and tunable families of steplengths. In particular, we analyze and extend the adaptive Barzilai–Borwein method to a new family of stepsizes. While this family exploits negative values for the target, we also consider positive targets. We present a convergence analysis for quadratic problems extending results by Dai and Liao (IMA J Numer Anal 22(1):1–10, 2002), and carry out experiments outlining the potential of the approaches.

Two New Preconditioned Conjugate Gradient Methods for Minimization Problems

In application to general function, each of the conjugate gradient and Quasi-Newton methods has particular advantages and disadvantages. Conjugate gradient (CG) techniques are a class of unconstrained optimization algorithms with strong local and global convergence qualities and minimal memory needs. Quasi-Newton methods are reliable and efficient on a wide range of problems and they converge faster than the conjugate gradient method and require fewer function evaluations but they have the disadvantage of requiring substantially more storage and if the problem is ill-conditioned, they may take several iterations. A new class has been developed, termed preconditioned conjugate gradient (PCG) method. It is a method that combines two methods, conjugate gradient and Quasi-Newton. In this work, two new preconditioned conjugate gradient algorithms are proposed namely New PCG1 and New PCG2 to solve nonlinear unconstrained optimization problems. A new PCG1 combines conjugate gradient method Hestenes-Stiefel (HS) with new self-scaling symmetric Rank one (SR1), and a new PCG2 combines conjugate gradient method Hestenes-Stiefel (HS) with new self-scaling Davidon, Flecher and Powell (DFP). The algorithm uses the strong Wolfe line search condition. Numerical comparisons with standard preconditioned conjugate gradient algorithms show that for these new algorithms, computational scheme outperforms the preconditioned conjugate gradient.

An Accelerated Three Term Efficient Algorithm for Numerical Optimization

Background:
 A new optimization algorithm is presented. The method is biased with new non monotone line search an accelerated three term conjugate gradient method damped of Quasi Newton method compared to previews method design efficiency in provident of more than one factor for different optimization problem are more dramatic due to the ability of the technique to utile existing data.
 Materials and Methods:
 New monotone line search, new monotone line search, new modification of Damped Quasi-Newton method, Motivation and New Quasi-Newton Algorithm (MQ) and Global convergence.
 Results:
 In this work, we have n tendency to compare our new algorithm with same classical strategies like [7] by exploiting of unconstrained nonlinear optimization problem the functions obtained from Andrei [5, 6] Waziri and Sabiu (2015)[10] and La couzetul (2004)[3]. The numerical experiments demonstrate the performance of the proposed method. We selected seven relatively unconstrained problems with the size varies from 10 to 100. We consider the three sizes of each problem so that the total number of problem is 21 test problems. We stop the iteration when is satisfied All codes were written in Matlab R2017a and run on a pc with Intel COREi4 with a processor with 4GB of Ram and CPU 2.3GHZ we solved test problems using two different initial starting points.
 Conclusion:
 In this research article, a project on an accelerated three-term efficient algorithm for numerical optimization has presented the method as completely a derivative-free algorithm with less NOI and NOF and CPU time computed to the existing methods .using classical assumption the global convergence was also proved. Numerical results using the three terms efficient algorithm show that the algorithm is promising.

Trust-Tech Source-Point Method for Systematically Computing Multiple Local Optimal Solutions: Theory and Method.

In this article, a Trust-Tech source-point method is proposed to systematically compute multiple local optimal solutions (LOSs) for continuous unconstrained nonlinear optimization problems. This proposed method consists of four stages. Stage I finds one LOS (in which existing effective optimizers can be applied), stage II is the stage of escaping an LOS while stage III is the stage for entering the stability region (SR) of another stable equilibrium point (SEP) (i.e., another LOS). Stage IV computes other SEPs (i.e., LOSs) in corresponding SRs. A theoretical foundation for both stages II and III is developed, and these theoretical results are quite general on their own. The proposed method is numerically evaluated to compute multiple LOSs. For instance, a total of 5085 LOSs have been computed by the proposed Trust-Tech source point method on a 50-D test function. In addition, the proposed method can find the global optimal solutions of several test functions with 50 dimensions and 100 dimensions.

A robust BFGS algorithm for unconstrained nonlinear optimization problems

The traditional BFGS algorithm has been proved very efficient. It is convergent for convex nonlinear optimization problems. However, for non-convex nonlinear optimization problems, it is known that the BFGS algorithm may not be convergent. This paper proposes a robust BFGS algorithm in the sense that the algorithm superlinearly converges to a local minimum under some mild assumptions for both convex and non-convex nonlinear optimization problems. Numerical test on the CUTEst test set is reported to demonstrate the merit of the proposed robust BFGS algorithm. This result shows that the robust BFGS algorithm is very efficient and effective.

A Sufficient Descent Property to Improving a ThreeTerm Conjugate Gradient Algorithm

The nonlinear conjugate gradient (NLCGM) methods have received attention because due to their simplicity, low memory requirements, and global convergent property, which allows them to be used directly to solve large-scale nonlinear unconstrained optimization problems. We suggested a modification to the β KMAR k formula, applied with three-term conjugate gradient method that is both simple and effective, denoted by (TTKMAR), which has a sufficient descent property (SDP) and ensures global convergence (GCP) when we use any line search. The numerical efficiency of TTKMAR was assessed using a variety of standard test functions. TTCGM has been demonstrated to be more numerically efficient than two-term CG methods. This paper also quantifies the difference between TTCGM and two-term methods of performance. As a result, we compare our new modification to an efficient two-term and TTCGM in the numerical results. Finally, we conclude that our proposed modification.

Ritz approximate method for solving delay fractional optimal control problems

In this paper, a numerical method based on the Müntz–Legendre polynomials for solving a class of time-delay fractional optimal control problems (TDFOCPs) is presented. The concept of fractional derivatives is considered in the Caputo sense. The key point of the applied method is that the initial and boundary conditions are imposed upon the approximations of state and control functions. First, the unknown state or control and their delayed functions are approximated by the Müntz–Legendre polynomials basis using the Ritz method then the next one is calculated through the given fractional differential equation. By substituting the estimated values in the cost function, the TDFOCP is converted to an unconstrained linear or nonlinear optimization problem. The convergence of the suggested method is extensively argued and several illustrative examples are presented to show the applicability and effectiveness of the method. It is shown that only a few terms of Müntz–Legendre polynomials are needed for obtaining the high accuracy and satisfactory results. The obtained results are compared with the available results in the literature to demonstrate the superiority of the applied approach. Furthermore, the approximated solutions agree with exact solutions and as α approaches an integer value, the method provides solution for the integer-order optimal control problem.

Mathematical Models and Nonlinear Optimization in Continuous Maximum Coverage Location Problem

This paper considers the maximum coverage location problem (MCLP) in a continuous formulation. It is assumed that the coverage domain and the family of geometric objects of arbitrary shape are specified. It is necessary to find such a location of geometric objects to cover the greatest possible amount of the domain. A mathematical model of MCLP is proposed in the form of an unconstrained nonlinear optimization problem. Python computational geometry packages were used to calculate the area of partial coverage domain. Many experiments were carried out which made it possible to describe the statistical dependence of the area calculation time of coverage domain on the number of covering objects. To obtain a local solution, the BFGS method with first-order differences was used. An approach to the numerical estimation of the objective function gradient is proposed, which significantly reduces computational costs, which is confirmed experimentally. The proposed approach is shown to solve the maximum coverage problem of a rectangular area by a family of ellipses.

A Descent Conjugate Gradient Method With Global Converges Properties for Non-Linear Optimization

Iterative methods such as the conjugate gradient method are well known methods for solving non-linear unconstrained minimization problems partially because of their capacity to handle large-scale unconstrained optimization problems rapidly, and partly due to their algebraic representation and implementation in computer programs. The conjugate gradient method has wide applications in a lot of fields such as machine learning, neural networks and many other fields. Fletcher and Reeves [1] expanded the approach to nonlinear problems in 1964. It is considered to be the first nonlinear conjugate gradient technique. Since then, lots of new other conjugate gradient methods have been proposed. In this work, we will propose a new coefficient conjugate gradient method to find the minimum of the non-linear unconstrained optimization problems based on parameter of Hestenes Stiefel. Section one in this work contains the derivative of new method. In section two, we will satisfy the descent and sufficient descent conditions. In section three, we will study the property of the global convergence of the new proposed. In the fourth section, we will give some numerical results by using some known test functions and compare the new method with Hestenes S. to demonstrate the effectiveness of the suggestion method. Finally, we will give conclusions.

Sufficient Descent Condition of the Polak-Ribière-Polyak (PRP) Conjugate Gradient Method without Line Search

Nonlinear conjugate gradient methods for unconstrained optimization problems are used in many aspects of theoretical and applied sciences. They are iterative methods, so at any iteration a step length is computed using a method called line search. In most cases, the sufficient descent condition plays an important role to prove the global convergence of a conjugate gradient method. Due to its outperformance in practical computation, the Polak-Ribière-Polyak (PRP) conjugate gradient method is widely used for solving nonlinear unconstrained optimization problems. However, the sufficient descent condition of PRP has not established without line search yet. In this study, we established the sufficient descent condition without line search based on the conditions 0 < bkPRP £ x bkFR and 1 < bkPRP £mbkFR, where 0 < x < 1 and m > 1. As a result, we found that under certain conditions, the sufficient descent condition is satisfied when the PRP implemented without line search

A Global Optimization Algorithm for Unconstrained Non-linear Optimization Problems Using Chaotic Neuron Map and Golden Number

This paper presents aninnovative global multi-variable optimization algorithm using one of the best chaotic sequences, the neuron map, a description of which is also provided in the paper. The algorithm uses neuron map in the first stage to move near the global minimum point, as well as in each iteration of the second stage of local search that is done using the N-dimensional golden section search algorithm. The generation and mapping of the neuron variables to the optimization variables along with the stagewise search for the global minimum is explained conscientiously in the work. Numerical results on some benchmark functions and the comparison with a latest state-of-the-art algorithm ispresented in order to demonstrate the efficiency of the proposed algorithm.

A Novel Second-Order Adaptive Grid Method for Singularly Perturbed Convection–Diffusion Equations

In this paper, a singularly perturbed convection–diffusion equation is studied. At first, the original problem is transformed into a parameterized singularly perturbed Volterra integro-differential equation by using an integral transform. Then, a second-order finite difference method on an arbitrary mesh is given. The stability and local truncation error estimates of the discrete schemes are analyzed. Based on the mesh equidistribution principle and local truncation error estimation, an adaptive grid algorithm is given. In addition, in order to calculate the parameters of the transformation equation, a nonlinear unconstrained optimization problem is constructed. Numerical experiments are given to illustrate the effectiveness of our presented adaptive grid algorithm.

BFEMP: Interpenetration-free MPM–FEM coupling with barrier contact

This paper introduces BFEMP, a new approach for monolithically coupling the Material Point Method (MPM) with the Finite Element Method (FEM) through barrier energy-based particle–mesh frictional contact using a variational time-stepping formulation. The fully implicit time integration of the coupled system is recast into a barrier-augmented unconstrained nonlinear optimization problem. A modified line-search Newton’s method is adopted to strictly prevent material points from penetrating the FEM domain, ensuring convergence and feasibility regardless of the time step size or the mesh resolutions. The proposed coupling scheme also reduces to a new approach for imposing separable frictional kinematic boundaries for MPM when all nodal displacements in the FEM domain are prescribed with Dirichlet boundary conditions. Compared to standard implicit time integration, the extra algorithmic components associated with the contact treatment only depend on simple point-segment (or point-triangle in 3D) geometric queries which robustly handle arbitrary FEM mesh boundaries represented with codimension-1 simplices. Experiments and analyses are performed to demonstrate the robustness and accuracy of the proposed method.

An Improved HS Type Spectral Conjugate Gradient Method

The spectral conjugate gradient (SCG) method was one of the most commomly used method to solve nonlinear unconstrained optimization problems. In this paper, an improved HS spectral conjugate gradient method is proposed. Under common assumptions, the descent and global convergence of this method are proved.

Point set registration for assembly feature pose estimation using simulated annealing nested Gauss-Newton optimization

PurposeThis paper aims to propose a fast and robust 3D point set registration method for pose estimation of assembly features with few distinctive local features in the manufacturing process.Design/methodology/approachThe distance between the two 3D objects is analytically approximated by the implicit representation of the target model. Specifically, the implicit B-spline surface is adopted as an interface to derive the distance metric. With the distance metric, the point set registration problem is formulated into an unconstrained nonlinear least-squares optimization problem. Simulated annealing nested Gauss-Newton method is designed to solve the non-convex problem. This integration of gradient-based optimization and heuristic searching strategy guarantees both global robustness and sufficient efficiency.FindingsThe proposed method improves the registration efficiency while maintaining high accuracy compared with several commonly used approaches. Convergence can be guaranteed even with critical initial poses or in partial overlapping conditions. The multiple flanges pose estimation experiment validates the effectiveness of the proposed method in real-world applications.Originality/valueThe proposed registration method is much more efficient because no feature estimation or point-wise correspondences update are performed. At each iteration of the Gauss–Newton optimization, the poses are updated in a singularity-free format without taking the derivatives of a bunch of scalar trigonometric functions. The advantage of the simulated annealing searching strategy is combined to improve global robustness. The implementation is relatively straightforward, which can be easily integrated to realize automatic pose estimation to guide the assembly process.