Total Number Of Function Evaluations Research Articles

An optimal eighth-order multipoint numerical iterative method to find simple root of scalar nonlinear equations

An optimal eighth-order multipoint numerical iterative method is constructed to find the simple root of scalar nonlinear equations. It is a three-point numerical iterative method that uses three evaluations of func-tion f(¢) associated with a scalar nonlinear equation and one of its deriv-atives f0 (¢). The four functional evaluations are required to achieve the eighth-order convergence. According to Kung-Traub conjecture (KTC), an iterative numerical multipoint method without memory can achieve maximum order of convergence 2n¡1 where n is the total number of func-tion evaluations in a single instance of the method. Therefore, following the KTC, the proposed method in this article is optimal.

A comparison of Quasi-Newton methods considering line search conditions in unconstrained minimization

Optimization algorithms are essential tools in all fields such as engineering, business, science and even in nature. Their performance and efficiency may differ from problem-to-problem. In this respect, the current paper aims to provide an understanding of the three well-known Quasi-Newton methods relation with various line search conditions on a performance basis. To this end, the fifteen Quasi-Newton method-line search condition combinations, which are composed of Symmetric-Rank-1 (SR-1), Broyden-Fletcher-Goldfarb-Shanno (BFGS) and Davidon-Fletcher-Powell (DFP) Quasi-Newton methods and Backtracking (BC), Armijo-Backtracking (ABC), Goldstein (GC), Weak Wolfe (WWC) and Strong Wolfe (SWC) conditions, are subjected to totally 195 computational experiments using thirteen test functions including smooth and non-smooth ones. During the experiments, the number of function evaluations at every iteration for all the combinations are kept track and the total number of function evaluations, when each combination converges, are set as a first performance metric. In addition to that, the eigenvalues of Hessian approximation matrix are checked if the updated matrix is positive definite or not. In case of having a negative definite Hessian approximation, the eigenvalue modification procedure is employed to guarantee that the updated matrix is positive definite, which in turn progress along the descent direction. Besides, for purpose of using in the performance evaluations, the number of negative definite matrix generations, as a second performance metric, are recorded for all the combinations. To conduct a reliable and an efficient performance analysis on the combinations, the performance and data profiles are used along with the number of negative definite Hessian matrix update generations. Based on the mathematical analysis through those data, the BFGS-GC combination is the fastest one whereas the slowest one is the DFP-SWC combination. For an optimal choice, it is determined that the BFGS-WWC combination is a great candidate. It is also observed that the BFGS and DFP methods operate very well as long as the curvature condition is satisfied by any line search conditions. However, this statement is not valid for the SR-1 method.

A Benchmark Study on Steepest Descent and Conjugate Gradient Methods-Line Search Conditions Combinations in Unconstrained Optimization

In this paper, it is aimed to computationally conduct a performance benchmarking for the steepest descent and the three well-known conjugate gradient methods (i.e., Fletcher-Reeves, Polak-Ribiere and Hestenes-Stiefel) along with six different step length calculation techniques/conditions, namely Backtracking, Armijo-Backtracking, Goldstein, weakWolfe, strongWolfe, Exact local minimizer in the unconstrained optimization. To this end, a series of computational experiments on a test function set is completed using the combinations of those optimization methods and line search conditions. During these experiments, the number of function evaluations for every iteration are monitored and recorded for all the optimization method-line search condition combinations. The total number of function evaluations are then set a performance measure when the combination in question converges to the functions minimums within the given convergence tolerance. Through those data, the performance and data profiles are created for all the optimization method-line search condition combinations with the purpose of a reliable and an efficient benchmarking. It has been determined that, for this test function set, the steepest descent-Goldstein combination is the fastest one whereas the steepest descent-exact local minimizer is the most robust one with a high convergence accuracy. By making a trade-off between convergence speed and robustness, it has been identified that the steepest descent-weak Wolfe combination is the optimal choice for this test function set.

Geometri Uydurma için En İyi Eşlenik Gradyan Yönteminin Tespit Edilmesi

In this study, it is aimed to determine the optimal conjugate gradient (CG) method for the geometry fitting of 2D measured profiles. To this end, the three well-known CG methods such as the Fletcher-Reeves, Polak-Ribiere and Hestenes-Stiefel were employed. For testing those methods performances, the five primitive geometries accommodating circle, square, triangle, ellipse and rectangle were first built with a 3D printer, and then they were scanned with a coordinate measuring machine (CMM) to achieve their 2D profiles. The nonlinear least squares procedure was implemented to minimize the error between those measured data and modeled ones. An iterative line search was utilized for this task. The search direction was calculated using the above-mentioned CG methods. During the geometry fitting process, the number of function evaluations at each iteration were computed and the total number of function evaluations were set to be a performance measure of the CG method in question when it converged. By using these performance measures, the performance and data profiles were created to efficiently determine the optimal CG method. Based on performance profiles, it can be stated that the Fletcher-Reeves and Polak-Ribiere methods are the fastest ones on three test geometries out of five. In addition to that, all the CG methods were able to complete the geometry fitting of 80% of test geometries. On the other hand, by examining the data profiles, it was determined that the Polak-Ribiere and Hestenes-Stiefel methods achieve their maximum capabilities of the completing geometry fitting (i.e., 80%) with much lower number of function evaluations than the Fletcher-Reeves method. Besides, in most geometries, the Polak-Ribiere method outperformed the others, thereby it was determined to be the optimal one for the geometry fitting. As a conclusion, the reported results in this work might help the end-users who study on the CMM data processing to conduct an efficient geometry fitting.

Convergence and Sample Complexity of Gradient Methods for the Model-Free Linear–Quadratic Regulator Problem

Model-free reinforcement learning attempts to find an optimal control action for an unknown dynamical system by directly searching over the parameter space of controllers. The convergence behavior and statistical properties of these approaches are often poorly understood because of the nonconvex nature of the underlying optimization problems and the lack of exact gradient computation. In this article, we take a step toward demystifying the performance and efficiency of such methods by focusing on the standard infinite-horizon linear–quadratic regulator problem for continuous-time systems with unknown state-space parameters. We establish exponential stability for the ordinary differential equation (ODE) that governs the gradient-flow dynamics over the set of stabilizing feedback gains and show that a similar result holds for the gradient descent method that arises from the forward Euler discretization of the corresponding ODE. We also provide theoretical bounds on the convergence rate and sample complexity of the random search method with two-point gradient estimates. We prove that the required simulation time for achieving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -accuracy in the model-free setup and the total number of function evaluations both scale as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\log \, (1/\epsilon)$</tex-math></inline-formula> .

Performance evaluation of a conjugate gradient method considering step length computation techniques in geometry fitting of coordinate measuring machine data

This paper focuses on performance evaluation of Hestenes–Stiefel conjugate gradient (CG) method along with two well-known line search conditions, which are weak Wolfe conditions (WWC) and strong Wolfe conditions (SWC), and a new step length computation algorithm in geometry fitting of five primitive geometries (i.e., circle, square, equilateral triangle, ellipse and rectangle). A coordinate measuring machine (CMM) is used to acquire 2D profiles of these geometries. Total number of function evaluations, when the CG method-step length computation technique combination being used successfully completes the geometry fitting, are considered to be performance criteria of the combination. It has been seen that the step length computation techniques for the line search crucially affect the success and performance of the Hestenes–Stiefel CG method in the geometry fitting. Therefore, they require a special care for optimal computational efficiency in addition to the line search direction.

Multifactorial Evolutionary Algorithm Based on Improved Dynamical Decomposition for Many-Objective Optimization Problems

In multiobjective optimization, it is generally known that the boom in computational complexity and search spaces came with a rise in the number of objectives, and this leads to a decrease in selection pressure and the deterioration of the evolutionary process. It follows then that the many-objective optimization problem (MaOP) has become one of the most challenging topics in the field of intelligent optimization. Recently, the multifactorial evolutionary algorithm (MFEA) and its variations, which have shown excellent performance in knowledge transfer across related problems, may provide a new and effective way for solving MaOPs. In this article, a novel MFEA based on improved dynamical decomposition (MFEA/IDD), which integrates the advantages of multitasking optimization and decomposition-based evolutionary algorithms, is proposed. Specifically, in the improved dynamical decomposition strategy (IDD) method, the bi-pivot strategy is designed to provide a good mechanism for balancing between convergence and diversity instead of the single-pivot strategy. Furthermore, a novel MFEA-based approach embedding the IDD strategy is developed to reduce the total running time for solving multiple MaOPs simultaneously. Compared with seven state-of-the-art algorithms, the efficacy of our proposed method is validated experimentally on the benchmarks WFG, DTLZ, and MAF with three to ten objectives, along with a series of real-world cases. The results reveal that the MFEA/IDD is well placed in balancing convergence and diversity while reducing the total number of function evaluations for solving MaOPs.

PERFORMANCE ASSESSMENT OF STEEPEST DESCENT METHOD CONSIDERING GRADIENT BASED LINE SEARCH CONDITIONS IN GEOMETRY FITTING OF 2D MEASURED PROFILES

This paper focuses on exploring effect of the step length, used in the line search, computation techniques on the performance of steepest descent (SD) method in the geometry fitting of the 2D measured profiles. To this end, the three step length computation techniques or line search conditions accommodating Weak Wolfe (WWC), Strong Wolfe (SWC) and the exact minimizer finder have been implemented during the fitting process. To test the line search conditions performances, the 2D primitive geometry test set consisting of five different geometries such as circle, square, triangle, ellipse and rectangle have been employed. The 2D profiles of those geometries have been extracted using coordinate measuring machine (CMM) with high precision. For performance assessments, the total number of function evaluations when the SD method-line search condition combination satisfies the required converge tolerance have been used. By means of those data, the performance profiles have been created to conduct reliable and efficient assessments on the line search conditions. The results have shown that the step length computation technique plays a crucial role for the SD method performance. Based on the performance profiles, it has been determined that the fastest line search condition is the WWC. Besides that, it has been revealed that the optimum technique is the exact minimizer finder for the geometry fitting process in the study.

A Modified Perry's Conjugate Gradient Method Based on Powell's Equation for Solving Large-Scale Unconstrained Optimization

It is known that the conjugate gradient method is still a popular method for many researchers who are focused in solving the large-scale unconstrained optimization problems and nonlinear equations because the method avoids the computation and storage of some matrices so the memory's requirements of the method are very small. In this work, a modified Perry conjugate gradient method which fulfills a global convergence with standard assumptions is shown and analyzed. The idea of new method is based on Perry method by using the equation which is founded via Powell in 1978. The weak Wolfe–Powell search conditions are used to choose the optimal line search, under the line search and suitable conditions, we prove both descent and sufficient descent conditions. In particular, numerical results show that the new conjugate gradient method is more effective and competitive when compared to other of standard conjugate gradient methods including: - CG- Hestenes and Stiefel (H/S) method, CG-Perry method, CG- Dai and Yuan (D/Y) method. The comparison is completed under a group of standard test problems with various dimensions from the CUTEst test library and the comparative performances of the methods are evaluated by total the number of iterations and the total number of function evaluations.

Performance Analysis of Steepest Descent-Line Search Condition Combinations in Nonlinear Least Squares Fitting of CMM Data

This paper presents a benchmarking study on the steepest descent (SD) method considering three different line search conditions including Backtracking (BC), Armijo-Backtracking (ABC) and Goldstein (GC) in nonlinear least squares fitting of measured data obtained from coordinate measuring machine (CMM). Within this scope, five primitive geometries such as circle, square, rectangle, triangle and ellipse were built via 3D printer. Those geometries were then scanned with CMM to acquire their 2D profiles to be fitted. To find best fitting parameters for each geometry, the nonlinear least squares approach along with the above-mentioned optimization method-line search condition combinations were employed. During the fitting process, the total number of function evaluations, when the combination converges to required tolerance, were used as a performance metric of the combination in question. With those data, the performance and data profiles for each combination were created to be able to carry out a reliable performance evaluation. Based on those profiles, it has been seen that the SD-ABC combination is the fastest one. In addition, it is successful on all the geometries while the others are not. For the second fastest combination, the SD-BC combination stands out. However, its successful rate is only 80%, which means it fails on a geometry. On the other hand, the SD-GC combination takes the last place in this study. All those results have shown that the line search conditions have a great contribution to the success and performance of the optimization algorithm being used. Besides, their performance may differ from problem-to-problem. The end-users should consider these facts to find best optimization method-line search condition combination for their problems.

An enhanced symbiotic organisms search algorithm for design optimization of trusses with frequency constraints

An enhanced symbiotic organisms search (ESOS) algorithm is developed and presented. Modifications to the basic symbiotic organisms search algorithm are carried out in all three phases of the algorithm with the aim of balancing the exploitation and exploration capabilities of the algorithm. To verify validity and capability of the ESOS algorithm in solving general optimization problems, the CEC2014 set of 22 benchmark functions is first optimized and the results are compared with other metaheuristic algorithms. The ESOS algorithm is then used to optimize the sizing and shape of five benchmark trusses with multiple frequency constraints. The best (minimum) mass, mean mass, standard deviation of the mass, total number of function evaluations, and the values of frequency constraints are then compared with those of a number of other metaheuristic solutions available in the literature. It is shown that the proposed ESOS algorithm is generally more efficient in optimizing the shape and sizing of trusses with dynamic frequency constraints compared to other reported metaheuristic algorithms, including the basic symbiotic organisms search and its other recently proposed improved variants such as the improved symbiotic organisms search algorithm (ISOS) and modified symbiotic organisms search algorithm (MSOS).

Optimization of structural and mechanical engineering problems using the enriched ViS-BLAST method

In this paper, an enhanced Violation-based Sensitivity analysis and Border-Line Adaptive Sliding Technique (ViS-BLAST) will be utilized for optimization of some well-known structural and mechanical engineering problems. ViS-BLAST has already been introduced by the authors for solving truss optimization problems. For those problems, this method showed a satisfactory enactment both in speed and efficiency. The Enriched ViS-BLAST or EVB is introduced to be vastly applicable to any solvable constrained optimization problem without any specific initialization. It uses one-directional step-wise searching technique and mostly limits exploration to the vicinity of FNF border and does not explore the entire design space. It first enters the feasible region very quickly and keeps the feasibility of solutions. For doing this important, EVB groups variables for specifying the desired searching directions in order to moving toward best solutions out or inside feasible domains. EVB was employed for solving seven numerical engineering design problems. Results show that for problems with tiny or even complex feasible regions with a larger number of highly non-linear constraints, EVB has a better performance compared to some records in the literature. This dominance was evaluated in terms of the feasibility of solutions, the quality of optimum objective values found and the total number of function evaluations performed.

Self-adaptive salp swarm algorithm for engineering optimization problems

Salp swarm algorithm is a recent introduction in the field of swarm intelligent algorithms and has proved its worth over various research domains. Though it is a competitive algorithm but it has been found that salp swarm algorithm suffers from various problems including poor exploitation, slow convergence and unbalanced exploration and exploitation operation. In present work, four major modifications have been added to salp swarm algorithm in order to make it self-adaptive and the proposed algorithm has been named as adaptive salp swarm algorithm. The modifications include division of generations and logarithmic adaptive parameters to control the extent of exploration and exploitation, enhanced exploitation phase to improve the local search and linearly decreasing population adaptation to reduce the total number of function evaluations. The performance of the proposed algorithm is tested on benchmark problems and further applied for optimization of transmission parameters in cognitive radio system. From the experimental results, it has been found that the proposed adaptive salp swarm algorithm is highly competitive and provides better results when compared with bat algorithm, grey wolf optimization, teacher learning based algorithm, dragonfly algorithm and others. Convergence profiles and statistical tests further validate the results.

Efficient method for solving highly oscillatory ordinary differential equations with applications to physical systems

We present a novel numerical routine (oscode) with a C++ and Python interface for the efficient solution of one-dimensional, second-order, ordinary differential equations with rapidly oscillating solutions. The method is based on a Runge-Kutta-like stepping procedure that makes use of the Wentzel-Kramers-Brillouin (WKB) approximation to skip regions of integration where the characteristic frequency varies slowly. In regions where this is not the case, the method is able to switch to a made-to-measure Runge-Kutta integrator that minimises the total number of function evaluations. We demonstrate the effectiveness of the method with example solutions of the Airy equation and an equation exhibiting a burst of oscillations, discussing the error properties of the method in detail. We then show the method applied to physical systems. First, the one-dimensional, time-independent Schr\"odinger equation is solved as part of a shooting method to search for the energy eigenvalues for a potential with quartic anharmonicity. Then, the method is used to solve the Mukhanov-Sasaki equation describing the evolution of cosmological perturbations, and the primordial power spectrum of the perturbations is computed in different cosmological scenarios. We compare the performance of our solver in calculating a primordial power spectrum of scalar perturbations to that of BINGO, an efficient code specifically designed for such applications.

A sequential surrogate-based multiobjective optimization method: effect of initial data set

Process optimization based on high-fidelity computer simulations or real experimentation is commonly expensive. Therefore, surrogate models are frequently used to reduce the computational or experimental cost. However, surrogate models need to achieve a maximum accuracy with a limited number of sampled points. Sequential sampling is a procedure in which sequentially surrogates are fitted and each surrogate defines the points that need to be sampled and used to fit the next model. For optimization purposes, points are sampled on regions of high potential for the optimal solutions. In this work, we first compared the effect of using different initial sets of points (experimental designs) in a sequential surrogate-based multiobjective optimization method. The optimization method is tested on five benchmark problems and the performance is quantified based on the total number of function evaluations and the quality of the final Pareto Front. Then an industrial applications on titanium welding is presented to show the use of the method. The case study is based on real experimental data.

Development of General-Purpose Root-Finding Module for General Mission Analysis Tool

A general-purpose root-finding module for designing spacecraft trajectories is developed to have similar accuracy to that of other well-known root-finding modules, and greater speed. Three quasi-Newton root-finding algorithms are implemented: the Newton–Raphson method, the Broyden’s method and the Generalized Broyden’s method. Based on the proposed root-finding module, General Mission Analysis Tool (GMAT) of National Aeronautics and Space Administration’s Goddard Space Flight Center (NASA/GSFC) is functionally extended by integrating the Broyden’s method and the Generalized Broyden’s method into its differential corrector module. Non-trivial spacecraft trajectory design problems, such as Lambert’s problem for trans-lunar trajectory and minimum-time transfer problem to the Mars are solved to analyze the performance of the proposed module. The numerical performances of each root-finding algorithm are quantitatively analyzed by the total number of function evaluations, the total number of iterations, convergence error, mean convergence rate, and running time. The overall comparative analysis shows that the Broyden’s method and the Generalized Broyden’s method are about 20–40% faster than the Newton–Raphson method and solutions from each algorithm have similar numerical accuracy. We also show that for selected test problems, the Generalized Broyden’s method converges in less running time than others with similar numerical accuracy in GMAT. The updated differential corrector module was released in R2014a version of GMAT.

Comparing two techniques for developing higher order two-point iterative methods for solving quadratic equations

Kung–Traub conjecture states that an iterative method without memory for finding the simple zero of a scalar equation could achieve convergence order $$2^{d-1}$$ , where d is the total number of function evaluations. Babajee (Algorithms 9:1, 2016) proposed some higher order two-point methods which fail the conjecture. He developed these methods for solving quadratic equations using weight functions. Recently, Ahmad (Algorithms 9:30, 2016) showed that the proposed method in Babajee (2016) was reported by him (Ahmad, in Researchgate, https://doi.org/10.13140/RG.2.1.1519.2487 , 2015) in which he used a loop to develop his methods. He also showed Babajee’s method is a member of his methods developed in Ahmad (2015). In this paper, we compare the two techniques and adopt the technique of weight functions to develop higher order Jarratt and Ostrowski’s methods for solving quadratic equations. We prove the local convergence of the methods by induction. Numerical experiments are carried out to compare the new methods with some existing methods. We apply our methods to find the optimal launch angle in a projectile problem.

Towards faster convergence of evolutionary multi-criterion optimization algorithms using Karush Kuhn Tucker optimality based local search

Evolutionary multi-criterion optimization (EMO) algorithms emphasize non-dominated and less crowded solutions in a population iteratively until the population converges close to the Pareto optimal set. During the search process, non-dominated solutions are differentiated only by their local crowding or contribution to hypervolume or using a similar other metric. Thus, during evolution and even at the final iteration, the true convergence behavior of each non-dominated solutions from the Pareto optimal set is unknown. Recent studies have used Karush Kuhn Tucker (KKT) optimality conditions to develop a KKT Proximity Measure (KKTPM) for estimating proximity of a solution from Pareto optimal set for a multi-objective optimization problem. In this paper, we integrate KKTPM with a recently proposed EMO algorithm to enhance its convergence properties towards the true Pareto optimal front. Specifically, we use KKTPM to identify poorly converged non-dominated solutions in every generation and apply an achievement scalarizing function based local search procedure to improve their convergence. Assisted by the KKTPM, the modified algorithm is designed in a way that maintains the total number of function evaluations as low as possible while making use of local search where it is most needed. Simulations on both constrained and unconstrained multi- and many objectives optimization problems demonstrate that the hybrid algorithm significantly improves the overall convergence properties. This study brings evolutionary optimization closer to mainstream optimization field and should motivate researchers to utilize KKTPM measure further within EMO and other numerical optimization algorithms.

Comment on: On the Kung-Traub Conjecture for Iterative Methods for Solving Quadratic Equations. Algorithms 2016, 9, 1

Kung-Traub conjecture states that an iterative method without memory for finding the simple zero of a scalar equation could achieve convergence order 2 d − 1 , and d is the total number of function evaluations. In an article “Babajee, D.K.R. On the Kung-Traub Conjecture for Iterative Methods for Solving Quadratic Equations, Algorithms 2016, 9, 1, doi:10.3390/a9010001”, the author has shown that Kung-Traub conjecture is not valid for the quadratic equation and proposed an iterative method for the scalar and vector quadratic equations. In this comment, we have shown that we first reported the aforementioned iterative method.

ParEGO extensions for multi-objective optimization of expensive evaluation functions

This paper deals with multi-objective optimization in the case of expensive objective functions. Such a problem arises frequently in engineering applications where the main purpose is to find a set of optimal solutions in a limited global processing time. Several algorithms use linearly combined criteria to use directly mono-objective algorithms. Nevertheless, other algorithms, such as multi-objective evolutionary algorithm (MOEA) and model-based algorithms, propose a strategy based on Pareto dominance to optimize efficiently all criteria. A widely used model-based algorithm for multi-objective optimization is Pareto efficient global optimization (ParEGO). It combines linearly the objective functions with several random weights and maximizes the expected improvement (EI) criterion. However, this algorithm tends to favor parameter values suitable for the reduction of the surrogate model error, rather than finding non-dominated solutions. The contribution of this article is to propose an extension of the ParEGO algorithm for finding the Pareto Front by introducing a double Kriging strategy. Such an innovation allows to calculate a modified EI criterion that jointly accounts for the objective function approximation error and the probability to find Pareto Set solutions. The main feature of the resulting algorithm is to enhance the convergence speed and thus to reduce the total number of function evaluations. This new algorithm is compared against ParEGO and several MOEA algorithms on a standard benchmark problems. Finally, an automotive engineering problem allowing to illustrate the applicability of the proposed approach is given as an example of a real application: the parameter setting of an indirect tire pressure monitoring system.