Global Optimization Test Problems Research Articles

Dynamic Exploration–Exploitation Pareto Approach for high-dimensional expensive black-box optimization

Surrogate Optimization (SO) plays a vital role in optimizing performance parameters for computationally expensive simulations. However, SO encounters significant challenges in high-dimensional spaces due to the curse of dimensionality, hampering effective point sampling around global optima. In this paper, we introduce “Dynamic Exploration–Exploitation Pareto Approach (DEEPA),” a novel SO method that combines Pareto sampling with a dynamic discretization schema to optimize high-dimensional black-box functions. Unlike traditional SO methods that heavily rely on specific surrogate models, Pareto sampling offers a more adaptable approach. Improvement-based acquisition functions, frequently employed in black-box optimization, are sensitive to model accuracy and tend to prioritize exploitation, potentially missing valuable regions of interest in complex landscapes. Furthermore, they can encounter challenges when dealing with high-dimensional problems due to the curse of dimensionality. DEEPA leverages dynamic coordinate importance to generate samples effectively, providing a solution for addressing high-dimensionality and complex functions. We employ feature selection strategies to assign importance probabilities to perturb each coordinate, demonstrating the impact of importance-based perturbation on convergence to a near-optimal region. We showcase DEEPA’s versatility in fixed-batch evaluation environments using complex global optimization test problems with various topological properties. We compare DEEPA’s performance with state-of-the-art black-box optimization algorithms, and our experimental results demonstrate DEEPA’s superior performance, particularly in complex problems with multiple local minima.

DIRECTGO : A New DIRECT -Type MATLAB Toolbox for Derivative-Free Global Optimization

In this work, we introduce DIRECTGO , a new MATLAB toolbox for derivative-free global optimization. DIRECTGO collects various deterministic derivative-free DIRECT -type algorithms for box-constrained, generally constrained, and problems with hidden constraints. Each sequential algorithm is implemented in two ways: using static and dynamic data structures for more efficient information storage and organization. Furthermore, parallel schemes are applied to some promising algorithms within DIRECTGO . The toolbox is equipped with a graphical user interface (GUI), ensuring the user-friendly use of all functionalities available in DIRECTGO . Available features are demonstrated in detailed computational studies using a comprehensive DIRECTGOLib v1.0 library of global optimization test problems. Additionally, 11 classical engineering design problems illustrate the potential of DIRECTGO to solve challenging real-world problems. Finally, the appendix gives examples of accompanying MATLAB programs and provides a synopsis of its use on the test problems with box and general constraints.

Global optimization test problems based on random field composition

The development and identification of effective optimization algorithms for non-convex real-world problems is a challenge in global optimization. Because theoretical performance analysis is difficult, and problems based on models of real-world systems are often computationally expensive, several artificial performance test problems and test function generators have been proposed for empirical comparative assessment and analysis of metaheuristic optimization algorithms. These test problems however often lack the complex function structures and forthcoming difficulties that can appear in real-world problems. This communication presents a method to systematically build test problems with various types and degrees of difficulty. By weighted composition of parameterized random fields, challenging test functions with tunable function features such as, variance contribution distribution, interaction order, and nonlinearity can be constructed. The method is described, and its applicability to optimization performance analysis is described by means of a few basic examples. The method aims to set a step forward in the systematic generation of global optimization test problems, which could lead to a better understanding of the performance of optimization algorithms on problem types with particular characteristics. On request an introductive MATLAB implementation of a test function generator based on the presented method is available.

Nonlinear Symbolic Transformations for Simplifying Optimization Problems

The theory of nonlinear optimization traditionally studies numeric computations. However, increasing attention is being paid to involve computer algebra into mathematical programming. One can identify two possibilities of applying symbolic techniques in this field. Computer algebra can help the modeling phase by producing alternate mathematical models via symbolic transformations. The present paper concentrates on this direction. On the other hand, modern nonlinear solvers use more and more information about the structure of the problem through the optimization process leading to hybrid symbolic-numeric nonlinear solvers. This paper presents a new implementation of a symbolic simplification algorithm for unconstrained nonlinear optimization problems. The program can automatically recognize helpful transformations of the mathematical model and detect implicit redundancy in the objective function. We report computational results obtained for standard global optimization test problems and for other artificially constructed instances. Our results show that a heuristic (multistart) numerical solver takes advantage of the automatically produced transformations. New theoretical results will also be presented, which help the underlying method to achieve more complicated transformations.

Continuous Global Optimization inR

This article surveys currently available implementations in R for continuous global optimization problems. A new R package globalOptTests is presented that provides a set of standard test problems for continuous global optimization based on C functions by Ali, Khompatraporn, and Zabinsky (2005). 48 of the objective functions contained in the package are used in empirical comparison of 18 R implementations in terms of the quality of the solutions found and speed.

Nonlinear transformations for the simplification of unconstrained nonlinear optimization problems

Formalization decisions in mathematical programming could significantly influence the complexity of the problem, and so the efficiency of the applied solver methods. This widely accepted statement induced investigations for the reformulation of optimization problems in the hope of getting easier to solve problem forms, e.g. in integer programming. These transformations usually go hand in hand with relaxation of some constraints and with the increase in the number of the variables. However, the quick evolution and the widespread use of computer algebra systems in the last few years motivated us to use symbolic computation techniques also in the field of global optimization. We are interested in potential simplifications generated by symbolic transformations in global optimization, and especially in automatic mechanisms producing equivalent expressions that possibly decrease the dimension of the problem. As it was pointed out by Csendes and Rapcsak (J Glob Optim 3(2):213–221, 1993), it is possible in some cases to simplify the unconstrained nonlinear objective function by nonlinear coordinate transformations. That means mostly symbolic replacement of redundant subexpressions expecting less computation, while the simplified task remains equivalent to the original in the sense that a conversion between the solutions of the two forms is possible. We present a proper implementation of the referred theoretical algorithm in a modern symbolic programming environment, and testing on some examples both from the original publications and from the set of standard global optimization test problems to illustrate the capabilities of the method.

A new genetic algorithm for global optimization of multimodal continuous functions

In this paper a new genetic algorithm is developed to find the near global optimal solution of multimodal nonlinear optimization problems. The algorithm defined makes use of a real encoded crossover and mutation operator. The performance of GA is tested on a set of twenty-seven nonlinear global optimization test problems of variable difficulty level. Results are compared with some well established popular GAs existing in the literature. It is observed that the algorithm defined performs significantly better than the existing ones.

A multi-start opposition-based particle swarm optimization algorithm with adaptive velocity for bound constrained global optimization

In this paper we present a multi-start particle swarm optimization algorithm for the global optimization of a function subject to bound constraints. The procedure consists of three main steps. In the initialization phase, an opposition learning strategy is performed to improve the search efficiency. Then a variant of the adaptive velocity based on the differential operator enhances the optimization ability of the particles. Finally, a re-initialization strategy based on two diversity measures for the swarm is act in order to avoid premature convergence and stagnation. The strategy uses the super-opposition paradigm to re-initialize particles in the swarm. The algorithm has been evaluated on a set of 100 global optimization test problems. Comparisons with other global optimization methods show the robustness and effectiveness of the proposed algorithm.

INFLUENCE OF LIPSCHITZ BOUNDS ON THE SPEED OF GLOBAL OPTIMIZATION

Global optimization methods based on Lipschitz bounds have been analyzed and applied widely to solve various optimization problems. In this paper a bound for Lipschitz function is proposed, which is computed using function values at the vertices of a simplex and the radius of the circumscribed sphere. The efficiency of a branch and bound algorithm with proposed bound and combinations of bounds is evaluated experimentally while solving a number of multidimensional test problems for global optimization. The influence of different bounds on the performance of a branch and bound algorithm has been investigated.

Integrated experimental design and nonlinear optimization to handle computationally expensive models under resource constraints

In many real-world applications of optimization, the underlying descriptive system model is defined by computationally expensive functions: simulation modules, numerical models and other "black box" model components are typical examples. In such cases, the model development and optimization team often has to rely on optimization carried out under severe resource constraints. To address this important issue, recently a Regularly Spaced Sampling (RSS) module has been added to the Lipschitz Global Optimizer (LGO) solver suite. RSS generates non-collapsing space filling designs, and produces corresponding solution estimates: this information is passed along to LGO for refinement within the given resource (function evaluation and/or runtime) limitations. Obviously, the quality of the solution obtained will essentially depend both on model instance difficulty and on the admissible computational effort. In spite of this general caveat, our results based on solving a selection of non-trivial global optimization test problems suggest that even a moderate amount of well-placed sampling effort enhanced by limited optimization can lead at least to reasonable or even to high quality results. Our numerical tests also indicate that LGO's overall efficiency is often increased by using RSS as a presolver, both in resource-constrained and in completed LGO runs.

Cluster‐structured particle swarm optimization with interaction and adaptation

AbstractParticle swarm optimization (PSO) is one of the most powerful methods for solving unconstrained and constrained global optimization problems. A cluster‐structured PSO with interaction and adaptation is proposed in this paper, and the cluster structure, interaction, and adaptation of the proposed PSO are analyzed by numerical simulations. The feasibility and advantages of the proposed cluster‐structured PSO are demonstrated by numerical simulations using some typical global optimization test problems. © 2011 Wiley Periodicals, Inc. Electron Comm Jpn, 94(11): 9–17, 2011; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10379

An Inflationary PSwarm Algorithm for Space Trajectory Design

A hybrid optimization algorithm, we call it Inflationary PSwarm Algorithm, is applied to solve the space trajectory design with gravity assists and multiple impulses. In this paper we will consider some simplified models proposed by the European Space Agency as test problems for global optimization. PSwarm is a pattern search method that incorporates a particle swarm scheme for dissemination of points in the feasible region in the (optional) search step. PSwarm is highly competitive with other global optimization methods based only on function values. An enhanced PSwarm with the capability of finding a global minimizer is proposed in this paper. The experiment results show the good performance obtained with our method and the improvement in terms of efficiency and computational cost.

Parallel branch and bound for global optimization with combination of Lipschitz bounds

The solution of multidimensional Lipschitz optimization problem requires a lot of computing time and memory resources. Parallel OpenMP and MPI versions of branch and bound algorithm with simplicial partitions and Lipschitz bounds were created, investigated and compared in this paper. The efficiency of the developed parallel algorithms is investigated by solving multidimensional test problems for global optimization.

Cluster-Structured Particle Swarm Optimization with Interaction and Adaptation

Particle Swarm Optimization (PSO) is one of the most powerful methods for solving unconstrained and constrained global optimization problems. While a cluster-structured PSO with interaction and adaptation is proposed in this paper, the cluster structure, interaction and adaptation of the proposed PSO are analyzed through some numerical simulations. The feasibility and the advantage of the proposed cluster-structured PSO are demonstrated through numerical simulations using some typical global optimization test problems.

Mixture surrogate models based on Dempster-Shafer theory for global optimization problems

Recent research in algorithms for solving global optimization problems using response surface methodology has shown that it is in general not possible to use one surrogate model for solving different kinds of problems. In this paper the approach of applying Dempster-Shafer theory to surrogate model selection and their combination is described. Various conflict redistribution rules have been examined with respect to their influence on the results. Furthermore, the implications of the surrogate model type, i.e. using combined, single or a hybrid of both, have been studied. The suggested algorithms were applied to several well-known global optimization test problems. The results indicate that the used approach leads for all problems to a thorough exploration of the variable domain, i.e. the vicinities of global optima could be detected, and that the global minima could in most cases be approximated with high accuracy.

Asynchronous parallel hybrid optimization combining DIRECT and GSS

In this paper, we explore hybrid parallel global optimization using Dividing Rectangles (DIRECT) and asynchronous generating set search (GSS). Both DIRECT and GSS are derivative-free and so require only objective function values; this makes these methods applicable to a wide variety of science and engineering problems. DIRECT is a global search method that strategically divides the search space into ever-smaller rectangles, sampling the objective function at the centre point for each rectangle. GSS is a local search method that samples the objective function at trial points around the current best point, i.e. the point with the lowest function value. Latin hypercube sampling can be used to seed GSS with a good starting point. Using a set of global optimization test problems, we compare the parallel performance of DIRECT and GSS with hybrids that combine the two methods. Our experiments suggest that the hybrid methods are much faster than DIRECT and scale better when more processors are added. This improvement in performance is achieved without any sacrifice in the quality of the solution – the hybrid methods find the global optimum whenever DIRECT does.

Pattern discrete and mixed Hit-and-Run for global optimization

We develop new Markov chain Monte Carlo samplers for neighborhood generation in global optimization algorithms based on Hit-and-Run. The success of Hit-and-Run as a sampler on continuous domains motivated Discrete Hit-and-Run with random biwalk for discrete domains. However, the potential for efficiencies in the implementation, which requires a randomization at each move to create the biwalk, lead us to a different approach that uses fixed patterns in generating the biwalks. We define Sphere and Box Biwalks that are pattern-based and easily implemented for discrete and mixed continuous/discrete domains. The pattern-based Hit-and-Run Markov chains preserve the convergence properties of Hit-and-Run to a target distribution. They also converge to continuous Hit-and-Run as the mesh of the discretized variables becomes finer, approaching a continuum. Moreover, we provide bounds on the finite time performance for the discrete cases of Sphere and Box Biwalks. We embed our samplers in an Improving Hit-and-Run global optimization algorithm and test their performance on a number of global optimization test problems.

Convexity and Concavity Detection in Computational Graphs: Tree Walks for Convexity Assessment

We examine symbolic tools associated with two modeling systems for mathematical programming, which can be used to automatically detect the presence or absence of convexity and concavity in the objective and constraint functions, as well as convexity of the feasible set in some cases. The coconut solver system [Schichl, H. 2004a. COCONUT: COntinuous CONstraints—Updating the technology] focuses on nonlinear global continuous optimization and possesses its own modeling language and data structures. The Dr. Ampl meta-solver [Fourer, R., D. Orban. 2007. Dr. Ampl—A meta solver for optimization. Technical Report G-2007-10, GERAD, Montréal] aims to analyze nonlinear differentiable optimization models and hooks into the ampl Solver Library [Gay, D. M. 2002. Hooking your solver to AMPL]. Our symbolic convexity analysis may be supplemented, when it returns inconclusive results, with a numerical phase that may detect nonconvexity. We report numerical results using these tools on sets of test problems for both global and local optimization.

Down-hill Simplex Method Based Differential Evolution

Differential Evolution (DE) is based on both an evolutionary strategy and a parallel direct search method employing a population. DE is an effective optimization method available for solving global optimization problem over continuous space. DE has a few control parameters that have to be set by users. This paper describes a new DE using Down-hill Simplex Method. Then we consider average distance of a new DE and we examine DE to choice a proposal method or the other proposal method according to the average distance. The feasibility and advantage of the proposed DE are demonstrated through some numerical simulations using four different typical global optimization test problems.

Investigation of selection strategies in branch and bound algorithm with simplicial partitions and combination of Lipschitz bounds

Speed and memory requirements of branch and bound algorithms depend on the selection strategy of which candidate node to process next. The goal of this paper is to experimentally investigate this influence to the performance of sequential and parallel branch and bound algorithms. The experiments have been performed solving a number of multidimensional test problems for global optimization. Branch and bound algorithm using simplicial partitions and combination of Lipschitz bounds has been investigated. Similar results may be expected for other branch and bound algorithms.