Continuous Global Optimization Research Articles

Consistency of Sequential Bayesian Sampling Policies

We consider Bayesian information collection, in which a measurement policy collects information to support a future decision. This framework includes ranking and selection, continuous global optimization, and many other problems in sequential experimental design. We give a sufficient condition under which measurement policies sample each measurement type infinitely often, ensuring consistency, i.e., that a globally optimal future decision is found in the limit. This condition is useful for verifying consistency of adaptive sequential sampling policies that do not do forced random exploration, making consistency difficult to verify by other means. We demonstrate the use of this sufficient condition by showing consistency of two previously proposed ranking and selection policies: optimal computing budget allocation (OCBA) for linear loss, and the knowledge-gradient policy with independent normal priors. Consistency of the knowledge-gradient policy was shown previously, while the consistency result for OCBA is new.

Convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization

This paper presents some simple technical conditions that guarantee the convergence of a general class of adaptive stochastic global optimization algorithms. By imposing some conditions on the probability distributions that generate the iterates, these stochastic algorithms can be shown to converge to the global optimum in a probabilistic sense. These results also apply to global optimization algorithms that combine local and global stochastic search strategies and also those algorithms that combine deterministic and stochastic search strategies. This makes the results applicable to a wide range of global optimization algorithms that are useful in practice. Moreover, this paper provides convergence conditions involving the conditional densities of the random vector iterates that are easy to verify in practice. It also provides some convergence conditions in the special case when the iterates are generated by elliptical distributions such as the multivariate Normal and Cauchy distributions. These results are then used to prove the convergence of some practical stochastic global optimization algorithms, including an evolutionary programming algorithm. In addition, this paper introduces the notion of a stochastic algorithm being probabilistically dense in the domain of the function and shows that, under simple assumptions, this is equivalent to seeing any point in the domain with probability 1. This, in turn, is equivalent to almost sure convergence to the global minimum. Finally, some simple results on convergence rates are also proved.

A global optimization algorithm for protein surface alignment

BackgroundA relevant problem in drug design is the comparison and recognition of protein binding sites. Binding sites recognition is generally based on geometry often combined with physico-chemical properties of the site since the conformation, size and chemical composition of the protein surface are all relevant for the interaction with a specific ligand. Several matching strategies have been designed for the recognition of protein-ligand binding sites and of protein-protein interfaces but the problem cannot be considered solved.ResultsIn this paper we propose a new method for local structural alignment of protein surfaces based on continuous global optimization techniques. Given the three-dimensional structures of two proteins, the method finds the isometric transformation (rotation plus translation) that best superimposes active regions of two structures. We draw our inspiration from the well-known Iterative Closest Point (ICP) method for three-dimensional (3D) shapes registration. Our main contribution is in the adoption of a controlled random search as a more efficient global optimization approach along with a new dissimilarity measure. The reported computational experience and comparison show viability of the proposed approach.ConclusionsOur method performs well to detect similarity in binding sites when this in fact exists. In the future we plan to do a more comprehensive evaluation of the method by considering large datasets of non-redundant proteins and applying a clustering technique to the results of all comparisons to classify binding sites.

Scalability of generalized adaptive differential evolution for large-scale continuous optimization

Differential evolution (DE) has become a very powerful tool for global continuous optimization problems. Parameter adaptations are the most commonly used techniques to improve its performance. The adoption of these techniques has assisted the success of many adaptive DE variants. However, most studies on these adaptive DEs are limited to some small-scale problems, e.g. with less than 100 decision variables, which may be quite small comparing to the requirements of real-world applications. The scalability performance of adaptive DE is still unclear. In this paper, based on the analyses of similarities and drawbacks of existing parameter adaptation schemes in DE, we propose a generalized parameter adaptation scheme. Applying the scheme to DE results in a new generalized adaptive DE (GaDE) algorithm. The scalability performance of GaDE is evaluated on 19 benchmark functions with problem scale from 50 to 1,000 decision variables. Based on the comparison with three other algorithms, GaDE is very competitive in both the performance and scalability aspects.

Speeding up continuous GRASP

Abstract Continuous GRASP (C-GRASP) is a stochastic local search metaheuristic for finding cost-efficient solutions to continuous global optimization problems subject to box constraints (Hirsch et al., 2007). Like a greedy randomized adaptive search procedure (GRASP), a C-GRASP is a multi-start procedure where a starting solution for local improvement is constructed in a greedy randomized fashion. In this paper, we describe several improvements that speed up the original C-GRASP and make it more robust. We compare the new C-GRASP with the original version as well as with other algorithms from the recent literature on a set of benchmark multimodal test functions whose global minima are known. Hart’s sequential stopping rule (1998) is implemented and C-GRASP is shown to converge on all test problems.

Hybrid optimization with improved tabu search

Finding a global optimum of an unknown system has attracted a great deal of interest in many engineering problems. In this settings, meta-heuristics are very common as efficient approaches for solving complex real-world problems in global continuous optimization problems (GCOPs) as they can approximate solutions without considering mathematical constraints such as differentiability. In this study, we propose a method based on tabu search (TS) and Nelder–Mead (NM) search strategy in application to GCOPs. To increase the robustness of the proposed method, we add a new phase, referred to as partitioning phase, before diversification which is realized by the TS. The TS is improved and then followed by the NM search strategy. The partitioning phase aims at distributing initial random solutions in the search space. By doing this, we increase the robustness of the method. The TS has an interesting ability of covering a wide solution space by promoting the search far away from the current solution and consecutively decreasing the possibility of trapping in local minima. The neighbour search-strategy of the TS is improved to accelerate the speed of finding the near optimum solution. Instead of just generating random neighbours around the current solution, we generate some neighbours in the direction of the previous move as well as some neighbours in the previous best crown. When certain criteria are reached for the diversification of the search space, the NM search strategy is carried out with the focus on the intensification of the solution found in the diversification phase. We assess the performance of the algorithm for a range of standard test functions available in the literature and compare the obtained results with those available in the literature. There are two main advantages of the proposed method; first, it can apply to any GCOP without considering any constraints and secondly, it shows better performance (in terms of function evaluation, success rate, and average error) for the functions with less than four input variables and relatively small or medium input domains. In other cases, the method still has acceptable perfomance and produces the results that are comparable with the results produced by other methods.

Iterated local search with Powell’s method: a memetic algorithm for continuous global optimization

In combinatorial solution spaces Iterated Local Search (ILS) turns out to be exceptionally successful. The question arises: is ILS also capable of improving the optimization process in continuous solution spaces? To demonstrate that hybridization leads to powerful techniques in continuous domains, we introduce a hybrid meta-heuristic that integrates Powell’s direct search method. It combines direct search with elements from population based evolutionary optimization. The approach is analyzed experimentally on a set of well known test problems and compared to a state-of-the-art technique, i.e., a restart variant of the Covariance Matrix Adaptation Evolution Strategy with increasing population sizes (G-CMA-ES). It turns out that the population-based Powell-ILS is competitive to the CMA-ES, in some cases even significantly faster and behaves more robust than the pure strategy of Powell in multimodal fitness landscapes. Further experiments on the perturbation mechanism, population sizes, and problems with noise complete the analysis of the hybrid methodology and lead to parameter recommendations.

Convexity and Concavity Detection in Computational Graphs: Tree Walks for Convexity Assessment

We examine symbolic tools associated with two modeling systems for mathematical programming, which can be used to automatically detect the presence or absence of convexity and concavity in the objective and constraint functions, as well as convexity of the feasible set in some cases. The coconut solver system [Schichl, H. 2004a. COCONUT: COntinuous CONstraints—Updating the technology] focuses on nonlinear global continuous optimization and possesses its own modeling language and data structures. The Dr. Ampl meta-solver [Fourer, R., D. Orban. 2007. Dr. Ampl—A meta solver for optimization. Technical Report G-2007-10, GERAD, Montréal] aims to analyze nonlinear differentiable optimization models and hooks into the ampl Solver Library [Gay, D. M. 2002. Hooking your solver to AMPL]. Our symbolic convexity analysis may be supplemented, when it returns inconclusive results, with a numerical phase that may detect nonconvexity. We report numerical results using these tools on sets of test problems for both global and local optimization.

A local search method for continuous global optimization

A large number of algorithms introduced in the literature to find the global minimum of a real function rely on iterative executions of searches of a local minimum. Multistart, tunneling and some versions of simulated annealing are methods that produce well-known procedures. A crucial point of these algorithms is to decide whether to perform or not a new local search. In this paper we look for the optimal probability value to be set at each iteration so that by moving from a local minimum to a new one, the average number of function evaluations evals is minimal. We find that this probability has to be 0 or 1 depending on the number of function evaluations required by the local search and by the size of the level set at the current point. An implementation based on the above result is introduced. The values required to calculate evals are estimated from the history of the algorithm at running time. The algorithm has been tested both for sample problems constructed by the GKLS package and for problems often used in the literature. The outcome is compared with recent results.

CODEQ: an effective metaheuristic for continuous global optimisation

CODEQ is a new, parameter-free meta-heuristic algorithm that is hybrid of concepts from chaotic search, opposition-based learning, differential evolution (DE) and quantum mechanics. The performance of the proposed approach is investigated and compared with other well-known population-based optimisation approaches when applied to solve 19 benchmark functions. The conducted experiments show that CODEQ presents excellent results with almost no parameter tuning. Moreover, the performance of CODEQ when applied to high-dimensional problems is investigated with excellent results. Finally, the application of CODEQ to constrained and real-world engineering problems is investigated with encouraging results.

Hybrid discrete dynamically dimensioned search (HD‐DDS) algorithm for water distribution system design optimization

The dynamically dimensioned search (DDS) continuous global optimization algorithm by Tolson and Shoemaker (2007) is modified to solve discrete, single‐objective, constrained water distribution system (WDS) design problems. The new global optimization algorithm for WDS optimization is called hybrid discrete dynamically dimensioned search (HD‐DDS) and combines two local search heuristics with a discrete DDS search strategy adapted from the continuous DDS algorithm. The main advantage of the HD‐DDS algorithm compared with other heuristic global optimization algorithms, such as genetic and ant colony algorithms, is that its searching capability (i.e., the ability to find near globally optimal solutions) is as good, if not better, while being significantly more computationally efficient. The algorithm's computational efficiency is due to a number of factors, including the fact that it is not a population‐based algorithm and only requires computationally expensive hydraulic simulations to be conducted for a fraction of the solutions evaluated. This paper introduces and evaluates the algorithm by comparing its performance with that of three other algorithms (specific versions of the genetic algorithm, ant colony optimization, and particle swarm optimization) on four WDS case studies (21‐ to 454‐dimensional optimization problems) on which these algorithms have been found to perform well. The results obtained indicate that the HD‐DDS algorithm outperforms the state‐of‐the‐art existing algorithms in terms of searching ability and computational efficiency. In addition, the algorithm is easier to use, as it does not require any parameter tuning and automatically adjusts its search to find good solutions given the available computational budget.

Continuous GRASP with a local active-set method for bound-constrained global optimization

Global optimization seeks a minimum or maximum of a multimodal function over a discrete or continuous domain. In this paper, we propose a hybrid heuristic--based on the CGRASP and GENCAN methods--for finding approximate solutions for continuous global optimization problems subject to box constraints. Experimental results illustrate the relative effectiveness of CGRASP---GENCAN on a set of benchmark multimodal test functions.

A simulated annealing driven multi-start algorithm for bound constrained global optimization

A derivative-free simulated annealing driven multi-start algorithm for continuous global optimization is presented. We first propose a trial point generation scheme in continuous simulated annealing which eliminates the need for the gradient-based trial point generation. We then suitably embed the multi-start procedure within the simulated annealing algorithm. We modify the derivative-free pattern search method and use it as the local search in the multi-start procedure. We study the convergence properties of the algorithm and test its performance on a set of 50 problems. Numerical results are presented which show the robustness of the algorithm. Numerical comparisons with a gradient-based simulated annealing algorithm and three population-based global optimization algorithms show that the new algorithm could offer a reasonable alternative to many currently available global optimization algorithms, specially for problems requiring ‘direct search’ type algorithm.

An evolutionary method for complex-process optimization

In this paper we present a new evolutionary method for complex-process optimization. It is partially based on the principles of the scatter search methodology, but it makes use of innovative strategies to be more effective in the context of complex-process optimization using a small number of tuning parameters. In particular, we introduce a new combination method based on path relinking, which considers a broader area around the population members than previous combination methods. We also use a population-update method which improves the balance between intensification and diversification. New strategies to intensify the search and to escape from suboptimal solutions are also presented. The application of the proposed evolutionary algorithm to different sets of both state-of-the-art continuous global optimization and complex-process optimization problems reveals that it is robust and efficient for the type of problems intended to solve, outperforming the results obtained with other methods found in the literature.

A framework for parallel large-scale global optimization

The paper describes the design and implementation of BNB-Solver, an object-oriented framework for discrete and continuous parallel global optimization. The framework supports exact branch-and-bound algorithms, heuristic methods and hybrid approaches. BNB-Solver provides a support for distributed and shared memory architectures. The implementation for distributed memory machines is based on MPI and thus can run on almost any computational cluster. In order to take advantages of multicore processors we provide a separate multi-threaded implementation for shared memory platforms. We introduce a novel collaborative scheme for combining exact and heuristic search methods that provides the support for sophisticated parallel heuristics and convenient balancing between exact and heuristic methods. In the experimental results section we discuss a nonlinear programming solver and a highly efficient knapsack solver that significantly outperforms existing parallel implementations.

Continuous Global Optimization in Multiview 3D Reconstruction

In this article, we introduce a new global optimization method to the field of multiview 3D reconstruction. While global minimization has been proposed in a discrete formulation in form of the maxflow-mincut framework, we suggest the use of a continuous convex relaxation scheme. Specifically, we propose to cast the problem of 3D shape reconstruction as one of minimizing a spatially continuous convex functional. In qualitative and quantitative evaluation we demonstrate several advantages of the proposed continuous formulation over the discrete graph cut solution. Firstly, geometric properties such as weighted boundary length and surface area are represented in a numerically consistent manner: The continuous convex relaxation assures that the algorithm does not suffer from metrication errors in the sense that the reconstruction converges to the continuous solution as the spatial resolution is increased. Moreover, memory requirements are reduced, allowing for globally optimal reconstructions at higher resolutions. We study three different energy models for multiview reconstruction, which are based on a common variational template unifying regional volumetric terms and on-surface photoconsistency. The three models use data measurements at increasing levels of sophistication. While the first two approaches are based on a classical silhouette-based volume subdivision, the third one relies on stereo information to define regional costs. Furthermore, this scheme is exploited to compute a precise photoconsistency measure as opposed to the classical estimation. All three models are compared on standard data sets demonstrating their advantages and shortcomings. For the third one, which gives the most accurate results, a more exhaustive qualitative and quantitative evaluation is presented.

A modified integral global optimization method and its asymptotic convergence

In this paper, we propose a new integral global optimization algorithm for finding the solution of continuous minimization problem, and prove the asymptotic convergence of this algorithm. In our modified method we use variable measure integral, importance sampling and main idea of the cross-entropy method to ensure its convergence and efficiency. Numerical results show that the new method is very efficient in some challenging continuous global optimization problems.

Global minimum cost design of a welded square stiffened plate supported at four corners

This paper presents the application of a refined version of the original Snyman–Fatti (SF) global continuous optimization algorithm (Snyman and Fatti, J Optimiz Theory Appl 54:121–141, 1987) to the optimal design of welded square stiffened plates. In particular we investigate square plates of square symmetry subjected to uniformly distributed normal static loads, supported at four corners, and stiffened by a square symmetrical orthogonal grid of ribs. Halved rolled I-section stiffeners are used welded to the base plate by double fillet welds. Profiles of different size are used for internal and edge stiffeners. A cost calculation method, developed by the first two authors and mainly used for welded structures (Farkas and Jarmai 2003), allows for the computation of cost for different proposed designs of the welded stiffened plates. The cost function includes material, welding as well as painting costs, and is formulated according to the fabrication sequence. Design variables include base plate thickness as well as the dimensions of the edge and internal stiffeners. Constraints on stress in the base plate and in stiffeners, as well as on deflection of edge stiffeners and of internal stiffeners are considered. For this purpose the Snyman–Fatti (SF) global unconstrained trajectory method is adapted to handle constraints of this type. For control purposes a particle swarm optimization (PSO) algorithm is also applied to confirm the results given by the SF algorithm. Since the torsional stiffness of open section stiffeners is very small, the stiffened plates are modelled as a torsionless gridwork. We present an algorithm for calculating the moments and deflections for torsionless gridworks with different number of internal stiffeners, using the force method.

Dynamic Globally Concavized Filled Function Method for Continuous Global Optimization

We consider a box-constrained continuous global minimization problem. A new definition of filled function, namely that of globally concavized filled function, is proposed. A new two-parameter class of globally concavized filled functions A(x,k,p) is constructed, which has the same global minimizers as the problem on the solution domain if p is large enough. The minimization of A(x,k,p) can escape successfully from a previously converged local minimizer by taking increasing values of k. A dynamic globally concavized filled function method is designed based on these functions and the convergence property is proved. Numerical experiments on a set of standard testing functions show that the resulting method is competitive with some well-known global minimization methods.

Solving systems of nonlinear equations with continuous GRASP

A method for finding all roots of a system of nonlinear equations is described. Our method makes use of C-GRASP, a recently proposed continuous global optimization heuristic. Given a nonlinear system, we solve a corresponding adaptively modified global optimization problem multiple times, each time using C-GRASP, with areas of repulsion around roots that have already been found. The heuristic makes no use of derivative information. We illustrate the approach using systems found in the literature.