Stochastic Global Optimization Algorithm Research Articles

A comparison of global optimization algorithms applied to a ride comfort optimization problem

This study presents a comparison of global optimization algorithms applied to an industrial engineering optimization problem. Three global stochastic optimization algorithms using continuous variables, i.e. the domain elimination method, the zooming method and controlled random search, have been applied to a previously studied ride comfort optimization problem. Each algorithm is executed three times and the total number of objective function evaluations needed to locate a global optimum is averaged and used as a measure of efficiency. The results show that the zooming method, with a proposed modification, is most efficient in terms of number of objective function evaluations and ability to locate the global optimum. Each design variable is thereafter given a set of discrete values and two optimization algorithms using discrete variables, i.e. a genetic algorithm and simulated annealing, are applied to the discrete ride comfort optimization problem. The results show that the genetic algorithm is more efficient than the simulated annealing algorithm for this particular optimization problem.

Global optimization methods for chemical process design: Deterministic and stochastic approaches

Process optimization often leads to nonconvex nonlinear programming problems, which may have multiple local optima. There are two major approaches to the identification of the global optimum: deterministic approach and stochastic approach. Algorithms based on the deterministic approach guarantee the global optimality of the obtained solution, but are usually applicable to small problems only. Algorithms based on the stochastic approach, which do not guarantee the global optimality, are applicable to large problems, but inefficient when nonlinear equality constraints are involved. This paper reviews representative deterministic and stochastic global optimization algorithms in order to evaluate their applicability to process design problems, which are generally large, and have many nonlinear equality constraints. Finally, modified stochastic methods are investigated, which use a deterministic local algorithm and a stochastic global algorithm together to be suitable for such problems.

Use of intelligent-particle swarm optimization in electromagnetics

The paper describes a new stochastic heuristic algorithm for global optimization. The new optimization algorithm, called intelligent-particle swarm optimization (IPSO), offers more intelligence to particles by using concepts such as: group experiences, unpleasant memories (tabu to be avoided), local landscape models based on virtual neighbors, and memetic replication of successful behavior parameters. The new individual complexity is amplified at the group level and consequently generates a more efficient optimization procedure. A simplified version of the IPSO algorithm was implemented and compared with the classical PSO algorithm for a simple test function and for the Loney's solenoid.

Nonconvex Stochastic Optimization for Model Reduction

In this paper a global stochastic optimization algorithm, which is almost surely (a.s.) convergent, is applied to the model reduction problem. The proposed method is compared with the balanced truncation and Hankel norm approximation methods by examples in step responses and in approximation errors as well. Simulation shows that the proposed algorithm provides better results.

A direct search variant of the simulated annealing algorithm for optimization involving continuous variables

A memory-based simulated annealing algorithm is proposed which fundamentally differs from the previously developed simulated annealing algorithms for continuous variables by the fact that a set of points rather than a single working point is used. The implementation of the new method does not need differentiability properties of the function being optimized. The method is well tested on a range of problems classified as easy, moderately difficult and difficult. The new algorithm is compared with other simulated annealing methods on both test problems and practical problems. Results showing an improved performance in finding the global minimum are given. Scope and purpose The inherent difficulty of global optimization problems lies in finding the very best optimum (maximum or minimum) from a multitude of local optima. Many practical global optimization problems of continuous variables are non-differentiable and noisy and even the function evaluation may involve simulation of some process. For such optimization problems direct search approaches are the methods of choice. Simulated annealing is a stochastic global optimization algorithm, initially designed for combinatorial (discrete) optimization problems. The algorithm that we propose here is a simulated annealing algorithm for optimization problems involving continuous variables. It is a direct search method. The strengths of the new algorithm are: it does not require differentiability or any other properties of the function being optimized and it is memory-based. Therefore, the algorithm can be applied to noisy and/or not exactly known functions. Although the algorithm is stochastic in nature, it can memorise the best solution. The new simulated annealing algorithm has been shown to be reliable, fast, general purpose and efficient for solving some difficult global optimization problems.

Automatic calibration of a distributed catchment model

Parameters of hydrologic models often are not exactly known and therefore have to be determined by calibration. A manual calibration depends on the subjective assessment of the modeler and can be very time-consuming though. Methods of automatic calibration can improve these shortcomings. Yet, the high number of parameters in distributed models makes special demands on the optimization. In this paper a strategy of imposing constraints on the parameters to limit the number of independently calibrated values is outlined. Subsequently, an automatic calibration of the version SWAT-G of the model SWAT (Soil and Water Assessment Tool) with a stochastic global optimization algorithm, the Shuffled Complex Evolution algorithm, is presented for a mesoscale catchment.

SPT: a stochastic tunneling algorithm for global optimization

A stochastic approach to solving unconstrained continuous-function global optimization problems is presented. It builds on the tunneling approach to deterministic optimization presented by Barhen and co-workers (Bahren and Protopopescu, in: State of the Art in Global Optimization, Kluwer, 1996; Barhen et al., Floudas and Pardalos (eds.), TRUST: a deterministic algorithm for global optimization, 1997) by combining a series of local descents with stochastic searches. The method uses a rejection-based stochastic procedure to locate new local minima descent regions and a fixed Lipschitz-like constant to reject unpromising regions in the search space, thereby increasing the efficiency of the tunneling process. The algorithm is easily implemented in low-dimensional problems and scales easily to large problems. It is less effective without further heuristics in these latter cases, however. Several improvements to the basic algorithm which make use of approximate estimates of the algorithms parameters for implementation in high-dimensional problems are also discussed. Benchmark results are presented, which show that the algorithm is competitive with the best previously reported global optimization techniques. A successful application of the approach to a large-scale seismology problem of substantial computational complexity using a low-dimensional approximation scheme is also reported.

Improvement of pure random search in global optimization

In this paper, the improvement of pure random search is studied. By taking some information of the function to be minimized into consideration, the authors propose two stochastic global optimization algorithms. Some numerical experiments for the new stochastic global optimization algorithms are presented for a class of test problems.

Convergence of a global stochastic optimization algorithm with partial step size restarting

This work develops a class of stochastic global optimization algorithms that are Kiefer-Wolfowitz (KW) type procedures with an added perturbing noise and partial step size restarting. The motivation stems from the use of KW-type procedures and Monte Carlo versions of simulated annealing algorithms in a wide range of applications. Using weak convergence approaches, our effort is directed to proving the convergence of the underlying algorithms under general noise processes.

Convergence of a global stochastic optimization algorithm with partial step size restarting

This work develops a class of stochastic global optimization algorithms that are Kiefer-Wolfowitz (KW) type procedures with an added perturbing noise and partial step size restarting. The motivation stems from the use of KW-type procedures and Monte Carlo versions of simulated annealing algorithms in a wide range of applications. Using weak convergence approaches, our effort is directed to proving the convergence of the underlying algorithms under general noise processes.

A Stochastic Algorithm for Constrained Global Optimization

We present a stochastic algorithm to solve numerically the problem of finding the global minimizers of a real valued function subject to lower and upper bounds. This algorithm looks for the global minimizers following the paths of a suitable system of stochastic differential equations. Numerical experience on several test problems known in literature is shown.

Global optimization by Darwin and Boltzmann mixed strategy

This paper presents a Darwin and Boltzmann mixed strategy to solve the global optimization problems. The algorithm is based on the integration of the Darwin strategy and the Boltzmann annealing strategy, it is a hybrid of the Stochastic Evolution (SE) and the Simulated Annealing (SA). The proposed algorithm is proved to converge asymptotically to the global optimal solutions and its approximation implementation has shown to be polynomial in complexity. Experimental results show that the proposed algorithm is more efficient than the SA algorithm and is comparable to other methods on six well-known test problems. Scope and purpose Global optimization concerns with the computation and characterization of global optimums of nonlinear functions. They are widely used in the areas of engineering, operations research, statistics, computer science, and molecular biology. Extensive efforts have been made to address the issue and many global optimization methods have been developed. Global optimization methods are different from local optimization methods in the sense that they are to find the overall optimal solution or global optimum over a bounded set. Depending on whether or not they incorporate any stochastic elements, global optimization methods are classified into deterministic and stochastic ones. Current research in stochastic methods for global optimization focuses on new methods to improve efficiency and to establish trade-offs between accuracy, reliability and computational burden. This paper aims at proposing a new stochastic algorithm for global optimization. The proposed algorithm is analysed and compared with several existing methods on six well-known test problems and the experimental results are given.

A new theoretical framework for analyzing stochastic global optimization algorithms

In this paper, we develop a new theoretical framework by means of the absorbin Markov process theory for analyzing some stochastic global optimization algorithms. Applying the framework to the pure random search, we prove that the pure random search converges to the global minimum in probability and its time has geometry distribution. We also analyze the pure adaptive search by this framework and turn out that the pure adaptive search converges to the global minimum in probability and its time has Poisson distribution.

Modelling by use of global stochastic optimization algorithms *

Abstract In this paper a global stochastic optimization algorithm, which is a.s. convergent, is applied to system modelling. For model reduction it is shown that the algorithm provides satisfactory results and better approximations in comparison with the Pade approximation. When modelling by neural networks the algorithm used for weight selection leads the neural network to have a satisfactory tracking property

Ab InitioFolding of Peptides by the Optimal-Bias Monte Carlo Minimization Procedure

Prediction of three-dimensional structures of proteins and peptides by global optimization of the free energy estimate has been attempted without much success for over thirty years. The key problems were the insufficient accuracy of the free energy estimate and the giant size of the conformational space. Global optimization of the free energy function of a peptide in internal coordinate space is a powerful method of structure prediction that outperforms both Molecular Dynamics, bound by the continuity requirement, and Monte Carlo, bound by the Boltzmann ensemble generation requirement. We demonstrate that stochastic global optimization algorithms of the first order, i.e., with local minimization after each iteration (e.g., Monte Carlo-Minimization), have a greater chance of finding the global minimum after a fixed number of function evaluations. Recently, the principle of optimal bias was mathematically justified and the Optimal-Bias Monte Carlo-Minimization algorithm (a.k.a. Biased Probability Monte Carlo-minimization) was successfully applied to theoreticalab initiofolding of several peptides, resulting in more than a 10-fold increase in efficiency compared to the Monte Carlo-Minimization method. The square-root bias is shown to be comparable in performance with the previously derived linear bias. A 23-residue peptide of beta-beta-alpha structure can be predicted from any random starting conformation.

Rates of Convergence for a Class of Global Stochastic Optimization Algorithms

Inspired and motivated by the recent advances in simulated annealing algorithms, this paper analyzes the convergence rates of a class of recursive algorithms for global optimization via Monte Carlo methods. By using perturbed Liapunov function methods, stability results of the algorithms are established. Then the rates of convergence are ascertained by examining the asymptotic properties of suitably scaled estimation error sequences.

Application of Stochastic Global Optimization Algorithms to Practical Problems

We describe global optimization problems from three different fields representing many-body potentials in physical chemistry, optimal control of a chemical reactor, and fitting a statistical model to empirical data. Historical background for each of the problems as well as the practical significance of the first two are given. The problems are solved by using eight recently developed stochastic global optimization algorithms representing controlled random search (4 algorithms), simulated annealing (2 algorithms), and clustering (2 algorithms). The results are discussed, and the importance of global optimization in each respective field is focused.

An adaptive stochastic global optimization algorithm for one-dimensional functions

In this paper a new algorithm is proposed, based upon the idea of modeling the objective function of a global optimization problem as a sample path from a Wiener process. Unlike previous work in this field, in the proposed model the parameter of the Wiener process is considered as a random variable whose conditional (posterior) distribution function is updated on-line. Stopping criteria for Bayesian algorithms are discussed and detailed proofs on finite-time stopping are provided.

Algorithm 744: a stochastic algorithm for global optimization with constraints

A stochastic algorithm is presented for finding the global optimum of a function of n variables subject to general constraints. The algorithm is intended for moderate values of n , but it can accommodate objective and constraint functions that are discontinuous and can take advantage of parallel processors. The performance of this algorithm is compared to that of the Nelder-Mead Simplex algorithm and a Simulated Annealing algorithm on a variety of nonlinear functions. In addition, one-, two-, four-, and eight-processor versions of the algorithm are compared using 64 of the nonlinear problems with constraints collected by Hock and Schittkowski. In general, the algorithm is more robust than the Simplex algorithm, but computationally more expensive. The algorithm appears to be as robust as the Simulated Annealing algorithm, but computationally cheaper. Issues discussed include algorithm speed and robustness, applicability to both computer and mathematical models, and parallel efficiency.

On a new stochastic global optimization algorithm based on censored observations

In this paper a new algorithm is proposed for global optimization problems. The main idea is that of modifying a standard clustering approach by sequentially sampling the objective function while adaptively deciding an appropriate sample size. Theoretical as well as computational results are presented.