Stochastic Global Optimization Research Articles

Integrated Differential Evolution for Global Optimization and Its Performance for Modeling Vapor–Liquid Equilibrium Data

Parameter estimation in thermodynamic models has been of great interest in chemical engineering because of its complex nature, including nonlinearity, flat objective function in the neighborhood of the global optimum, badly scaled model, and nondifferential term(s) in the equations. In this article, a stochastic global optimization algorithm called integrated differential evolution (IDE) is introduced to solve the parameter estimation problems for modeling vapor–liquid equilibrium (VLE) data. In IDE, the choice of mutation strategy and associated parameters are adapted according to the learning experience from previous generations. The tabu list used in IDE can avoid revisiting the same area and avoid unnecessary function evaluations. A novel and effective stopping criterion based on the number of rejected points during the generation of a trial vector is tested and compared with other criteria. Furthermore, IDE uses a local optimizer after the global search to find the optimum accurately and efficiently. The performance of IDE for benchmark functions and VLE modeling is compared with that of other stochastic algorithms such as DE, DE with tabu list, particle swarm optimization, simulated annealing, and the deterministic algorithm Branch and Reduce Optimization Navigator (BARON). IDE is shown to be better than or comparable to these algorithms for parameter estimation in modeling VLE data.

Evaluation of integrated differential evolution and unified bare-bones particle swarm optimization for phase equilibrium and stability problems

Phase equilibrium calculations and phase stability analysis play a significant role in the simulation, design and optimization of separation processes in chemical engineering. These are very challenging problems due to the high non-linearity of thermodynamic models. Global optimization methods are required in order to solve these complex, non-convex optimization problems. Recently, stochastic global optimization algorithms were applied to solve these problems. However, these optimization methods have some parameters that need to be tuned in order to obtain good reliability and efficiency. In this study, we introduce three global optimization algorithms developed by our group for phase and chemical equilibrium calculations, namely, unified bare-bones particle swarm optimization (UBBPSO), integrated differential evolution (IDE) and IDE without tabu list and radius (IDE_N), which have fewer control parameters to be tuned. The performance of these three stochastic algorithms is tested and compared in order to identify their relative strengths for phase equilibrium and phase stability problems. The phase equilibrium problems include both without and with chemical reactions. Our results show that the effectiveness of the stochastic methods tested depends on the stopping criterion. Overall, IDE has achieved better performance for the phase equilibrium, chemical equilibrium and phase stability problems.

Solution of Jiles–Atherton vector hysteresis parameters estimation by modified Differential Evolution approaches

Differential Evolution (DE) is a simple and efficient stochastic global optimization algorithm of evolutionary computation field, which involves the evolution of a population of solutions using operators such as mutation, crossover, and selection. The basic idea of DE is to adapt the search during the evolutionary process. At the start of the evolution, the perturbations are large since parent populations are far away from each other. As the evolutionary process matures, the population converges to a small region and the perturbations adaptively become small. DE approaches have been successfully applied to solve a wide range of optimization problems. In this paper, the parameters set of the Jiles–Atherton vector hysteresis model is obtained with an approach based on modified Differential Evolution (MDE) approaches using generation-varying control parameters based on generation of random numbers with uniform distribution. Several evaluated MDE approaches perform better than the classical DE methods and a genetic algorithm approach in terms of the quality and stability of the final solutions in optimization of vector Jiles–Atherton vector hysteresis model from a workbench containing a rotational single sheet tester.

Short-term scheduling of cascade reservoirs using an immune algorithm-based particle swarm optimization

This paper presents a new approach for short-term hydropower scheduling of reservoirs using an immune algorithm-based particle swarm optimization (IA-PSO). IA-PSO is employed by coupling the immune information processing mechanism with the particle swarm optimization algorithm in order to achieve a better global solution with less computational effort. With the IA-PSO technique, the hydro-electrical optimization model of reservoirs is formulated as a high-dimensional, dynamic, nonlinear and stochastic global optimization problem of a multi-reservoir hydropower system. The purpose of the proposed methodology is to maximize total hydropower production. Here it is applied to a reservoir system on the Qingjiang River, in the Yangtze watershed, that consists of two reservoirs. The results are compared with the results obtained through conventional operation method, the dynamic programming and the standard PSO algorithm. From the comparative results, it is found that the IA-PSO approach provides the most globally optimum solution at a faster convergence speed.

GlobalMIT: learning globally optimal dynamic bayesian network with the mutual information test criterion

Dynamic Bayesian networks (DBN) are widely applied in modeling various biological networks including the gene regulatory network (GRN). Due to the NP-hard nature of learning static Bayesian network structure, most methods for learning DBN also employ either local search such as hill climbing, or a meta stochastic global optimization framework such as genetic algorithm or simulated annealing. This article presents GlobalMIT, a toolbox for learning the globally optimal DBN structure from gene expression data. We propose using a recently introduced information theoretic-based scoring metric named mutual information test (MIT). With MIT, the task of learning the globally optimal DBN is efficiently achieved in polynomial time. The toolbox, implemented in Matlab and C++, is available at http://code.google.com/p/globalmit. vinh.nguyen@monash.edu; madhu.chetty@monash.edu Supplementary data is available at Bioinformatics online.

Crystallographic structure and morphology of bithiophene-fluorene polymer nanocrystals

Nanocrystals of the polymer poly(9,9-dioctylfluorenyl-co-bithiophene) (F8T2) with a molecular weight of 3.2 kg/mol are grown in a para-xylene solution. The typical morphology of the crystals is needle like with typical widths of 50 nm and lengths of about 200 nm. The crystal structure and morphology are stable up to a temperature of 353 K. The structure solution is obtained by x-ray powder diffraction (XRD) pattern with data modelling by a stochastic global optimization procedure which allows simultaneous indexing and molecular packing determination. Final Rietveld refinement was applied on the most promising crystal structure with a = 1.376 nm, b = 3.105 nm, c = 2.690 nm and ß = 109.5° within the space group C2/c choosing the polymer backbone parallel to the b-axis. The structural motifs of the molecular packing could be identified: aromatic units within a single polymer chain are slightly bent relative to the chain axis, octyl side chains are aligned along the polymer backbone and aromatic units of neighbouring molecules display a strong tendency to stack parallel to each other. XRD results of F8T2 with a molecular weight of 19 kg/mol reveal the same peak positions compared to the 3.2 kg/mol material, showing that both materials crystallise similarly and can be described by the same crystallographic unit cell. The smaller peak intensities together with the broader peak widths, however, show that the ability of crystal formation for the 19 kg/mol material is reduced.

Preface: Special issue SAGO08

The international workshop, Stochastic and Applied Global Optimization, was held during 19–22 July 2008. What made this workshop different to those previously held in global optimization, was the opportunities offered to the participants to view the game and natural beauty of South Africa alongside discussing research. The workshop was organized in the Skukuza Camp of the South African national game reserve, the Kruger National Park. The Skukuza Camp is located in the southern part of the Kruger National Park and is well known for its lion sightings. The presentations were scheduled in single stream sessions in such a way that there was ample time for the participants to view game in the mornings and late evenings. The participants were treated to the rare sight of each of the ‘big five’ (Lion, Rhinoceros, Buffalo, Elephant and Leopard). Round trips from Johannesburg to the Skukuza Camp were organized for the all participants to see the spectacular scenic beauties of South Africa. A total of 46 scholars from twelve countries attended the workshop. The participants were invited to submit their contribution to a special issue. Papers dealing with all aspects of global optimization, ranging from theories to applications, were submitted. The usual referee process, where three experts per paper were invited to referee, led to the final acceptance of six papers. Papers in both stochastic and heuristic methods for global optimization, in combinatorial optimization, in scheduling, and application areas such as location analysis are representative of the special issue. We are grateful to Professor Panos Pardalos, the Editor-in-chief of the Journal of Global Optimization, for granting this special issue and for his encouragement and guidance during the preparations and publication of the issue.

Global optimization using dimensional jumping and fuzzy adaptive simulated annealing

This paper proposes a new multi-start, stochastic global optimization algorithm that uses dimensional reduction techniques based upon approximations of space-filling curves, simulated annealing and particle swarm optimization, aiming at finding global minima of real-valued functions that are not necessarily well behaved, that is, are not required to be differentiable, continuous, or even satisfying Lipschitz conditions. The overall idea is as follows: given a real-valued function F with a multidimensional and compact domain D , the method builds an equivalent one-dimensional problem by composing F with a linearization of D , searches for a small population of candidates and returns to the original high-dimensional domain, this time with a small set of promising starting points.

A hybrid global optimization algorithm and its application to parameter estimation problems

AbstractGlobal optimization has been an area of active research for the past few decades. There are many different global optimization algorithms that have been proposed and studied. Among these, differential evolution (DE) is a promising population‐based stochastic global optimization algorithm. It has been gaining popularity in the recent past due to its capability to handle non‐convex, non‐differentiable and multimodal objective functions. In this work, a hybrid global optimization (HGO) method with a novel stopping criterion is proposed. The proposed algorithm integrates DE with a taboo list of taboo search and self‐adaptive strategies. The taboo list/check prevents revisiting the same area, thus increasing the population diversity and computational efficiency. The self‐adaptive strategy reduces the algorithm parameters to be provided and makes the algorithm more robust. Furthermore, a novel stopping criterion based on the number of rejected points is developed. The effectiveness of the proposed stopping criterion and HGO is assessed on more than 30 benchmark problems with 2–30 variables. The performance of HGO is compared with some state‐of‐the‐art global optimization algorithms recently published in the literature. The results show that the reliability of finding the global optimum by HGO is slightly better than that by the latest DE techniques in the literature, and the number of function evaluations (NFE) of HGO is significantly reduced. Finally, HGO is tested for parameter estimation in dynamic models. In conclusion, the proposed HGO algorithm with the novel stopping criterion is found to be reliable, efficient and suitable for application problems. Copyright © 2011 Curtin University of Technology and John Wiley & Sons, Ltd.

Model to study the effect of workforce on a safety equipment and its optimization

Many industrial sectors are concerned on developing optimal maintenance planning because of the importance of maintenance on the economy and safety. Traditionally, the maintenance planning is formulated in terms of a multi-objective optimization problem where reliability, availability, maintainability and cost act as decision criteria and surveillance tests and maintenance strategies act as decision variables. However, the appropriate development of each maintenance strategy depends not only on the maintenance intervals but also on the resources available to implement such strategies. To solve the multi-objective optimization problem Particle Swarm Optimization (PSO) can be used. PSO is a stochastic global optimization technique inspired by social behavior of bird flocking or fish schooling. In this paper the multi-objective optimization of the maintenance of a nuclear power plant safety equipment using PSO is presented.

Application of PSO Algorithm in Parameter Optimization of Biodegradable Medical Polymer Degradation Model

The particle swarm optimization (PSO) algorithm is a stochastic global optimization technique based on swarm intelligence. It possesses advantages such as being a simple principle, few parameters and easy to be realized. In this paper, an optimization model is established to solve the difficulty in selecting parameters and improve the simulation accuracy of the biodegradable medical polymer degradation model. When modeling, the particle swarm optimization (PSO) algorithm is proposed to solve the model and calculate undetermined parameters of the biodegradable medical polymer degradation equations. A comparative analysis of the calculation results is progressed. It shows that parameters determined by optimization model make the simulation results of degradation model more close to the experiment results. Using this method to solve the model is more accurate and efficiency than determining parameters artificially. It also shows that the particle swarm optimization algorithm used to optimize parameters have practical significance and application value.

SOLVING CONSTRAINED GLOBAL OPTIMIZATION VIA ARTIFICIAL IMMUNE SYSTEM

Artificial immune systems (AISs) are computational intelligence (CI) oriented methods using information based on biological immune systems. In this study, an AIS, which combines the metaphor of clonal selection with idiotypic network theories, is developed. Although they are contradictory approaches, clonal selection and idiotypic network may prove useful in designing a stochastic global optimization tool. The AIS method consists of idiotypic network selection, somatic hypermuation, receptor editing and bone marrow operators. The idiotypic network selection operator determines the number of good solutions. The somatic hypermutation and receptor editing operators comprise the searching mechanisms for the exploration of the solution space. Diversity on the population of solutions is ensured by the bone marrow operator. The performance of the proposed AIS method is tested on a set of global constrained optimization problems (GCO), comprising of four benchmark nonlinear programming problems and four generalized polynomial programming (GPP) problems, where GPP problems are nonconvex optimization problems. The best solution found by the AIS algorithm is compared with the known global optimum. Numerical results show that the proposed method converged to the global optimal solution to each tested CGO problem. Moreover, this study compares the numerical results obtained by the AIS approach with those taken from published CI approaches, such as alternative AIS methods and genetic algorithms.

Competitive trapping in complex state spaces

In complex state space dynamics at finite time scales, the trapping in certain regions of state space is of great importance, e.g. in the field of protein folding or in the application of stochastic global optimization algorithms. Here, we analyze the influence of the density of states on the features of the trapping process. In particular, we compare the trapping power of a valley with a power-law density of states to one with an exponentially growing density of states. The outcome of this competition crucially depends on the annealing speed and shows that the clear difference between these two paradigmatic densities of states observed at very slow (near-equilibrium) annealing is lost for fast non-equilibrium processes, and that the outcome of the relaxation can strongly depend on the time scale of the process and subtle features of the density of states.

SIMOP: Application to global MINLP stochastic optimization

SIMOP (SIMulation for OPtimization) is an application that automatically creates FORTRAN simulation models for the stochastic optimizers MSGA and MSIMPSA. This paper presents the next step in SIMOP's development timeline, namely its expansion to cope with mixer-integer nonlinear problems (MINLP). From the perspective of MINLP formulations' relevance to the Chemical Engineering field, an introduction to SIMOP's MINLP modeling features is given. Its application is addressed by three case studies. The best results match those previously obtained with MSGA and MSIMPSA when non-automatically coded MINLP simulations where employed, thus validating the present approach. The introduction of discrete variables benefits the performance of both stochastic optimizers, contrary to what should a priori be expected, since a MINLP formulation promotes an accurate search on lower and upper bounds for several continuous variables.

A comparative study of common and self-adaptive differential evolution strategies on numerical benchmark problems

Abstract Differential Evolution (DE) is a population-based stochastic global optimization technique that requires the adjustment of a very few parameters in order to produce results. However, the control parameters involved in DE are highly dependent on the optimization problem; in practice, their fine-tuning is not always an easy task. The self-adaptive differential evolution (SADE) variants are those that do not require the pre-specified choice of control parameters. On the contrary, control parameters are selfadapted by using the previous learning experience. In this paper, we discuss and evaluate popular common and self-adaptive differential evolution (DE) algorithms. In particular, we present an empirical comparison between two self-adaptive DE variants and common DE methods. In order to assure a fair comparison, we test the methods by using a number of well-known unimodal and multimodal, separable and non-separable, benchmark optimization problems for different dimensions and population size. The results show that SADE variants outperform, or at least produce similar results, to common differential evolution algorithms in terms of solution accuracy and convergence speed. The advantage of using the self-adaptive methods is that the user does not need to adjust control parameters. Therefore, the total computational effort is significantly reduced.

Evolutionary Computation and Its Applications in Neural and Fuzzy Systems

Neural networks and fuzzy systems are two soft‐computing paradigms for system modelling. Adapting a neural or fuzzy system requires to solve two optimization problems: structural optimization and parametric optimization. Structural optimization is a discrete optimization problem which is very hard to solve using conventional optimization techniques. Parametric optimization can be solved using conventional optimization techniques, but the solution may be easily trapped at a bad local optimum. Evolutionary computation is a general‐purpose stochastic global optimization approach under the universally accepted neo‐Darwinian paradigm, which is a combination of the classical Darwinian evolutionary theory, the selectionism of Weismann, and the genetics of Mendel. Evolutionary algorithms are a major approach to adaptation and optimization. In this paper, we first introduce evolutionary algorithms with emphasis on genetic algorithms and evolutionary strategies. Other evolutionary algorithms such as genetic programming, evolutionary programming, particle swarm optimization, immune algorithm, and ant colony optimization are also described. Some topics pertaining to evolutionary algorithms are also discussed, and a comparison between evolutionary algorithms and simulated annealing is made. Finally, the application of EAs to the learning of neural networks as well as to the structural and parametric adaptations of fuzzy systems is also detailed.

Tuning Genetic Algorithm Parameters to Improve Convergence Time

Fermentation processes by nature are complex, time‐varying, and highly nonlinear. As dynamic systems their modeling and further high‐quality control are a serious challenge. The conventional optimization methods cannot overcome the fermentation processes peculiarities and do not lead to a satisfying solution. As an alternative, genetic algorithms as a stochastic global optimization method can be applied. For the purpose of parameter identification of a fed‐batch cultivation of S. cerevisiae altogether four kinds of simple and four kinds of multipopulation genetic algorithms have been considered. Each of them is characterized with a different sequence of implementation of main genetic operators, namely, selection, crossover, and mutation. The influence of the most important genetic algorithm parameters—generation gap, crossover, and mutation rates has—been investigated too. Among the considered genetic algorithm parameters, generation gap influences most significantly the algorithm convergence time, saving up to 40% of time without affecting the model accuracy.

A hybrid metaheuristic approach to the university course timetabling problem

This paper describes the development of a novel metaheuristic that combines an electromagnetic-like mechanism (EM) and the great deluge algorithm (GD) for the University course timetabling problem. This well-known timetabling problem assigns lectures to specific numbers of timeslots and rooms maximizing the overall quality of the timetable while taking various constraints into account. EM is a population-based stochastic global optimization algorithm that is based on the theory of physics, simulating attraction and repulsion of sample points in moving toward optimality. GD is a local search procedure that allows worse solutions to be accepted based on some given upper boundary or `level'. In this paper, the dynamic force calculated from the attraction-repulsion mechanism is used as a decreasing rate to update the `level' within the search process. The proposed method has been applied to a range of benchmark university course timetabling test problems from the literature. Moreover, the viability of the method has been tested by comparing its results with other reported results from the literature, demonstrating that the method is able to produce improved solutions to those currently published. We believe this is due to the combination of both approaches and the ability of the resultant algorithm to converge all solutions at every search process.

A hybrid shuffled complex evolution approach based on differential evolution for unconstrained optimization

Numerous optimization methods have been proposed for the solution of the unconstrained optimization problems, such as mathematical programming methods, stochastic global optimization approaches, and metaheuristics. In this paper, a metaheuristic algorithm called Modified Shuffled Complex Evolution (MSCE) is proposed, where an adaptation of the Downhill Simplex search strategy combined with the differential evolution method is proposed. The efficiency of the new method is analyzed in terms of the mean performance and computational time, in comparison with the genetic algorithm using floating-point representation (GAF) and the classical shuffled complex evolution (SCE-UA) algorithm using six benchmark optimization functions. Simulation results and the comparisons with SCE-UA and GAF indicate that the MSCE improves the search performance on the five benchmark functions of six tested functions.

Convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization

This paper presents some simple technical conditions that guarantee the convergence of a general class of adaptive stochastic global optimization algorithms. By imposing some conditions on the probability distributions that generate the iterates, these stochastic algorithms can be shown to converge to the global optimum in a probabilistic sense. These results also apply to global optimization algorithms that combine local and global stochastic search strategies and also those algorithms that combine deterministic and stochastic search strategies. This makes the results applicable to a wide range of global optimization algorithms that are useful in practice. Moreover, this paper provides convergence conditions involving the conditional densities of the random vector iterates that are easy to verify in practice. It also provides some convergence conditions in the special case when the iterates are generated by elliptical distributions such as the multivariate Normal and Cauchy distributions. These results are then used to prove the convergence of some practical stochastic global optimization algorithms, including an evolutionary programming algorithm. In addition, this paper introduces the notion of a stochastic algorithm being probabilistically dense in the domain of the function and shows that, under simple assumptions, this is equivalent to seeing any point in the domain with probability 1. This, in turn, is equivalent to almost sure convergence to the global minimum. Finally, some simple results on convergence rates are also proved.