Stochastic Global Optimization Method Research Articles

Trajectory Optimization With Particle Swarm Optimization for Manipulator Motion Planning

Optimization-based methods have been recently proposed to solve motion planning problems with complex constraints. Previous methods have used optimization methods that may converge to a local minimum. In this study, particle swarm optimization (PSO) is proposed for trajectory optimization. PSO is a population-based stochastic global optimization method inspired by group behaviors in wildlife, and has the advantages of simplicity and fast convergence. Trajectory modifications are encoded in particles that are optimized with PSO. The normalized step cost (NSC) concept is used for the initialization of the particles in PSO. A method for reusing previously optimized parameters is also developed. The optimized parameters are stored together with corresponding NSC vectors, and when a constraint violation occurs, the parameters associated with an NSC vector that is similar to the query NSC vector are selected. The selected vector is used for initializing the particles. The reuse of the previously optimized parameters improves the convergence of the PSO in motion planning. The effectiveness of these methods is shown with simulations and an experiment using a three-dimensional problem with constraints. The proposed algorithm successfully optimized a trajectory while satisfying the constraints and is less likely to converge to a local minimum.

Searching for new TiO2 crystal phases with better photoactivity

Using the recently developed stochastic surface walking global optimization method, this work explores the potential energy surface of TiO2 crystals aiming to search for likely phases with higher photocatalytic activity. Five new phases of TiO2 are identified and the lowest energy phase transition pathways connecting to the most abundant phases (rutile and anatase) are determined. Theory shows that a high-pressure phase, α-PbO2-like form (TiO2II) acts as the key intermediate in between rutile and anatase. The phase transition of anatase to rutile belongs to the diffusionless Martensitic phase transition, occurring through a set of habit planes, rutile(101)//TiO2II(001), and TiO2II(100)//anatase(112). With regard to the photocatalytic activity, three pure phases (#110, pyrite and fluorite) are found to possess the band gap narrower than rutile, but they are unstable at the low-pressure condition. Instead, a mixed anatase-TiO2II phase is found to have good stability and narrower band gap than both parent phases. Because of the phase separation, the mixed phase is also expected to improve the photocatalytic performance by reducing the probability of the electron-hole pair recombination.

A Subspace Semi-Definite programming-based Underestimation (SSDU) method for stochastic global optimization in protein docking.

We propose a new stochastic global optimization method targeting protein docking problems. The method is based on finding a general convex polynomial underestimator to the binding energy function in a permissive subspace that possesses a funnel-like structure. We use Principal Component Analysis (PCA) to determine such permissive subspaces. The problem of finding the general convex polynomial underestimator is reduced into the problem of ensuring that a certain polynomial is a Sum-of-Squares (SOS), which can be done via semi-definite programming. The underestimator is then used to bias sampling of the energy function in order to recover a deep minimum. We show that the proposed method significantly improves the quality of docked conformations compared to existing methods.

A new simple, fast and efficient algorithm for global optimization over continuous search-space problems: Radial Movement Optimization

Optimization of non-linear and non-differentiable problems has been considered as an important issue for mathematicians and engineers. A new stochastic global optimization method for non-linear and non-differentiable problems is proposed and extensively explained, in this article. It is a swarm-based method which uses spherical boundaries in a vector search-space to explore for the optimal solution. Having a few numbers of parameter to be adjusted, being robust and fast, needing small memory storage size and capability of escaping from local optima, are the main features of this new algorithm. To analyze and evaluate the capability of this novel method, ten benchmark functions are chosen and the results are compared with two existing optimization algorithms which are Differential Evolution and Particle Swarm Optimization. Comparisons are made based on the consistency in obtaining optimal solutions, computation time and convergence profile. Results show the capability of the proposed method in finding a proper solution in a very short time and also escaping from local optima of the solution-space.

Modeling multiple experiments using regularized optimization: A case study on bacterial glucose utilization dynamics

The aim of inverse modeling is to capture the systems׳ dynamics through a set of parameterized Ordinary Differential Equations (ODEs). Parameters are often required to fit multiple repeated measurements or different experimental conditions. This typically leads to a multi-objective optimization problem that can be formulated as a non-convex optimization problem. Modeling of glucose utilization of Lactococcus lactis bacteria is considered using in vivo Nuclear Magnetic Resonance (NMR) measurements in perturbation experiments. We propose an ODE model based on a modified time-varying exponential decay that is flexible enough to model several different experimental conditions. The starting point is an over-parameterized non-linear model that will be further simplified through an optimization procedure with regularization penalties. For the parameter estimation, a stochastic global optimization method, particle swarm optimization (PSO) is used. A regularization is introduced to the identification, imposing that parameters should be the same across several experiments in order to identify a general model. On the remaining parameter that varies across the experiments a function is fit in order to be able to predict new experiments for any initial condition. The method is cross-validated by fitting the model to two experiments and validating the third one. Finally, the proposed model is integrated with existing models of glycolysis in order to reconstruct the remaining metabolites. The method was found useful as a general procedure to reduce the number of parameters of unidentifiable and over-parameterized models, thus supporting feature selection methods for parametric models.

A note on effective phase stability calculations using a Gradient-Based Cuckoo Search algorithm

Stochastic global optimization methods have been successfully used to perform phase stability calculations. However, these methods may show some drawbacks in challenging phase stability problems. In this study, we made use of the gradient of the tangent plane distance function to improve the performance of Cuckoo Search (CS) algorithm, which is a promising nature-inspired stochastic global optimization method, for the calculation of phase stability analysis. The new modified algorithm, Gradient-Based Cuckoo Search (GBCS), was evaluated for solving several challenging phase stability problems. Its performance at different numerical effort levels and the effect of stopping criterion have been analyzed. GBCS was found to perform better than the original CS algorithm. In comparison with other stochastic optimization methods using an improvement objective function-based stopping criterion, GBCS proved to be the most reliable without any reduction in efficiency.

Design of non-uniform concentric circular antenna arrays with optimal sidelobe level reduction using biogeography-based optimization

This paper presents the design of non-uniform concentric circular antenna arrays (CCAAs) of isotropic radiators with optimum sidelobe level (SLL) reduction. The biogeography-based optimization (BBO) method is used to determine an optimum set of excitation amplitudes that provide a radiation pattern with optimum SLL reduction with the constraint of a fixed major lobe beamwidth. The BBO method represents a new global evolutionary algorithm for optimization problems in electromagnetics. It is shown that the BBO results provide an SLL reduction that is comparable to that obtained using well-known algorithms, such as the particle swarm optimization (PSO), genetic algorithm (GA), and evolutionary programming (EP). Moreover, BBO results are compared with those obtained using the Matlab function Fmincon which uses a sequential quadratic programming (SQP) method. The comparison shows that the design of non-uniformly excited CCAAs using the SQP method provides a SLL reduction that is better than that obtained using global stochastic optimization methods, indicating that global optimization techniques might not really be needed in this problem.

Gradient-Based Cuckoo Search for Global Optimization

One of the major advantages of stochastic global optimization methods is the lack of the need of the gradient of the objective function. However, in some cases, this gradient is readily available and can be used to improve the numerical performance of stochastic optimization methods specially the quality and precision of global optimal solution. In this study, we proposed a gradient-based modification to the cuckoo search algorithm, which is a nature-inspired swarm-based stochastic global optimization method. We introduced the gradient-based cuckoo search (GBCS) and evaluated its performance vis-à-vis the original algorithm in solving twenty-four benchmark functions. The use of GBCS improved reliability and effectiveness of the algorithm in all but four of the tested benchmark problems. GBCS proved to be a strong candidate for solving difficult optimization problems, for which the gradient of the objective function is readily available.

Automatic termination of parallel optimization runs of global stochastic optimization methods by upper limit criterion

Many combinations of adjustable parameters should be tested in optimization experiments of biochemical networks to find the smallest subset of parameters enabling the best improvements of objective function both in case of design task and parameter estimation task. In case of optimization with global stochastic optimization methods one of the problems is the termination of the optimization run looking for a good compromise between spent computational resources and probability that the best found value of objective function will be the global optimum. Longer runs increase the possibility to each the global optimum. Automatic termination criteria in case of consensus or stagnation of parallel optimization runs have been proposed as criteria for automatic termination. Varying the consensus and delay time settings different probability of reaching global optimum and duration of optimization can be reached. It is proposed to modify automatic optimization termination criteria of parallel optimization runs applying upper limit agreement of a number of parallel optimization runs. Automatic application of upper limit agreement would reduce the duration of scanning of the whole space of combination of adjustable parameters. This approach is tested on the yeast glycolysis model with six adjustable parameters using COPASI, CoRunner and ConvAn software for five parallel optimization runs per combination of adjustable parameters.

Determinition of best set of adjustable parameters with full search and limited search methods

In case of optimization of biochemical networks the best combination of adjustable parameters has to be found keeping low the costs of modification of biochemical network and reducing the risk of unpredicted side effects of processes outside the scope of the model. Dynamic models of biochemical networks usually are in form of system of differential equations and therefore can not be optimized analytically. Usually they are optimized using computationally demanding global stochastic optimization methods which may have hardly predictable duration of optimization. Optimization of all the possible combinations is necessary to find the best set of adjustable parameters per number of parameters in combination. Due to the combinatorial explosion the full search often is not feasible approach if computational and time resources are limited. In this study the performance of full search is compared with forward selection and three metabolic control based methods which need significantly less combinations to be examined. It is found that forward selection is a good compromise in case of larger number of adjustable parameters where the number of possible combinations exceeds the available resources needed for full search in the adjustable parameter space. It is proposed to combine full search for small number of adjustable parameters with forward selection at larger number of adjustable parameters per combination.

Assessment of capabilities and limitations of stochastic global optimization methods for modeling mean activity coefficients of ionic liquids

In this paper, we have studied and analyzed the capabilities and limitations of seven stochastic global optimization methods to model mean activity coefficients of ammonium aqueous electrolytes using the electrolyte NRTL model. These stochastic methods are: simulated annealing (SA), genetic algorithm (GA), differential evolution (DE), particle swarm optimization (PSO), DE with tabu list (DETL), harmony search (HS) and bare bones PSO (BBPSO). Convergence characteristics and performances of these optimization methods have been tested and compared using different stopping conditions. Overall, our results indicate that SA, DETL and BBPSO offer better performance and promising convergence characteristics for solving parameter estimation problems involved in the modeling of thermodynamic properties of ionic liquids (ILs).

One-step approach for heat exchanger network retrofitting using integrated differential evolution

Heat exchanger network (HEN) retrofitting is more important and challenging than HEN synthesis since it involves modifying existing network for improved energy efficiency. Additional factors to be considered include spatial constraints, relocation and re-piping costs, reassignment and effective use of existing heat exchanger areas. The previous studies using stochastic global optimization algorithms are mainly focused on two-level approach: the first level uses a stochastic algorithm for optimizing structure, and the second level uses either a stochastic or a deterministic algorithm for optimizing continuous variables. In this study, we propose and test one-step approach where a stochastic global optimization method, namely, integrated differential evolution (IDE), handles both discrete and continuous variables together. Thus, HEN structure and retrofitting model parameters are simultaneously optimized by IDE, which avoids the algorithm trapping at a local optimum and also improves the computational efficiency. Results on HEN applications show that the proposed approach gives better solutions.

OBTAINING APPROXIMATE VALUES OF EXTERIOR ORIENTATION ELEMENTS OF MULTI-INTERSECTION IMAGES USING PARTICLE SWARM OPTIMIZATION

Abstract. In this paper, an efficient global optimization algorithm in the field of artificial intelligence, named Particle Swarm Optimization (PSO), is introduced into close range photogrammetric data processing. PSO can be applied to obtain the approximate values of exterior orientation elements under the condition that multi-intersection photography and a small portable plane control frame are used. PSO, put forward by an American social psychologist J. Kennedy and an electrical engineer R.C. Eberhart, is a stochastic global optimization method based on swarm intelligence, which was inspired by social behavior of bird flocking or fish schooling. The strategy of obtaining the approximate values of exterior orientation elements using PSO is as follows: in terms of image coordinate observed values and space coordinates of few control points, the equations of calculating the image coordinate residual errors can be given. The sum of absolute value of each image coordinate is minimized to be the objective function. The difference between image coordinate observed value and the image coordinate computed through collinear condition equation is defined as the image coordinate residual error. Firstly a gross area of exterior orientation elements is given, and then the adjustment of other parameters is made to get the particles fly in the gross area. After iterative computation for certain times, the satisfied approximate values of exterior orientation elements are obtained. By doing so, the procedures like positioning and measuring space control points in close range photogrammetry can be avoided. Obviously, this method can improve the surveying efficiency greatly and at the same time can decrease the surveying cost. And during such a process, only one small portable control frame with a couple of control points is employed, and there are no strict requirements for the space distribution of control points. In order to verify the effectiveness of this algorithm, two experiments are carried out. In the first experiment, images of a standard grid board are taken according to multi-intersection photography using digital camera. Three points or six points which are located on the left-down corner of the standard grid are regarded as control points respectively, and the exterior orientation elements of each image are computed through PSO, and compared with these elements computed through bundle adjustment. In the second experiment, the exterior orientation elements obtained from the first experiment are used as approximate values in bundle adjustment and then the space coordinates of other grid points on the board can be computed. The coordinate difference of grid points between these computed space coordinates and their known coordinates can be used to compute the accuracy. The point accuracy computed in above experiments are ±0.76mm and ±0.43mm respectively. The above experiments prove the effectiveness of PSO used in close range photogrammetry to compute approximate values of exterior orientation elements, and the algorithm can meet the requirement of higher accuracy. In short, PSO can get better results in a faster, cheaper way compared with other surveying methods in close range photogrammetry.

Hybrid particle swarm optimization transplanted into a hyper-heuristic structure for solving examination timetabling problem

Examination timetabling is a discrete, multi-objective and combinatorial optimization problem which tends to be solved with a cooperation of stochastic search approaches such as evolutionary algorithms (EAs) and heuristic methods such as sequential graph coloring heuristics. This research investigates the use of discrete particle swarm optimization (DPSO) for solving examination timetabling problem. A combination of mutation, specialist recombination operator and graph coloring heuristics are used to update position of particles in the DPSO. A new local search method, called two staged hill climbing, is proposed and is utilized to hybridize the DPSO algorithm. Three structures for the DPSO and three strategies to hybridize it are proposed. On one hand, since the proposed DPSO algorithms such as hyper-heuristics methods employ a strategy to manage a set of constructive low-level heuristics, they can be classified as hyper-heuristic systems and, on the other hand, the DPSO is a stochastic global optimization method from class of EAs. The proposed algorithms are tested on a set of Carter benchmark problems to set the parameters of algorithms and also to compare different methods. The obtained results demonstrate that the proposed hill climbing local search, in spite of its simplicity, has a better performance than original hill climbing method. Among different graph coloring heuristics, those of algorithms which employ the saturation degree heuristic lead to the better results. Also among different proposed algorithms, the first structure of DPSO and third strategy of hybridizing obtain a better performance than the other structures and strategies. In a later part of the comparative experiment, performance comparisons of the proposed algorithms with some other hyper-heuristic and EA methods are done. The obtained results confirm that the proposed hybrid algorithm has a better, or at least comparable, performance than other hyper-heuristic systems. Also it obtains the best results among hyper-heuristic systems on some problems. Also in comparison of other EAs, it has a completely comparable performance.

Application of Genetic Algorithm to the calculation of parameters for NRTL and Two-Suffix Margules models in ternary extraction ionic liquid systems

The stochastic global optimization methods have been extensively used in the fluid phase equilibrium calculations. Among these methods, Genetic Algorithm (GA) can be applied to calculate fitting parameters of activity coefficient models in equilibrium systems. In the present work, based on the GA method, the parameters of NRTL and Two-Suffix Margules models have been calculated for 20 ternary extraction systems containing ionic liquids. The values of the parameters of these models along with the root mean square deviations (rmsd) are reported. The obtained results, in terms of rmsd for these models are satisfactory, with the overall values of 0.0039 and 0.0195 for 169 tie-lines for NRTL and Two-Suffix Margules models, respectively.

Solving Constrained Global Optimization Problems by Using Hybrid Evolutionary Computing and Artificial Life Approaches

This work presents a hybrid real‐coded genetic algorithm with a particle swarm optimization (RGA‐PSO) algorithm and a hybrid artificial immune algorithm with a PSO (AIA‐PSO) algorithm for solving 13 constrained global optimization (CGO) problems, including six nonlinear programming and seven generalized polynomial programming optimization problems. External RGA and AIA approaches are used to optimize the constriction coefficient, cognitive parameter, social parameter, penalty parameter, and mutation probability of an internal PSO algorithm. CGO problems are then solved using the internal PSO algorithm. The performances of the proposed RGA‐PSO and AIA‐PSO algorithms are evaluated using 13 CGO problems. Moreover, numerical results obtained using the proposed RGA‐PSO and AIA‐PSO algorithms are compared with those obtained using published individual GA and AIA approaches. Experimental results indicate that the proposed RGA‐PSO and AIA‐PSO algorithms converge to a global optimum solution to a CGO problem. Furthermore, the optimum parameter settings of the internal PSO algorithm can be obtained using the external RGA and AIA approaches. Also, the proposed RGA‐PSO and AIA‐PSO algorithms outperform some published individual GA and AIA approaches. Therefore, the proposed RGA‐PSO and AIA‐PSO algorithms are highly promising stochastic global optimization methods for solving CGO problems.

Automatic termination of parallel optimization runs of stochastic global optimization methods in consensus or stagnation cases

Dynamic models give detailed information about the influence of many parameters on the behaviour of the biochemical process of interest. Parameter optimization of dynamic models is used in parameter estimation tasks and in design tasks. A drawback of the popular family of global stochastic optimization methods is the stochastic nature of the convergence of the best value of objective function to the global optimum or a value close to that. Therefore the optimization can take long time until a stable value of objective function is reached. Even then the risk of stagnation far from global optimum remains. That sets force to look for efficient approaches to reduce optimization time and discover cases of poor performance of optimization methods. Parallel optimization runs of identical optimization tasks can be used to reduce the impact of stochastic processes used in stochastic optimization methods. Consensus and stagnation criteria are proposed to terminate a set of parallel optimization runs when it is assessed that no significant improvements of the best value of the objective function are expected. Four automatically detectable cases of behaviour of a group of parallel optimization runs are analysed: 1) reaching of consensus criterion (consensus case), 2) stagnation of all optimization runs without reaching the consensus criterion (stagnation case), 3) stagnation at the initial value of the objective function, 4) lack of feasible solution. The proposed approach can be used automating the termination of optimization process when no further progress of the best value of objective function is expected. Suitability of particular optimization method with its settings for particular optimization task can be assessed analysing the dynamics of objective function's best values of parallel runs.

An efficient constraint handling method with integrated differential evolution for numerical and engineering optimization

Constrained optimization problems are very important as they are encountered in many engineering applications. Equality constraints in them are challenging to handle due to tiny feasible region. Additionally, global optimization is required for finding global optimum when the objective function and constraints are nonlinear. Stochastic global optimization methods can handle non-differentiable and multi-modal objective functions. In this paper, a new constraint handling method for use with such methods is proposed for solving equality and/or inequality constrained problems. It incorporates adaptive relaxation of constraints and the feasibility approach for selection. The recent integrated differential evolution (IDE) with the proposed constraint handling technique is tested for solving benchmark problems with constraints, and then applied to many chemical engineering application problems with equality and inequality constraints. The results show that the proposed constraint handling method with IDE (C-IDE) is reliable and efficient for solving constrained optimization problems, even with equality constraints.

A generalization of the Solis–Wets method

In this paper we focus on the application of global stochastic optimization methods to extremum estimators. We propose a general stochastic method—the master method—which includes several stochastic optimization algorithms as a particular case. The proposed method is sufficiently general to include the Solis–Wets method, the improving hit-and-run algorithm, and a stochastic version of the zigzag algorithm. A matrix formulation of the master method is presented and some specific results are given for the stochastic zigzag algorithm. Convergence of the proposed method is established under a mild set of conditions, and a simple regression model is used to illustrate the method.

Inverse estimation of inlet parameters in an axisymmetric cylindrical combustor with thermal radiation effect

The major objective of the present study is to extend the application of an inverse analysis to a more realistic engineering problem with complex combustion process than traditional simple heat transfer problem. In order to do this, unknown inlet parameters for burner in an axisymmetric cylindrical combustor, such as species mass fractions and inlet velocities of fuel/oxidizer, are estimated from measurement data using the inverse analysis. To simulate more realistic phenomena, thermal radiation effect is considered additionally. For efficient estimation in a practical diffusion-controlled turbulent combustion problem, the repulsive particle swarm optimization (RPSO) method is implemented as an inverse solver which belongs to the class of stochastic evolutionary global optimization methods. Based on the results of the present study, it is expected that useful information can be successfully predicted using the inverse analysis even in a design optimization of the real combustion systems accompanying thermal radiation effect.