Stochastic Global Optimization Research Articles

Improved sub‐band adaptive thresholding function for denoising of satellite image based on evolutionary algorithms

In this study, an improved method based on evolutionary algorithms for denoising of satellite images is proposed. In this approach, the stochastic global optimisation techniques such as Cuckoo Search (CS) algorithm, artificial bee colony (ABC), and particle swarm optimisation (PSO) technique and their different variants are exploited for learning the parameters of adaptive thresholding function required for optimum performance. It was found that the CS algorithm and ABC algorithm-based denoising approach give better performance in terms of edge preservation index or edge keeping index (EPI or EKI) peak signal-to-noise ratio (PSNR) and signal-to-noise ratio (SNR) as compared to PSO-based denoising approach. The proposed technique has been tested on satellite images. The quantitative (EPI, PSNR and SNR) and visual (denoised images) results show superiority of the proposed technique over conventional and state-of-the-art image denoising techniques.

A New Method on Data Clustering Based on Hybrid K-Harmonic Means and Imperialist Competitive Algorithm

Data clustering is one of the commonest data mining techniques. The K-means algorithm is one of the most wellknown clustering algorithms thatare increasingly popular due to the simplicity of implementation and speed of operation. However, its performancecouldbe affected by some issues concerningsensitivity to the initialization and getting stuck in local optima. The K-harmonic means clustering method manages the issue of sensitivity to initialization but the local optimaissue still compromises the algorithm. Particle Swarm Optimization algorithm is a stochastic global optimization technique which is a good solution to the above-mentioned problems. In the present article, the PSOKHM, a hybrid algorithm which draws upon the advantages of both of the algorithms, strives not only to overcome the issue of local optima in KHM but also the slow convergence speed of PSO. In this article, the proposed GSOKHM method, which is a combination of PSO and the evolutionary genetic algorithmwithin PSOKHM,has been positedto enhancethe PSO operation. To carry out this experiment, four real datasets have been employed whose results indicate thatGSOKHMoutperforms PSOKHM.

Solving nonlinear systems of functional equations with fuzzy adaptive simulated annealing

This paper proposes a method for finding solutions of arbitrarily nonlinear systems of functional equations through stochastic global optimization. The original problem (equation solving) is transformed into a global optimization one by synthesizing objective functions whose global minima, if they exist, are also solutions to the original system. The global minimization task is carried out by the stochastic method known as fuzzy adaptive simulated annealing, triggered from different starting points, aiming at finding as many solutions as possible. To demonstrate the efficiency of the proposed method, solutions for several examples of nonlinear systems are presented and compared with results obtained by other approaches. We consider systems composed of n equations on Euclidean spaces ℝn (n variables: x1, x2, x3, ⋯, xn).

Design of decanter in heterogeneous azeotropic distillation column by minimizing Gibbs free energy

ABSTRACTIn this paper, liquid–liquid equilibrium in decanter of heterogeneous azeotropic distillation is solved by minimizing Gibbs free energy, by using global optimization technique. The azeotropic distillation column is modeled by solving material balance, equilibrium and summation equations, and repulsive particle swarm optimization (RPSO), a stochastic global optimization formalism, is employed to predict stable steady state solution in decanter. To verify the performance of the RPSO algorithm, it is compared with equation solving method for liquid–liquid equilibria by considering two azeotropic systems, namely (1) benzene–ethanol–water and (2) furfural–water. The proposed methodology shows feasibility of the RPSO algorithm in predicting liquid–liquid equilibrium in decanter of heterogeneous azeotropic distillation. © 2013 Curtin University of Technology and John Wiley & Sons, Ltd.

Solving Unconstrained Global Optimization Problems via Hybrid Swarm Intelligence Approaches

Stochastic global optimization (SGO) algorithms such as the particle swarm optimization (PSO) approach have become popular for solving unconstrained global optimization (UGO) problems. The PSO approach, which belongs to the swarm intelligence domain, does not require gradient information, enabling it to overcome this limitation of traditional nonlinear programming methods. Unfortunately, PSO algorithm implementation and performance depend on several parameters, such as cognitive parameter, social parameter, and constriction coefficient. These parameters are tuned by using trial and error. To reduce the parametrization of a PSO method, this work presents two efficient hybrid SGO approaches, namely, a real-coded genetic algorithm-based PSO (RGA-PSO) method and an artificial immune algorithm-based PSO (AIA-PSO) method. The specific parameters of the internal PSO algorithm are optimized using the external RGA and AIA approaches, and then the internal PSO algorithm is applied to solve UGO problems. The performances of the proposed RGA-PSO and AIA-PSO algorithms are then evaluated using a set of benchmark UGO problems. Numerical results indicate that, besides their ability to converge to a global minimum for each test UGO problem, the proposed RGA-PSO and AIA-PSO algorithms outperform many hybrid SGO algorithms. Thus, the RGA-PSO and AIA-PSO approaches can be considered alternative SGO approaches for solving standard-dimensional UGO problems.

Automatic termination of parallel optimization runs of global stochastic optimization methods by upper limit criterion

Many combinations of adjustable parameters should be tested in optimization experiments of biochemical networks to find the smallest subset of parameters enabling the best improvements of objective function both in case of design task and parameter estimation task. In case of optimization with global stochastic optimization methods one of the problems is the termination of the optimization run looking for a good compromise between spent computational resources and probability that the best found value of objective function will be the global optimum. Longer runs increase the possibility to each the global optimum. Automatic termination criteria in case of consensus or stagnation of parallel optimization runs have been proposed as criteria for automatic termination. Varying the consensus and delay time settings different probability of reaching global optimum and duration of optimization can be reached. It is proposed to modify automatic optimization termination criteria of parallel optimization runs applying upper limit agreement of a number of parallel optimization runs. Automatic application of upper limit agreement would reduce the duration of scanning of the whole space of combination of adjustable parameters. This approach is tested on the yeast glycolysis model with six adjustable parameters using COPASI, CoRunner and ConvAn software for five parallel optimization runs per combination of adjustable parameters.

Determinition of best set of adjustable parameters with full search and limited search methods

In case of optimization of biochemical networks the best combination of adjustable parameters has to be found keeping low the costs of modification of biochemical network and reducing the risk of unpredicted side effects of processes outside the scope of the model. Dynamic models of biochemical networks usually are in form of system of differential equations and therefore can not be optimized analytically. Usually they are optimized using computationally demanding global stochastic optimization methods which may have hardly predictable duration of optimization. Optimization of all the possible combinations is necessary to find the best set of adjustable parameters per number of parameters in combination. Due to the combinatorial explosion the full search often is not feasible approach if computational and time resources are limited. In this study the performance of full search is compared with forward selection and three metabolic control based methods which need significantly less combinations to be examined. It is found that forward selection is a good compromise in case of larger number of adjustable parameters where the number of possible combinations exceeds the available resources needed for full search in the adjustable parameter space. It is proposed to combine full search for small number of adjustable parameters with forward selection at larger number of adjustable parameters per combination.

Assessment of capabilities and limitations of stochastic global optimization methods for modeling mean activity coefficients of ionic liquids

In this paper, we have studied and analyzed the capabilities and limitations of seven stochastic global optimization methods to model mean activity coefficients of ammonium aqueous electrolytes using the electrolyte NRTL model. These stochastic methods are: simulated annealing (SA), genetic algorithm (GA), differential evolution (DE), particle swarm optimization (PSO), DE with tabu list (DETL), harmony search (HS) and bare bones PSO (BBPSO). Convergence characteristics and performances of these optimization methods have been tested and compared using different stopping conditions. Overall, our results indicate that SA, DETL and BBPSO offer better performance and promising convergence characteristics for solving parameter estimation problems involved in the modeling of thermodynamic properties of ionic liquids (ILs).

Design and optimization of an ethanol dehydration process using stochastic methods

Due to the increasing demand for renewable fuels that are economically attractive, as well as part of the quest for energy alternatives to replace carbon-based fuels, the purification of ethanol plays a key role. This paper presents the design and optimization of a dehydration process for ethanol, using two separation sequences: a conventional arrangement and an alternative arrangement based on liquid–liquid extraction. Both sequences were optimized using a stochastic global optimization algorithm (differential evolution) implemented in Mathworks Matlab and coupled to rigorous process simulations carried out in Aspen Plus. The economic feasibility of the two configurations was studied by changing the ethanol–water composition in the analyzed feed stream. The results clearly demonstrate that significant savings are possible by extraction when the ethanol content in the feed stream exceeds 10%mol (22wt%).

Professor Ryszard Zieliński's contribution to Monte Carlo methods and random number generators. Uniform asymptotics in statistics

The aim of the paper is to summarize contributions of Ryszard Zielinski to two important areas of research. First, we discuss his work related to Monte Carlo methods. Ryszard Zielinski was particularly interested in Monte Carlo optimization. About 10 of his papers concerned stochastic algorithms for seeking extrema. He examined methods related to stochastic approximation, random search and global optimization. We stress that Zielinski often considered computational problems from a statistical perspective. In several articles he explicitly indicated that optimization can be reformulated as a statistical estimation problem. We also discuss relation between the family of Simulated Annealing algorithms on the one hand and some procedures examined earlier by Ryszard Zielinski on the other. Another topic belonging to Monte Carlo methods, in which Ryszard Zielinski has achieved interesting results, is construction of random number generators and examination of their statistical properties. Zielinski proposed an aperiodic generator based on Weil sequences and showed how it can be efficiently implemented. Later he constructed an algorithm which uses several such generators and produces pseudo-random sequences with better statistical properties. The second area of Zielinski’s work discussed here is related to uniform limit theorems of mathematical statistics. We stress the methodological motivation behind the research in this direction. In Zielinski’s view, asymptotic results should hold uniformly with respect to the family of probability distributions under consideration. In his opinion, this requirement comes from the very nature of statistical models and the needs of practical applications. Zielinski examined uniform versions the Weak Law of Large Numbers, Strong Law of Large Numbers and Central Limit Theorem in several statistical models. Some results were rather unexpected. He also gave a necessary and sufficient condition for uniform consistency of sample quantiles. Two papers of Ryszard Zielinski were devoted to uniform consistency of smoothed versions of empirical cumulative distribution function. In one of them he proved a version of Dvoretzky-Kiefer-Wolfowitz inequality.

One-step approach for heat exchanger network retrofitting using integrated differential evolution

Heat exchanger network (HEN) retrofitting is more important and challenging than HEN synthesis since it involves modifying existing network for improved energy efficiency. Additional factors to be considered include spatial constraints, relocation and re-piping costs, reassignment and effective use of existing heat exchanger areas. The previous studies using stochastic global optimization algorithms are mainly focused on two-level approach: the first level uses a stochastic algorithm for optimizing structure, and the second level uses either a stochastic or a deterministic algorithm for optimizing continuous variables. In this study, we propose and test one-step approach where a stochastic global optimization method, namely, integrated differential evolution (IDE), handles both discrete and continuous variables together. Thus, HEN structure and retrofitting model parameters are simultaneously optimized by IDE, which avoids the algorithm trapping at a local optimum and also improves the computational efficiency. Results on HEN applications show that the proposed approach gives better solutions.

Economic Dispatch Using Improved Differential Evolution Approach: A Case Study of the Algerian Electrical Network

Differential evolution (DE) is a simple but powerful evolutionary optimization algorithm with continually outperforming many of the already existing stochastic and direct search global optimization techniques. DE algorithm is a new optimization method that can handle non-differentiable, nonlinear, and multimodal objective functions. This paper presents an efficient modified differential evolution algorithm for solving economic dispatch problem. A new mutation strategy of the conventional DE is suggested to improve the performance and avoid premature convergence. Numerical results on the IEEE 30 bus test system and the practical Algerian 59 bus system show that the proposed approach is faster and more robust compared with those reported recently in the literature. The comparison results prove the capability of the proposed method in real-time implementation for the economic dispatch problem.

Optimization of Cognitive Radio System Using Simulated Annealing

Cognitive radio (CR) technology has introduced a revolution in wireless communication network and it is capable to operate in a continuously varying radio frequency environment that depends on multiple parameters. In this paper, optimization of CR system has been achieved using simulated annealing (SA) Technique. SA is a stochastic global optimization technique that exploits an analogy between the way in which a metal cools and freezes into a minimum energy crystalline structure. SA has been used to meet the quality of service (QoS) that is defined by the user in terms of minimum transmit power, minimum bit error rate, maximum throughput, minimum interference and maximum spectral efficiency. The results obtained by SA are compared with the genetic algorithm (GA) results for the various QoS parameters and it has been observed that SA is outperforming GA in CR system optimization.

Toward Optimal Operation Conditions of Freeze-Drying Processes via a Multilevel Approach

Freeze drying (lyophilization) offers an attractive dehydration method for valuable food and biological products, because it is capable of preserving product quality and biological activity while extending their shelf life. However, despite these benefits in terms of product quality, freeze drying is also a notoriously energy-intensive and time-consuming process. This requires an expensive operation to construct an efficient optimal decision-making tool able to drive the operation through the most effective paths that minimize time and maximize product quality. Here we propose an integrated approach to operational design and control of the freeze-drying process that combines dynamic modeling with efficient optimized off-line and on-line control. The required mass and energy balance equations still contain inherent nonlinearity, even in their lumped parameter version. This results in a set of complex dynamic, computationally costly optimization problems solved by selected global stochastic optimization algorithms. Real-time disturbances and model uncertainties are addressed via the proposed hierarchical multilevel approach, allowing recalculation of the required control strategies. The framework developed has been revealed as a useful tool to systematically define off-line and on-line optimal operation policies for many food and biological processing units.

Deterministic and stochastic global optimization techniques for planar covering with ellipses problems

Problems of planar covering with ellipses are tackled in this work. Ellipses can have a fixed angle or each of them can be freely rotated. Deterministic global optimization methods are developed for both cases, while a stochastic version of the method is also proposed for large instances of the latter case. Numerical results show the effectiveness and efficiency of the proposed methods.

OBTAINING APPROXIMATE VALUES OF EXTERIOR ORIENTATION ELEMENTS OF MULTI-INTERSECTION IMAGES USING PARTICLE SWARM OPTIMIZATION

Abstract. In this paper, an efficient global optimization algorithm in the field of artificial intelligence, named Particle Swarm Optimization (PSO), is introduced into close range photogrammetric data processing. PSO can be applied to obtain the approximate values of exterior orientation elements under the condition that multi-intersection photography and a small portable plane control frame are used. PSO, put forward by an American social psychologist J. Kennedy and an electrical engineer R.C. Eberhart, is a stochastic global optimization method based on swarm intelligence, which was inspired by social behavior of bird flocking or fish schooling. The strategy of obtaining the approximate values of exterior orientation elements using PSO is as follows: in terms of image coordinate observed values and space coordinates of few control points, the equations of calculating the image coordinate residual errors can be given. The sum of absolute value of each image coordinate is minimized to be the objective function. The difference between image coordinate observed value and the image coordinate computed through collinear condition equation is defined as the image coordinate residual error. Firstly a gross area of exterior orientation elements is given, and then the adjustment of other parameters is made to get the particles fly in the gross area. After iterative computation for certain times, the satisfied approximate values of exterior orientation elements are obtained. By doing so, the procedures like positioning and measuring space control points in close range photogrammetry can be avoided. Obviously, this method can improve the surveying efficiency greatly and at the same time can decrease the surveying cost. And during such a process, only one small portable control frame with a couple of control points is employed, and there are no strict requirements for the space distribution of control points. In order to verify the effectiveness of this algorithm, two experiments are carried out. In the first experiment, images of a standard grid board are taken according to multi-intersection photography using digital camera. Three points or six points which are located on the left-down corner of the standard grid are regarded as control points respectively, and the exterior orientation elements of each image are computed through PSO, and compared with these elements computed through bundle adjustment. In the second experiment, the exterior orientation elements obtained from the first experiment are used as approximate values in bundle adjustment and then the space coordinates of other grid points on the board can be computed. The coordinate difference of grid points between these computed space coordinates and their known coordinates can be used to compute the accuracy. The point accuracy computed in above experiments are ±0.76mm and ±0.43mm respectively. The above experiments prove the effectiveness of PSO used in close range photogrammetry to compute approximate values of exterior orientation elements, and the algorithm can meet the requirement of higher accuracy. In short, PSO can get better results in a faster, cheaper way compared with other surveying methods in close range photogrammetry.

Application Of Genetic Algorithm To Parameter Estimation In Liquid-liquid Phase Equilibrium Modeling

Parameter estimation problems for liquid–liquid equilibrium data modeling are challenging due to the nonlinearity of thermodynamic models. In this work, the stochastic global optimization technique, namely Genetic Algorithm is applied to calculate the interaction and non randomness parameters of NRTL model. Forty ternary extraction systems containing benzene, hexane and different ionic liquids as solvent at various temperatures reported in literature are selected. The parameters of NRTL model are calculated and the global rmsd value of 0.0076 for 357 tie-lines shows that these parameters can be considered as the global parameters of NRTL model in the studied ternary systems. The global parameters are applied in all systems and rmsd values for them are reported. The results showed that Genetic Algorithm as a powerful and effective tool can be used to optimize highly nonlinear problems in phase equilibrium modeling.

On the capabilities and limitations of harmony search for parameter estimation in vapor–liquid equilibrium modeling

The modeling of vapor–liquid equilibrium data using local composition models is an interesting and challenging global optimization problem in the context of chemical engineering and applied thermodynamics. Until now, several deterministic and stochastic global optimization strategies have been used for modeling vapor–liquid equilibrium (VLE) data. Stochastic optimization methods may offer several advantages for solving global optimization problems and, until now, some meta-heuristics have been tested for modeling phase equilibrium data. However, these optimization strategies usually show a robust performance but, in some challenging problems, they may fail to locate the global optimum. In particular, Harmony Search (HS) is a direct-search method with attractive characteristics for its use in phase equilibrium modeling and calculations. However, to the best of our knowledge, this stochastic optimization strategy has not been used to perform this type of thermodynamic calculations. This study introduces the HS method for solving the non-linear parameter estimation problem involved in the modeling of VLE data. Specifically, the performance of this meta-heuristic has been tested and analyzed using several sets of binary VLE data with local composition models and both the classical approach of the least squares regression and the error-in-variable formulation. Results of this study are used to identify the capabilities and limitations of HS for VLE data modeling. In summary, HS is a promising meta-heuristic for processing these phase equilibrium data using the classical least square formulation and may offer a better performance than those obtained using current stochastic methods such as Genetic Algorithm or Particle Swarm Optimization. However, the reliability of the traditional HS is poor for VLE parameter estimation using the error-in-variable formulation. Finally, this paper discusses and analyzes alternatives to improve the performance of HS in VLE data modeling especially for the error-in-variable approach. Results indicate that the HS variants called the Improved Harmony Search and the Global-Best Harmony Search offer a better performance for solving EIV parameter estimation problems.

Hybrid particle swarm optimization transplanted into a hyper-heuristic structure for solving examination timetabling problem

Examination timetabling is a discrete, multi-objective and combinatorial optimization problem which tends to be solved with a cooperation of stochastic search approaches such as evolutionary algorithms (EAs) and heuristic methods such as sequential graph coloring heuristics. This research investigates the use of discrete particle swarm optimization (DPSO) for solving examination timetabling problem. A combination of mutation, specialist recombination operator and graph coloring heuristics are used to update position of particles in the DPSO. A new local search method, called two staged hill climbing, is proposed and is utilized to hybridize the DPSO algorithm. Three structures for the DPSO and three strategies to hybridize it are proposed. On one hand, since the proposed DPSO algorithms such as hyper-heuristics methods employ a strategy to manage a set of constructive low-level heuristics, they can be classified as hyper-heuristic systems and, on the other hand, the DPSO is a stochastic global optimization method from class of EAs. The proposed algorithms are tested on a set of Carter benchmark problems to set the parameters of algorithms and also to compare different methods. The obtained results demonstrate that the proposed hill climbing local search, in spite of its simplicity, has a better performance than original hill climbing method. Among different graph coloring heuristics, those of algorithms which employ the saturation degree heuristic lead to the better results. Also among different proposed algorithms, the first structure of DPSO and third strategy of hybridizing obtain a better performance than the other structures and strategies. In a later part of the comparative experiment, performance comparisons of the proposed algorithms with some other hyper-heuristic and EA methods are done. The obtained results confirm that the proposed hybrid algorithm has a better, or at least comparable, performance than other hyper-heuristic systems. Also it obtains the best results among hyper-heuristic systems on some problems. Also in comparison of other EAs, it has a completely comparable performance.

Gene regulatory network modeling via global optimization of high-order dynamic Bayesian network

BackgroundDynamic Bayesian network (DBN) is among the mainstream approaches for modeling various biological networks, including the gene regulatory network (GRN). Most current methods for learning DBN employ either local search such as hill-climbing, or a meta stochastic global optimization framework such as genetic algorithm or simulated annealing, which are only able to locate sub-optimal solutions. Further, current DBN applications have essentially been limited to small sized networks.ResultsTo overcome the above difficulties, we introduce here a deterministic global optimization based DBN approach for reverse engineering genetic networks from time course gene expression data. For such DBN models that consist only of inter time slice arcs, we show that there exists a polynomial time algorithm for learning the globally optimal network structure. The proposed approach, named GlobalMIT+, employs the recently proposed information theoretic scoring metric named mutual information test (MIT). GlobalMIT+ is able to learn high-order time delayed genetic interactions, which are common to most biological systems. Evaluation of the approach using both synthetic and real data sets, including a 733 cyanobacterial gene expression data set, shows significantly improved performance over other techniques.ConclusionsOur studies demonstrate that deterministic global optimization approaches can infer large scale genetic networks.