Stochastic Annealing Research Articles

Re-use of samples in stochastic annealing

The re-use of samples in stochastic black box optimisation is a double-edged sword. On the one hand it has the potential of substantially reducing the number of simulation runs required, on the other hand it introduces dependencies between iterations of the optimisation algorithm that may misguide the search. This paper proposes a principled way to re-use samples in stochastic annealing, a generalisation of simulated annealing for stochastic black-box optimisation problems. We propose three alternative algorithms that all, despite the stochastic errors when evaluating a solution via simulation, obey the detailed balance equation and thus have the same assurance of converging to the true optimum as standard simulated annealing in the deterministic case. This is achieved by optimising in the product space of solution and error. We compare our new algorithms in terms of their acceptance ratio as well as empirically on two stochastic combinatorial optimisation problems and find that the best algorithms are of order twice as efficient as the state-of-the-art. Furthermore, one algorithm we propose can work with non-Gaussian and even unknown error distributions.

Achieving the Social Optimum in a Nonconvex Cooperative Aggregative Game: A Distributed Stochastic Annealing Approach.

This brief designs a distributed stochastic annealing algorithm for nonconvex cooperative aggregative games, whose players' cost functions not only depend on players' own decision variables but also rely on the sum of players' decision variables. To seek the social optimum of cooperative aggregative games, a distributed stochastic annealing algorithm is proposed, where the local cost functions are nonconvex and the communication topology between players is time-varying. The weak convergence to the social optimum of the algorithm is further analyzed. A numerical example is finally given to illustrate the effectiveness of the proposed algorithm.

Combinatorial optimization by weight annealing in memristive hopfield networks

The increasing utility of specialized circuits and growing applications of optimization call for the development of efficient hardware accelerator for solving optimization problems. Hopfield neural network is a promising approach for solving combinatorial optimization problems due to the recent demonstrations of efficient mixed-signal implementation based on emerging non-volatile memory devices. Such mixed-signal accelerators also enable very efficient implementation of various annealing techniques, which are essential for finding optimal solutions. Here we propose a “weight annealing” approach, whose main idea is to ease convergence to the global minima by keeping the network close to its ground state. This is achieved by initially setting all synaptic weights to zero, thus ensuring a quick transition of the Hopfield network to its trivial global minima state and then gradually introducing weights during the annealing process. The extensive numerical simulations show that our approach leads to a better, on average, solutions for several representative combinatorial problems compared to prior Hopfield neural network solvers with chaotic or stochastic annealing. As a proof of concept, a 13-node graph partitioning problem and a 7-node maximum-weight independent set problem are solved experimentally using mixed-signal circuits based on, correspondingly, a 20 × 20 analog-grade TiO2 memristive crossbar and a 12 × 10 eFlash memory array.

Adaptive large neighborhood search for solving the circle bin packing problem

We address a new packing problem variant known as the 2D circle bin packing problem (2D-CBPP), which involves packing circles into multiple square bins as densely as possible so as to minimize the number of bins used. To this end, we propose an adaptive large neighborhood search (ALNS) algorithm, which uses our Greedy Algorithm with Corner Occupying Action (GACOA) to construct an initial layout. The greedy solution is usually in a local optimum trap, and ALNS enables multiple neighborhood searches that depend on a stochastic annealing schedule to avoid local minimum traps. Specifically, ALNS perturbs the current layout to escape local optima by iteratively reassigning some circles and accepting the new layout with some probability during the search. The acceptance probability is adjusted adaptively using simulated annealing, which fine-tunes the search direction in order to reach the global optimum. We benchmark computational results against GACOA in heterogeneous instances. In all cases, ALNS outperforms GACOA in improving the objective function, and in several cases, the number of bins used for packing is significantly reduced.

Joint leisure travel optimization with user-generated data via perceived utility maximization

The lack of personalized solutions for managing the demand of joint leisure trips in cities in real time hinders the optimization of transportation system operations. Joint leisure activities can account for up to 60% of trips in cities and unlike fixed trips (i.e., trips to work where the arrival time and the trip destination are predefined), leisure activities offer more optimization flexibility since the activity destination and the arrival times of individuals can vary.To address this problem, a perceived utility model derived from non-traditional data such as smartphones/social media for representing users’ willingness to travel a certain distance for participating in leisure activities at different times of day is presented. Then, a stochastic annealing search method for addressing the exponential complexity optimization problem is introduced. The stochastic annealing method suggests the preferred location of a joint leisure activity and the arrival times of individuals based on the users’ preferences derived from the perceived utility model. Test-case implementations of the approach used 14-month social media data from London and showcased an increase of up to 3 times at individuals’ satisfaction while the computational complexity is reduced to almost linear time serving the real-time implementation requirements.

Stochastic annealing optimization of uncertain aeroelastic system

Dealing with optimization when uncertainty is introduced in the objective function through probabilistic models is a difficult problem. When the gradient of the objective function is not available, there exist several extensions of classical heuristics used for deterministic problems. This paper addresses the case of the annealing algorithm. We adapt a discrete stochastic optimization algorithm to the problem of uncertain aeroelastic optimization. The stochastic annealing algorithm used here optimizes the sum of the expected value of the objective function and a penalized term which takes into account the error bandwidth of the estimator used for evaluating the expected value of the objective function. The approach is tested on a simple wing mockup described by a fish bone finite element model. Uncertainties are introduced in the stiffness of the beams. The purpose of the optimization is to find the best mass configuration which maximizes the critical pressure value of the wing. We show the applicability of this approach in the aeroelasticity context.

Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule

Simulated annealing has been widely used in the solution of optimization problems. As known by many researchers, the global optima cannot be guaranteed to be located by simulated annealing unless a logarithmic cooling schedule is used. However, the logarithmic cooling schedule is so slow that no one can afford to use this much CPU time. This article proposes a new stochastic optimization algorithm, the so-called simulated stochastic approximation annealing algorithm, which is a combination of simulated annealing and the stochastic approximation Monte Carlo algorithm. Under the framework of stochastic approximation, it is shown that the new algorithm can work with a cooling schedule in which the temperature can decrease much faster than in the logarithmic cooling schedule, for example, a square-root cooling schedule, while guaranteeing the global optima to be reached when the temperature tends to zero. The new algorithm has been tested on a few benchmark optimization problems, including feed-forward neural network training and protein-folding. The numerical results indicate that the new algorithm can significantly outperform simulated annealing and other competitors. Supplementary materials for this article are available online.

Efficient Design under Uncertainty of Renewable Power Generation Systems Using Partitioning and Regression in the Course of Optimization

Renewable power generation systems are significantly affected by uncertainty due to intense variability often observed in energy sources. Uncertainty should be considered during design to enable optimum performance within constantly changing conditions. However, the resulting computational complexity and effort is high, especially in view of flowsheets integrating multiple subsystems. To address this challenge, the presented work proposes the partitioning of the space representing uncertain realizations to facilitate the development and continuous update of a surrogate model in the course of optimization. A wide exploration of this strategy reveals and addresses important issues in the implementation of the partitioning and model regression layers. Formal statistical associations are examined regarding the beneficial implications of partitioning to computational efficiency and surrogate model development. The proposed strategy is presented as part of a Simulated Annealing algorithm. This is tested in terms of computational efficiency and solution robustness against an adaptation of Stochastic Annealing, which addresses computational intensity through a different approach while depending entirely on a full system model. Results are illustrated through numerical examples and case studies on a stand-alone, hybrid system using renewable energy sources for power generation and storage.

Approximate stochastic annealing for online control of infinite horizon Markov decision processes

We present an online simulation-based algorithm called Approximate Stochastic Annealing (ASA) for solving infinite-horizon finite state-action space Markov decision processes (MDPs). The algorithm estimates the optimal policy by sampling at each iteration from a probability distribution function over the policy space, which is updated iteratively based on the Q-function estimates obtained via a recursion of Q-learning type. By exploiting a novel connection of ASA to the stochastic approximation method, we show that the sequence of distribution functions generated by the algorithm converges to a degenerated distribution that concentrates only on the optimal policy. Numerical examples are also provided to illustrate the algorithm.

Performance evaluation and segmentation for synthetic aperture radar image using mixture multiscale autoregressive model and bootstrap technique

An unsupervised segmentation and its performance evaluation technique are proposed for synthetic aperture radar (SAR) image based on the mixture multiscale autoregressive (MMAR) model and the bootstrap method. The segmentation-evaluation techniques consist of detecting the number of image regains, estimating MMAR parameters by using bootstrap stochastic annealing expectation-maximization (BSAEM) algorithm, and classifying pixels into region by using Bayesian classifier. Experimental results demonstrate that the evaluation operation is robust, and the proposed segmentation method is superior to the traditional single resolution techniques, and considerably reduces the computing time over the EM algorithm.

Optimum design and operation under uncertainty of power systems using renewable energy sources and hydrogen storage

The presented work addresses the design and optimization under uncertainty of power generation systems using renewable energy sources and hydrogen storage. A systematic design approach is proposed that enables the simultaneous consideration of synergies developed among numerous sub-systems within an integrated power generation system and the uncertainty involved in the system operation. The Stochastic Annealing optimization algorithm is utilized to handle the increased combinatorial complexity and to enable the consideration of different types of uncertainty in the performed optimization. A parallel adaptation of this algorithm is proposed to address the associated computational requirements through execution in a Grid computing environment. The proposed developments are implemented in a system that consists of photovoltaic panels, wind generators, accumulators, an electrolyzer, storage tanks, a compressor, a fuel cell and a diesel generator. Numerous design and operating parameters are considered as decision variables, while uncertain parameters are associated with weather fluctuations and operating efficiency of the employed sub-systems. The obtained results indicate robust performance under realizable system designs, in response to external or internal operating variations.

Simulated annealing in the presence of noise

In many practical optimization problems, evaluation of a solution is subject to noise, e.g., due to stochastic simulations or measuring errors. Therefore, heuristics are needed that are capable of handling such noise. This paper first reviews the state-of-the-art in applying simulated annealing to noisy optimization problems. Then, two new algorithmic variants are proposed: an improved version of stochastic annealing that allows for arbitrary annealing schedules, and a new approach called simulated annealing in noisy environments (SANE). The latter integrates ideas from statistical sequential selection in order to reduce the number of samples required for making an acceptance decision with sufficient statistical confidence. Finally, SANE is shown to significantly outperform other state-of-the-art simulated annealing techniques on a stochastic travelling salesperson problem.

Improved Genetic Algorithms for Deterministic Optimization and Optimization under Uncertainty. Part II. Solvent Selection under Uncertainty

The existence of a combinatorial search space in molecular design poses a challenge for traditional deterministic optimization methods. This paper presents an innovative approach based on improved genetic algorithms to optimally design solvents or extracting agents to separate the binary mixture of acetic acid and water in the presence of separation and physical property constraints. The UNIFAC−VLE model and Hansen's solubility parameter model are used to estimate the mixture properties. Since the mixture properties are predicted using group contribution methods, in which the group properties are derived from the regression of experimental data, uncertainties are inherent in the predictions. To account for these uncertainties, an “uncertainty factor” is introduced, which is defined as the ratio of experimental value to the value computed by the model. This uncertainty factor then is propagated through the model. A deterministic optimization framework is developed to compare the performance of two algorithms: (1) the improved genetic algorithm and (2) efficient stochastic annealing. Uncertainties are propagated through the stochastic framework. The results of the stochastic framework, with respect to the two algorithms, have also been analyzed and compared. As a stochastic optimization technique, the improved genetic algorithm outperforms its counterpart, the stochastic annealing technique. Using this method, new solvents with better targeted properties are found with less computational time.

Instantaneous Frequency Estimation When the Amplitude is a Stochastic Process Using Stochastic Calculus and Bootstrapping

Stochastic calculus methods are used to estimate the frequencies of a polynomial sinusoid when the amplitude is modeled as an Ornstein-Uhlenbeck process. Using stochastic calculus, one is able to develop a stochastic differential equation that relates the observations to the frequencies. The likelihood function is obtained through Girsanov theory and the Radon-Nikodym derivative. Maximum likelihood estimates are obtained numerically using stochastic annealing. Bootstrapping is used to improve the estimate of the frequencies.

Characterization of the probabilistic traveling salesman problem.

We show that stochastic annealing can be successfully applied to gain new results on the probabilistic traveling salesman problem. The probabilistic "traveling salesman" must decide on an a priori order in which to visit n cities (randomly distributed over a unit square) before learning that some cities can be omitted. We find the optimized average length of the pruned tour follows E(L(pruned))=sqrt[np](0.872-0.105p)f(np), where p is the probability of a city needing to be visited, and f(np)-->1 as np--> infinity. The average length of the a priori tour (before omitting any cities) is found to follow E(L(a priori))=sqrt[n/p]beta(p), where beta(p)=1/[1.25-0.82 ln(p)] is measured for 0.05< or =p< or =0.6. Scaling arguments and indirect measurements suggest that beta(p) tends towards a constant for p<0.03. Our stochastic annealing algorithm is based on limited sampling of the pruned tour lengths, exploiting the sampling error to provide the analog of thermal fluctuations in simulated (thermal) annealing. The method has general application to the optimization of functions whose cost to evaluate rises with the precision required.

Hammersley stochastic annealing: efficiency improvement for combinatorial optimization under uncertainty

This paper presents hierarchical improvements to combinatorial stochastic annealing algorithms using a new and efficient sampling technique. The Hammersley Sequence Sampling (HSS) technique is used for updating discrete combinations, reducing the Markov chain length, determining the number of samples automatically, and embedding better confidence intervals of the samples. The improved algorithm, Hammersley stochastic annealing, can significantly improve computational efficiency over traditional stochastic programming methods. This new method can be a useful tool for large-scale combinatorial stochastic programming problems. A real-world case study involving solvent selection under uncertainty illustrates the usefulness of this new algorithm.

Integrated Solvent Selection and Recycling for Continuous Processes

This paper presents a real world case study where a multiobjective programming (MOP) framework under uncertainty is used for simultaneous integration of environmentally benign solvent (EBS) selection and in-process solvent (IPS) recycling. At the EBS selection level within this framework, the Hammersley stochastic annealing algorithm is applied to design candidate EBSs under uncertainty. This algorithm can efficiently optimize stochastic combinatorial optimization problems and generate a different set of candidate EBSs from that of the deterministic EBS selection model. At the IPS recycling level, Aspen Plus with a nonlinear programming technique is used to optimize the acetic acid recovery process. Then, these EBS selection and IPS recycling models are integrated under the MOP framework. At the MOP level, an efficient constraint MOP algorithm is employed to evenly approximate the Pareto solution surface (i.e., tradeoff surface). Four objectivesacetic acid recovery, process flexibility, and two environmental impacts based on LC50 and LD50are evaluated. The resulting MOP framework provides very distinctive chemical and process design alternatives (i.e., Pareto optimal solutions), and uncertainties in this framework significantly affect the size and shape of the Pareto set. This novel MOP framework can be applied to any large-scale stochastic mixed-integer nonlinear optimization problems because this framework is computationally efficient even in the case of combinatorial explosion and uncertainty inclusion.

Efficient Combinatorial Optimization under Uncertainty. 2. Application to Stochastic Solvent Selection

Solvent selection is an important step in process synthesis, design, or process modification. The computer-aided molecular design (CAMD) approach, based on the reverse use of group contribution methods, provides a promising tool for solvent selection. However, uncertainties inherent in these techniques and associated models are often neglected. This paper, part 2 of the series, presents a new approach to solvent selection under uncertainty using the Hammersley stochastic annealing (HSTA) algorithm. A real world case study of acetic acid extraction from water, based on two stochastic CAMD models, namely, the infinite dilution activity coefficient model and the solubility parameter model, is presented. This example illustrates the importance of uncertainty in CAMD and demonstrates the usefulness of this HSTA approach to obtain robust decisions.

Efficient Combinatorial Optimization under Uncertainty. 1. Algorithmic Development

This paper presents hierarchical improvements to the combinatorial stochastic annealing algorithm using a new and efficient sampling technique. The Hammersley sequence sampling technique is used for updating discrete combinations, reducing the Markov chain length, determining the number of samples automatically, and embedding better confidence intervals of the samples. The improved algorithm, Hammersley stochastic annealing, can significantly improve computational efficiency over traditional stochastic programming methods. Three distinctive example functions considered proved the efficiency improvement of Hammersley stochastic annealing to be up to 99.3% better than the traditional counterparts. Thus, this new method can be a useful tool for large-scale combinatorial stochastic programming problems. Application of this new algorithm to a real world problem of solvent selection under uncertainty is presented in part 2 of this series.

Stochastic sampling algorithms for state estimation of jump Markov linear systems

Jump Markov linear systems are linear systems whose parameters evolve with time according to a finite-state Markov chain. Given a set of observations, our aim is to estimate the states of the finite-state Markov chain and the continuous (in space) states of the linear system. The computational cost in computing conditional mean or maximum a posteriori (MAP) state estimates of the Markov chain or the state of the jump Markov linear system grows exponentially in the number of observations. We present three globally convergent algorithms based on stochastic sampling methods for state estimation of jump Markov linear systems. The cost per iteration is linear in the data length. The first proposed algorithm is a data augmentation (DA) scheme that yields conditional mean state estimates. The second proposed scheme is a stochastic annealing (SA) version of DA that computes the joint MAP sequence estimate of the finite and continuous states. Finally, a Metropolis-Hastings DA scheme based on SA is designed to yield the MAP estimate of the finite-state Markov chain. Convergence results of the three above-mentioned stochastic algorithms are obtained. Computer simulations are carried out to evaluate the performances of the proposed algorithms. The problem of estimating a sparse signal developing from a neutron sensor based on a set of noisy data from a neutron sensor and the problem of narrow-band interference suppression in spread spectrum code-division multiple-access (CDMA) systems are considered.