Adaptive Annealing Schedule Research Articles

Self-Tuning Lam Annealing: Learning Hyperparameters While Problem Solving

The runtime behavior of Simulated Annealing (SA), similar to other metaheuristics, is controlled by hyperparameters. For SA, hyperparameters affect how “temperature” varies over time, and “temperature” in turn affects SA’s decisions on whether or not to transition to neighboring states. It is typically necessary to tune the hyperparameters ahead of time. However, there are adaptive annealing schedules that use search feedback to evolve the “temperature” during the search. A classic and generally effective adaptive annealing schedule is the Modified Lam. Although effective, the Modified Lam can be sensitive to the scale of the cost function, and is sometimes slow to converge to its target behavior. In this paper, we present a novel variation of the Modified Lam that we call Self-Tuning Lam, which uses early search feedback to auto-adjust its self-adaptive behavior. Using a variety of discrete and continuous optimization problems, we demonstrate the ability of the Self-Tuning Lam to nearly instantaneously converge to its target behavior independent of the scale of the cost function, as well as its run length. Our implementation is integrated into Chips-n-Salsa, an open-source Java library for parallel and self-adaptive local search.

Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimation

We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a sequential factorization of the marginal likelihood in such settings by using stochastic gradient Markov Chain Monte Carlo techniques. This approach is far more efficient than traditional marginal likelihood estimation techniques such as nested sampling and annealed importance sampling due to its use of mini-batches to approximate the likelihood. Stability of the estimates is provided by an adaptive annealing schedule. The resulting stochastic gradient annealed importance sampling (SGAIS) technique, which is the key contribution of our paper, enables us to estimate the marginal likelihood of a number of models considerably faster than traditional approaches, with no noticeable loss of accuracy. An important benefit of our approach is that the marginal likelihood is calculated in an online fashion as data becomes available, allowing the estimates to be used for applications such as online weighted model combination.

An improved simulated annealing algorithm in fracture network modeling

Fracture network modeling is a crucial step in development of hydrocarbon resources. For proposing development plan and design of stimulation operations fracture network should be known. This paper proposes a new improved simulated annealing algorithm for the problem of fracture network modeling. This algorithm improves computation performance without degrading solution quality so much by incorporating a method for estimation of the initial value of temperature-like parameter, using an adaptive Markov chain length (inner iterations) and suggesting a new fast and adaptive annealing schedule. Masihi et al. (2007) proposed an objective function for fracture network modeling based on mechanism of fracturing. We implemented this objective function through a proposed algorithm to generate models that incorporate field observations. The applicability of the methodology is assessed through a case study and the results are satisfying.

Experimental and numerical investigation of an adaptive simulated annealing technique in optimization of warm tube hydroforming

Simulated annealing technique – which has attracted significant attention as being suitable for combinatorial optimization problems – has been used in many engineering applications. In current research, this method is used in optimization of a complicated forming process. Warm tube hydroforming is a new technology in the forming of lightweight aluminum and magnesium alloys in automotive, electronics and leisure industries. Owing to the complex nature of this process at high temperatures, and difficulty of obtaining the optimized forming conditions, this process has not received the credit it deserves yet. A new method for optimization of the warm tube hydroforming process is developed by using the simulated annealing algorithm with a novel adaptive annealing schedule. The optimal pressure loading paths are obtained for AA6061 aluminum alloy tubes with different wall thicknesses and corner fillets. The results obtained by simulated annealing technique are verified numerically and experimentally.

Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning

We present a theory of population based optimization methods using approximations of search distributions. We prove convergence of the search distribution to the global optima for the factorized distribution algorithm (FDA) if the search distribution is a Boltzmann distribution and the size of the population is large enough. Convergence is defined in a strong sense––the global optima are attractors of a dynamical system describing mathematically the algorithm. We investigate an adaptive annealing schedule and show its similarity to truncation selection. The inverse temperature β is changed inversely proportionally to the standard deviation of the population. We extend FDA by using a Bayesian hyper-parameter. The hyper-parameter is related to mutation in evolutionary algorithms. We derive an upper bound on the hyper-parameter to ensure that FDA still generates the optima with high probability. We discuss the relation of the FDA approach to methods used in statistical physics to approximate a Boltzmann distribution and to belief propagation in probabilistic reasoning. In the last part are sparsely connected. Our empirical results are as good or even better than any other method used for this problem.

熱力学的選択ルールを用いた巡回セールスマン問題の遺伝的解法

For successful applications of the genetic algorithm, there are two important points to be considered. The first point is the design of the fitness landscape introduced by the representation of the solution as a gene and searching operations such as crossover and mutation. The second is control of the convergence brought about by the selection operation. In the conventional implementation of GA, these two points are mutually dependent, i.e., a suitable selection pressure varies largely depending on, e.g., the crossover operator. Hence, it requires much trial-and-error effort to find a nice configuration of GA. The authors apply a novel selection rule, the Thermodynamical Genetic Algorithm (TDGA) proposed by N. Mori et al. (1995) to the traveling salesman problem (TSP), and propose an adaptive annealing schedule of the temperature in TDGA. Computer simulation with several crossover operators for TSP shows that TDGA reduces the mutual dependency between the fitness landscape and the convergence process.

A comparison of several cooling schedules for simulated annealing implemented on a residual statics problem

A study of several cooling schedules for the simulated annealing optimization algorithm, as applied to a residual statics problem in seismic data processing, is undertaken and discussed. A variety of schedules which depend on user‐controlled parameters and an adaptive annealing schedule which depends on the system itself are considered. The algorithm, with each schedule implemented in turn, is tested on a residual statics estimation where the optimization function is a coherence function between adjacent common depth point (CDP) gathers. Results of this study suggest the usefulness of a hybrid cooling schedule to reduce the number of cycles. Constrained adaptive annealing schedule offers a viable alternative.