Self-Tuning Lam Annealing: Learning Hyperparameters While Problem Solving

Vincent A Cicirello

doi:10.3390/app11219828

Abstract

The runtime behavior of Simulated Annealing (SA), similar to other metaheuristics, is controlled by hyperparameters. For SA, hyperparameters affect how “temperature” varies over time, and “temperature” in turn affects SA’s decisions on whether or not to transition to neighboring states. It is typically necessary to tune the hyperparameters ahead of time. However, there are adaptive annealing schedules that use search feedback to evolve the “temperature” during the search. A classic and generally effective adaptive annealing schedule is the Modified Lam. Although effective, the Modified Lam can be sensitive to the scale of the cost function, and is sometimes slow to converge to its target behavior. In this paper, we present a novel variation of the Modified Lam that we call Self-Tuning Lam, which uses early search feedback to auto-adjust its self-adaptive behavior. Using a variety of discrete and continuous optimization problems, we demonstrate the ability of the Self-Tuning Lam to nearly instantaneously converge to its target behavior independent of the scale of the cost function, as well as its run length. Our implementation is integrated into Chips-n-Salsa, an open-source Java library for parallel and self-adaptive local search.

Highlights

Optimization problems of industrial relevance are often NP-Hard, such as applications of various classical problems such as the optimization variants of NP-Complete problems like the traveling salesperson, graph coloring, largest common subgraph, and bin packing, among many others [1].Approaches that are guaranteed to provide optimal solutions to such problems have worst case run-times that are exponential
The Modified Lam can be sensitive to the scale of the cost function, and is sometimes slow to converge to its target behavior
We present a novel variation of the Modified Lam that we call Self-Tuning Lam, which uses early search feedback to auto-adjust its self-adaptive behavior

Summary

Introduction

Approaches that are guaranteed to provide optimal solutions to such problems have worst case run-times that are exponential. It is common to turn to metaheuristics, such as genetic algorithms [2] and other forms of evolutionary computation, simulated annealing [3,4,5], tabu search [6], ant colony optimization [7], stochastic local search [8], among many others. Not guaranteed to optimally solve the problem, they can often find near optimal solutions, or at least sufficiently optimal solutions, in a fraction of the time of the alternatives. They are usually characterized by an anytime [9] property, where solution quality improves with additional runtime. One can run such an algorithm as long as time allows, utilizing whatever solution is found at the time it is needed

Objectives

Methods

Results

Conclusion