Generalized Hill Climbing Algorithms Research Articles

Performance Analysis of Cyclical Simulated Annealing Algorithms

Generalized hill climbing (GHC) algorithms provide a framework for modeling local search algorithms for addressing intractable discrete optimization problems. Measures for assessing the finite-time performance of GHC algorithms have been developed using this framework, including the expected number of iterations to visit a predetermined objective function value level. This paper analyzes how the expected number of iterations to visit a predetermined objective function value level can be estimated for cyclical simulated annealing. Cyclical simulated annealing uses a cooling schedule that cycles through a set of temperature values. Computational results with traveling salesman problem instances taken from TSPLIB show how the expected number of iterations to visit solutions with predetermined objective function levels can be estimated for cyclical simulated annealing.

Tabu Guided Generalized Hill Climbing Algorithms

This paper formulates tabu search strategies that guide generalized hill climbing (GHC) algorithms for addressing NP-hard discrete optimization problems. The resulting framework, termed tabu guided generalized hill climbing (TG2HC) algorithms, uses a tabu release parameter that probabilistically accepts solutions currently on the tabu list. TG2HC algorithms are modeled as a set of stationary Markov chains, where the tabu list is fixed for each outer loop iteration. This framework provides practitioners with guidelines for developing tabu search strategies to use in conjunction with GHC algorithms that preserve some of the algorithms’ known performance properties. In particular, sufficient conditions are obtained that indicate how to design iterations of problem-specific tabu search strategies, where the stationary distributions associated with each of these iterations converge to the distribution with zero weight on all non-optimal solutions.

Analyzing the Performance of Generalized Hill Climbing Algorithms

Generalized hill climbing algorithms provide a framework to describe and analyze metaheuristics for addressing intractable discrete optimization problems. The performance of such algorithms can be assessed asymptotically, either through convergence results or by comparison to other algorithms. This paper presents necessary and sufficient convergence conditions for generalized hill climbing algorithms. These conditions are shown to be equivalent to necessary and sufficient convergence conditions for simulated annealing when the generalized hill climbing algorithm is restricted to simulated annealing. Performance measures are also introduced that permit generalized hill climbing algorithms to be compared using random restart local search. These results identify a solution landscape parameter based on the basins of attraction for local optima that determines whether simulated annealing or random restart local search is more effective in visiting a global optimum. The implications and limitations of these results are discussed.

Global Optimization Performance Measures for Generalized Hill Climbing Algorithms

Generalized hill climbing algorithms provide a framework for modeling several local search algorithms for hard discrete optimization problems. This paper introduces and analyzes generalized hill climbing algorithm performance measures that reflect how effectively an algorithm has performed to date in visiting a global optimum and how effectively an algorithm may perform in the future in visiting such a solution. These measures are also used to obtain a necessary asymptotic convergence (in probability) condition to a global optimum, which is then used to show that a common formulation of threshold accepting does not converge. These measures assume particularly simple forms when applied to specific search strategies such as Monte Carlo search and threshold accepting.

On the convergence of generalized hill climbing algorithms

Generalized hill climbing (GHC) algorithms provide a general local search strategy to address intractable discrete optimization problems. GHC algorithms include as special cases stochastic local search algorithms such as simulated annealing and the noising method, among others. In this paper, a proof of convergence of GHC algorithms is presented, that relaxes the sufficient conditions for the most general convergence proof for stochastic local search algorithms in the literature. Note that classical convergence proofs for stochastic local search algorithms require either that an exponential distribution be used to model the acceptance of candidate solutions along a search trajectory, or that the Markov chain model of the algorithm must be reversible. The proof in this paper removes these limitations, by introducing a new path concept between global and local optima. Convergence is based on the asymptotic behavior of path probabilities between local and global optima. Examples are given to illustrate the convergence conditions. Implications of this result are also discussed.

Finite-Time Performance Analysis of Static Simulated Annealing Algorithms

Generalized hill climbing (GHC) algorithms provide a framework for modeling local search algorithms for addressing intractable discrete optimization problems. Current theoretical results are based on the assumption that the goal when addressing such problems is to find a globally optimal solution. However, from a practical point of view, solutions that are close enough to a globally optimal solution (where close enough is measured in terms of the objective function value) for a discrete optimization problem may be acceptable. This paper introduces β-acceptable solutions, where β is a value greater than or equal to the globally optimal objective function value. Moreover, measures for assessing the finite-time performance of GHC algorithms, in terms of identifying β-acceptable solutions, are defined. A variation of simulated annealing (SA), termed static simulated annealing (S2A), is analyzed using these measures. S2A uses a fixed cooling schedule during the algorithm's execution. Though S2A is provably nonconvergent, its finite-time performance can be assessed using the finite-time performance measures defined in terms of identifying β-acceptable solutions. Computational results with a randomly generated instance of the traveling salesman problem are reported to illustrate the results presented. These results show that upper and lower estimates for the number of iterations to reach a β-acceptable solution within a specified number of iterations can be obtained, and that these estimates are most accurate for moderate and high fixed temperature values for the S2A algorithm.

A class of convergent generalized hill climbing algorithms

Generalized hill climbing (GHC) algorithms have been presented as a modeling framework for local search strategies applied to address intractable discrete optimization (minimization) problems. GHC algorithms include simulated annealing (SA), pure local search (LS), and threshold accepting (TA), among others, as special cases. A particular class of GHC algorithms is designed for discrete optimization problems where the objective function value of a globally optimal solution is known (in this case, the task is to identify an associated optimal solution). This class of GHC algorithms is shown to converge, and six examples are provided that illustrate the diversity of GHC algorithms within this class of convergent algorithms. Implications of these results are discussed.

A convergence analysis of generalized hill climbing algorithms

Generalized hill climbing (GHC) algorithms provide a well-defined framework for describing the performance of local search algorithms for discrete optimization problems. Necessary and sufficient convergence conditions for GHC algorithms are presented. These convergence conditions are derived using a new iteration classification scheme for GHC algorithms. The implications of the necessary and the sufficient convergence conditions fur GHC algorithms with respect to existing convergence theory for simulated annealing are also discussed.

A New Neighborhood Function for Discrete Manufacturing Process Design Optimization Using Generalized Hill Climbing Algorithms

Discrete manufacturing process design optimization can be difficult, due to the large number of manufacturing process design sequences and associated input parameter setting combinations that exist. Generalized hill climbing algorithms have been introduced to address such manufacturing design problems. Initial results with generalized hill climbing algorithms required the manufacturing process design sequence to be fixed, with the generalized hill climbing algorithm used to identify optimal input parameter settings. This paper introduces a new neighborhood function that allows generalized hill climbing algorithms to be used to also identify the optimal discrete manufacturing process design sequence among a set of valid design sequences. The neighborhood function uses a switch function for all the input parameters, hence allows the generalized hill climbing algorithm to simultaneously optimize over both the design sequences and the inputs parameters. Computational results are reported with an integrated blade rotor discrete manufacturing process design problem under study at the Materials Process Design Branch of the Air Force Research Laboratory, Wright Patterson Air Force Base (Dayton, Ohio, USA). [S1050-0472(00)01002-3]

Expert system approach in design of mechanical components

The present investigation is aimed toward the development of knowledge-based aids for the design of mechanical systems. We have developed and implemented the knowledgebased aid system, which includes MEET and DPMED. The basic approach of MEET follows along the lines ofDesign=Refinement+ Constraint Propagation. This approach has been proven successful in the circuit design domain. Our attempts to utilize MEET have convinced us that we need to extend this methodology to solve mechanical design problems. The DPMED methodology has been applied to design gear-pairs, v-belts, bearings, and shafts. Rules for selecting materials, critical design criteria, and so on are incorporated as part of the rule-system. In order for DPMED to select the design parameter values within the feasible design space, design criteria need to be investigated. Based on these criteria and input/output specifications, DPMED attempts to perform parameter selections. DPMED uses a general hill-climbing algorithm to guide the search.

Optimal classification into groups: An approach for solving the taxonomy problem

Many problems which involve organizing data into homogeneous groups can be approached by defining a function for evaluating the amount of structure present in a given partition of the set of data, and then attempting to find that partition for which the function is optimized. In general, different functions are required for different types of problems. Two possible measures of structure are presented here, together with a discussion of the types of problems to which each applies. Also described is a general hill-climbing algorithm which can be used with any measure of structure to attempt to climb to the optimum partition. The algorithm does not require a numerical measure, but only a decision as to which of two partitions is more highly structured, or more valuable. Areas of application include biological taxonomy; isolation of disease syndromes in medicine; information retrieval; business applications such as “types” of sales offices, TV audiences, etc.; anthropology (categorization of civilizations); and sociology (categorization of tribes). An analysis of results on a set of botanical data, written by Dr Ghillean Prance of the New York Botanical Gardens, is included as an Appendix.

An Approach to Organizing Data into Homogeneous Groups

Many problems which involve organizing data into homogeneous groups can be approached by defining a function for measuring or evaluating the structure present in a given partition of the set of data, and then attempting to find that partition for which the function is optimized. In general, different functions are required for different types of problems. One possible measure of structure is presented here, together with a discussion of the types of problems to which it applies. Also described is a general hill-climbing algorithm which can be used with any measure of structure to attempt to climb to the optimum partition. The algorithm does not require a numerical measure, but only a decision as to which of two partitions is more highly structured, or more valuable. Areas of application include biological taxonomy; isolation of disease syndromes in medicine; information retrieval; business applications such as “types” of sales offices, TV audiences, etc.; anthropology (categorization of civilizations); and sociology (categorization of tribes).