Hyper-heuristic Research Articles

A new Hyper-heuristic based on Adaptive Simulated Annealing and Reinforcement Learning for the Capacitated Electric Vehicle Routing Problem

A new Hyper-heuristic based on Adaptive Simulated Annealing and Reinforcement Learning for the Capacitated Electric Vehicle Routing Problem

A reinforcement learning-based hyper-heuristic for AGV task assignment and route planning in parts-to-picker warehouses

A reinforcement learning-based hyper-heuristic for AGV task assignment and route planning in parts-to-picker warehouses

Knowledge-transfer based genetic programming algorithm for multi-objective dynamic agile earth observation satellite scheduling problem

Knowledge-transfer based genetic programming algorithm for multi-objective dynamic agile earth observation satellite scheduling problem

A review of reinforcement learning based hyper-heuristics.

The reinforcement learning based hyper-heuristics (RL-HH) is a popular trend in the field of optimization. RL-HH combines the global search ability of hyper-heuristics (HH) with the learning ability of reinforcement learning (RL). This synergy allows the agent to dynamically adjust its own strategy, leading to a gradual optimization of the solution. Existing researches have shown the effectiveness of RL-HH in solving complex real-world problems. However, a comprehensive introduction and summary of the RL-HH field is still blank. This research reviews currently existing RL-HHs and presents a general framework for RL-HHs. This article categorizes the type of algorithms into two categories: value-based reinforcement learning hyper-heuristics and policy-based reinforcement learning hyper-heuristics. Typical algorithms in each category are summarized and described in detail. Finally, the shortcomings in existing researches on RL-HH and future research directions are discussed.

Multi-armed bandit-based hyper-heuristics for combinatorial optimization problems

Multi-armed bandit-based hyper-heuristics for combinatorial optimization problems

Parallel hyper heuristic algorithm based on reinforcement learning for the corridor allocation problem and parallel row ordering problem

Parallel hyper heuristic algorithm based on reinforcement learning for the corridor allocation problem and parallel row ordering problem

Hyper-heuristics applications to manufacturing scheduling: overview and opportunities

Hyper-heuristics applications to manufacturing scheduling: overview and opportunities

When move acceptance selection hyper-heuristics outperform Metropolis and elitist evolutionary algorithms and when not

Selection hyper-heuristics (HHs) are automated algorithm selection methodologies that choose between different heuristics during the optimisation process. Recently, selection HHs choosing between a collection of elitist randomised local search heuristics with different neighbourhood sizes have been shown to optimise standard unimodal benchmark functions from evolutionary computation in the optimal expected runtime achievable with the available low-level heuristics. In this paper, we extend our understanding of the performance of HHs to the domain of multimodal optimisation by considering a Move Acceptance HH (MAHH) from the literature that can switch between elitist and non-elitist heuristics during the run. In essence, MAHH is a non-elitist search heuristic that differs from other search heuristics in the source of non-elitism.We first identify the range of parameters that allow MAHH to hillclimb efficiently and prove that it can optimise the standard hillclimbing benchmark function OneMax in the best expected asymptotic time achievable by unbiased mutation-based randomised search heuristics. Afterwards, we use standard multimodal benchmark functions to highlight function characteristics where MAHH outperforms elitist evolutionary algorithms and the well-known Metropolis non-elitist algorithm by quickly escaping local optima, and ones where it does not. Since MAHH is essentially a non-elitist random local search heuristic, the paper is of independent interest to researchers in the fields of artificial intelligence and randomised search heuristics.

A hyper-heuristic based reinforcement-learning algorithm to train feedforward neural networks

A hyper-heuristic based reinforcement-learning algorithm to train feedforward neural networks

Selection hyper-heuristics for the multi and many-objective quadratic assignment problem

Selection hyper-heuristics for the multi and many-objective quadratic assignment problem

Beyond Hyper-Heuristics: A Squared Hyper-Heuristic Model for Solving Job Shop Scheduling Problems

Hyper-heuristics (HHs) stand as a relatively recent approach to solving optimization problems. There are different kinds of HHs. One of them deals with how low-level heuristics must be combined to deliver an improved solution to a set of problem instances. Literature commonly refers to them as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">selection hyper-heuristics</i> . One of their advantages is that the strengths of each heuristic can be fused into a high-level solver. However, one of their drawbacks is that sometimes this generalization scheme does not suffice. Additionally, it is not easy to reuse these HHs since the model cannot be easily tweaked. So, in this work, we develop a hyper-heuristic model with an additional layer of generalization. The rationale behind it is to preserve the general structure of selecting an adequate solver for a particular situation but to use HHs instead of low-level heuristics. We call this model a Squared Hyper-Heuristic (SHH). To validate our proposal, we pursue a four-stage methodology that covers several testing scenarios. Our data reveal that, under proper conditions, our model can outperform the base HHs. Moreover, it is flexible enough to allow for an increased number of layers so that the complexity of the final model can be tuned. Additionally, different kinds of instances can be used to train each stage of the model, thus setting the groundwork for developing a transfer learning approach for hyper-heuristics.

A novel hybrid-load AGV for JIT-based sustainable material handling scheduling with time window in mixed-model assembly line

ABSTRACT Since global warming and the needs for sustainable production models, this paper focuses on a Just-in-Time (JIT)-based sustainable material handling scheduling problem (JSMHSP) with time window and capacity constraints for mixed-model assembly lines in the automobile industry. A novel Hybrid-load Automated Guided Vehicle (H-AGV) is proposed to fulfil material handling tasks between supermarkets and assembly lines. The motivation is to minimise the total line-side inventory and the total energy consumption, which corresponds to JIT and environmental objectives. Due to the NP-hard nature of the proposed scheduling problem, a Deep Q network and Non-dominated sorting-based Hyper-Heuristic (DN-HH) algorithm is presented to solve the bi-objective scheduling problem, which benefits from the synergy of the Deep Q Network (DQN) and Hyper-Heuristic (HH). In the DQN, the states and rewards are designed according to the characteristics of the scheduling problem. To improve the performance of DQN, the experience pool (EP) and the target network are presented to improve the convergence speed. Computational results reveal that the proposed DN-HH algorithm outperforms the other two compared algorithms in both solution quality and convergence speed and the performance of the H-AGV is better than that of the other two types of AGVs.

On the cooperation of meta-heuristics for solving many-objective problems: An empirical analysis including benchmark and real-world problems

On the cooperation of meta-heuristics for solving many-objective problems: An empirical analysis including benchmark and real-world problems

Dynamic scheduling of manufacturing systems: a product-driven approach using hyper-heuristics

ABSTRACT Dynamic scheduling of manufacturing systems is encountered in many real-world industries such as the food and pharmaceutical industries. The scheduling of these systems must not only be efficient but also reactive to cope with dynamic job arrivals and machine breakdowns. Over the last decade, products within Product-Driven Control Systems (PDCS) have become smart entities capable of actively handling the manufacturing process. In this paper, a PDCS based on the design of Smart Products is proposed. A generic model of the decisional strategy allows Smart Products to characterize different decisional contexts and thus switch efficiently from one scheduling rule to another using a novel Hyper-Heuristics (HH) based approach. The implementation and testing of the proposed PDCS on hybrid flexible flow-shops with multiple constraints inspired by the pharmaceutical industry are presented. The comparative study with 168 combinations of scheduling rules from the literature highlighted the superiority of the HH to minimize the Mean Completion Time. Furthermore, the proposed approach enhanced both the global performance and the reactivity of the manufacturing control system.

A multi-objective hyper heuristic framework for integrated optimization of carrier-based aircraft flight deck operations scheduling and resource configuration

A multi-objective hyper heuristic framework for integrated optimization of carrier-based aircraft flight deck operations scheduling and resource configuration

Two multi-start heuristics for the k-traveling salesman problem

This paper is concerned with the k-traveling salesman problem (k-TSP), which is a variation of widely studied traveling salesman problem (TSP). Given a set of n cities including a home city and a fixed value k such that $$1 < k \le n$$ , this problem seeks a tour of minimum length which starts and ends at the home city and visits k cities (including the home city) exactly out of these n cities. Finding a feasible solution to k-TSP involves finding a subset of k cities including the home city first, and then a circular permutation representing a tour of these k cities. In this paper, we have proposed two multi-start heuristic approaches for the k-TSP. The first approach is based on general variable neighborhood search algorithm (GVNS), whereas the latter approach is a hyper-heuristic (HH) approach. A variable neighborhood descent strategy operating over two neighborhood structures is utilized for doing the local search in the GVNS. As part of the hyper-heuristic, two low level heuristics are considered. To the best of our knowledge, these are the first metaheuristic and hyper-heuristic approaches for the k-TSP. To evaluate the performance of our approaches, a set of benchmark instances is created utilizing instances from TSPLIB. Computational results on these benchmark instances show HH approach to be better than GVNS approach.

Evaluation of Hyper-Heuristic Method Using Random-Hill Climbing Algorithm in the Examination Timetabling Problem

Examination timetabling is included in the category of Nondeterministic Polynomial-Hard (NP-Hard) problems, namely problems that cannot be solved by conventional methods in finding optimal solutions. One solution to this problem is to use the Simple Random - Hill Climbing - Hyper Heuristic (SR-HC-HH) approach. But this approach still cannot produce an optimum solution. The researcher presents a critical analysis of the performance evaluation of the solution method used. The stages of this research include: (1) problem identification; (2) literature study; (3) data retrieval and understanding; (4) translation of mathematical models into data structures; (5) evaluation of the SR-HC-HH algorithm; (6) trial implementation; (7) algorithmic experiment parameters; and (8) analysis of results and conclusions. SR-HC-HH algorithm used in Hyper-Heuristics-based applications is able to solve timetabling problems in the Examination Timetabling Problem domain in the ITC-2007 dataset, but it is still not optimal. Parameters that can be changed in this study are the number of iterations (time limit) and algorithms for selection of Low Level Heuristics. Changing parameters in the trial scenario can also affect the results of a more optimum solution. The parameters to be explored in this study include LLH selection strategies and move acceptance in hyper-heuristics.

How the Duration of the Learning Period Affects the Performance of Random Gradient Selection Hyper-Heuristics

Recent analyses have shown that a random gradient hyper-heuristic (HH) using randomised local search (RLSk) low-level heuristics with different neighbourhood sizes k can optimise the unimodal benchmark function LeadingOnes in the best expected time achievable with the available heuristics, if sufficiently long learning periods τ are employed. In this paper, we examine the impact of the learning period on the performance of the hyper-heuristic for standard unimodal benchmark functions with different characteristics: Ridge, where the HH has to learn that RLS1 is always the best low-level heuristic, and OneMax, where different low-level heuristics are preferable in different areas of the search space. We rigorously prove that super-linear learning periods τ are required for the HH to achieve optimal expected runtime for Ridge. Conversely, a sub-logarithmic learning period is the best static choice for OneMax, while using super-linear values for τ increases the expected runtime above the asymptotic unary unbiased black box complexity of the problem. We prove that a random gradient HH which automatically adapts the learning period throughout the run has optimal asymptotic expected runtime for both OneMax and Ridge. Additionally, we show experimentally that it outperforms any static learning period for realistic problem sizes.

Applying Social Choice Theory to Solve Engineering Multi-objective Optimization Problems

Multi-objective optimization problems usually do not have a single unique optimal solution, for either discrete or continuous domains. Furthermore, there are usually many possible available algorithms for solving these problems, and one typically does not know in advance which of these will be the most effective for solving a particular problem instance. Hyper-heuristics (HHs) are often used as a means to make this choice. In particular, the underlying idea of HHs is to run several algorithms or heuristics and dynamically decide, based on different criteria, which problem or part of the problem should be solved by which algorithm or heuristic. On the other hand, the domain of social choice theory studies how to design collective decision processes by aggregating individual inputs into collective ones. In this paper, we explore the use of social choice theory in creating HHs. By using HHs based on different voting methods, like Borda, Copeland and Kemeny–Young, we show how we can solve both continuous and discrete engineering multi-objective optimization problems and discuss the results obtained by each of these methods. Our obtained results show that our strategy has found solutions that are at least equals to the ones generated by the best algorithm among the studied ones, and sometimes even overcomes these results.

A Hyper Heuristics Technique for Data Partitioning and Scheduling to Heterogeneous Systems using Genetic Algorithm and Improved Particle Swarm Optimization

Development of the load partitioning for multiple round load distribution and effective scheduling of partitioned load to heterogeneous processor is primary goal of distributed and parallel system. In this paper, we propose hyper heuristics scheduling algorithm for load partitioning using genetic and improved particle swarm optimization techniques.A communication model is used to predict the optimal activation order, optimal number of processor and optimal number of rounds of the load. Heuristics Based Scheduling Algorithm is proposed using Hyper Heuristic Scheduling which is used to find the candidate solution (low level heuristic) to form Scheduling Solutions (heuristics algorithms) for large scale system with diversity operator as sequence dependent and sequence independent scheduling. For this solution, processing time of the entire processing load will be reduced. Hybrid Real Code genetic algorithm(HRGA) computes optimal activation order with cross over and mutation operator without considering the processor latency and different types of variation in the perturbation parameters. In order to optimize this issue, we utilizeImproved Particle swarm optimization (IPSO) determine the load fraction for generating activation order in terms of dynamically predicting fitness value of the processor with certain number. The Simulation analysis demonstrates the proposed model performance in terms of mean, standard deviation, computational complexity and Average Execution Time comparing against hybrid real coded genetic algorithm