Hyper-heuristic Approach Research Articles

Low-cost heuristics for matrix bandwidth reduction combined with a Hill-Climbing strategy

This paper studies heuristics for the bandwidth reduction of large-scale matrices in serial computations. Bandwidth optimization is a demanding subject for a large number of scientific and engineering applications. A heuristic for bandwidth reduction labels the rows and columns of a given sparse matrix. The algorithm arranges entries with a nonzero coefficient as close to the main diagonal as possible. This paper modifies an ant colony hyper-heuristic approach to generate expert-level heuristics for bandwidth reduction combined with a Hill-Climbing strategy when applied to matrices arising from specific application areas. Specifically, this paper uses low-cost state-of-the-art heuristics for bandwidth reduction in tandem with a Hill-Climbing procedure. The results yielded on a wide-ranging set of standard benchmark matrices showed that the proposed strategy outperformed low-cost state-of-the-art heuristics for bandwidth reduction when applied to matrices with symmetric sparsity patterns.

Evaluation of Multi-Objective Optimization Algorithms for NMR Chemical Shift Assignment.

An automated NMR chemical shift assignment algorithm was developed using multi-objective optimization techniques. The problem is modeled as a combinatorial optimization problem and its objective parameters are defined separately in different score functions. Some of the heuristic approaches of evolutionary optimization are employed in this problem model. Both, a conventional genetic algorithm and multi-objective methods, i.e., the non-dominated sorting genetic algorithms II and III (NSGA2 and NSGA3), are applied to the problem. The multi-objective approaches consider each objective parameter separately, whereas the genetic algorithm followed a conventional way, where all objectives are combined in one score function. Several improvement steps and repetitions on these algorithms are performed and their combinations are also created as a hyper-heuristic approach to the problem. Additionally, a hill-climbing algorithm is also applied after the evolutionary algorithm steps. The algorithms are tested on several different datasets with a set of 11 commonly used spectra. The test results showed that our algorithm could assign both sidechain and backbone atoms fully automatically without any manual interactions. Our approaches could provide around a 65% success rate and could assign some of the atoms that could not be assigned by other methods.

A simple hyper-heuristic approach for a variant of many-to-many hub location-routing problem

A simple hyper-heuristic approach for a variant of many-to-many hub location-routing problem

Complex University Timetabling Using Iterative Forward Search Algorithm and Great Deluge Algorithm

University timetabling is an issue that has received more attention in the field of operations research. Course scheduling is the process of arranging time slots and room for a class by paying attention to existing limitations. This problem is an NP-Hard problem, which means the computation time to find a solution increases exponentially with the size of the problem. Solutions to problems of this kind generally use a heuristic approach, which tries to find a sufficiently good (not necessarily optimal) solution in a reasonable time. We go through two stages in solving the timetabling problem. The first stage is to schedule all classes without breaking any predefined rules. The second stage optimizes the timetable generated in the first stage. This study attempts to solve the class timetabling problem issued in a competition called the 2019 International Timetabling Competition (ITC 2019). In the first stage, we use the Iterative Forward Search (IFS) algorithm to eliminate timetable candidates and to generate a schedule. In the second stage, we employ the Great Deluge algorithm with a hyper-heuristic approach to optimize the solution produced in the first stage. We have tested the method using 30 datasets by taking 1,000,000 iterations on each dataset. The result is an application that does schedule elimination and uses the IFS algorithm to produce a schedule that does not violate any of the hard constraints on 30 ITC 2019 datasets. The implementation of the Great Deluge algorithm optimizes existing schedules with an average penalty reduction of 42%.

Genetic programming-based hyper-heuristic approach for solving dynamic job shop scheduling problem with extended technical precedence constraints

Extended technical precedence constraints (ETPC) in dynamic job shop scheduling problem (DJSP) are the precedence constraints existing between different jobs instead of the conventional technical precedence constraints existing in the operations of the same job. This paper presents the mathematical programming model of the DJSP with ETPC to minimize the mean weighted tardiness of the jobs. The mathematical model contributes to the solution and modelling of the DJSP with ETPC and it is used to solve small-sized problems to optimality. To solve industry-sized problems, a constructive heuristic called the dispatching rule (DR) is employed. This paper investigates the use of genetic programming (GP) as a hyper-heuristic in the automated generation of the problem-specific DRs for solving the problem under consideration. The genetic programming-based hyper heuristic (GPHH) approach constructs the DRs which are learned from the training instances and then verified on the test instances by the simulation experiments. To enhance the efficiency of the approach when evolving effective DRs to solve the problem, the approach is improved with strategies which consist of a problem-specific attribute selection for GP and a threshold condition mechanism for fitness evaluation. The simulation results verify the effectiveness and efficiency of the evolved DRs to the problem under consideration by comparing against the existing classical DRs. The statistical analysis of the simulation results shows that the evolved DRs outperform the selected benchmark DRs on the problem under study. The sensitivity analysis also shows that the DRs generated by the GPHH approach are robust under different scheduling performance measures. Moreover, the effects of the model parameters, including the percentage of jobs with ETPC and the machine utilization, on the performance of the DRs are investigated.

Q-learning and hyper-heuristic based algorithm recommendation for changing environments

A considerable amount of research has been devoted to solving static optimization problems via bio-inspired metaheuristic algorithms. However, most of the algorithms assume that all problem-related data remain unchanged during the optimization process, which is not a realistic assumption. Recently, dynamic optimization problems (DOPs) grabbed remarkable attention from the research community. However, the literature still lacks clear guidelines on selecting the most appropriate bio-inspired algorithm under changing environments. Due to the availability of many design choices, the selection of a suitable bio-inspired metaheuristic algorithm becomes an immediate challenge. This study proposes an algorithm recommendation architecture using Q-learning and hyper-heuristic approaches to help decision-makers select the most suitable bio-inspired algorithm for a given problem. To this end, Artificial Bee Colony (ABC), Manta Ray Foraging Optimization (MRFO), Salp Swarm Algorithm (SSA), and Whale Optimization Algorithm (WOA) are employed as low-level optimizers so that the Q-learning and hyper-heuristic automatically select the optimizer in each cycle of the optimization process. The proposed algorithms are implemented in dynamic multidimensional knapsack problems, a natural extension of the well-known 0–1 knapsack problem. The performances of the recommender and standalone bio-inspired algorithms are evaluated through a comprehensive experimental analysis including appropriate statistical tests. Thus, the significant differences among the algorithms are revealed. The obtained results point out the efficiencies of the Q-learning-based algorithm recommender and MRFO in solving the dynamic multidimensional knapsack problem.

A decomposition-based multi-objective genetic programming hyper-heuristic approach for the multi-skill resource constrained project scheduling problem

In this paper, an efficient decomposition-based multi-objective genetic programming hyper-heuristic (MOGP-HH/D) approach is proposed for the multi-skill resource constrained project scheduling problem (MS-RCPSP) with the objectives of minimizing the makespan and the total cost simultaneously. First, the decomposition mechanism is presented to improve the diversity of solutions. Second, a single-list encoding scheme and an improved repair-based decoding scheme are designed to represent individuals and construct feasible schedules, respectively. Third, ten adaptive heuristics are developed elaborately to constitute a list of low-level heuristics (LLHs). Fourth, genetic programming is employed as the high-level heuristic (HLH) to generate a promising heuristics sequence from the LLHs set flexibly. Finally, the Taguchi method of design-of-experiment (DOE) is conducted to analyze the performance of parameter settings. The effectiveness of MOGP-HH/D is evaluated on a typical benchmark dataset and computational results exhibit the superiority of the proposed algorithm over the existing methods in solving multi-objective MS-RCPSP.

Ensemble bat algorithm based on Hyper heuristic approach for solving unconstrained optimization problems

Abstract Maintaining convergence and diversification in solving optimization is one of the most important challenges facing metaheuristic algorithms in general and the bat algorithm in particular. Many researchers have suggested some improvements to preserve the ability of the algorithm to find good solutions in a timely manner and also to move away as much as possible from landing on the local optimization zone. In this paper, a hyper-heuristic method was proposed to incorporate the behavior of three optimized algorithms from the bat algorithm. The method is based on the distribution of a specific implementation probability for each used algorithm and then updating this probability iteratively according to the results of each algorithm, and then we use random selection to determine the algorithm used in the current iteration. Some nonlinear models proposed in CEC2005 used to compare the efficiency of the proposed algorithm and compare its results with some state-of-the-art algorithms.

Surrogate-Assisted Evolutionary Multitask Genetic Programming for Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (JSS) is an important combinatorial optimization problem with complex routing and sequencing decisions under dynamic environments. Genetic programming (GP), as a hyperheuristic approach, has been successfully applied to evolve scheduling heuristics for JSS. However, its training process is time consuming, and it faces the retraining problem once the characteristics of job shop scenarios vary. It is known that multitask learning is a promising paradigm for solving multiple tasks simultaneously by sharing knowledge among the tasks. To improve the training efficiency and effectiveness, this article proposes a novel surrogate-assisted evolutionary multitask algorithm via GP to share useful knowledge between different scheduling tasks. Specifically, we employ the phenotypic characterization for measuring the behaviors of scheduling rules and building a surrogate for each task accordingly. The built surrogates are used not only to improve the efficiency of solving each single task but also for knowledge transfer in multitask learning with a large number of promising individuals. The results show that the proposed algorithm can significantly improve the quality of scheduling heuristics for all scenarios. In addition, the proposed algorithm manages to solve multiple tasks collaboratively in terms of the evolved scheduling heuristics for different tasks in a multitask scenario.

Sentinel: A Hyper-Heuristic for the Generation of Mutant Reduction Strategies

Mutation testing is an effective approach to evaluate and strengthen software test suites, but its adoption is currently limited by the mutants' execution computational cost. Several strategies have been proposed to reduce this cost (a.k.a. mutation cost reduction strategies), however none of them has proven to be effective for all scenarios since they often need an ad-hoc manual selection and configuration depending on the software under test (SUT). In this paper, we propose a novel multi-objective evolutionary hyper-heuristic approach, dubbed Sentinel, to automate the generation of optimal cost reduction strategies for every new SUT. We evaluate Sentinel by carrying out a thorough empirical study involving 40 releases of 10 open-source real-world software systems and both baseline and state-of-the-art strategies as a benchmark. We execute a total of 4,800 experiments, and evaluate their results with both quality indicators and statistical significance tests, following the most recent best practice in the literature. The results show that strategies generated by Sentinel outperform the baseline strategies in 95% of the cases always with large effect sizes. They also obtain statistically significantly better results than state-of-the-art strategies in 88% of the cases, with large effect sizes for 95% of them. Also, our study reveals that the mutation strategies generated by Sentinel for a given software version can be used without any loss in quality for subsequently developed versions in 95% of the cases. These results show that Sentinel is able to automatically generate mutation strategies that reduce mutation testing cost without affecting its testing effectiveness (i.e. mutation score), thus taking off from the tester's shoulders the burden of manually selecting and configuring strategies for each SUT.

Correlation Coefficient-Based Recombinative Guidance for Genetic Programming Hyperheuristics in Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (JSS) is a challenging combinatorial optimization problem due to its complex environment. In this problem, machine assignment and operation sequencing decisions need to be made simultaneously under the dynamic environments. Genetic programming (GP), as a hyperheuristic approach, has been successfully used to evolve scheduling heuristics for dynamic flexible JSS. However, in traditional GP, recombination between parents may disrupt the beneficial building blocks by choosing the crossover points randomly. This article proposes a recombinative mechanism to provide guidance for GP to realize effective and adaptive recombination for parents to produce offspring. Specifically, we define a novel measure for the importance of each subtree of an individual, and the importance information is utilized to decide the crossover points. The proposed recombinative guidance mechanism attempts to improve the quality of offspring by preserving the promising building blocks of one parent and incorporating good building blocks from the other. The proposed algorithm is examined on six scenarios with different configurations. The results show that the proposed algorithm significantly outperforms the state-of-the-art algorithms on most tested scenarios, in terms of both final test performance and convergence speed. In addition, the rules obtained by the proposed algorithm have good interpretability.

A hyper-heuristic approach based upon a hidden Markov model for the multi-stage nurse rostering problem

The importance of the nurse rostering problem in complex healthcare environments should not be understated. The nurses in a hospital should be assigned to the most appropriate shifts and days so as to meet the demands of the hospital as well as to satisfy the requirements and requests of the nurses as much as possible. Nurse rostering represents a challenging and demanding combinatorial optimisation problem. To address it, general and efficient methodologies, such as selection hyper-heuristics, have emerged. In this paper, we will consider the multi-stage nurse rostering formulation, posed by the second international nurse rostering competition’s problem. We introduce a sequence-based selection hyper-heuristic that utilises a statistical Markov model. The proposed methodology incorporates a dedicated algorithm for building feasible initial solutions and a series of low-level heuristics with different dynamics that respect the characteristics of the competition’s problem formulation. Empirical results and analysis suggest that the proposed approach has significant potential for difficult problem instances.

HYPER HEURISTIC EVOLUTIONARY APPROACH FOR CONSTRUCTING DECISION TREE CLASSIFIERS

Decision tree models have earned a special status in predictive modeling since these are considered comprehensible for human analysis and insight. Classification and Regression Tree (CART) algorithm is one of the renowned decision tree induction algorithms to address the classification as well as regression problems. Finding optimal values for the hyper parameters of a decision tree construction algorithm is a challenging issue. While making an effective decision tree classifier with high accuracy and comprehensibility, we need to address the question of setting optimal values for its hyper parameters like the maximum size of the tree, the minimum number of instances required in a node for inducing a split, node splitting criterion and the amount of pruning. The hyper parameter setting influences the performance of the decision tree model. As researchers, we know that no single setting of hyper parameters works equally well for different datasets. A particular setting that gives an optimal decision tree for one dataset may produce a sub-optimal decision tree model for another dataset. In this paper, we present a hyper heuristic approach for tuning the hyper parameters of Recursive and Partition Trees (rpart), which is a typical implementation of CART in statistical and data analytics package R. We employ an evolutionary algorithm as hyper heuristic for tuning the hyper parameters of the decision tree classifier. The approach is named as Hyper heuristic Evolutionary Approach with Recursive and Partition Trees (HEARpart). The proposed approach is validated on 30 datasets. It is statistically proved that HEARpart performs significantly better than WEKA’s J48 algorithm in terms of error rate, F-measure, and tree size. Further, the suggested hyper heuristic algorithm constructs significantly comprehensible models as compared to WEKA’s J48, CART and other similar decision tree construction strategies. The results show that the accuracy achieved by the hyper heuristic approach is slightly less as compared to the other comparative approaches.

A Survey of University Course Timetabling Problem: Perspectives, Trends and Opportunities

The timetabling problem is common to academic institutions such as schools, colleges or universities. It is a very hard combinatorial optimisation problem which attracts the interest of many researchers. The university course timetabling problem (UCTTP) is difficult to address due to the size of the problem and several challenging hard and soft constraints. Over the years, various methodologies were proposed to solve UCTTP. The purpose of this survey paper is to provide the most recent scientific review of the methodologies applied to UCTTP. The paper unveils a classification of methodologies proposed in recent years based on chronology and datasets used. Perspectives, trends, challenges and opportunities in UCTTP are also presented. It is observed that meta-heuristic approaches are popular among researchers. This is followed closely by hybrid methodologies. Hyper-heuristic approaches are also able to produce effective results. Another observation is that the state-of-art methodologies in the scientific literature are not fully utilised in a real-world environment perhaps due to the limited flexibility of these methodologies.

A Methodology to Determine the Subset of Heuristics for Hyperheuristics through Metalearning for Solving Graph Coloring and Capacitated Vehicle Routing Problems

In this work, we focus on the problem of selecting low‐level heuristics in a hyperheuristic approach with offline learning, for the solution of instances of different problem domains. The objective is to improve the performance of the offline hyperheuristic approach, identifying equivalence classes in a set of instances of different problems and selecting the best performing heuristics in each of them. A methodology is proposed as the first step of a set of instances of all problems, and the generic characteristics of each instance and the performance of the heuristics in each one of them are considered to define the vectors of characteristics and make a grouping of classes. Metalearning with statistical tests is used to select the heuristics for each class. Finally, we used the Naive Bayes to test the set instances with k‐fold cross‐validation, and we compared all results statistically with the best‐known values. In this research, the methodology was tested by applying it to the problems of capacitated vehicle routing (CVRP) and graph coloring (GCP). The experimental results show that the proposed methodology can improve the performance of the offline hyperheuristic approach, correctly identifying the classes of instances and applying the appropriate heuristics in each case. This is based on the statistical comparison of the results obtained with those of the state of the art of each instance.

Enhancing Hyperheuristics for the Knapsack Problem through Fuzzy Logic.

Hyperheuristics rise as powerful techniques that get good results in less computational time than exact methods like dynamic programming or branch and bound. These exact methods promise the global best solution, but with a high computational time. In this matter, hyperheuristics do not promise the global best solution, but they promise a good solution in a lot less computational time. On the contrary, fuzzy logic provides the tools to model complex problems in a more natural way. With this in mind, this paper proposes a fuzzy hyperheuristic approach, which is a combination of a fuzzy inference system with a selection hyperheuristic. The fuzzy system needs the optimization of its fuzzy rules due to the lack of expert knowledge; indeed, traditional hyperheuristics also need an optimization of their rules. The fuzzy rules are optimized by genetic algorithms, and for the rules of the traditional methods, we use particle swarm optimization. The genetic algorithm will also reduce the number of fuzzy rules, in order to find the best minimal fuzzy rules, whereas traditional methods already use very few rules. Experimental results show the advantage of using our approach instead of a traditional selection hyperheuristic in 3200 instances of the 0/1 knapsack problem.

A STUDY OF THE EFFECT OF THE PARAMETERS FOR OPTIMIZING PROFIT USING SIMULATED ANNEALING TO SOLVE SHELF SPACE ALLOCATION PROBLEM

Shelf space allocation has always remained a crucial issue for any retail store, as space is a limited resource. This work proposes a model that uses a hyper-heuristic approach to allocate products on shelves to maximize the retailer's profit. This work has concentrated on providing a solution specifically for a consumer packaged goods store. There exist multiple conflicting objectives and constraints which influence the profit. The consequence is a non-linear programming model having a complex objective function, which is solved by using multiple neighborhood approaches using simulated annealing as simulated annealing is a useful tool for solving complex combinatorial optimization problems. Detailed analysis of the proposed technique of using annealing and reheating has revealed the effectiveness in profit maximization in the shelf space allocation problem. Various simulated annealing parameters have been studied in this paper, which provides optimum values for maximizing profit.

A hyper-heuristic based ensemble genetic programming approach for stochastic resource constrained project scheduling problem

In project scheduling studies, to the best of our knowledge, the hyper-heuristic collaborative scheduling is first-time applied to project scheduling with random activity durations. A hyper-heuristic based ensemble genetic programming (HH-EGP) method is proposed for solving stochastic resource constrained project scheduling problem (SRCPSP) by evolving an ensemble of priority rules (PRs). The proposed approach features with (1) integrating the critical path method into the resource-based policy class to generate schedules; (2) improving the existing single hyper-heuristic project scheduling research to construct a suitable solution space for solving SRCPSP; and (3) bettering genetic evolution of each subpopulation from a decision ensemble with three different local searches in corporation with discriminant mutation and discriminant population renewal. In addition, a sequence voting mechanism is designed to deal with collaborative decision-making in the scheduling process for SRCPSP. The benchmark PSPLIB is performed to verify the advantage of the HH-EGP over heuristics, meta-heuristics and the single hyper-heuristic approaches.

Meta Analysis of Heuristic Approaches for Optimizing Node Localization and Energy Efficiency in Wireless Sensor Networks

Background: In the literature node localization and energy efficiency are intrinsic problems often experienced in wireless sensor networks (WSNs). Consequently, various heuristic approaches have been proposed to allay the challenges faced by WSNs. However, there is little to nothing in the literature to support which of the heuristic approaches is best in optimizing node localization and energy efficiency problems in WSN. The aim of this paper is to assess the best heuristic approach to date on resolving the node localization and energy efficiency in WSNs. Method: The extraction of the relevant articles was designed following the technique of preferred reporting items for systematic reviews and meta-analyses (PRISMA). All the included research articles were searched from the widely used databases of Google Scholar and Web of Science. All statistical analysis was performed with the fixed-effects model and the random-effects model implementation in RStudio. The overall pooled global estimate and categorization of performance for the heuristic approaches were presented in forest plots. Results: A total of 18 studies were included in this meta-analysis and the overall pooled estimated categorization of the heuristic approaches was 35% (95% CI (13%, 67%)). According to subgroup analysis the pooled estimation of heuristic approach with hyper-heuristic was 71% (95% CI: 6% to 99%), I2 = 100%) while the hybrid heuristic, was 31% (95% CI: 3% to 87%, I2 = 100%) and metaheuristic was 21%(95% CI: 9% to 41%, I2 = 100%). Conclusion: It can be concluded based on the experimental results that hyper-heuristic approach outclassed the hybrid heuristic and metaheuristic approaches in optimizing node localization and energy efficiency in WSNs.

Hyper-Heuristics based on Reinforcement Learning, Balanced Heuristic Selection and Group Decision Acceptance

In this paper, we introduce a multi-objective selection hyper-heuristic approach combining Reinforcement Learning, (meta)heuristic selection, and group decision-making as acceptance methods, referred to as Hyper-Heuristic based on Reinforcement LearnIng, Balanced Heuristic Selection and Group Decision AccEptance (HRISE), controlling a set of Multi-Objective Evolutionary Algorithms (MOEAs) as Low-Level (meta)Heuristics (LLHs). Along with the use of multiple MOEAs, we believe that having a robust LLH selection method as well as several move acceptance methods at our disposal would lead to an improved general-purpose method producing most adequate solutions to the problem instances across multiple domains. We present two learning hyper-heuristics based on the HRISE framework for multi-objective optimisation, each embedding a group decision-making acceptance method under a different rule: majority rule (HRISE_M) and responsibility rule (HRISE_R). A third hyper-heuristic is also defined where both a random LLH selection and a random move acceptance strategy are used. We also propose two variants of the late acceptance method and a new quality indicator supporting the initialisation of selection hyper-heuristics using low computational budget. An extensive set of experiments were performed using 39 multi-objective problem instances from various domains where 24 are from four different benchmark function classes, and the remaining 15 instances are from four different real-world problems. The cross-domain search performance of the proposed learning hyper-heuristics indeed turned out to be the best, particularly HRISE_R, when compared to three other selection hyper-heuristics, including a recently proposed one, and all low-level MOEAs each run in isolation.