A selection hyper-heuristic algorithm with Q-learning mechanism

Abstract

The selection of an algorithm in the real world of the application domain is a challenging problem as no specific algorithm exists capable of solving all issues to a satisfactory requirement. Selecting a suitable algorithm presents major challenges such as solving problems requiring expert knowledge or trial-and-error algorithms, which have hindered advancements in this field. In this work, we introduce a novel method that uniquely addresses these challenges by integrating hyper-heuristic and Q-learning mechanism techniques. A selection hyper-heuristic algorithm with Q-learning (QLSHH) is proposed to select appropriate low-level heuristic (LLH) for the computation stages of the optimization process. The Q-learning mechanism guided by the feedback of the solution state was designed according to the environment. Four low-level heuristics (LLHs) were proposed according to the optimization mechanism for continuous optimization problems. The QLSHH learns the successful experience in the optimization process through Q-learning to select the appropriate LLH at each decision point. The results tested on the CEC 2017 and CEC 2020 benchmark suite show that the QLSHH outperforms the other nine comparison algorithms on 50% of the functions and the experimental results of algorithm complexity show that the proposed algorithm is the fastest compared with other algorithms.