Q-learning embedded sine cosine algorithm (QLESCA)

Abstract

Sine Cosine Algorithm (SCA) was recognized as a lightweight, efficient, and has a clear math principal optimizer. However, SCA still suffers from a set of problems such as stagnation at local optima, a slow convergence curve, and a lack of efficient balancing between exploration and exploitation search modes. To mitigate these limitations and improve SCA performance, this study introduces a new version of SCA called QLESCA that smartly controls SCA parameters through an embedded Q-learning algorithm at run time. Each QLESCA agent evolves independently, and it has its own Q-table. The Q-table contains nine different states computed based on population density and distance from the micro population leader. As such, nine different actions are generated by Q-table to control QLESCA parameters, namely r1 and r3. These QLESCA parameters are responsible for adaptive switching from exploration/exploitation and vice versa. For each Q-table action, a reward value is given to a well-performing agent and a penalty to a non-performing agent. To verify the proposed algorithm's performance, QLESCA was evaluated with 23 continuous benchmarks, 20 large scale benchmark optimization functions, and three engineering design problems. In addition, the conducted analysis was compared with various SCA variant algorithms and other state-of-the-art swarm-based optimization methods. The numerical results demonstrate that the QLESCA was superior in terms of achieved fitness value. Statistical results confirm that QLESCA significantly outperforms other optimization algorithms. Additionally, the convergence curve outcomes show that the proposed QLESCA optimization obtains fast convergence against other conducted algorithms.