Problem feature based meta-heuristics with Q-learning for solving urban traffic light scheduling problems

Abstract

An urban traffic light scheduling problem (UTLSP) is studied by using problem feature based meta-heuristics with Q-learning. The goal is to minimize the network-wise total delay time within a time window by finding a high-quality schedule of traffic lights. First, a dynamic flow model is used to describe the UTLSP in a scheduling framework. Second, four improved meta-heuristics combining Q-learning are proposed, including harmony search (HS), water cycle algorithm (WCA), Jaya, and artificial bee colony (ABC) algorithms. Five problem feature based local search operators are constructed. During the iterative process, Q-learning is employed to select the local search operators with strong competitiveness. Two ensemble strategies are proposed to combine meta-heuristics and Q-learning. Finally, experiments are conducted based on real traffic data. The performance of the improved meta-heuristics with Q-learning is verified by solving eighteen cases with different scales. Numerical results and comparisons show that the proposed algorithms have statistical improvements over their peers. The proposed feature-based ABC with Q-learning has the strongest competitiveness among all compared ones.