Problem Feature-Based Meta-Heuristics with Reinforcement Learning for Solving Urban Traffic Light Scheduling Problems

Abstract

This paper studies the urban traffic light scheduling problems (UTLSP). A dynamic flow model is used to describe the UTLSP in a scheduling framework. The goal is to find a reasonable schedule of traffic lights that minimizes the network-wise total delay time within a time window. In this paper, an improved harmony search (HS) and water cycle algorithm (WCA) are proposed for solving the UTLSP. Firstly, five feature-based search operators are developed to improve the search performance of optimization methods. Secondly, an integration of meta-heuristics and reinforcement learning (RL) is proposed, where Q-learning is employed to select the local search operator with strong competitiveness. Finally, experiments are conducted based on real traffic data in Singapore. The HS, WCA, and their variants are evaluated by solving eight transport network cases. Numerical results show that the proposed algorithms are able to obtain statistical improvements compared to their peers.