Extracting Decision Tree From Trained Deep Reinforcement Learning in Traffic Signal Control

Abstract

Deep reinforcement learning (DRL) has achieved impressive success in traffic signal control systems (TSCS). However, since a key component of many DRL models is the complex deep neural networks (DNNs), it hinders humans or experts from understanding and explaining the learned policy. Recently, many works have focused on developing interpretable techniques to compress or distill complex DNNs into smaller, faster, or more understandable models. The decision trees (DTs) are viewed as the de facto technique for interpretable and transparent machine learning, and can provide an easy-understanding decision path from the root to the leaf node. In this work, we utilize modified DTs to extract models with simpler hierarchical structures from premium policy achieved by DRL methods. First, we use a DRL algorithm to learn a premium policy for traffic signal control. Then, we collect a dataset with the learned premium policy by interacting with the environment. Finally, we extract the DTs with the collected dataset of state-actions pairs. We evaluate our method on a Simulation of Urban Mobility simulator in a simulation way. Simulation results show that the extracted DTs can generate human-understandable decision processes and provide explicit knowledge from the DNNs reference of the DRL.