Learning to Discover Task-Relevant Features for Interpretable Reinforcement Learning

Abstract

Reinforcement Learning (RL) agents are often fed with large-dimensional observations to achieve the ideal performance in complex environments. Unfortunately, the massive observation space usually contains useless or even adverse features, which leads to low sample efficiency. Existing methods rely on domain knowledge and cross-validation to discover efficient features which are informative for decision-making. To minimize the impact of prior knowledge, we propose a temporal-adaptive feature attention algorithm (TAFA). We adopt a non-linear attention module, automatically choosing task-relevant components of hand-crafted state features without any domain knowledge. Our experiments on MuJoCo and TORCS tasks show that the agent achieves competitive performance with state-of-the-art methods while successfully identifying the most task-relevant features for free. We believe our work takes a step towards the interpretability of RL. Our code is available at https://github.com/QiyuanZhang19/Temporal-Adaptive-Feature-Attention/tree/master.