A Survey of Collaborative Reinforcement Learning: Interactive Methods and Design Patterns

Abstract

Recently, methods enabling humans and Artificial Intelligent (AI) agents to collaborate towards improving the efficiency of Reinforcement Learning - also called Collaborative Reinforcement Learning (CRL) - have been receiving increasing attention. In this paper, we provide a long-term, in-depth survey, investigating human-AI collaborative methods based on both interactive reinforcement learning algorithms and human-AI collaborative frameworks, between 2011 and 2020. We elucidate and discuss synergistic analysis methods of both the growth of the field and the state-of-the-art; we suggest novel technical directions and new collaboration design ideas. Specifically, we provide a new CRL classification taxonomy, as a systematic modelling tool for selecting and improving new CRL designs. Furthermore, we propose generic CRL challenges providing the research community with a guide towards effective implementation of human-AI collaboration. The aim is to empower researchers to develop more efficient and natural human-AI collaborative methods that could utilise the different strengths of humans and AI.