Remote Reinforcement Learning over a Noisy Channel

Abstract

A collaborative multi-agent reinforcement learning (RL) problem is considered, where agents communicate over a noisy communication channel towards achieving a common goal. In particular, we consider a remote-controlled version of a single-agent RL problem, in which the system state is observed by a guide agent, while the actions are taken by a scout. The guide can communicate to the scout over a noisy communication link, reminiscent of a remote-controlled version of the single-agent RL problem. This transformation turns the original single-agent Markov decision process (MDP) into a two-agent partially observable MDP (POMDP). In conventional systems, communication and learning tasks are taken care of separately. We show the suboptimality of this approach, and propose a deep Q-learning solution that aims at learning the optimal policy taking into account the channel impairments.