A Review of Personalized Federated Reinforcement Learning

Abstract

Reinforcement learning and federated learning both provide strong theoretical support for the study of artificial intelligence. In recent years, an emerging federated reinforcement learning paradigm has been proposed and widely studied and applied. However, in the federated reinforcement learning architecture, the environment, data type and device performance of different agents may be different, which is called heterogeneity. The existence of heterogeneity factors may lead to slow convergence speed of the algorithm, poor generalization, and suboptimal quality of the trained model. Therefore, how to solve the negative impact of the heterogeneity problem on model training has become a hot content of research, and the most important method is to train personalized models for clients. This paper introduces the theory of federated reinforcement learning, as well as methods to cope with heterogeneity in federated reinforcement learning, and provides an overview of the applications of federated reinforcement learning. We conclude the paper with a summary and future perspectives.