Asynchronous Advantage Actor-Critic Algorithms Based on Residual Networks

Abstract

Deep reinforcement learning is one of the fastest-growing technologies in machine learning. The Asynchronous Advantage Actor-Critic algorithm completely uses the actor-critic framework and utilizes the idea of asynchronous training, which greatly speeds up the training and improves performance. Although A3C algorithm puts actor-critic into multiple threads to train synchronously, effectively utilizes computer resources and improves training effectiveness, it is still difficult to train in deep neural network. Deep networks have proved to be capable of extending to thousands of layers and still have improved performance. However, every one percent increase in accuracy almost doubles the cost of layers, so it is not easy for A3C to train both actor and critic networks. In response to this problem, we innovatively utilize the residual network to apply to the asynchronous advantage actor-critic algorithm and has achieved improvement greatly in the inverted pendulum problem.