Population based Reinforcement Learning

Abstract

Genetic algorithms have recently seen an increase in application due to their highly scalable nature. Enabling more efficient utilization of processing power that has become readily available. This study introduces Population based reinforcement learning (PBRL), a method that hybridizes a GA with a policy gradient reinforcement learning algorithm. This combination not only enables more scalable policy optimization, but also helps mitigate some of the common weaknesses of policy gradient algorithms. Furthermore, PBRL is also extended to include automatic hyper-parameter tuning, which is used to evaluate the impact that such tuning can have on the performance of the policy gradient algorithm being used. Experiments comparing these methods are conducted on a number of continuous control problems simulated by MuJoCo. Results show that PBRL is capable of outperforming a commonly used policy gradient algorithm, while also producing results in nearly one fifth the time. It is also observed that the addition of automatic hyperparameter tuning can be greatly beneficial for environments where well tuned hyper-parameters are not known.