Transferring Online Reinforcement Learning for Electric Motor Control From Simulation to Real-World Experiments

Abstract

Reinforcement learning (RL) based methods are an upcoming approach for the control of power systems such as electric drives. These data-driven techniques do not need an explicit plant model like most common state-of-the-art approaches. Instead, the control policy is continuously improved solely based on measurement feedback pursuing optimal control performance through learning. While the general feasibility of RL-based drive control algorithms has already been proven in simulation, this work focuses on transferring the methodology to real-world experiments. In the case of electric motor control, a strict real-time requirement, safety constraints, system delays and the limitations of embedded hardware frameworks are hurdles to overcome. Hence, several modifications to the general RL training setup are introduced in order to enable RL in real-world electric drive control problems. In particular, a rapid control prototyping toolchain is introduced allowing fast and flexible testing of arbitrary RL algorithms. This simulation-to-experiment pipeline is considered an important intermediate step towards introducing RL in embedded control for power electronic systems. To highlight the potential of RL-based drive control, extensive experimental investigations addressing the current control of a permanent magnet synchronous motor utilizing a deep deterministic policy gradient algorithm have been conducted. Despite the early state of research in this domain, promising control performance could be achieved.