Suspension Regulation of Medium-Low-Speed Maglev Trains Via Deep Reinforcement Learning

Abstract

This article studies the suspension regulation problem of medium-low-speed maglev trains (mlsMTs), which is not well solved by most model-based controllers. We propose a sample-based controller by reformulating it as a continuous Markov decision process (MDP) with unknown transition probabilities. Then, we propose a reinforcement learning algorithm with actor-critic neural networks to solve the MDP. To reduce estimation errors in the Q-function, we adopt a double Q-learning scheme and propose a novel initialization method to accelerate the convergence by exploiting the PID controller. Finally, we illustrate that the proposed controllers outperform the existing PID controller with a real dataset from the mlsMT in Changsha, China, and are even comparable to model-based controllers, which, however, assume that the complete information of the model is known, via simulations. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Impact Statement</i>—The control problem of levitation systems is essential for maglev trains. The advanced control methods require an exact dynamical model, which is difficult to establish in practice due to uncertainties and complex dynamics. Reinforcement learning (RL), as a model-free method, learns a controller directly from data. We are the first to propose deep RL algorithms to address the levitation control problem. By learning the real dataset provided by CRRC, the proposed algorithms outperform the well-known PID controller significantly. In particular, our controller responses 50 times faster than PID even without additional efforts on modeling. Moreover, the proposed initialization method can be applied in a variety of RL-based control problems to improve performance.