Reinforcement learning control of servo actuated centrally pivoted ball on a beam

Abstract

The objective of this work is to devise a controller using Reinforcement Learning (RL) agents, for unstable and complex control systems like the ball beam system. The reinforcement learning agent's job is to keep the ball's position as close as possible to a set point. The Reinforcement Learning agent learns through rewards. Every action is taken such that the reward value is maximized. The reward becomes maximum if setpoint and the current ball position are as close as possible. So, a ball position from the sensor, in terms of reward is taken as feedback to predict the next action. The predicted action is the angle of the beam which needs to be turned by the motor. The action space considered is of a continuous domain, and the Reinforcement Learning algorithms that have been used are Proximal Policy Optimization (PPO) and Deep Deterministic Policy Gradient (DDPG). Once the environment dynamics are defined, hyper-parameters of the reinforcement learning algorithms pertaining to this environment are tuned, and the model is trained. Servo motor is used as the actuation mechanism.