Neuro-Dynamic Programming in Control of the Ball and Beam System

Abstract

This paper presents a new approach to the control problem of the ball and beam system, with a Neuro-Dynamic Programming algorithm implemented as the main part of the control system. The controlled system is included in the group of underactuated systems, which are nonlinear dynamical objects with the number of control signals smaller than the number of degrees of freedom. This results in problems in the formulation of a stable control algorithm, that guarantees stabilization of the ball in the desired position on the beam. The type of ball and beam material has a noticeable influence on the difficulties in stabilization of the ball, because of a smaller rolling friction and big inertia of the used metallic ball in comparison to other, for example made of non-metallic materials. The main part of the proposed discrete control system is the Neuro-Dynamic Programming algorithm in a Dual-Heuristic Dynamic Programming configuration, realized in a form of two neural networks: the actor and the critic. Neuro-Dynamic Programming algorithms use the Reinforcement Learning idea for adaptation of artificial neural network weights. Additional elements of the control system are the PD controller and the supervisory term, that ensures stability of the closed system loop. The control algorithm works on-line and does not require a preliminary learning phase of the neural network weights. Performance of the control algorithm was verified using the physical system controlled by the dSpace digital signal processing board.