Online reinforcement learning with passivity-based stabilizing term for real time overhead crane control without knowledge of the system model

Abstract

Due to the existing uncertainties such as the payload mass and unmodeled dynamics in the overhead crane system, classical model-based control methods yielding fixed control gain can exhibit certain limitations. In this study, a novel model-free online Reinforcement Learning (RL) control method is proposed for the real-time overhead crane position regulation and anti-swing control problem, combining the benefits of adaptive control and optimal control. The crane control problem is first formulated as an optimal regulation problem with a user-specified objective function. Then, an improved neural-network updating rule with an additional passivity-based stabilizing term is developed to ensure the system stability during learning based on the overhead crane system passivity analysis. The proposed method, unlike other online RL algorithms, does not require the initial stabilizing control policy or prior knowledge of the crane mathematical model. Lyapunov approach is used to prove the closed-loop system stability, and the learned controller is shown to be near-optimal within a finite bound. Finally, simulation studies are conducted to demonstrate the effectiveness of the proposed method in the presence of system parameter variations and external disturbances, exhibiting satisfactory performance when compared to LQR and passivity-based control.