Model-Free Optimal Vibration Control of a Nonlinear System Based on Deep Reinforcement Learning

Abstract

Optimal control of nonlinear vibration requires precise knowledge of the system and the solution to Hamilton–Jacobi–Bellman (HJB) equation. However, in practical engineering applications, acquiring precise system parameters poses challenges, and the analytical solutions for the HJB equation are difficult to obtain. In this paper, two reinforcement learning algorithms, Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm and Soft Actor-Critic (SAC) algorithm, are employed to train neural network-based optimal controllers for the van der Pol vibration system in the presence of unknown system parameters. To validate their performance, the controllers undergo testing in a series of experiments, including assessments of free vibration, frequency sweep excitation, and Gaussian noise excitation. The results indicate that both the TD3-trained and SAC-trained neural network-based controller are capable of proficiently suppress the vibration of the van der Pol oscillator. Additionally, these two model-free controllers can approximate the optimal control law which solved based on the dynamic model of the nonlinear system.