Enhancing vehicle ride comfort through deep reinforcement learning with expert-guided soft-hard constraints and system characteristic considerations

Abstract

Currently, the research on controlling vehicle ride comfort primarily revolves around utilizing traditional algorithms for active or semi-active control of suspension systems. However, these methods often lack adaptability and necessitate a substantial allocation of human and material resources for system calibration and parameter tuning. With the advancement of cutting-edge computational methods, such as artificial intelligence (AI), being applied in engineering, new opportunities have arisen to tackle knowledge-intensive tasks like suspension control. This study aims to enhance vehicle ride comfort by proposing an active suspension control method that integrates deep reinforcement learning (DRL) while considering system characteristics. Firstly, we construct a Twin Delayed Deep Deterministic Policy Gradient (TD3) architecture to systematically explore control policies. Secondly, we propose an expert-guided soft-hard constraints model (TD3-SH) that synergistically incorporates multi-scale information such as displacement, velocity, acceleration, and control force. Additionally, in practical engineering applications, we introduce action delay mechanisms and hard constraint modules to address time delay and actuator dynamic constraints, thereby alleviating the challenges associated with subsequent parameter adjustments and other knowledge-intensive tasks. Finally, simulations demonstrate the effective mitigation of body vibrations in the low-frequency range and the subsequent improvement of ride comfort by TD3-SH. In comparison to the deep deterministic policy gradient (DDPG), TD3, and model predictive control (MPC) baselines, the proposed method showcases control performance improvements of 54.8%, 35.5%, and 18.3%, respectively. Moreover, the method exhibits ride comfort optimization exceeding 85% across diverse road conditions, showcasing its exceptional generalization and adaptive capacity. Furthermore, the optimization amount exceeding 58% can be sustained despite the constraints of time delay and actuator dynamics. Evidently, the proposed algorithm holds significant potential for engineering applications and is uniquely suited for complex tasks in the vehicle industry characterized by high uncertainty.