Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy‐duty truck control in the field

Abstract

AbstractTo develop cooperative adaptive cruise control (CACC), the choice of control approach often influences and can limit the choice of model structure, and vice versa. For heavy‐duty trucks, practical application of CACC in the field is heavily influenced by the accuracy of the used model. Deep learning and deep reinforcement learning (deep‐RL) have recently been used to demonstrate improved modeling and control performance for vehicles such as cars and quadrotors compared to state‐of‐the‐art. The literature on the application of deep learning and deep‐RL for heavy‐duty trucks in the field, which are significantly more complex than cars, is still sparse, however. In this article, we develop a two‐layer gray‐box deep learning model to capture longitudinal dynamics of heavy‐duty trucks while abstracting their complexity and present an approach to properly break the nested feedback loops in the model for training. We compare this model with three other alternative models and show that it achieves better general performance compared to a standard artificial neural network and results in and slower steady‐state acceleration and speed error growth rates, respectively. We then present an architecture to utilize these deep learning models within the deep‐RL framework and use it to develop baseline CACC controllers that can be zero‐shot transferred to the field. To carry out the work, we present a setup of differently configured trucks along with their interface architecture and stochastic driving cycle generators for data collection. Numerical validation of the approach demonstrated stationary and bounded modeling error, and demonstrated transfer of CACC controllers with consistent overshoot bounds and a stable approximately‐zero steady‐state error. Validation from field experiments demonstrated similarly consistent results. Compared to a state‐of‐the‐art benchmark, the deep‐RL controller achieved lower speed and time‐gap error variance but higher time‐gap error offset.