Self-Learning Cruise Control Using Kernel-Based Least Squares Policy Iteration

Abstract

This paper presents a novel learning-based cruise controller for autonomous land vehicles (ALVs) with unknown dynamics and external disturbances. The learning controller consists of a time-varying proportional-integral (PI) module and an actor-critic learning control module with kernel machines. The learning objective for the cruise control is to make the vehicle's longitudinal velocity follow a smoothed spline-based speed profile with the smallest possible errors. The parameters in the PI module are adaptively tuned based on the vehicle's state and the action policy of the learning control module. Based on the state transition data of the vehicle controlled by various initial policies, the action policy of the learning control module is optimized by kernel-based least squares policy iteration (KLSPI) in an offline way. The effectiveness of the proposed controller was tested on an ALV platform during long-distance driving in urban traffic and autonomous driving on off-road terrain. The experimental results of the cruise control show that the learning control method can realize data-driven controller design and optimization based on KLSPI and that the controller's performance is adaptive to different road conditions.