Deep Reinforcement Learning Based Adaptation of Pure-Pursuit Path-Tracking Control for Skid-Steered Vehicles

Abstract

The growing need for autonomous vehicles in the offroad space raises certain complexities that need to be considered more rigorously in comparison to onroad vehicle automation. Popular path control frameworks in onroad autonomy deployments such as the pure-pursuit controller use geometric and kinematic motion models to generate reference trajectories. However in the offroad settings these controllers, despite their merits (low design and computation requirements), could compute dynamically infeasible trajectories as several of the nominal assumptions made by these models don't hold true when operating in a 2.5D terrain. Outside of the notable challenges such as uncertainties and non-linearities/disturbances introduced by the unknown/unmapped 2.5D terrains, additional complexities arise from the use of vehicle architectures such as the skid-steer that experience lateral skidding for achieving simple curvilinear motion. Additionally, linear models of skid-steer vehicles often consist of high modeling uncertainty which renders traditional linear optimal and robust control techniques inadequate given their sensitivity to modeling errors. Nonlinear MPC has emerged as an upgrade, but needs to overcome realtime deployment challenges (including slow sampling time, design complexity, and limited computational resources). This provides an unique opportunity to utilize data-driven adaptive control methods in tailored application spaces to implicitly learn and hence compensate for the unmodeled aspects of the robot operation. In this study, we build an adaptive control framework called Deep Reinforcement Learning based Adaptive Pure Pursuit (DRAPP) where the base structure is that of a geometric Pure-Pursuit (PP) algorithm which is adapted through a policy learned using Deep Reinforcement Learning (DRL). An additional law that enforces a mechanism to account for the rough terrain is added to the DRL policy to prioritize smoother reference-trajectory generation (and thereby more feasible trajectories for lower-level controllers). The adaptive framework converges quickly and generates smoother references relative to a pure 2D-kinematic path tracking controller. This work includes extensive simulations and a bench marking of the DRAPP framework against Nonlinear Model Predictive Control (NMPC) that is an alternate popular choice in literature for this application.