Adaptive policy learning for data-driven powertrain control with eco-driving

Abstract

Modern powertrain control design practices rely on model-based approaches accompanied by costly calibrations in order to meet ever stringent energy use and emissions targets specified for standardized drive cycles. These practices struggle to capture the complexity and uncertainty of real-world driving. However, the deluge of operational data now available with connected vehicle technology presents opportunities to foster data-driven control design methods that adapt the powertrain control systems to their field use conditions. While the most attractive of these methods is reinforcement learning (RL), it is rarely directly applicable in physical applications due to its challenges of learning efficiency (sample complexity) and guaranteeing safety. In this paper, we propose and evaluate an adaptive policy learning (APL) framework that leverages existing source policies shipped with vehicles to accelerate initial learning while recovering the asymptotic performance of a reinforcement learning-based powertrain control agent trained from scratch. We present a critique of related residual policy learning approaches and detail our algorithmic implementations for two versions of the proposed framework. We find that the APL powertrain control agents offer in the order of 10% fuel economy improvement over a default powertrain controller of a commercial vehicle without compromising driver accommodation metrics. We demonstrate that the APL frameworks offer a viable approach towards potentially applying RL for real-world scenarios by addressing its learning efficiency issues.