An Autonomous Lane-Changing System With Knowledge Accumulation and Transfer Assisted by Vehicular Blockchain

Abstract

Inappropriate lane following and changing behaviors of connected and autonomous vehicles (CAVs) can result in accidents, such as rear-end collision and side collision. To remedy that, the use of deep reinforcement learning (DRL) for autonomous driving decisions is currently a widely used promising solution. In this case, the accuracy and effectiveness of such a machine learning (ML) model is quite essential for this artificial intelligence (AI)-enabled CAVs. This article proposes a blockchain-based collective learning (BCL) framework for autonomous lane-changing systems. Four key issues, namely, learning efficiency, data security, users’ privacy, as well as communication burden, are addressed by applying collective learning, vehicular blockchain, and knowledge transfer. First, we model the lane-changing problem as a DRL process and learn the autonomous lane-changing strategy through the deep deterministic policy gradient (DDPG) algorithm. Second, a single CAV involves a limited number of driving scenarios, and the independent learning method has the problem of inefficiency. Therefore, we propose a collective learning framework to utilize the “collective intelligence” shared by CAVs. Third, a vehicular blockchain is then applied to ensure the security and privacy of the user and data. In addition, the introduction of the blockchain can incentivize more users to participate in collective learning. Finally, in order to accelerate the learning process and achieve higher level performance while further reducing the communication burden, we use the corresponding knowledge extracted from the ML model such as human learning, as privileged information for sharing instead of directly sharing local ML models. Extensive simulation results validate the effectiveness and efficiency of our proposal in terms of learning efficiency, driving safety, as well as system security and robustness.