Communication-efficient decision-making of digital twin assisted Internet of vehicles: A hierarchical multi-agent reinforcement learning approach

Abstract

The connected autonomous vehicle is considered an effective way to improve transport safety and efficiency. To overcome the limited sensing and computing capabilities of individual vehicles, we design a digital twin assisted decision-making framework for Internet of Vehicles, by leveraging the integration of communication, sensing and computing. In this framework, the digital twin entities residing on edge can effectively communicate and cooperate with each other to plan sub-targets for their respective vehicles, while the vehicles only need to achieve the sub-targets by generating a sequence of atomic actions. Furthermore, we propose a hierarchical multiagent reinforcement learning approach to implement the framework, which can be trained in an end-to-end way. In the proposed approach, the communication interval of digital twin entities could adapt to time-varying environment. Extensive experiments on driving decision-making have been performed in traffic junction scenarios of different difficulties. The experimental results show that the proposed approach can largely improve collaboration efficiency while reducing communication overhead.