Deep reinforcement learning-based cooperative interactions among heterogeneous vehicular networks

Abstract

Most real-world vehicle nodes can be structured into an interconnected network of vehicles. Through structuring these services and vehicle device interactions into multiple types, such internet of vehicles becomes multidimensional heterogeneous overlay networks. The heterogeneousness of the overlays makes it difficult for the overlay networks to coordinate with each other to improve their performance. Therefore, it poses an interesting but critical challenge to the effective analysis of heterogeneous virtual vehicular networks. A variety of virtual vehicular networks can be easily deployed onto the native network by applying the concept of SDN (Software Defined Networking). These virtual networks reflect their heterogeneousness due to their different performance goals, and they compete for the same physical resources of the underlying network, so that a sub-optimal performance of the virtual networks may be achieved. Therefore, we propose a Deep Reinforcement Learning (DRL) approach to make the virtual networks cooperate with each other through the SDN controller. A cooperative solution based on the asymmetric Nash bargaining is proposed for co-existing virtual networks to improve their performance. Moreover, the Markov Chain model and DRL resolution are introduced to leverage the heterogeneous performance goals of virtual networks. The implementation of the approach is introduced, and simulation results confirm the performance improvement of the latency sensitive, loss-rate sensitive and throughput sensitive heterogeneous vehicular networks using our cooperative solution.