A Novel Deep Q-Learning-Based Air-Assisted Vehicular Caching Scheme for Safe Autonomous Driving

Abstract

The safety driving-related content demands of vehicle users increase rapidly, especially with the development of autonomous driving. It is significantly necessary to obtain the safety-related transportation information of an area when vehicles are drove there, whether or not they are controlled by human being. However, vehicular content caching can bring issues in distributed-fashion, such as high response delay and low content response ratio because of the poor traffic condition and the obstructions of buildings. As a consequence, we adopt UAVs (Unmanned Aerial Vehicles) to assist the driving safety-related content caching for vehicles. Besides, since the power energy and the caching storage of UAVs are limited, it is needed to design an optimal caching scheme to guarantee the driving safety-related content demands of vehicle users as well as reduce the energy consumption of UAVs. In this article, we propose a novel deep Q-learning based air-assisted vehicular caching scheme to respond to the driving safety-related content requests of vehicle users. First, a three-layered content response architecture is introduced, where an airship is leveraged to take charge of the scheduling of UAVs to improve the content response. Then, a multi-objective mathematical model is built to describe the specific problem of the proposed scheme. Finally, deep Q-learning is applied to solve the multi-objective problem by learning from the history content requests of vehicle users. Extensive experiments have been conducted which show the proposed scheme outperforms its counterparts in terms of content hit ratio, response delay, being scheduling probability and packet buffering time.