Securing UAV-to-Vehicle Communications: A Curiosity-Driven Deep Q-learning Network (C-DQN) Approach

Abstract

Unmanned aerial vehicle (UAV) will open up new application fields in smart city-based intelligent transportation systems (ITSs), e.g., traffic management, disaster rescue, police patrol, etc. However, the broadcast and line-of-sight nature of air-to-ground wireless channels give rise to a new challenge to the information security of UAV-to-vehicle (U2V) communications. This paper considers U2V communications subject to multi-eavesdroppers on the ground in urban scenarios. We aim to maximize the secrecy rates in physical layer security perspective while considering both the energy consumption and flight zone limitation, by jointly optimizing the UAV's trajectory, the transmit power of the UAV, and the jamming power sent by the roadside unit (RSU). This joint optimization problem is modeled as a Markov decision process (MDP), considering time-varying characteristics of the wireless channels. A curiosity-driven deep reinforcement learning (DRL) algorithm is subsequently utilized to solve the above MDP, in which the agent is reinforced by an extrinsic reward supplied by the environment and an intrinsic reward defined as the prediction error of the consequence after executing its actions. Extensive simulation results show that compared to the DRL without intrinsic rewards, the proposed scheme can have excellent performance in terms of the average reward, learning efficiency, and generalization to other scenarios.