Privacy-Aware Multiagent Deep Reinforcement Learning for Task Offloading in VANET

Abstract

Offloading task to roadside units (RSUs) provides a promising solution for enhancing the real-time data processing capacity and reducing energy consumption of vehicles in the vehicular ad-hoc network (VANET). Recently, multi-agent deep reinforcement learning (MADRL)-based offloading approaches have been widely used for task offloading in VANET. However, existing MADRL-based approaches suffer from offloading preference inference (OPI) attack, which utilizes the vulnerability in the policy learning process of MADRL to mislead vehicles to offload tasks to malicious RSUs. In this paper, we first formulate a joint optimization of offloading action and transmitting power with the objective of minimizing the system cost, including local and edge costs, under the privacy requirement of protecting offloading preference during offloading policy learning process in VANET. Despite the non-convexity and centralized of this joint optimization problem, we propose a privacy-aware MADRL (PA-MADRL) approach to solve it, which can allow the offload decision of each vehicle to reach the Nash Equilibrium (NE) without leaking offloading preference. The key to resisting the OPI attack is to protect the offloading preference by 1)elaborately constructing the noise based on ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\beta,\Phi$</tex-math> </inline-formula> )-differential privacy mechanism and 2) adding it to the action selection and policy updating process of vanilla MADRL. We conduct a detailed theoretical analysis of the convergence and privacy guarantee of the proposed PA-MADRL, and extensive simulations are conducted to demonstrate the effectiveness, privacy-protecting capacity, and cost-efficiency of PA-MADRL approach.