Decentralized Parallel SGD With Privacy Preservation in Vehicular Networks

Abstract

With the prosperity of vehicular networks and intelligent transport systems, vast amount of data can be easily collected by vehicular devices from their users and widely spread in the vehicular networks for the purpose of solving large-scale machine learning problems. Hence how to preserve the data privacy of users during the learning process has become a public concern. To address this concern, under the celebrated framework of differential privacy (DP), we present in this paper a decentralized parallel stochastic gradient descent (D-PSGD) algorithm, called DP <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>2</b></sup> -SGD, which can offer protection for privacy of users in vehicular networks. With thorough analysis we show that DP <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>2</b></sup> -SGD satisfies (ε,δ)- DP while the learning efficiency is the same as D-PSGD without privacy preservation. We also propose a refined algorithm called EC-SGD by introducing an error-compensate strategy. Extensive experiments show that EC-SGD can further improve the convergence efficiency over DP <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>2</b></sup> -SGD in reality.