Differential Privacy Technology Research Articles

Research on machine learning technology with privacy protection strategy in recommendation field

Technologies such as machine learning can achieve accurate personalized recommendations. However, due to the collection and utilization of a large amount of user information in this process, people are widely worried about data security and privacy issues. This paper first introduces two key issues of privacy protection in the field of machine learning, namely data privacy and model privacy. On this basis, this paper introduces and analyzes homomorphic encryption, differential privacy and federated learning, and compares their advantages and disadvantages. Among them, homomorphic encryption technology has a large computational cost, differential privacy technology has a negative impact on system accuracy, and federated learning technology has a high training and communication cost. Therefore, it will be the future research direction to study more efficient and accurate recommendation models.

Differential privacy in edge computing-based smart city Applications:Security issues, solutions and future directions

Fast-growing smart city applications, such as smart delivery, smart community, and smart health, are generating big data that are widely distributed on the internet. IoT (Internet of Things) systems are at the centre of smart city applications, as traditional cloud computing is insufficient for satisfying the critical requirements of smart IoT systems. Due to the nature of smart city applications, massive IoT data may contain sensitive information; hence, various privacy-preserving methods, such as anonymity, federated learning, and homomorphic encryption, have been utilised over the years. Furthermore, limited concern has been given to the resource consumption for data privacy-preserving in edge computing environments, which are resource-constrained when compared with cloud data centres. In particular, differential privacy (DP) has been an effective privacy-preserving method in the edge computing environment. However, there is no dedicated study on DP technology with a focus on smart city applications in the edge computing environment. To fill this gap, this paper provides a comprehensive study on DP in edge computing-based smart city applications, covering various aspects, such as privacy models, research methods, mechanisms, and applications. Our study focuses on five areas of data privacy, including data transmitting privacy, data processing privacy, data model training privacy, data publishing privacy, and location privacy. In addition, we investigate many potential applications of DP in smart city application scenarios. Finally, future directions of DP in edge computing are envisaged. We hope this study can be a useful roadmap for researchers and practitioners in edge computing enable smart city applications.

Multiuser Incomplete Preference K-Nearest Neighbor Query Method Based on Differential Privacy in Road Network

In view of the existing research in the field of k-nearest neighbor query in the road network, the incompleteness of the query user’s preference for data objects and the privacy protection of the query results are not considered, this paper proposes a multiuser incomplete preference k-nearest neighbor query algorithm based on differential privacy in the road network. The algorithm is divided into four parts; the first part proposes a multiuser incomplete preference completion algorithm based on association rules. The algorithm firstly uses the frequent pattern tree proposed in this paper to mine frequent item sets, then uses frequent item sets to mine strong correlation rules, and finally completes multiuser incomplete preference based on strong correlation rules. The second part proposes attribute preference weight coefficient based on multiuser’ s different preferences and clusters users accordingly. The third part compares the dominance of the query object, filters the data with low dominance, and performs a k-neighbor query. The fourth part proposes a privacy budget allocation method based on differential privacy technology. The method uses the Laplace mechanism to add noise to the result release and balance the privacy and availability of data. Theoretical research and experimental analysis show that the proposed method can better deal with the multiuser incomplete preference k-nearest neighbor query and privacy protection problems in the road network.

Model Protection Scheme Against Distillation Attack in Internet of Vehicles

Aiming at the problems of model security and user data disclosure caused by the deep learning model in the Internet of Vehicles scenario, which can be stolen by malicious roadside units or base stations and other attackers through knowledge distillation and other techniques, this paper proposes a scheme to strengthen prevent against distillation. The scheme exploits the idea of model reinforcement such as model self-learning and attention mechanism to maximize the difference between the pre-trained model and the normal model without sacrificing performance. It also combines local differential privacy technology to reduce the effectiveness of model inversion attacks. Our experimental results on several datasets show that this method is effective for both standard and data-free knowledge distillation, and provides better model protection than passive defense.

A differential privacy trajectory data storage and publishing scheme based on radix tree

SummaryWith the rapid development of information technology, location‐based services (LBSs) are becoming more and more popular, and at the same time, the users' privacy issues are becoming more and more serious. Therefore, this article proposes a differential private trajectory data storage and publishing scheme based on radix tree (TPRT). Firstly, the original trajectory is preprocessed by considering semantic information and simplified by dividing the stay area to reduce the risk of privacy leakage. Then, differential privacy technology is used to add Laplace noise to sensitive locations to achieve generalization. Finally, a radix tree structure satisfying differential privacy is defined, which alleviates the problem of large space consumption caused by tree height. To protect privacy, Laplace noise is added to the sensitive counts of the radix tree and limited by the Markov chain and privacy degree. Compared with several existing methods, our method has better performance and higher data availability.

Personalized Privacy Protection-Preserving Collaborative Filtering Algorithm for Recommendation Systems

With the rapid development of ubiquitous data collection and data analysis, data privacy in a recommended system is facing more and more challenges. Differential privacy technology can provide strict privacy protection while reducing the risk of privacy leakage, but it also introduces unwanted noise, which makes the performance of the recommender system worsen. Among different users, the degree of their sensitivity to privacy is usually different. Thus, through considering the impact of users’ personalized requirements, the collaborative filtering algorithm can be designed to reduce the amount of unwanted noise. Taking the above assertions into account, we propose a collaborative filtering algorithm based on personalized privacy protection. First, it locally classifies ratings by privacy sensitivity on the user side, then utilizes the random flip mechanism to protect the privacy-sensitive ratings. Then, after the server catches the perturbed rating data, we reconstruct the joint item-item distribution through the Bayesian estimation method. Experimental results show that our proposed algorithm can significantly improve the recommendation performance of recommendation systems while protecting users’ privacy.

Data release for machine learning via correlated differential privacy

Traditional correlated differential privacy technology usually introduces too much noise, reducing data availability. Besides, machine learning often confronts training sets of high-dimensional data, which brings heavy computing overhead. Aiming at the first issue, we design a more reasonable correlation analysis method. This method combines feature matching algorithms with information entropy-based feature importance to accurately calculate the correlated degree of records, reducing data correlation and correlated sensitivity and improving the data’s utility. It is a novel evaluation method of the correlation of records that can alleviate the limitations of traditional calculating correlation methods. Based on this method, we provide a data release solution to reduce the data dimensionality and improve the training efficiency of machine learning by combining the maximum information coefficient with differential privacy. Furthermore, we introduce an optimization algorithm based on mutual information to choose the best principal components to improve the efficiency of our data release solution. To demonstrate the proposed solution’s effectiveness and performance compared to existing schemes, we conducted experiments on three real-world datasets. The experimental results show that our scheme reduces the data correlation by up to 80% compared to existing schemes. Moreover, the accuracy of machine learning is improved by 10% to 20% for the same privacy budget.

Multitentacle Federated Learning Over Software-Defined Industrial Internet of Things Against Adaptive Poisoning Attacks

Software-defined industrial Internet of things (SD-IIoT) exploits federated learning to process the sensitive data at edges, while adaptive poisoning attacks threat the security of SD-IIoT. To address this problem, this article proposes a multi-tentacle federated learning (MTFL) framework, which is essential to guarantee the trustness of training data in SD-IIoT. In MTFL, participants with similar learning tasks are assigned to the same tentacle group. To identify adaptive poisoning attacks, a tentacle distribution-based efficient poisoning attack detection (TD-EPAD) algorithm is presented. And also, to minimize the impact of adaptive poisoning data, a stochastic tentacle data exchanging (STDE) protocol is also proposed. Simultaneously, to protect the tentacle’s privacy in STDE, all exchanged data will be processed by differential privacy technology. A MTFL prototype system is implemented, which provides extensive ablation experiments and comparison experiments, demonstrating that the accuracy of the global model under attack scenario can be improved with 40%.

Multi-Source Data Privacy Protection Method Based on Homomorphic Encryption and Blockchain

Multi-Source data plays an important role in the evolution of media convergence. Its fusion processing enables the further mining of data and utilization of data value and broadens the path for the sharing and dissemination of media data. However, it also faces serious problems in terms of protecting user and data privacy. Many privacy protection methods have been proposed to solve the problem of privacy leakage during the process of data sharing, but they suffer from two flaws: 1) the lack of algorithmic frameworks for specific scenarios such as dynamic datasets in the media domain; 2) the inability to solve the problem of the high computational complexity of ciphertext in multi-source data privacy protection, resulting in long encryption and decryption times. In this paper, we propose a multi-source data privacy protection method based on homomorphic encryption and blockchain technology, which solves the privacy protection problem of multi-source heterogeneous data in the dissemination of media and reduces ciphertext processing time. We deployed the proposed method on the Hyperledger platform for testing and compared it with the privacy protection schemes based on k-anonymity and differential privacy. The experimental results show that the key generation, encryption, and decryption times of the proposed method are lower than those in data privacy protection methods based on k-anonymity technology and differential privacy technology. This significantly reduces the processing time of multi-source data, which gives it potential for use in many applications.

Research on Federated Learning Data Sharing Scheme Based on燚ifferential燩rivacy

To realize data sharing, and to fully use the data value, breaking the data island between institutions to realize data collaboration has become a new sharing mode. This paper proposed a distributed data security sharing scheme based on C/S communication mode, and constructed a federated learning architecture that uses differential privacy technology to protect training parameters. Clients do not need to share local data, and they only need to upload the trained model parameters to achieve data sharing. In the process of training, a distributed parameter update mechanism is introduced. The server is mainly responsible for issuing training commands and parameters, and aggregating the local model parameters uploaded by the clients. The client mainly uses the stochastic gradient descent algorithm for gradient trimming, updates, and transmits the trained model parameters back to the server after differential processing. To test the performance of the scheme, in the application scenario where many medical institutions jointly train the disease detection system, the model is tested from multiple perspectives by taking medical data as an example. From the testing results, we can know that for this specific test dataset, when the parameters are properly configured, the lowest prediction accuracy rate is 90.261% and the highest accuracy rate is up to 94.352. It shows that the performance of the model is good. The results also show that this scheme realizes data sharing while protecting data privacy, completes accurate prediction of diseases, and has a good effect.

Visual Object Detection for Privacy-Preserving Federated Learning

Visual object detection is a computer vision technique based on deep learning. However, due to privacy issues, it is highly challenging to build an object detection model according to the current methods on the centrally stored training dataset. Federated learning is an effective approach to solving the challenge of training data collection by using distributed training. We propose FedVisionBC, a blockchain-based federated learning system for visual object detection that addresses the challenges of single point of failure, model poisoning attacks, and membership inference attacks in traditional federated learning. In the FedVisionBC system, we set up an aggregation node and a verification node instead of a central server to solve the single point of failure problem. We establish a security mechanism that uses encryption techniques, verification nodes, and smart contracts to resist model poisoning attacks. Experimental results show that FedVisionBC can accomplish the object detection task when the percentage of malicious clients is less than 60%. We also propose a new algorithm, ADPFedAvg, to prevent membership inference attacks, which relies on user-level differential privacy technology and the federal average algorithm. Experimental results indicate that ADPFedAvg can achieve a large-scale visual object detection model with differential privacy protection, while only a negligible cost in predictive accuracy.

Differential privacy performance evaluation under the condition of non-uniform noise distribution

Differential Privacy (DP) technology provides powerful and mathematically defined privacy protection for aggregated data. It makes the query results of adjacent datasets indistinguishable by introducing a proper noise level into the query results. The existing DP schemes usually uniformly add the noise which obeys a certain distribution to the target data set. Although they can effectively protect privacy, they may also lead to excessive privacy protection for local data in a data set, which greatly reduces the usability of query results. Based on the above issues, this paper proposes a novel DP scheme with non-uniform noise distribution. Our scheme divides the target data set and adds noise to specific data blocks. We analyzed the entire dataset to verify its privacy and usability. What we care about is whether the data set processed by the DP scheme with non-uniform noise distribution has sufficient privacy protection strength on the basis of improving usability. We obtain encouraging results from evaluating their performance on real data sets. Our results show that both theoretically and empirically, the scheme strikes a good balance between privacy and usability and can provide better privacy protection.

NLSP: A novel lattice‐based secure primitive for privacy‐preserving smart grid communications

SummaryAs the new generation of power scheme, smart grid is proposed to overcome the shortcomings of traditional systems, such as low efficiency and reliability. In this article, a novel lattice‐based secure primitive for privacy‐preserving smart grid communications is proposed, which has the remarkable characteristics, such as scalable multi‐dimensional fine‐grained power data structure and differential privacy security. First, combining with the lattice‐based data encryption technology, while effectively resisting quantum attacks, the method of simultaneous processing of multi‐dimensional data is innovated. Second, through combining the additive homomorphism of the lattice‐based cryptosystem and the Chinese remainder theorem, the data aggregation mechanism that can directly perform homomorphic operations on compressed ciphertext is constructed. Thanks to the above innovative design ideas, the proposed scheme not only significantly improves the efficiency of data communication and processing, greatly reduces the computational cost of the intermediate entity, but also realizes the data confidentiality and information privacy. Finally, observing the decentralized topology of communication nodes in the typical cyber‐physical system of smart grid, the localized differential privacy technology is leveraged to optimize and balance the utility, security, and efficiency of differential privacy. Extensive performance evaluations are conducted to illustrate that the proposed scheme outperforms the state‐of‐the‐art similar schemes in terms of computation complexity and communication cost.

Privacy-preserving edge computing offloading scheme based on whale optimization algorithm

Aiming at the problem of user’s task offloading in mobile edge computing and the potential leakage of location privacy during the offloading process, a privacy-preserving computing offloading scheme based on whale optimization algorithm is proposed. Using differential privacy technology to obfuscate the user's location information, the user can make task offloading decisions according to the obfuscated distance. Considering the delay, energy consumption, and their weighted sum, the offloading problem is modeled as a convex optimization problem. Then, the whale optimization algorithm is adopted to solve this optimization problem to achieve a balance between privacy protection and resource consumption. Experiments are conducted to verify the relationship between the degree of privacy leakage, the computation-offloading cost and real distance, privacy-preserving impact factor, the respective weights of time delay and energy consumption The experimental results show that the offloading scheme proposed in this paper has good performance in terms of cost and privacy protection.

Multiparty Data Publishing via Blockchain and Differential Privacy

Data are distributed between different parties. Collecting data from multiple parties for analysis and mining will serve people better. However, it also brings unprecedented privacy threats to the participants. Therefore, safe and reliable data publishing among multiple data owners is an urgent problem to be solved. We mainly study the problem of privacy protection in data publishing. For a centralized scenario, we propose the LDA-DP algorithm. First, the within-class mean vectors and the pooled within-class scatter matrix are perturbed by the Gaussian noise. Second, the optimal projection direction vector with differential privacy is obtained by the Fisher criterion. Finally, the low-dimensional projection data of the original data are obtained. For distributed scenarios, we propose the Mul-LDA-DP algorithm based on a blockchain and differential privacy technology. First, the within-class mean vectors and within-class scatter matrices of local data are perturbed by the Gaussian noise and uploaded to the blockchain network. Second, the projection direction vector is calculated in the blockchain network and returned to the data owner. Finally, the data owner uses the projection direction vector to generate low-dimensional projection data of the original data and upload it to the blockchain network for publishing. Furthermore, in a distributed scenario, we propose a correlated noise generation scheme that uses the additivity of the Gaussian distribution to mitigate the effects of noise and can achieve the same noise level as the centralized scenario. We measure the utility of the published data by the SVM misclassification rate. We conduct comparative experiments with similar algorithms on different real data sets. The experimental results show that the data released by the two algorithms can maintain good utility in SVM classification.

Multi-Source Selection Transfer Learning with Privacy-Preserving

Transfer learning has ability to create learning task of weakly labeled or unlabeled target domain by using knowledge of source domain to help, which can effectively improve the performance of target learning task. At present, the increased awareness of privacy protection restricts access to data sources and poses new challenges to the development of transfer learning. However, the research on privacy protection in transfer learning is very rare. The existing work mainly uses differential privacy technology and does not consider the distribution difference between data sources, or does not consider the conditional probability distribution of data, which causes negative transfer to harm the effect of algorithm. Therefore, this paper proposes multi-source selection transfer learning algorithm with privacy-preserving MultiSTLP, which is used in scenarios where target domain contains unlabeled data sets with only a small amount of group probability information and multiple source domains with a large number of labeled data sets. Group probability means that the class label of each sample in target data set is unknown, but the probability of each class in a given data group is available, and multiple source domains indicate that there are more than two source domains. The number of data set contains more than two data sets of source domain and one data set of target domain. The algorithm adapts to the marginal probability distribution and conditional probability distribution differences between domains, and can protect the privacy of target data and improve classification accuracy by fusing the idea of multi-source transfer learning and group probability into support vector machine. At the same time, it can select the representative dataset in source domains to improve efficiency relied on speeding up the training process of algorithm. Experimental results on several real datasets show the effectiveness of MultiSTLP, and it also has some advantages compared with the state-of-the-art transfer learning algorithm.

Hybrid differential privacy based federated learning for Internet of Things

Wireless sensor networks have been widely used to achieve fine-grained information collection. However, numerous data acquisition and processing of sensors bring some privacy issues. Federated learning is a promising and privacy-friendly framework that trains a model across multiple devices or edge nodes holding local data samples without transferring their data to the server. It is not enough to protect privacy only by maintaining data locality, so differential privacy technology is often used to protect privacy in federated learning. However, different users have different privacy requirements, so it is inappropriate to use the same privacy protection scheme, assuming that all users trust or distrust the server. The former has poor accuracy, while the latter has poor privacy. This paper proposes a secure and reliable federated learning algorithm by integrating hybrid differential privacy into federated learning. We divide users into two categories according to their different privacy needs. In addition, we analyze the convergence and privacy bounds of the proposed algorithm and propose an adaptive gradient clip scheme and improved composition method to reduce the effects of noise and clip, respectively. The validity of the algorithm is verified by theoretical analysis and experimental evaluation on real-world datasets.

An intelligent and privacy-enhanced data sharing strategy for blockchain-empowered Internet of Things

With the development of the Internet of Things (IoT), the massive data sharing between IoT devices improves the Quality of Service (QoS) and user experience in various IoT applications. However, data sharing may cause serious privacy leakages to data providers. To address this problem, in this study, data sharing is realized through model sharing, based on which a secure data sharing mechanism, called BP2P-FL, is proposed using peer-to-peer federated learning with the privacy protection of data providers. In addition, by introducing the blockchain to the data sharing, every training process is recorded to ensure that data providers offer high-quality data. For further privacy protection, the differential privacy technology is used to disturb the global data sharing model. The experimental results show that BP2P-FL has high accuracy and feasibility in the data sharing of various IoT applications.

Research on Improvement of the Click-Through Rate Prediction Model Based on Differential Privacy

Click-through rate prediction is crucial in network applications such as recommendation systems and online networks. Designing feature extraction schemes to obtain features and modeling users’ click behavior are used to estimate the probability of users clicking on recommended items. The AutoInt model is a recent and effective research finding. It constructs combined features by referencing the multi-head attention mechanism but does not fully mine meaningful high-order cross-features and ignores user privacy protection. To address this problem, this study proposes the differential privacy bidirectional long short-term memory network (DP-Bi-LSTM-AutoInt) model, which is an improved AutoInt model. A bidirectional long short-term memory network is added after the embedding layer to deeply mine the nonlinear relationship between user click behaviors and construct high-order features. Further, differential privacy technology is adopted for user privacy protection, and the Gaussian mechanism is used to randomly perturb the gradient descent algorithm of the model. Using the Criteo dataset to conduct experiments, the experimental results show that the accuracy of the Bi-LSTM-AutoInt model proposed herein is improved by 0.65 % compared to the original AutoInt model. When the privacy budget is greater than 3.0, the accuracies of the DP-Bi-LSTM-AutoInt and Bi-LSTM-AutoInt models are nearly equivalent. However, the DP-Bi-LSTM-AutoInt model algorithm is more secure and reliable than the AutoInt model.

A Differentially Private Federated Learning Model against Poisoning Attacks in Edge Computing

Federated learning is increasingly popular, as it allows us to circumvent challenges due to data islands, by training a global model using data from one or more data owners/sources. However, in edge computing, resource-constrained end devices are vulnerable to be compromised and abused to facilitate poisoning attacks. Privacy-preserving is another important property to consider when dealing with sensitive user data on end devices. Most existing approaches only consider either defending against poisoning attacks or supporting privacy, but not both properties simultaneously. In this paper, we propose a differentially private federated learning model against poisoning attacks, designed for edge computing deployment. First, we design a weight-based algorithm to perform anomaly detection on the parameters uploaded by end devices in edge nodes, which improves detection rate using only small-size validation datasets and minimizes the communication cost. Then, differential privacy technology is leveraged to protect the privacy of both data and model in an edge computing setting. We also evaluate and compare the detection performance in the presence of random and customized malicious end devices with the state-of-the-art, in terms of attack resiliency, communication and computation costs. Experimental results demonstrate that our scheme can achieve an optimal tradeoff between security, efficiency and accuracy.