A survey of optimization algorithms for differential privacy in Federated Learning

Personal data privacy is a great concern by governments across the world as citizens generate huge amount of data continuously and industries using this for betterment of user centric services. There must be a reasonable balance between data privacy and utility of data. Differential privacy is a promise by data collector to the customer’s personal privacy. Centralised Differential Privacy (CDP) is performing output perturbation of user’s data by applying required privacy budget. This promises the inclusion or exclusion of individual’s data in data set not going to create significant change for a statistical query output and it offers -Differential privacy guarantee. CDP is holding a strong belief on trusted data collector and applying global sensitivity of the data. Local Differential Privacy (LDP) helps user to locally perturb his data and there by guaranteeing privacy even with untrusted data collector. Many differential privacy algorithms handles parameters like privacy budget, sensitivity and data utility in different ways and mostly trying to keep trade-off between privacy and utility of data. This paper evaluates differential privacy algorithms in regard to the privacy support it offers according to the sensitivity of the data. Generalized application of privacy budget is found ineffective in comparison to the sensitivity based usage of privacy budget.

Until now, a lot of clustering algorithms for differential privacy (DP) have been proposed. Practically, there still exist difficulties in implementing these algorithms in a big data platform. In this paper, we proposed a clustering algorithm for privacy preservation on MapReduce. The algorithm is implemented from two aspects. Firstly, the optimized Canopy algorithm is implemented to get the optimal number of clusters and the initial center points on MapReduce. Secondly, the DP K-means algorithm is implemented to get the final clusters on MapReduce. As a result, the proposed algorithm can generate the optimal clustering number that is same with the standard classified data set and can achieve better accuracy of the clusters with the suitable privacy budget \(\varepsilon \).

It is a challenging problem to preserve the friendly-correlations between individuals when publishing social-network data. To alleviate this problem, uncertain graph has been presented recently. The main idea of uncertain graph is converting an original graph into an uncertain form, where the correlations between individuals is an associated probability. However, the existing methods of uncertain graph lack rigorous guarantees of privacy and rely on the assumption of adversary's knowledge. In this paper we first introduced a general model for constructing uncertain graphs. Then, we proposed an algorithm under the model which is based on differential privacy and made an analysis of algorithm's privacy. Our algorithm provides rigorous guarantees of privacy and against the background knowledge attack. Finally, the algorithm we proposed satisfied differential privacy and showed feasibility in the experiments. And then, we compare our algorithm with (k, e)-obfuscation algorithm in terms of data utility, the importance of nodes for network in our algorithm is similar to (k, e)-obfuscation algorithm.

This article is concerned with a distributed algorithm for seeking the Nash equilibrium in noncooperative games with partial-decision information, which simultaneously addresses the protection of individual privacy and ensures fast algorithmic convergence. First, a differential privacy mechanism is used in the fully distributed consensus-based projected pseudo-gradient algorithm to obfuscate shared messages over the communication network and quantify the algorithm's privacy level. To achieve fast convergence, a novel relaxed inertial method is designed, consisting of two steps with independently designed parameters: 1) a relaxation step and 2) an inertia step. The adaptive inertia coefficient in the inertia step is designed based on the iteration error of the players' estimated decisions and a decaying sequence, with the only requirement being the non-negativity of its internal parameters. Compared to existing approaches, our algorithm exhibits high flexibility in parameter selection. Furthermore, we analyze the algorithm's convergence and differential privacy under both linearly decaying and fixed stepsizes within a unified framework, providing sufficient conditions that are independent of the number of players. Finally, numerical simulations validate the algorithm's potential, demonstrating significant improvements in convergence rate, accuracy, and privacy level.

Hospital information security, especially the management of hospital information, is of great significance to improve hospital quality, promote resource sharing, and enhance hospital competitiveness. Despite their unique advantages in preventing transmission data leakage when dealing with medical data, federated learning algorithms still have some shortcomings. Based on this, this study proposes to combine the improved TVFedmul algorithm with the federated learning technique to enhance the efficiency of information aggregation and also proposes to utilize the Gaussian difference privacy algorithm to enhance the protection of private data. Four datasets from cancer rehabilitation data are utilized as research samples in experiments. Compared with the FedAvg algorithm, the TVFedmul algorithm is relatively leading in accuracy, e.g., the accuracy enhancement on the same-distribution dataset of renal cancer reaches 3.03%, and the performance enhancement in the C-domain of the non-simultaneous-distribution dataset of breast cancer reaches 14.2%. The TVFedmul algorithm’s model aggregation speed is also faster, which can effectively improve the efficiency of information aggregation. Although the privacy mechanism of the Gaussian differential privacy algorithm affects the accuracy of the model, its accuracy convergence is not much different from that of federated learning without differential privacy, implying that the Gaussian differential privacy algorithm utilizes a small performance loss to provide more valuable privacy protection.

Data are often correlated in real-world datasets. Existing data privacy algorithms did not consider data correlation an inherent property of datasets. This data correlation caused privacy leakages that most researchers left unnoticed. Such privacy leakages are often caused by homogeneity, background knowledge, and linkage attacks, and the probability of such attacks increases with the magnitude of correlation among data. This problem further got magnified by the large size of real-world datasets, and we refer to these large datasets as ’Big Data.’ Several researchers proposed algorithms using machine learning models, correlation analysis, and data privacy algorithms to prevent privacy leakages due to correlation in large-sized data. The current proposed work first analyses the correlation among data. We studied the Mutual Information Correlation analysis technique and the distance correlation analysis technique for data correlation analysis. We found out distance correlation analysis technique to be more accurate for high-dimensional data. It then divides the data into blocks using the correlation computed earlier and applies the differential privacy algorithm to ensure the data privacy expectations. The results are derived based upon multiple parameters such as data utility, mean average error, variation with data size, and privacy budget values. The results showed that the proposed methodology provides better data utility when compared to the works of other researchers. Also, the data privacy commitments offered by the proposed method are comparable to the other results. Thus, the proposed methodology gives a better data utility while maintaining the required data privacy commitments.

This paper investigates differential privacy in federated learning. This topic has been actively examined in conventional network environments, but few studies have investigated it in the Internet of Vehicles, especially considering various mobility patterns. In particular, this work aims to measure and enumerate the trade-off between accuracy of performance and the level of data protection and evaluate how mobility patterns affect it. To this end, this paper proposes a method considering three factors: learning models, vehicle mobility, and a privacy algorithm. By taking into account mobility patterns, local differential privacy is enhanced with an adaptive clipping method and applied to a mobility-based federated learning model. Experiments run the model on vehicular networks with two different mobility scenarios representing a non-accident traffic situation and traffic events, respectively. Results show that our privacy-enhanced federated learning models degrade accuracy performance by 2.96–3.26% on average, which is compared to the performance drop (42.97% on average) in conventional federated learning models.

This study addresses the privacy problems of data provided by multiple data owners for range query processing on the cloud. Although existing methods preserve data privacy against the cloud or data analysts who receive query responses, protecting data privacy from both remains a challenge. Combining differential privacy (DP) and homomorphic encryption (HE) to construct differentially private outputs over encrypted raw data is a promising way to avoid data privacy leakage to the cloud with encryption while protecting data privacy from data analysts with DP. Although DP adopts several partitioning algorithms to achieve small noise, partitioning cannot be executed once the data is encrypted. In this paper, we propose a new HE-friendly privacy-preserving partitioning algorithm satisfying DP. Although HE enables operations over encrypted data, the execution time of such primitive arithmetic operations is approximately 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">9</sup> times slower than without encryption. Therefore, it is mandatory to reduce the calculation complexity. The proposed partitioning method, which only compares every next-to-each-other data to merge, achieves $\mathcal{O}\left( n \right)$ calculation complexity, where n is the domain size of the input histograms, whereas the greedy algorithm requires $\mathcal{O}\left( {{2^n}} \right)$. The experimental evaluation showed that the execution time of the proposed algorithm for 4,096-domain-size data was approximately 4 h and 35 min, which was acceptable when creating a data summary for the range query processing system and not targeting on-the-fly adoption of DP. Additionally, we confirmed that the accuracy of the proposed algorithm was equivalent to that of the state-of-the-art partitioning algorithm.

With the advancement of technology, there is an increasing emphasis on the personal privacy and security of elderly consumers. This article focuses on the personal privacy and security protection of elderly consumers. Based on the K-means (KM) clustering algorithm, the optimal value was obtained using the monarch butterfly optimization (MBO) algorithm. The migration operator and adjustment operator of the MBO algorithm were enhanced using differential variation algorithm and adaptive methods to obtain a modified monarch butterfly optimization (MMBO) algorithm. Then, to ensure secure protection during clustering, differential privacy (DP) was employed to add noise perturbation to data to obtained a method called DPMMBO-KM algorithm. In experiments on the UCI dataset, it was found that the MMBO-KM algorithm had better clustering performance. Taking the Iris dataset as an example, the MMBO-KM algorithm achieved the highest accuracy of 93.21%. In the application to recommendation systems, the DPMMBO-KM algorithm achieved higher F1 values under different privacy budgets; the average was 0.06. The results demonstrate that the improved clustering algorithm designed in this article can improve clustering results while ensuring personal privacy and data security, and also perform well in recommendation systems.

With the continuous development of big data and the popularization of data processing technology, the classification of a large amount of data has become an inevitable trend. This paper first introduces the current development of differential privacy, and then introduces the definition and nature of differential privacy. Then, this paper summarizes various types of classification algorithms for differential privacy in data analysis, analyzes the principle of technology, and focuses on the privacy protection algorithm of classification, regression, unsupervised learning. Finally, through the analysis and comparison of the performance, scalability and applicability of these algorithms, put forward my own views on the development of differential privacy algorithm in data analysis.

Using the machine learning technology to explore the potential value of Big Data brings us into a smarter world, and the way data is mined through data sharing patterns also threatens the privacy of personal data. Differential privacy is a prevalent mechanism to effectively protect the personal data privacy due to the strict and the provable privacy definition, although there are several achievements have reached by combining the differential privacy and traditional machine learning algorithms in a stand-alone mode, little to talk about the distributed environment. To fill this gap, this paper proposes a method to embed the differential privacy mechanism into distributed platform, respectively implements the DPLloyd, GUPT k-means and GUPT logistic regression on the platform of Spark. The evaluation demonstrates that the approach barely interferes the effect of distributed machine learning algorithms and thus achieves the goal of differential privacy.

Differential Privacy and Multilayer Grouping Consensus Algorithm for Social Network Privacy and Security Management

We describe a framework for designing efficient active learning algorithms that are tolerant to random classification noise and are differentially-private. The framework is based on active learning algorithms that are statistical in the sense that they rely on estimates of expectations of functions of filtered random examples. It builds on the powerful statistical query framework of Kearns (JACM 45(6):983---1006, 1998). We show that any efficient active statistical learning algorithm can be automatically converted to an efficient active learning algorithm which is tolerant to random classification noise as well as other forms of uncorrelated noise. The complexity of the resulting algorithms has information-theoretically optimal quadratic dependence on $$1/(1-2\eta )$$1/(1-2?), where $$\eta $$? is the noise rate. We show that commonly studied concept classes including thresholds, rectangles, and linear separators can be efficiently actively learned in our framework. These results combined with our generic conversion lead to the first computationally-efficient algorithms for actively learning some of these concept classes in the presence of random classification noise that provide exponential improvement in the dependence on the error $$\epsilon $$∈ over their passive counterparts. In addition, we show that our algorithms can be automatically converted to efficient active differentially-private algorithms. This leads to the first differentially-private active learning algorithms with exponential label savings over the passive case.

Differentially Private Federated Learning (DP-FL) has garnered attention as a collaborative machine learning approach that ensures formal privacy. Most DP-FL approaches ensure DP at the record-level within each silo for cross-silo FL. However, a single user's data may extend across multiple silos, and the desired user-level DP guarantee for such a setting remains unknown. In this study, we present Uldp-FL, a novel FL framework designed to guarantee user-level DP in cross-silo FL where a single user's data may belong to multiple silos. Our proposed algorithm directly ensures user-level DP through per-user weighted clipping, departing from group-privacy approaches. We provide a theoretical analysis of the algorithm's privacy and utility. Additionally, we improve the utility of the proposed algorithm with an enhanced weighting strategy based on user record distribution and design a novel private protocol that ensures no additional information is revealed to the silos and the server. Experiments on real-world datasets show substantial improvements in our methods in privacy-utility trade-offs under user-level DP compared to baseline methods. To the best of our knowledge, our work is the first FL framework that effectively provides user-level DP in the general cross-silo FL setting.

The adoption of differential privacy is growing but the complexity of designing private, efficient and accurate algorithms is still high. We propose a novel programming framework and system, ∈ktelo, for implementing both existing and new privacy algorithms. For the task of answering linear counting queries, we show that nearly all existing algorithms can be composed from operators, each conforming to one of a small number of operator classes. While past programming frameworks have helped to ensure the privacy of programs, the novelty of our framework is its significant support for authoring accurate and efficient (as well as private) programs. We describe the design and architecture of the ∈ktelo system and show that ∈ktelo is expressive enough to describe many algorithms from the privacy literature. ∈ktelo allows for safer implementations through code reuse and allows both privacy novices and experts to more easily design new algorithms. We demonstrate the use of ∈ktelo by designing new algorithms offering state-of-the-art accuracy and runtime.