Laplace Mechanism Research Articles

Bivariate Correlation-Based Attack: A Challenge for Privacy Preservation of Location Sequence Data

Abstract. Data correlation is a critical issue for location sequence data privacy protection. Series-indistinguishability provides a theoretical basis for the differential privacy protection of temporally correlated location data and is implemented by the correlated Laplace mechanism (CLM), which has become a novel privacy protection method. However, location sequence data is essentially a multivariate time series with data correlation not only within each dimension but also between different dimensions. The CLM-based location privacy scheme only adapts to the auto-correlation in each dimension and ignores cross-correlation between different dimensions, which compromises its privacy performance. To evaluate its actual privacy protection performance, we propose a bivariate correlation-based attack (BCA) utilizing a filtering method and theoretically derive the optimal filter parameters. Based on simulations and real data experiments, the results show that the privacy performance of the CLM-based privacy scheme is significantly reduced under BCA, confirming that this scheme cannot achieve complete series-indistinguishability for bivariate sequences. Furthermore, the results suggest that BCA is an effective tool for privacy performance evaluation.

Density-based clustering with differential privacy

In recent years, differentially private clustering has received increasing attention. However, most existing differentially private clustering algorithms cannot achieve better results when handling non-convex datasets. To enhance knowledge extraction from data while protecting users' sensitive information, we propose a density-based clustering algorithm with differential privacy. Specifically, we incorporate differential privacy mechanisms into the density-based clustering paradigm to enhance the effectiveness of differentially private clustering on non-convex datasets. Firstly, to avoid privacy leakage, we employ the Laplace mechanism for inject noise into the density during the density estimation stage. Then, we design a privacy budget allocation scheme in the cluster expansion stage to make it harder for attackers to access private information. Theoretical analysis demonstrates that our algorithm satisfies ϵ-differential privacy. Experimental outcomes in synthetic and real-world datasets show that our introduced algorithm can obtain high-quality clustering results when dealing with non-convex datasets. In the approximation experiments, it is evident that our algorithm outperforms others in terms of approximation.

Asymptotic in a class of network models with an increasing sub-Gamma degree sequence

. For differential privacy under sub-Gamma noise, we derive the asymptotic properties of a class of network models with binary values with a general link function. In this article, we release the degree sequences of the binary networks under a general noisy mechanism, with the discrete Laplace mechanism as a special case. We establish the asymptotic result, including both consistency and asymptotically normality, of the parameter estimator when the number of parameters goes to infinity in a class of network models. Simulations and a real-data example are provided to illustrate the asymptotic results.

Bidirectional LSTM-Based Privacy Preserving Method for Trajectory Generation

To ensure the privacy of trajectory data while improving its usability upon release, machine learning algorithms can be applied to process trajectory data, thereby enhancing its usability. Addressing the issue of trajectory data release usability, we propose a trajectory privacy protection scheme that combines Bidirectional Long Short-Term Memory (BILSTM) networks and differential privacy (DP). The scheme prepossess the trajectory data using BILSTM to improve its usability. For the generated trajectory data, the Laplace mechanism in differential privacy is applied to add noise, thereby achieving privacy protection. The generalized trajectory data-set obtained is then released. This scheme ensures good data usability and offers certain efficiency advantages.

Towards achieving geo‐indistinguishability for 3D GPS location: A 3D Laplace mechanism approach

SummaryAs the scope of human exploration continues to expand from land to space and the oceans, location‐based data analysis and services are facing unprecedented opportunities and challenges. The wide application of various services based on spatial location in the fields of medical, transportation, financial, social and so forth not only provides great convenience but also exposes the disadvantages of location privacy leakage. Most present location privacy protection solutions are only relevant to 2D locations. However, with the widespread use of positioning‐enabled aircraft, sensing devices and acquisition equipment, there is an urgent need to achieve privacy protection for 3D spatial location data. Therefore, this article proposes a 3D geo‐indistinguishability perturbation method for 3D GPS Locations. The suggested approach calculates GPS coordinate perturbation accurately using the 3D Laplace mechanism, allowing for point‐to‐point perturbation while protecting the privacy of 3D spatial GPS coordinates. Experimental examination of real spatial location datasets demonstrates that the proposed 3D spatial location perturbation method outperforms other existing methods in terms of location service quality loss, data availability, and operational efficiency with the same privacy budget.

Instance-Wise Laplace Mechanism via Deep Reinforcement Learning (Student Abstract)

Recent research has shown a growing interest in per-instance differential privacy (pDP), highlighting the fact that each data instance within a dataset may incur distinct levels of privacy loss. However, conventional additive noise mechanisms apply identical noise to all query outputs, thereby deteriorating data statistics. In this study, we propose an instance-wise Laplace mechanism, which adds non-identical Laplace noises to the query output for each data instance. A challenge arises from the complex interaction of additive noise, where the noise introduced to individual instances impacts the pDP of other instances, adding complexity and resilience to straightforward solutions. To tackle this problem, we introduce an instance-wise Laplace mechanism algorithm via deep reinforcement learning and validate its ability to better preserve data statistics on a real dataset, compared to the original Laplace mechanism.

Efficient Cross-Project Software Defect Prediction Based on Federated Meta-Learning

Software defect prediction is an important part of software development, which aims to use existing historical data to predict future software defects. Focusing on the model performance and communication efficiency of cross-project software defect prediction, this paper proposes an efficient communication-based federated meta-learning (ECFML) algorithm. The lightweight MobileViT network is used as the meta-learner of the Model Agnostic Meta-Learning (MAML) algorithm. By learning common knowledge on the local data of multiple clients, and then fine-tuning the model, the number of unnecessary iterations is reduced, and communication efficiency is improved while reducing the number of parameters. The gradient information model is encrypted using the differential privacy of the Laplace mechanism, and the optimal privacy budget is determined through experiments. Experiments on three public datasets (AEEEM, NASA, and Relink) verified the effectiveness of ECFML in terms of parameter quantity, convergence, and model performance of cross-project software defect prediction.

An Efficient and Differential Privacy-Based Scheme for Aggregating Mobility Datasets

Mobile smart devices, such as mobile phones, wearable devices, and in-vehicle navigation systems, bring us convenience and have become necessities in modern daily life. The built-in global positioning system (GPS) of these mobile devices collects the users’ mobility data to support path planning, navigation and other location-related applications, which also inevitably causes privacy issues. Previous research has shown that employing count-min sketch (CMS) to aggregate mobility datasets is a valid privacy-preserving method for resisting the reconstruction attack on population distributions. However, as the utility/accessibility of the protected datasets is excessively correlated with the size of CMS, decreasing the data transmission cost has become an unsolved issue of that approach. In this paper, we propose an efficient scheme with differential privacy to protect mobility datasets, which releases the privacy-preserving population distributions and achieves better utility as well as a much smaller data transmission cost compared to the CMS-based method. Our proposed scheme is comprised of two collaborative components, global sketch and temporal sketch. The global sketch is responsible for aggregating the raw mobility data and decreasing the data transmission cost, while the temporal sketch is in charge of guaranteeing the utility of the population distributions aggregated by the global sketch. Besides, to enhance the privacy preservation, we employ the Laplace mechanism to make the transmitted data satisfy ϵ-differential privacy. Through our analysis and empirical experiments, compared to the other three state-of-the-art privacy-preserving methods on mobility datasets, our scheme could preserve the privacy of the mobility datasets with much less data transmission cost under the same utility loss.

Privacy-Preserving Classification on Deep Learning with Exponential Mechanism

How to protect the privacy of training data in deep learning has been the subject of increasing amounts of related research in recent years. Private Aggregation of Teacher Ensembles (PATE) uses transfer learning and differential privacy methods to provide a broadly applicable data privacy framework in deep learning. PATE combines the Laplacian mechanism and the voting method to achieve deep learning privacy classification. However, the Laplacian mechanism may greatly distort the histogram vote counts of each class. This paper proposes a novel exponential mechanism with PATE to ensure the privacy protection. This proposed method improves the protection effect and accuracy through the screening algorithm and uses the differential privacy combination theorems to reduce the total privacy budget. The data-dependent analysis demonstrates that the exponential mechanism outperforms the original Laplace mechanism. Experimental results show that the proposed method can train models with improved accuracy while requiring a smaller privacy budget when compared to the original Pate framework.

Differential Privacy Preservation for Continuous Release of Real-Time Location Data.

Continuous real-time location data is very important in the big data era, but the privacy issues involved is also a considerable topic. It is not only necessary to protect the location privacy at each release moment, but also have to consider the impact of data correlation. Correlated Laplace Mechanism (CLM) is a sophisticated method to implement differential privacy on correlated time series. This paper aims to solve the key problems of applying CLM in continuous location release. Based on the finding that the location increment is approximately stationary in many scenarios, a location correlation estimation method based on the location increment is proposed to solve the problem of nonstationary location data correlation estimation; an adaptive adjustment model for the CLM filter based on parameter quantization idea (QCLM) as well as its effective implementation named QCLM-Lowpass utilizing the lowpass spectral characteristics of location data series is proposed to solve the problem of output deviations due to the undesired transient response of the CLM filter in time-varying environments. Extensive simulations and real data experiments validate the effectiveness of the proposed approach and show that the privacy scheme based on QCLM-Lowpass can offer a better balance between the ability to resist correlation-based attacks and data availability.

PRIVIC: A privacy-preserving method for incremental collection of location data

With recent advancements in technology, the threats of privacy violations of individuals' sensitive data are surging. Location data, in particular, have been shown to carry a substantial amount of sensitive information. A standard method to mitigate the privacy risks for location data consists in adding noise to the true values to achieve geo-indistinguishability (geo-ind). However, geo-ind alone is not sufficient to cover all privacy concerns. In particular, isolated locations are not sufficiently protected by the state-of-the-art Laplace mechanism (LAP) for geo-ind. In this paper, we focus on a mechanism based on the Blahut-Arimoto algorithm (BA) from the rate-distortion theory. We show that BA, in addition to providing geo-ind, enforces an elastic metric that mitigates the problem of isolation. Furthermore, BA provides an optimal trade-off between information leakage and quality of service. We then proceed to study the utility of BA in terms of the statistics that can be derived from the reported data, focusing on the inference of the original distribution. To this purpose, we de-noise the reported data by applying the iterative Bayesian update (IBU), an instance of the expectation-maximization method. It turns out that BA and IBU are dual to each other, and as a result, they work well together, in the sense that the statistical utility of BA is quite good and better than LAP for high privacy levels. Exploiting these properties of BA and IBU, we propose an iterative method, PRIVIC, for a privacy-friendly incremental collection of location data from users by service providers. We illustrate the soundness and functionality of our method both analytically and with experiments.

Differential privacy scheme using Laplace mechanism and statistical method computation in deep neural network for privacy preservation

Mountainous amounts of information are now available in hospitals, finance, counter-terrorism, education and many other sectors. Those information can offer a rich source for analysis and decision making. Such information contains user's sensitive and personal data as well. This emanates direct conflict with the user's privacy. Individual's privacy is their right. The existing privacy preserving algorithms works mainly on the numerical data and doesn't care about the categorical data. In addition, there is a heavy trade-off between privacy preservation and data utility. To overcome these issues, a deep neural network - statistical differential privacy (DNN−SDP) algorithm is proposed as the solution to disguise the individual's private and sensitive data. Both the numerical and categorical based human-specific data are considered and fed to the input layer of the neural network. The statistical methods weight of evidence and information value is applied in the hidden layer along with the random weight (wi) to get the initial perturbed data. This initially perturbed data is taken by Laplace computation based differential privacy mechanism as the input and provides the final perturbed data. Census income, bank marketing and heart disease datasets are used for experimentation. While comparing with the state-of-the-art methods, DNN−SDP algorithm provides 97.4% of accuracy with 98.2% of precision, 99% of recall rate and 98.7% of F-measure value. In addition, the fall-out rate, miss rate and false omission rate of the proposed algorithm are less than 4.1%. The DNN−SDP algorithm guarantees the privacy preservation along with data utility.

A Novel Privacy-Preserving Graph Convolutional Network via Secure Matrix Multiplication

Graph convolutional network (GCN) is one of the most representative methods in the realm of graph neural networks (GNNs). In the convolution process, GCN combines the structural information of the networks with the features of nodes. In practice, the structure information of the networks and the features of nodes may be controlled by different parties, which can render GCN ineffective when information sharing is constrained by privacy concerns or licensing issues. Therefore, it is of significant importance to design an effective and secure scheme for GCN that can collaboratively merge information from both parties while safeguarding their sensitive data. In this paper, we introduce S-GCN (i.e., Secure GCN Scheme), which employs secure matrix multiplication (SMM) to compute the product of two matrices in a privacy-preserving manner. The S-GCN scheme requires frequent utilization of SMM to merge information from both parties, resulting in high time and space complexity. To address this issue, we introduce SF-GCN (i.e., Secure and Fast GCN Scheme), which minimizes the use of SMM. Additionally, both S-GCN and SF-GCN may be susceptible to privacy breaches when dealing with dense networks. Hence, we further enhance security by introducing differential privacy through the Laplacian mechanism. Experimental results demonstrate that the proposed schemes do not significantly reduce accuracy in downstream tasks, and, more importantly, effectively protect the privacy information of both parties.

Optimizing the Numbers of Queries and Replies in Convex Federated Learning With Differential Privacy

Federated learning (FL) empowers distributed clients to collaboratively train a shared machine learning model through exchanging parameter information. Despite the fact that FL can protect clients' raw data, malicious users can still crack original data with disclosed parameters. To amend this flaw, differential privacy (DP) is incorporated into FL clients to disturb original parameters, which however can significantly impair the accuracy of the trained model. In this work, we study an imperative question which has been vastly overlooked by existing works: what are the optimal numbers of queries and replies in FL with DP so that the final model accuracy is maximized. In FL, the parameter server (PS) needs to query participating clients for multiple global iterations to complete training. Each client responds a query from the PS by conducting a local iteration. We consider FL that will uniformly and randomly select participating clients to conduct local iterations with the FedSGD algorithm. Our work investigates how many times the PS should query clients and how many times each client should reply the PS by incorporating two most extensively used DP mechanisms (i.e., the Laplace mechanism and Gaussian mechanisms). Through conducting convergence rate analysis, we can determine the optimal numbers of queries and replies in FL with DP so that the final model accuracy can be maximized. Finally, extensive experiments are conducted with publicly available datasets: MNIST and FEMNIST, to verify our analysis and the results demonstrate that properly setting the numbers of queries and replies can significantly improve the final model accuracy in FL with DP.

Universal optimality and robust utility bounds for metric differential privacy1

We study the privacy-utility trade-off in the context of metric differential privacy. Ghosh et al. introduced the idea of universal optimality to characterise the “best” mechanism for a certain query that simultaneously satisfies (a fixed) ε-differential privacy constraint whilst at the same time providing better utility compared to any other ε-differentially private mechanism for the same query. They showed that the Geometric mechanism is universally optimal for the class of counting queries. On the other hand, Brenner and Nissim showed that outside the space of counting queries, and for the Bayes risk loss function, no such universally optimal mechanisms exist. Except for the universal optimality of the Laplace mechanism, there have been no generalisations of these universally optimal results to other classes of differentially-private mechanisms. In this paper, we use metric differential privacy and quantitative information flow as the fundamental principle for studying universal optimality. Metric differential privacy is a generalisation of both standard (i.e., central) differential privacy and local differential privacy, and it is increasingly being used in various application domains, for instance in location privacy and in privacy-preserving machine learning. Similar to the approaches adopted by Ghosh et al. and Brenner and Nissim, we measure utility in terms of loss functions, and we interpret the notion of a privacy mechanism as an information-theoretic channel satisfying constraints defined by ε-differential privacy and a metric meaningful to the underlying state space. Using this framework we are able to clarify Nissim and Brenner’s negative results by (a) that in fact all privacy types contain optimal mechanisms relative to certain kinds of non-trivial loss functions, and (b) extending and generalising their negative results beyond Bayes risk specifically to a wide class of non-trivial loss functions. Our exploration suggests that universally optimal mechanisms are indeed rare within privacy types. We therefore propose weaker universal benchmarks of utility called privacy type capacities. We show that such capacities always exist and can be computed using a convex optimisation algorithm. Further, we illustrate these ideas on a selection of examples with several different underlying metrics.

Generalized multi-agent competitive reinforcement learning with differential augmentation

Observation generalization in multi-agent reinforcement learning is essential for real-world applications as it enables agents to formulate effective policies to deal with environmental uncertainty. However, most existing studies seldom consider the generalization ability of agents in competitive scenarios, which is not in line with the actual process of intelligent groups in real applications. We propose a generalized multi-agent reinforcement learning framework (GMARL) by incorporating the data augmentation mechanism of differential privacy, which enhances the agent’s adaptability and improves the model’s stability. Specifically, we introduce a Laplacian mechanism with temporary adjacency sensitivity into the training and add an observation supplement to the agent’s input. Under this mechanism, the agent intelligently considers the environment fluctuation, thus guarding against the adverse effects of the uncertain changes in the environment states. We show the rigorous guarantee of the difference balance between the observation supplement and the original data, and propose a series of variants upon the GMARL framework. Extensive experimental results on five benchmark multi-agent competitive environments against with five state-of-the-art algorithms validate the effectiveness and robustness of our proposed method.

Local Differential Privacy Image Generation Using Flow-Based Deep Generative Models

Diagnostic radiologists need artificial intelligence (AI) for medical imaging, but access to medical images required for training in AI has become increasingly restrictive. To release and use medical images, we need an algorithm that can simultaneously protect privacy and preserve pathologies in medical images. To address this, we introduce DP-GLOW, a hybrid that combines the local differential privacy (LDP) algorithm with GLOW, one of the flow-based deep generative models. By applying a GLOW model, we disentangle the pixelwise correlation of images, which makes it difficult to protect privacy with straightforward LDP algorithms for images. Specifically, we map images to the latent vector of the GLOW model, where each element follows an independent normal distribution. We then apply the Laplace mechanism to this latent vector to achieve ϵ-LDP, which is one of the LDP algorithms. Moreover, we applied DP-GLOW to chest X-ray images to generate LDP images while preserving pathologies. The ϵ-LDP-processed chest X-ray images obtained with DP-GLOW indicate that we have obtained a powerful tool for releasing and using medical images for training AI.

A Privacy-Preserving State Estimation Scheme for Smart Grids

With the appearance of electric energy market deregulation, there exists a growing concern over the potential privacy leakage of commercial data among competing power companies where data sharing is essential in the applications such as smart grid state estimation. Most of the existing solutions are either perturbation-based or conventional cryptography-based where a trusted central 3rd party would often be required. This paper proposes privacy-preserving state estimation protocols for DC and AC models. The proposed idea is to distribute the overall task of the system state estimation into sub-tasks which can be performed by local sub-grid operators with their private data. A masking method is designed inside a homomorphic encryption scheme which is then used to ensure both the input and output data privacy during the collaboration process among individual sub-task players. Security is achieved via the computationally indistinguishable post-quantum security guaranteed by a levelled homomorphic encryption scheme over real numbers and the differential privacy of the output estimated states provided by the Laplace mechanism perturbation integrated into the masking linear transformation. Simulation results are presented to demonstrate the validity of our proposed privacy-preserving system state estimation protocols.

An Improved Density Peak Clustering Algorithm Based on Chebyshev Inequality and Differential Privacy

This study aims to improve the quality of the clustering results of the density peak clustering (DPC) algorithm and address the privacy protection problem in the clustering analysis process. To achieve this, a DPC algorithm based on Chebyshev inequality and differential privacy (DP-CDPC) is proposed. Firstly, the distance matrix is calculated using cosine distance instead of Euclidean distance when dealing with high-dimensional datasets, and the truncation distance is automatically calculated using the dichotomy method. Secondly, to solve the difficulty in selecting suitable clustering centers in the DPC algorithm, statistical constraints are constructed from the perspective of the decision graph using Chebyshev inequality, and the selection of clustering centers is achieved by adjusting the constraint parameters. Finally, to address the privacy leakage problem in the cluster analysis, the Laplace mechanism is applied to introduce noise to the local density in the process of cluster analysis, enabling the privacy protection of the algorithm. The experimental results demonstrate that the DP-CDPC algorithm can effectively select the clustering centers, improve the quality of clustering results, and provide good privacy protection performance.

Multiuser Incomplete Preference K-Nearest Neighbor Query Method Based on Differential Privacy in Road Network

In view of the existing research in the field of k-nearest neighbor query in the road network, the incompleteness of the query user’s preference for data objects and the privacy protection of the query results are not considered, this paper proposes a multiuser incomplete preference k-nearest neighbor query algorithm based on differential privacy in the road network. The algorithm is divided into four parts; the first part proposes a multiuser incomplete preference completion algorithm based on association rules. The algorithm firstly uses the frequent pattern tree proposed in this paper to mine frequent item sets, then uses frequent item sets to mine strong correlation rules, and finally completes multiuser incomplete preference based on strong correlation rules. The second part proposes attribute preference weight coefficient based on multiuser’ s different preferences and clusters users accordingly. The third part compares the dominance of the query object, filters the data with low dominance, and performs a k-neighbor query. The fourth part proposes a privacy budget allocation method based on differential privacy technology. The method uses the Laplace mechanism to add noise to the result release and balance the privacy and availability of data. Theoretical research and experimental analysis show that the proposed method can better deal with the multiuser incomplete preference k-nearest neighbor query and privacy protection problems in the road network.