Local Differential Privacy Model Research Articles

Community detection in decentralized social networks with local differential privacy

Community detection under local differential privacy has been a research hotspot recently. Most existing methods primarily rely on the edge local differential privacy model (edge-LDP), compromising privacy strength for high utility. In addition, they adopt a way of agglomerating nodes one by one to construct communities (or clusters) through multiple iterations, suffering from significant noise and low detection efficiency. Moreover, the error accumulation caused by large-scale iterations is not conducive to detection accuracy. To address these issues, we propose a privacy-preserving community detection method, CD-LDP, based on node-LDP, achieving efficient and accurate community construction by aggregating grouped units instead of individual nodes and delimiting the interaction to three rounds. Specifically, we propose a privacy-preserving global social network construction method to gain connectivity information among all nodes through two rounds of interactions. In addition, a novel perturbation mechanism based on the star social network structure is devised, which improves the accuracy of the global social network by reducing the probability of generating fake edges. Secondly, we design a privacy-preserving node grouping approach leveraging clustering coefficients and Haar discrete wavelet transform in the third round, achieving low noise injection and high-quality node grouping through lossless dimensionality reduction. Lastly, we develop a fast unit merging algorithm combining modularity and Huffman trees to realize efficient and accurate community building by reducing the aggregation possibility of nodes with low similarity. Theoretical analysis and experimentation on real-world datasets testify that our solution can efficiently extract high-quality community structures while satisfying node-LDP.

PAC learning halfspaces in non-interactive local differential privacy model with public unlabeled data

In this paper, we study the problem of PAC learning halfspaces in the non-interactive local differential privacy model (NLDP). To breach the barrier of exponential sample complexity, previous results studied a relaxed setting where the server has access to some additional public but unlabeled data. We continue in this direction. Specifically, we consider the problem under the standard setting instead of the large margin setting studied before. Under different mild assumptions on the underlying data distribution, we propose two approaches that are based on the Massart noise model and self-supervised learning and show that it is possible to achieve sample complexities that are only linear in the dimension and polynomial in other terms for both private and public data, which significantly improve the previous results. Our methods could also be used for other private PAC learning problems.

Collecting Multi-type and Correlation-Constrained Streaming Sensor Data with Local Differential Privacy

Local differential privacy (LDP) is a promising privacy model for distributed data collection. It has been widely deployed in real-world systems (e.g. Chrome, iOS, macOS). In LDP-based mechanisms, an aggregator collects private values perturbed by each user and then analyses these values to estimate their statistics, such as frequency and mean. Most existing works focus on simple scalar value types, such as boolean and categorical values. However, with the emergence of smart sensors and Internet of Things, high-dimensional data are gaining increasing popularity. In many cases where more than one type of sensor data are collected simultaneously, correlations exist between various attributes of such data, e.g. temperature and luminance. To ensure LDP for high-dimensional data, existing solutions either partition the privacy budget ϵ among these correlated attributes or adopt sampling, both of which dilute the density of useful information and thus result in poor data utility. In this paper, we propose a relaxed LDP model, namely, univariate dominance local differential privacy (UDLDP), for high-dimensional data. We quantify the correlations between attributes and present a correlation-bounded perturbation (CBP) mechanism that optimizes the partitioning of privacy budget on each correlated attribute. Furthermore, we extend CBP to support sampling, which is a common bandwidth reduction technique in sensor networks and Internet of Things. We derive the best allocation strategy of sampling probabilities among attributes in terms of data utility, which leads to the correlation-bounded perturbation mechanism with sampling (CBPS). Finally, we discuss how to collect and leverage the correlation from real-time data stream with a by-round algorithm to enhance the utility. The performance of the proposed mechanisms is evaluated and compared with state-of-the-art LDP mechanisms on real-world and synthetic datasets.

VC-PPQ: Privacy-Preserving Q-Learning Based Video Caching Optimization in Mobile Edge Networks

Mobile edge caching has great advantages in alleviating video traffic pressure and reducing transmission delay, which is considered as a hopeful solution to improve video resource utilization and achieve faster service response. However, due to insufficient and non-real time acquisition of the status between edge servers and mobile users, it is still difficult for existing solutions to improve the video quality of experience (VQoE) for mobile users. Besides, due to the mobility of users and malicious third-party attacks, users may suffer the risk of personal privacy (e.g. location, trajectory, preference, etc.) leakage when requesting video services that are scattered in different areas. In order to efficiently deal with these problems, this paper proposes a privacy-preserving Q-learning based video caching optimization framework in mobile edge networks (VC-PPQ), including a privacy scheme for user data and a dynamic video caching algorithm. Specifically, in the first task, the local differential privacy model is leveraged to protect the locations and preferences of users. Then, a data aggregation model is provided to make trade-off between aggregation accuracy and privacy protection. In the second task, we further obtain the video popularity in user's area by a computation service model, then formulate a caching objective function combined with transcoding technologies. Afterwards, Q-learning is adopted to obtain the goal of caching optimization, which can achieve low delay and high hit ratio. Finally, extensive simulations prove that our VC-PPQ framework delivers prominent performance advantages in terms of user privacy and caching services, which greatly improves the VQoE of users.

Optimized setting of privacy budget in a recommendation system with local differential privacy

Recommendation system can help users find the data they need from the massive amounts of data. At the same time, uploading original user data to the server may reveal user privacy. We utilize local differential privacy techniques to provide privacy protection for users in the recommendation system. In the local differential privacy model, the degree of privacy protection is measured by the privacy budget, and a high privacy budget usually means high analysis accuracy. To help users minimize privacy loss and maximize recommendation accuracy, we model the privacy budget setting problem as a multiarmed bandit problem and propose the upper confidence bound learning policy to help each user choose the privacy budget. Considering that users have different sensitivity levels to different data, we modify the above policy. Experimental results reveal that the proposed policy can help users choose an appropriate privacy budget, which can effectively increase the total user payoff.

LDPCD: A Novel Method for Locally Differentially Private Community Detection.

As one of the cores of data analysis in large social networks, community detection has become a hot research topic in recent years. However, user's real social relationship may be at risk of privacy leakage and threatened by inference attacks because of the semitrusted server. As a result, community detection in social graphs under local differential privacy has gradually aroused the interest of industry and academia. On the one hand, the distortion of user's real data caused by existing privacy-preserving mechanisms can have a serious impact on the mining process of densely connected local graph structure, resulting in low utility of the final community division. On the other hand, private community detection requires to use the results of multiple user-server interactions to adjust user's partition, which inevitably leads to excessive allocation of privacy budget and large error of perturbed data. For these reasons, a new community detection method based on the local differential privacy model (named LDPCD) is proposed in this paper. Due to the introduction of truncated Laplace mechanism, the accuracy of user perturbation data is improved. In addition, the community divisive algorithm based on extremal optimization (EO) is also refined to reduce the number of interactions between users and the server. Thus, the total privacy overhead is reduced and strong privacy protection is guaranteed. Finally, LDPCD is applied in two commonly used real-world datasets, and its advantage is experimentally validated compared with two state-of-the-art methods.

Federated matrix factorization with privacy guarantee

Matrix factorization (MF) approximates unobserved ratings in a rating matrix, whose rows correspond to users and columns correspond to items to be rated, and has been serving as a fundamental building block in recommendation systems. This paper comprehensively studies the problem of matrix factorization in different federated learning (FL) settings, where a set of parties want to cooperate in training but refuse to share data directly. We first propose a generic algorithmic framework for various settings of <u>f</u>ederated <u>m</u>atrix <u>f</u>actorization (FMF) and provide a theoretical convergence guarantee. We then systematically characterize privacy-leakage risks in data collection, training, and publishing stages for three different settings and introduce privacy notions to provide end-to-end privacy protections. The first one is vertical federated learning (VFL), where multiple parties have the ratings from the same set of users but on disjoint sets of items. The second one is horizontal federated learning (HFL), where parties have ratings from different sets of users but on the same set of items. The third setting is local federated learning (LFL), where the ratings of the users are only stored on their local devices. We introduce adapted versions of FMF with the privacy notions guaranteed in the three settings. In particular, a new private learning technique called embedding clipping is introduced and used in all the three settings to ensure differential privacy. For the LFL setting, we combine differential privacy with secure aggregation to protect the communication between user devices and the server with a strength similar to the local differential privacy model, but much better accuracy. We perform experiments to demonstrate the effectiveness of our approaches.

Interaction is Necessary for Distributed Learning with Privacy or Communication Constraints

Local differential privacy (LDP) is a model where users send privatized data to an untrusted central server whose goal it to solve some data analysis task. In the non-interactive version of this model the protocol consists of a single round in which a server sends requests to all users then receives their responses. This version is deployed in industry due to its practical advantages and has attracted significant research interest. Our main result is an exponential lower bound on the number of samples necessary to solve the standard task of learning a large-margin linear separator in the non-interactive LDP model. Via a standard reduction this lower bound implies an exponential lower bound for stochastic convex optimization and specifically, for learning linear models with a convex, Lipschitz and smooth loss. These results answer the questions posed by Smith, Thakurta, and Upadhyay (IEEE Symposium on Security and Privacy 2017) and Daniely and Feldman (NeurIPS 2019). Our lower bound relies on a new technique for constructing pairs of distributions with nearly matching moments but whose supports can be nearly separated by a large margin hyperplane. These lower bounds also hold in the model where communication from each user is limited and follow from a lower bound on learning using non-adaptive statistical queries.

Frequency estimation under local differential privacy

Private collection of statistics from a large distributed population is an important problem, and has led to large scale deployments from several leading technology companies. The dominant approach requires each user to randomly perturb their input, leading to guarantees in the local differential privacy model. In this paper, we place the various approaches that have been suggested into a common framework, and perform an extensive series of experiments to understand the tradeoffs between different implementation choices. Our conclusion is that for the core problems of frequency estimation and heavy hitter identification, careful choice of algorithms can lead to very effective solutions that scale to millions of users.

Inferring ground truth from crowdsourced data under local attribute differential privacy

Nowadays, crowdsourcing gains an increasing popularity as it can be adopted to solve many challenging question answering tasks that are easy for humans but difficult for computers. Due to the variety in the quality of users, it is important to infer not only the underlying ground truth of these tasks but also the users ability from the answers given by users. This problem is called Ground Truth Inference and has been studied for many years. However, since the answers collected from the users may contain sensitive information, ground truth inference raises serious privacy concern. Due to this reason, the problem of ground truth inference under local differential privacy (LDP) model has been recently studied. However, this problem is still not well understood and even some basic questions have not been solved yet. First, it is still unknown what is the average error of the private estimators to the underlying ground truth. Secondly, we do not know whether we can infer the ability of each user under LDP model and what is the estimation error w.r.t. the underlying users ability. Finally, previous work only shows that their methods have better performance than the private major voting algorithm through experiments. However, there is still no theoretically result which shows this priority formally or mathematically. In this paper, we partially solve these problems by studying the ground truth inference problem under local attribute differential privacy (LADP) model, which is a relaxation of LDP model, and propose a new algorithm called private Dawid-Skene method, which is motivated by the classical Dawid-Skene method. Specifically, we first provide the estimation errors for both ability of users and the ground truth under some assumptions of the problem if the algorithm start with some appropriate initial vector. Moreover, we propose an explicit instance and show that the estimation error of the ground truth achieved by the private major voting algorithm is always greater than the error achieved by our method.

Set-valued data collection with local differential privacy based on category hierarchy.

Set-valued data is extremely important and widely used in sensor technology and application. Recently, privacy protection for set-valued data under differential privacy (DP) has become a research hotspot. However, the DP model assumes that the data center is trustworthy, consequently, increasingly attention has been paid to the application of the local differential privacy model (LDP) for set-valued data. Constrained by the local differential privacy model, most methods randomly respond to the subset of set-valued data, and the data collector conducts statistics on the received data. There are two main problems with this kind of method: one is that the utility function used in the random response loses too much information; the other is that the privacy protection of the set-valued data category is usually ignored. To solve these problems, this paper proposes a set-valued data collection method (SetLDP) based on the category hierarchy under the local differential privacy model. The core idea is to first make a random response to the existence of the category, continue to disturb the item count if the category exists, and finally randomly respond to a candidate itemset based on the new utility function. Theory analysis and experimental results show that the SetLDP can not only preserve more information, but also protect the category private information in set-valued data.

A Comprehensive Survey on Local Differential Privacy toward Data Statistics and Analysis.

Collecting and analyzing massive data generated from smart devices have become increasingly pervasive in crowdsensing, which are the building blocks for data-driven decision-making. However, extensive statistics and analysis of such data will seriously threaten the privacy of participating users. Local differential privacy (LDP) was proposed as an excellent and prevalent privacy model with distributed architecture, which can provide strong privacy guarantees for each user while collecting and analyzing data. LDP ensures that each user’s data is locally perturbed first in the client-side and then sent to the server-side, thereby protecting data from privacy leaks on both the client-side and server-side. This survey presents a comprehensive and systematic overview of LDP with respect to privacy models, research tasks, enabling mechanisms, and various applications. Specifically, we first provide a theoretical summarization of LDP, including the LDP model, the variants of LDP, and the basic framework of LDP algorithms. Then, we investigate and compare the diverse LDP mechanisms for various data statistics and analysis tasks from the perspectives of frequency estimation, mean estimation, and machine learning. Furthermore, we also summarize practical LDP-based application scenarios. Finally, we outline several future research directions under LDP.

On Sparse Linear Regression in the Local Differential Privacy Model

In this paper, we study the sparse linear regression problem under the Local Differential Privacy (LDP) model. We first show that polynomial dependency on the dimensionality <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> of the space is unavoidable for the estimation error in both non-interactive and sequential interactive local models, if the privacy of the whole dataset needs to be preserved. Similar limitations also exist for other types of error measurements and in the relaxed local models. This indicates that differential privacy in high dimensional space is unlikely achievable for the problem. With the understanding of this limitation, we then present two algorithmic results. The first one is a sequential interactive LDP algorithm for the low dimensional sparse case, called Locally Differentially Private Iterative Hard Thresholding (LDP-IHT), which achieves a near optimal upper bound. This algorithm is actually rather general and can be used to solve quite a few other problems, such as (Local) DP-ERM with sparsity constraints and sparse regression with non-linear measurements. The second one is for the restricted (high dimensional) case where only the privacy of the responses (labels) needs to be preserved. For this case, we show that the optimal rate of the error estimation can be made logarithmically dependent on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> (i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\log p$ </tex-math></inline-formula> ) in the local model, where an upper bound is obtained by a label-privacy version of LDP-IHT. Experiments on real world and synthetic datasets confirm our theoretical analysis.

Tight lower bound of sparse covariance matrix estimation in the local differential privacy model

In this paper, we study the sparse covariance matrix estimation problem in the local differential privacy model, and give a lower bound of Ω(s2log⁡pnϵ2) on the ϵ non-interactive private minimax risk in the metric of squared spectral norm, where s is the row sparsity of the underlying covariance matrix, n is the sample size, and p is the dimensionality of the data. We show that the lower bound is actually tight, as it matches a previous upper bound. Our main technique for achieving this lower bound is a general framework, called General Private Assouad Lemma, which is a considerable generalization of the previous private Assouad lemma and can be used as a general method for bounding the private minimax risk of matrix-related estimation problems.

Principal Component Analysis in the local differential privacy model

In this paper, we study the Principal Component Analysis (PCA) problem under the (distributed) non-interactive local differential privacy model. For the low dimensional case (i.e.,p≪n), we show the optimal rate of Θ(kpnϵ2) (omitting the eigenvalue terms) for the private minimax risk of the k-dimensional PCA using the squared subspace distance as the measurement, where n is the sample size and ϵ is the privacy parameter. For the high dimensional (i.e.,p≫n) row sparse case, we first give a lower bound of Ω(kslog⁡pnϵ2) on the private minimax risk, where s is the underlying sparsity parameter. Then we provide an efficient algorithm to achieve the upper bound of O(s2log⁡pnϵ2). Experiments on both synthetic and real world datasets confirm our theoretical guarantees.

Local differential privacy for social network publishing

Social networks often contain sensitive information. Releasing social network data could possibly seriously jeopardize individual privacy. Therefore, we need to protect privacy when publish social network data. However, the current differential privacy for social network data publishing seriously influences the structure of the social network. We propose a local differential privacy model for social network publishing that preserves community structure information. The model generates the synthetic social network data as published versions under the structural constraints of the edge probability reconstruction. We theoretically prove that the local differential privacy model satisfies the definition of differential privacy. We evaluate the efficacy of the proposed method using three real-life social network datasets and show that our method effectively preserves network structural properties, while ensuring a strong degree of privacy.

Locally private Jaccard similarity estimation

SummaryJaccard Similarity has been widely used to measure the distance between two sets (or preference profiles) owned by two different users. Yet, in the private data collection scenario, it requires the untrusted curator could only estimate an approximately accurate Jaccard similarity of the involved users but without being allowed to access their preference profiles. This paper aims to address the above requirements by considering the local differential privacy model. To achieve this, we initially focused on a particular hash technique, MinHash, which was originally invented to estimate the Jaccard similarity efficiently. We designed the PrivMin algorithm to achieve the perturbation of MinHash signature by adopting Exponential mechanism and build the Locally Differentially Private Jaccard Similarity Estimation (LDP‐JSE) protocol for allowing the untrusted curator to approximately estimate Jaccard similarity. Theoretical and empirical results demonstrate that the proposed protocol can retain a highly acceptable utility of the estimated similarity as well as preserving privacy.