Label Inference Research Articles

ISPPFL: An incentive scheme based privacy-preserving federated learning for avatar in metaverse

The metaverse, seen as the internet’s successor, is a digital world with its own values and economy, linked to the physical world. Accessed via digital avatars using VR equipment, it involves extensive personal data, raising concerns about data security and privacy. Federated learning is a distributed machine learning technique that ensures privacy by allowing metaverse avatars to share knowledge without revealing user data. However, recent research shows that federated learning still faces privacy threats, such as Source Inference Attacks (SIAs) in Horizontal Federated Learning (HFL) and Label Inference Attacks (LIAs) in Vertical Federated Learning (VFL). To solve these problems, in this paper, we propose the first Incentive Scheme based Privacy-Preserving Federated Learning for avatar in metaverse (ISPPFL). The framework is composed of a privacy risks auditor, perturbation generation mechanism and adaptive incentive mechanisms, which effectively defends against privacy risks. We conducted comprehensive experiments on two distinct datasets in various scenarios. The result demonstrated that, the ISPPFL can effectively defend against privacy attacks while maintain model accuracy. For SIAs under HFL, compared to the baseline, the introduction of perturbation and incentive mechanisms could minimize privacy risk indicator (PRI) to around 20%, while maintain the model performance. For LIAs under VFL, the PRI of the model with perturbation generation mechanisms could decrease by approximately 10% compared to the model trained without defenses. Simultaneously, with the introduction of adaptive incentive mechanisms, the PRI could drop from 90% to 69%. Ultimately, the paper also summarizes the completed work and proposes directions for future research.

A benchmark of health insurance fraud detection using machine learning techniques

Health insurance fraud is a complex problem that also has a significant financial impact. Recently, with the availability of large volumes of data and the evolution of computing power, machine learning techniques have become the preferred method for fraud detection. However, the main difficulty facing researchers in this field is the lack of real data sets and the absence of reliable fraud labels. Most published studies use aggregated provider-level or simulated data to test fraud detection algorithms, which may not deliver accurate results. The present study aims to provide a more accurate assessment of fraud detection methods by using real detailed health insurance claims data to compare six of the most common supervised classification algorithms including neural networks and the use of two categorical feature preparation methods. The study was conducted under the guidance of insurance experts, who provided the fraud label inference rules and reviewed the results. A comprehensive description of the benchmarking process and an interpretation of the results are provided in this paper. The results show that supervised classification can be used effectively to detect health insurance fraud, improving detection accuracy by a factor of 4.2 (84% recall for a positive rate of 20%).<p> </p>

Online learning system for English speech automatic recognition based on hidden Markov model algorithm and conditional random field algorithm

E-learning provides a flexible way of learning, which can provide corresponding learning content and tasks according to students’ interests, abilities and learning progress, so that students can choose learning content and learning progress more independently. In traditional teaching methods, teachers need to spend a lot of time and energy to evaluate students’ learning. The e-learning system can automatically record students’ learning behavior and learning results, and give feedback and evaluation in time during the learning process. Automatic speech recognition systems have been widely applied in various fields, and the demand for more user-friendly and convenient user operating systems is becoming increasingly urgent. This article proposes an English automatic speech recognition system based on hidden Markov model algorithm and conditional random field algorithm to improve recognition accuracy and stability, and applies it to the computer field. Label inference and optimization for speech recognition were carried out using the conditional random field algorithm. By establishing the correlation between features and labels, label inference was performed on the speech signal, which was compared with existing speech recognition systems to evaluate whether the recognition accuracy and stability of the new system had been improved. The results show that the English automatic speech recognition system based on hidden Markov model algorithm and conditional random field algorithm exhibits high performance in terms of recognition accuracy and stability. Compared with traditional speech recognition systems, this system has made significant improvements in the application of computer science.

Robust Meta-Representation Learning via Global Label Inference and Classification.

Few-shot learning (FSL) is a central problem in meta-learning, where learners must efficiently learn from few labeled examples. Within FSL, feature pre-training has become a popular strategy to significantly improve generalization performance. However, the contribution of pre-training to generalization performance is often overlooked and understudied, with limited theoretical understanding. Further, pre-training requires a consistent set of global labels shared across training tasks, which may be unavailable in practice. In this work, we address the above issues by first showing the connection between pre-training and meta-learning. We discuss why pre-training yields more robust meta-representation and connect the theoretical analysis to existing works and empirical results. Second, we introduce Meta Label Learning (MeLa), a novel meta-learning algorithm that learns task relations by inferring global labels across tasks. This allows us to exploit pre-training for FSL even when global labels are unavailable or ill-defined. Lastly, we introduce an augmented pre-training procedure that further improves the learned meta-representation. Empirically, MeLa outperforms existing methods across a diverse range of benchmarks, in particular under a more challenging setting where the number of training tasks is limited and labels are task-specific.

Defending Batch-Level Label Inference and Replacement Attacks in Vertical Federated Learning

In a vertical federated learning (VFL) scenario where features and models are split into different parties, it has been shown that sample-level gradient information can be exploited to deduce crucial label information that should be kept secret. An immediate defense strategy is to protect sample-level messages communicated with Homomorphic Encryption (HE), exposing only batch-averaged local gradients to each party. In this paper, we show that even with HE-protected communication, private labels can still be reconstructed with high accuracy by gradient inversion attack, contrary to the common belief that batch-averaged information is safe to share under encryption. We then show that backdoor attack can also be conducted by directly replacing encrypted communicated messages without decryption. To tackle these attacks, we propose a novel defense method, Confusional AutoEncoder (termed <b>CAE</b>), which is based on autoencoder and entropy regularization to disguise true labels. To further defend attackers with sufficient prior label knowledge, we introduce DiscreteSGD-enhanced CAE (termed <b>DCAE</b>), and show that DCAE significantly boosts the main task accuracy than other known methods when defending various label inference attacks.

Improved Gradient Inversion Attacks and Defenses in Federated Learning

Gradient inversion attacks can reconstruct the victim's private data once they have access to the victim's model and gradient. However, existing research is still immature, and many attacks are conducted in ideal conditions. It is unclear how damaging such attacks really are and how they can be effectively defended. In this paper, we first summarize the current relevant researches and their limitations. Then we design a general gradient inversion attack framework, which can attack both FedSGD and FedAVG. We propose approaches to enhance the label inference and image restoration, respectively. Our approach surpasses the SOTA attacks, by successfully attacking the batches from ImageNet while other methods fail to attack. Finally, we suggest several defense strategies without any utility loss from extensive experiments. We are confirmed that our work makes people aware of the privacy issues and can actively avoid the potential risks.

Similarity-Based Label Inference Attack Against Training and Inference of Split Learning

Similarity-Based Label Inference Attack Against Training and Inference of Split Learning

Secure Cascade Vertical Federated Learning over Distributed Labels Against Passive Label Inference Attack

Secure Cascade Vertical Federated Learning over Distributed Labels Against Passive Label Inference Attack

CGIR: Conditional Generative Instance Reconstruction Attacks Against Federated Learning

Data reconstruction attack has become an emerging privacy threat to Federal Learning (FL), inspiring a rethinking of FL's ability to protect privacy. While existing data reconstruction attacks have shown some effective performance, prior arts rely on different strong assumptions to guide the reconstruction process. In this work, we propose a novel Conditional Generative Instance Reconstruction Attack (CGIR attack) that drops all these assumptions. Specifically, we propose a batch label inference attack in non-IID FL scenarios, where multiple images can share the same labels. Based on the inferred labels, we conduct a “coarse-to-fine” image reconstruction process that provides a stable and effective data reconstruction. In addition, we equip the generator with a label condition restriction so that the contents and the labels of the reconstructed images are consistent. Our extensive evaluation results on two model architectures and five image datasets show that without the auxiliary assumptions, the CGIR attack outperforms the prior arts, even for complex datasets, deep models, and large batch sizes. Furthermore, we evaluate several existing defense methods. The experimental results suggest that pruning gradients can be used as a strategy to mitigate privacy risks in FL if a model tolerates a slight accuracy loss.

Joint Latent Space and Label Inference Estimation with Adaptive Fused Data and Label Graphs

Recently, structured computing has become an interesting topic in the world of artificial intelligence, especially in the field of machine learning, as most researchers focus on the development of graph-based semi-supervised learning models. In this article, we present a new framework for graph-based semi-supervised learning. We present a powerful method for simultaneous label inference and linear transform estimation. The targeted linear transformation is used to obtain a discriminant subspace. To improve semi-supervised learning, our framework focuses on exploiting the data structure and soft labels of the available unlabeled samples. In the iterative optimization scheme used, the prior estimation of the label increases the supervision information indirectly through an introduced informative matrix called the label graph, thus avoiding the use of hard confidence-based decisions as used in self-supervised methods. In addition, the estimation of labels and projected data is made more robust by using smoothing concepts based on hybrid graphs. For each type of smoothing, the hybrid graph is an adaptive fusion of the two graphs encoding the similarity of the data and the similarity of the labels. The proposed method leads to an improved discriminative linear transformation. Several experimental results on real image datasets confirm the effectiveness of the proposed method. They also show superior performance compared to semi-supervised methods that use integration and label inference simultaneously.

Secure vertical federated learning based on feature disentanglement

Federated learning (FL) faces many security threats. Although multiple robust FL frameworks have been proposed to defend against these malicious attacks in horizontal federated learning (HFL), security issues in vertical federated learning (VFL) have not been adequately studied. Recent studies show that VFL is vulnerable to inference attacks (e.g., label inference attacks), which puts VFL at risk. To solve this problem, we propose a new VFL framework SVFL (Secure Vertical Federated Learning) to defend against privacy breaches inspired by feature disentanglement. Specifically, in SVFL, the bottom models are feature extractors to extract samples’ features in the high-dimensional space, and the top model sews samples’ features of the same sample ID. Then, disentangling the samples’ features into the class-relevant feature and class-irrelevant one via two classifiers: one is to recognize the class-relevant feature by regular training, and another is to recognize the class-irrelevant feature by adversarial training. Our experiments show that SVFL not only defends against label inference attacks, no matter how many samples features a malicious participant occupies, but also improves the global model’s accuracy. Therefore, SVFL provides a privacy security guarantee for the vertical federated learning system.

Beyond model splitting: Preventing label inference attacks in vertical federated learning with dispersed training

Federated learning is an emerging paradigm that enables multiple organizations to jointly train a model without revealing their private data. As an important variant, vertical federated learning (VFL) deals with cases in which collaborating organizations own data of the same set of users but with disjoint features. It is generally regarded that VFL is more secure than horizontal federated learning. However, recent research (USENIX Security’22) reveals that it is still possible to conduct label inference attacks in VFL, in which attacker can acquire privately owned labels of other participants; even VFL constructed with model splitting (the kind of VFL architecture with higher security guarantee) cannot escape it. To solve this issue, in this paper, we propose the dispersed training framework. It utilizes secret sharing to break the correlations between the bottom model and the training data. Accordingly, even if the attacker receives the gradients in the training phase, he is incapable to deduce the feature representation of labels from the bottom model. Besides, we design a customized model aggregation method such that the shared model can be privately combined, and the linearity of secret sharing schemes ensures the training accuracy to be preserved. Theoretical and experimental analyses indicate the satisfactory performance and effectiveness of our framework.

Weakly Supervised Patch Label Inference Networks for Efficient Pavement Distress Detection and Recognition in the Wild

Automatic image-based pavement distress detection and recognition are vital for pavement maintenance and management. However, existing deep learning-based methods largely omit the specific characteristics of pavement images, such as high image resolution and low distress area ratio, and are not end-to-end trainable. In this paper, we present a series of simple yet effective end-to-end deep learning approaches named Weakly Supervised Patch Label Inference Networks (WSPLIN) for efficiently addressing these tasks under various application settings. WSPLIN transforms the fully supervised pavement image classification problem into a weakly supervised pavement patch classification problem for solutions. Specifically, WSPLIN first divides the pavement image under different scales into patches with different collection strategies and then employs a Patch Label Inference Network (PLIN) to infer the labels of these patches to fully exploit the resolution and scale information. Notably, we design a patch label sparsity constraint based on the prior knowledge of distress distribution and leverage the Comprehensive Decision Network (CDN) to guide the training of PLIN in a weakly supervised way. Therefore, the patch labels produced by PLIN provide interpretable intermediate information, such as the rough location and the type of distress. We evaluate our method on a large-scale bituminous pavement distress dataset named CQU-BPDD and the augmented Crack500 (Crack500-PDD) dataset, which is a newly constructed pavement distress detection dataset augmented from the Crack500. Extensive results demonstrate the superiority of our method over baselines in both performance and efficiency. The source codes of WSPLIN are released on https://github.com/DearCaat/ wsplin.

Local saliency consistency‐based label inference for weakly supervised salient object detection using scribble annotations

AbstractRecently, weak supervision has received growing attention in the field of salient object detection due to the convenience of labelling. However, there is a large performance gap between weakly supervised and fully supervised salient object detectors because the scribble annotation can only provide very limited foreground/background information. Therefore, an intuitive idea is to infer annotations that cover more complete object and background regions for training. To this end, a label inference strategy is proposed based on the assumption that pixels with similar colours and close positions should have consistent labels. Specifically, k‐means clustering algorithm was first performed on both colours and coordinates of original annotations, and then assigned the same labels to points having similar colours with colour cluster centres and near coordinate cluster centres. Next, the same annotations for pixels with similar colours within each kernel neighbourhood was set further. Extensive experiments on six benchmarks demonstrate that our method can significantly improve the performance and achieve the state‐of‐the‐art results.

A benchmark for automatic medical consultation system: frameworks, tasks and datasets.

In recent years, interest has arisen in using machine learning to improve the efficiency of automatic medical consultation and enhance patient experience. In this article, we propose two frameworks to support automatic medical consultation, namely doctor-patient dialogue understanding and task-oriented interaction. We create a new large medical dialogue dataset with multi-level fine-grained annotations and establish five independent tasks, including named entity recognition, dialogue act classification, symptom label inference, medical report generation and diagnosis-oriented dialogue policy. We report a set of benchmark results for each task, which shows the usability of the dataset and sets a baseline for future studies. Both code and data are available from https://github.com/lemuria-wchen/imcs21. Supplementary data are available at Bioinformatics online.

An optimization approach for worker selection in crowdsourcing systems

Crowdsourcing has attracted considerable attention in recent years. A large amount of labelled data can be obtained efficiently and cheaply from the crowdsourcing platform. Therefore, providing a complete, feasible and efficient paradigm for marking unlabelled data in crowdsourcing is in demand. Since the quality of crowd workers directly influences the quality of the labelled data, an optimization model subjecting to quality constraints is built in this paper, aiming to minimize cost function designed based on worker quality. Cost-controlled and quality-assured worker selection algorithms are proposed to solve crowd-labelling problems for static and dynamic crowdsourcing systems separately. Numerical examples based on collected data validate the feasibility of the proposed algorithms. Through extensive simulations, it is demonstrated the proposed method can keep a trade-off between the total cost and the accuracy of label inference which is assured via worker selection. In addition, the proposed approach can also adjust the parameters flexibly according to requesters’ demand for the accuracy or budget requirements, which shows its practicality.

Joint Label Inference and Discriminant Embedding.

Graph-based learning in semisupervised models provides an effective tool for modeling big data sets in high-dimensional spaces. It has been useful for propagating a small set of initial labels to a large set of unlabeled data. Thus, it meets the requirements of many emerging applications. However, in real-world applications, the scarcity of labeled data can negatively affect the performance of the semisupervised method. In this article, we present a new framework for semisupervised learning called joint label inference and discriminant embedding for soft label inference and linear feature extraction. The proposed criterion and its associated optimization algorithm take advantage of both labeled and unlabeled data samples in order to estimate the discriminant transformation. This type of criterion should allow learning more discriminant semisupervised models. Nine public image data sets are used in the experiments and method comparisons. These experimental results show that the performance of the proposed method is superior to that of many advanced semisupervised graph-based algorithms.

Simultaneous label inference and discriminant projection estimation through adaptive self-taught graphs

Processing structured data has become an interesting topic in recent years. The development of graph-based semi-supervised learning models has attracted much attention from machine learning researchers. In this paper, we present a novel approach for graph-based semi-supervised learning. We provide an effective method for simultaneous label recovery and linear transformation estimation. The targeted linear transformation is to obtain a discriminant subspace. The most important factor in this work to improve the semi-supervised learning is to exploit the data structure and soft labels of the available unlabeled samples. In the iterative optimization scheme used, the prior estimation of the labels increases the monitoring information in an indirect way through an introduced label-graph, avoiding the use of confidence-based hard decisions as used in self-supervised methods. It also enforces label smoothing and projected data smoothing through the use of hybrid graphs. For each smoothing type, the hybrid graph is an adaptive fusion of the two graphs encoding the similarity of the data information and the similarity of the label information. The proposed method leads to an improved discriminant linear transformation. Several experimental results on real image datasets confirm the effectiveness of the proposed method. This work also shows superior performance compared to semi-supervised methods that use simultaneous embedding and inference of labels.

An Iteratively Optimized Patch Label Inference Network for Automatic Pavement Distress Detection

We present a novel deep learning framework named the Iteratively Optimized Patch Label Inference Network (IOPLIN) for automatically detecting various pavement distresses that are not solely limited to specific ones, such as cracks and potholes. IOPLIN can be iteratively trained with only the image label via the Expectation-Maximization Inspired Patch Label Distillation (EMIPLD) strategy, and accomplish this task well by inferring the labels of patches from the pavement images. IOPLIN enjoys many desirable properties over the state-of-the-art single branch CNN models such as GoogLeNet and EfficientNet. It is able to handle images in different resolutions, and sufficiently utilize image information particularly for the high-resolution ones, since IOPLIN extracts the visual features from unrevised image patches instead of the resized entire image. Moreover, it can roughly localize the pavement distress without using any prior localization information in the training phase. In order to better evaluate the effectiveness of our method in practice, we construct a large-scale Bituminous Pavement Disease Detection dataset named CQU-BPDD consisting of 60,059 high-resolution pavement images, which are acquired from different areas at different times. Extensive results on this dataset demonstrate the superiority of IOPLIN over the state-of-the-art image classification approaches in automatic pavement distress detection. The source codes of IOPLIN are released on <uri>https://github.com/DearCaat/ioplin</uri>, and the CQU-BPDD dataset is able to be accessed on <uri>https://dearcaat.github.io/CQU-BPDD/</uri>.

A survey of large-scale graph-based semi-supervised classification algorithms

Semi-supervised learning is an effective method to study how to use both labeled data and unlabeled data to improve the performance of the classifier, which has become the hot field of machine learning in recent years. Graph-based semi-supervised learning is very promising among these Semi-supervised methods for its good performance and universality, it represents all the data as a graph and the label information is propagated from the labeled data to the unlabeled data along with the constructed graph. To improve the scalability of graph-based semi-supervised methods for large-scale data, an increasing number of methods with granulation mechanisms are proposed. However, there exist few papers to solely analyze the recent research progress. Following the process of graph-based semi-supervised learning, this paper concludes with the fundamental principles of the granulation mechanism for graph construction and label inference respectively. Moreover, a new taxonomy is generated according to the granulation criterion for each process. Therefore, it provides an important reference for the use and research of semi-supervised classification algorithms facing massive data.