Soft Labels Research Articles

Tensorized Label Learning on Anchor Graph

Graph-based multimedia data clustering has attracted much attention due to the impressive clustering performance for arbitrarily shaped multimedia data. However, existing graph-based clustering methods need post-processing to get labels for multimedia data with high computational complexity. Moreover, it is sub-optimal for label learning due to the fact that they exploit the complementary information embedded in data with different types pixel by pixel. To handle these problems, we present a novel label learning model with good interpretability for clustering. To be specific, our model decomposes anchor graph into the products of two matrices with orthogonal non-negative constraint to directly get soft label without any post-processing, which remarkably reduces the computational complexity. To well exploit the complementary information embedded in multimedia data, we introduce tensor Schatten p-norm regularization on the label tensor which is composed of soft labels of multimedia data. The solution can be obtained by iteratively optimizing four decoupled sub-problems, which can be solved more efficiently with good convergence. Experimental results on various datasets demonstrate the efficiency of our model.

Layer Attack Unlearning: Fast and Accurate Machine Unlearning via Layer Level Attack and Knowledge Distillation

Recently, serious concerns have been raised about the privacy issues related to training datasets in machine learning algorithms when including personal data. Various regulations in different countries, including the GDPR grant individuals to have personal data erased, known as ‘the right to be forgotten’ or ‘the right to erasure’. However, there has been less research on effectively and practically deleting the requested personal data from the training set while not jeopardizing the overall machine learning performance. In this work, we propose a fast and novel machine unlearning paradigm at the layer level called layer attack unlearning, which is highly accurate and fast compared to existing machine unlearning algorithms. We introduce the Partial-PGD algorithm to locate the samples to forget efficiently. In addition, we only use the last layer of the model inspired by the Forward-Forward algorithm for unlearning process. Lastly, we use Knowledge Distillation (KD) to reliably learn the decision boundaries from the teacher using soft label information to improve accuracy performance. We conducted extensive experiments with SOTA machine unlearning models and demonstrated the effectiveness of our approach for accuracy and end-to-end unlearning performance.

Unsupervised twitter social bot detection using deep contrastive graph clustering

The emergence of malicious Twitter social bots poses a considerable threat to the security of social networks, and the detection of evolving social bots has become challenging. State-of-the-art detection methods are usually supervised, but the label acquisition process suffers from time-consuming and inaccurate problems in addition to its inability to cope with the challenge of the continuous evolution of social bots. Moreover, structural relationships within social networks are under-explored using current unsupervised methods. To address current challenges, we propose an unsupervised social bot detection method based on deep contrastive graph clustering (BotDCGC). This approach utilises a graph attentional encoder and an inner product decoder to acquire user node embeddings. By incorporating information from both user account features and topological structure, the model employs a contrastive learning technique based on structure to discern node embeddings of distinct classes within the feature space. Subsequently, confident cluster assignments are used as soft labels to guide the embedding process by calculating the similarity between each node and the clustering centre, enabling the joint optimisation of user node embeddings and clustering results. Experiments show that BotDCGC is more effective than the state-of-the-art baselines with an accuracy of 0.8095 in the Twibot-20 dataset and 0.9334 in the Cresci-2015 dataset, and the experimental results indicate the effectiveness of the graph autoencoder module, contrastive learning and deep clustering module.

Data-Free Hard-Label Robustness Stealing Attack

The popularity of Machine Learning as a Service (MLaaS) has led to increased concerns about Model Stealing Attacks (MSA), which aim to craft a clone model by querying MLaaS. Currently, most research on MSA assumes that MLaaS can provide soft labels and that the attacker has a proxy dataset with a similar distribution. However, this fails to encapsulate the more practical scenario where only hard labels are returned by MLaaS and the data distribution remains elusive. Furthermore, most existing work focuses solely on stealing the model accuracy, neglecting the model robustness, while robustness is essential in security-sensitive scenarios, e.g, face-scan payment. Notably, improving model robustness often necessitates the use of expensive techniques such as adversarial training, thereby further making stealing robustness a more lucrative prospect. In response to these identified gaps, we introduce a novel Data-Free Hard-Label Robustness Stealing (DFHL-RS) attack in this paper, which enables the stealing of both model accuracy and robustness by simply querying hard labels of the target model without the help of any natural data. Comprehensive experiments demonstrate the effectiveness of our method. The clone model achieves a clean accuracy of 77.86% and a robust accuracy of 39.51% against AutoAttack, which are only 4.71% and 8.40% lower than the target model on the CIFAR-10 dataset, significantly exceeding the baselines. Our code is available at: https://github.com/LetheSec/DFHL-RS-Attack.

Robust Deep Neural Network for Learning in Noisy Multi-Label Food Images.

Deep networks can facilitate the monitoring of a balanced diet to help prevent various health problems related to eating disorders. Large, diverse, and clean data are essential for learning these types of algorithms. Although data can be collected automatically, the data cleaning process is time-consuming. This study aims to provide the model with the ability to learn even when the data are not completely clean. For this purpose, we extend the Attentive Feature MixUp method to enable its learning on noisy multi-label food data. The extension was based on the hypothesis that during the MixUp phase, when a pair of images are mixed, the resulting soft labels should be different for each ingredient, being larger for ingredients that are mixed with the background because they are better distinguished than when they are mixed with other ingredients. Furthermore, to address data perturbation, the incorporation of the Laplace approximation as a post-hoc method was analyzed. The evaluation of the proposed method was performed on two food datasets, where a notable performance improvement was obtained in terms of Jaccard index and F1 score, which validated the hypothesis raised. With the proposed MixUp, our method reduces the memorization of noisy multi-labels, thereby improving its performance.

CONSULT-II: accurate taxonomic identification and profiling using locality-sensitive hashing.

Taxonomic classification of short reads and taxonomic profiling of metagenomic samples are well-studied yet challenging problems. The presence of species belonging to groups without close representation in a reference dataset is particularly challenging. While k-mer-based methods have performed well in terms of running time and accuracy, they tend to have reduced accuracy for such novel species. Thus, there is a growing need for methods that combine the scalability of k-mers with increased sensitivity. Here, we show that using locality-sensitive hashing (LSH) can increase the sensitivity of the k-mer-based search. Our method, which combines LSH with several heuristics techniques including soft lowest common ancestor labeling and voting, is more accurate than alternatives in both taxonomic classification of individual reads and abundance profiling. CONSULT-II is implemented in C++, and the software, together with reference libraries, is publicly available on GitHub https://github.com/bo1929/CONSULT-II.

Image-based novel fault detection with deep learning classifiers using hierarchical labels

One important characteristic of modern fault classification systems is the ability to flag the system when faced with previously unseen fault types. This work considers the unknown fault detection capabilities of deep neural network-based fault classifiers. Specifically, we propose a methodology on how, when available, labels regarding the fault taxonomy can be used to increase unknown fault detection performance without sacrificing model performance. To achieve this, we propose to utilize soft label techniques to improve the state-of-the-art deep novel fault detection techniques during the training process and novel hierarchically consistent detection statistics for online novel fault detection. Finally, we demonstrated increased detection performance on novel fault detection in inspection images from the hot steel rolling process, with results well replicated across multiple scenarios and baseline detection methods.

Group benefits instance for data purification

Manually annotating datasets for training deep models is very labor-intensive and time-consuming. To overcome such inferiority, directly leveraging web images to conduct training data becomes a natural choice. Nevertheless, the presence of label noise in web data usually degrades the model performance. Existing methods for combating label noise are typically designed and tested on synthetic noisy datasets. However, they tend to fail to achieve satisfying results on real-world noisy datasets. To this end, we propose a method named GRIP to alleviate the noisy label problem for both synthetic and real-world datasets. Specifically, GRIP utilizes a group regularization strategy that estimates class soft labels to improve noise-robustness. Soft label supervision reduces overfitting on noisy labels and learns inter-class similarities to benefit classification. Furthermore, an instance purification operation globally identifies noisy labels by measuring the difference between each training sample and its class soft label. Through operations at both group and instance levels, our approach integrates the advantages of noise-robust and noise-cleaning methods and remarkably alleviates the performance degradation caused by noisy labels. Comprehensive experimental results on synthetic and real-world datasets demonstrate the superiority of GRIP over the existing state-of-the-art methods. The data and source code of this work have been made available at: https://github.com/NUST-Machine-Intelligence-Laboratory/GRIP.

FFLDGA-Net: Image retrieval method based on Feature Fusion Learnable Descriptor Graph Attention Network

Image retrieval aims to retrieve and return the image in the database that is most similar to the query image. However, the performance of image retrieval models is often hindered by the limited dimensionality of images, which lacks depth information about objects. To address this issue, we propose a novel image retrieval model called FFLDGA-Net (Feature Fusion-based Learnable Descriptor Graph Attention Network). This model aims to overcome the absence of depth information in images by fusing feature information from both image data and point cloud data. First, we introduce the LDGA-Net, which effectively improved the model’s ability to mine hard samples and negative samples. Then, we combine the multi-scale route convolution module with a one-dimensional path aggregation network to fuse point clouds and image features at multiple scales, and establish a high-dimensional relationship between features and low-dimensional features. To mitigate training noise, we incorporate a soft label strategy tailored to the dataset’s characteristics. Our experimental results on two benchmark datasets demonstrate the significant improvements achieved by FFLDGA-Net in image retrieval performance.

Semi-supervised medical image classification based on class prototype matching for soft pseudo labels with consistent regularization

Semi-supervised medical image classification based on class prototype matching for soft pseudo labels with consistent regularization

Tolerant Self-Distillation for image classification

Deep neural networks tend to suffer from the overfitting issue when the training data are not enough. In this paper, we introduce two metrics from the intra-class distribution of correct-predicted and incorrect-predicted samples to provide a new perspective on the overfitting issue. Based on it, we propose a knowledge distillation approach without pretraining a teacher model in advance named Tolerant Self-Distillation (TSD) for alleviating the overfitting issue. It introduces an online updating memory and selectively stores the class predictions of the samples from the past iterations, making it possible to distill knowledge across the iterations. Specifically, the class predictions stored in the memory bank serve as the soft labels for supervising the samples from the same class for the current iteration in a reverse way, i.e. the correct-predicted samples are supervised with the incorrect predictions while the incorrect-predicted samples are supervised with the correct predictions. Consequently, the premature convergence issue caused by the over-confident samples would be mitigated, which helps the model to converge to a better local optimum. Extensive experimental results on several image classification benchmarks, including small-scale, large-scale, and fine-grained datasets, demonstrate the superiority of the proposed TSD.

Enhanced Knowledge Distillation for Advanced Recognition of Chinese Herbal Medicine.

The identification and classification of traditional Chinese herbal medicines demand significant time and expertise. We propose the dual-teacher supervised decay (DTSD) approach, an enhancement for Chinese herbal medicine recognition utilizing a refined knowledge distillation model. The DTSD method refines output soft labels, adapts attenuation parameters, and employs a dynamic combination loss in the teacher model. Implemented on the lightweight MobileNet_v3 network, the methodology is deployed successfully in a mobile application. Experimental results reveal that incorporating the exponential warmup learning rate reduction strategy during training optimizes the knowledge distillation model, achieving an average classification accuracy of 98.60% for 10 types of Chinese herbal medicine images. The model boasts an average detection time of 0.0172 s per image, with a compressed size of 10 MB. Comparative experiments demonstrate the superior performance of our refined model over DenseNet121, ResNet50_vd, Xception65, and EfficientNetB1. This refined model not only introduces an approach to Chinese herbal medicine image recognition but also provides a practical solution for lightweight models in mobile applications.

An approach for handwritten Chinese text recognition unifying character segmentation and recognition

Text line recognition methods can be categorized into explicit segmentation based and implicit segmentation based ones. Explicit segmentation based methods require character-level annotation during training, while implicit segmentation based methods, trained on line-level annotated data, face alignment drift challenges. Though some methods have been proposed to address these challenges using weakly supervised object detection, they often rely on cumbersome pseudo-box generation processes and complex decoding. In this paper, we propose a unified framework to overcome these challenges, achieving high accuracy in text recognition and character segmentation. To eliminate the need of character-level annotated real text line data in training, we introduce a novel training paradigm that utilizes character-level annotated synthetic data and line-level annotated real data jointly. For synthetic data, candidate characters are explicitly aligned with labeled characters to generate hard labels for supervising model training. For real data, implicit alignment is produced by Connectionist Temporal Classification (CTC) mapping to provide soft labels for weakly-supervised model training. And for inference, we propose two decoding strategies leveraging the advantages of Non-Maximum Suppression (NMS) and CTC decoding. Extensive experiments on benchmark datasets demonstrate the superior performance of our method in text recognition and character localization, even with minimal amounts of character-level annotated line data.

Enhancing deep learning classification performance of tongue lesions in imbalanced data: mosaic-based soft labeling with curriculum learning

BackgroundOral potentially malignant disorders (OPMDs) are associated with an increased risk of cancer of the oral cavity including the tongue. The early detection of oral cavity cancers and OPMDs is critical for reducing cancer-specific morbidity and mortality. Recently, there have been studies to apply the rapidly advancing technology of deep learning for diagnosing oral cavity cancer and OPMDs. However, several challenging issues such as class imbalance must be resolved to effectively train a deep learning model for medical imaging classification tasks. The aim of this study is to evaluate a new technique of artificial intelligence to improve the classification performance in an imbalanced tongue lesion dataset.MethodsA total of 1,810 tongue images were used for the classification. The class-imbalanced dataset consisted of 372 instances of cancer, 141 instances of OPMDs, and 1,297 instances of noncancerous lesions. The EfficientNet model was used as the feature extraction model for classification. Mosaic data augmentation, soft labeling, and curriculum learning (CL) were employed to improve the classification performance of the convolutional neural network.ResultsUtilizing a mosaic-augmented dataset in conjunction with CL, the final model achieved an accuracy rate of 0.9444, surpassing conventional oversampling and weight balancing methods. The relative precision improvement rate for the minority class OPMD was 21.2%, while the relative F1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${F}_{1}$$\\end{document} score improvement rate of OPMD was 4.9%.ConclusionsThe present study demonstrates that the integration of mosaic-based soft labeling and curriculum learning improves the classification performance of tongue lesions compared to previous methods, establishing a foundation for future research on effectively learning from imbalanced data.

Exploiting Cross-Modal Prediction and Relation Consistency for Semisupervised Image Captioning.

The task of image captioning aims to generate captions directly from images via the automatically learned cross-modal generator. To build a well-performing generator, existing approaches usually need a large number of described images (i.e., supervised image-sentence pairs), requiring a huge effects on manual labeling. However, in real-world applications, a more general scenario is that we only have limited amount of described images and a large number of undescribed images. Therefore, a resulting challenge is how to effectively combine the undescribed images into the learning of cross-modal generator (i.e., semisupervised image captioning). To solve this problem, we propose a novel image captioning method by exploiting the cross-modal prediction and relation consistency (CPRC), which aims to utilize the raw image input to constrain the generated sentence in the semantic space. In detail, considering that the heterogeneous gap between modalities always leads to the supervision difficulty while using the global embedding directly, CPRC turns to transform both the raw image and corresponding generated sentence into the shared semantic space, and measure the generated sentence from two aspects: 1) prediction consistency: CPRC utilizes the prediction of raw image as soft label to distill useful supervision for the generated sentence, rather than employing the traditional pseudo labeling and 2) relation consistency: CPRC develops a novel relation consistency between augmented images and corresponding generated sentences to retain the important relational knowledge. In result, CPRC supervises the generated sentence from both the informativeness and representativeness perspectives, and can reasonably use the undescribed images to learn a more effective generator under the semisupervised scenario. The experiments show that our method outperforms state-of-the-art comparison methods on the MS-COCO "Karpathy" offline test split under complex nonparallel scenarios, for example, CPRC achieves at least 6% improvements on the CIDEr-D score.

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

A psychological evaluation method incorporating noisy label correction mechanism

Using machine learning and deep learning methods to analyze text data from social media can effectively explore hidden emotional tendencies and evaluate the psychological state of social media account owners. However, the label noise caused by mislabeling may significantly influence the training and prediction results of traditional supervised models. To resolve this problem, this paper proposes a psychological evaluation method that incorporates a noisy label correction mechanism and designs an evaluation framework that consists of a primary classification model and a noisy label correction mechanism. Firstly, the social media text data are transformed into heterogeneous text graphs, and a classification model combining a pre-trained model with a graph neural network is constructed to extract semantic features and structural features, respectively. After that, the Gaussian mixture model is used to select the samples that are likely to be mislabeled. Then, soft labels are generated for them to enable noisy label correction without prior knowledge of the noise distribution information. Finally, the corrected and clean samples are composed into a new data set and re-input into the primary model for mental state classification. Results of experiments on three real data sets indicate that the proposed method outperforms current advanced models in classification accuracy and noise robustness under different noise ratio settings, and can efficiently explore the potential sentiment tendencies and users’ psychological states in social media text data.

OWFD-UCPM: An open-world fault diagnosis scheme based on uncertainty calibration and prototype management

In real-world fault diagnosis tasks, it is never possible to enumerate all fault types beforehand since there are always unseen situations that may arise unexpectedly. Therefore, fault diagnosis is essentially an open-world recognition task. A desirable open-world fault diagnosis model must be able to perform open-set recognition (OSR) and incremental learning (IL). Classifier calibration and prototype replay are popular in two subtasks. However, existing calibration and replay strategies suffer from tampering with raw data and ignoring resource allocation, respectively. In this work, we formulate an open-world fault diagnosis scheme to address both issues. Firstly, we calibrate the uncertainty of the classifier’s prediction by assigning soft labels to the samples based on their distance from the class center. Then, Shannon entropy is employed to quantify the uncertainty as an estimate of the probability that the sample belongs to the unknown. Secondly, we adaptively manage the memory of old and new known classes according to the training accuracy. Then, based on this memory budget, a dissimilarity-based sparse subset selection algorithm is utilized to pick more diverse exemplars as prototypes for replay. Furthermore, we extend the open-set F-Measure (OSFM) to the weighted OSFM to make it applicable even when the classes are imbalanced. Experimental results on multiphase flow facility and wastewater treatment plant demonstrate the feasibility and superiority of the proposed method.

AmbiFC: Fact-Checking Ambiguous Claims with Evidence

Abstract Automated fact-checking systems verify claims against evidence to predict their veracity. In real-world scenarios, the retrieved evidence may not unambiguously support or refute the claim and yield conflicting but valid interpretations. Existing fact-checking datasets assume that the models developed with them predict a single veracity label for each claim, thus discouraging the handling of such ambiguity. To address this issue we present AmbiFC,1 a fact-checking dataset with 10k claims derived from real-world information needs. It contains fine-grained evidence annotations of 50k passages from 5k Wikipedia pages. We analyze the disagreements arising from ambiguity when comparing claims against evidence in AmbiFC, observing a strong correlation of annotator disagreement with linguistic phenomena such as underspecification and probabilistic reasoning. We develop models for predicting veracity handling this ambiguity via soft labels, and find that a pipeline that learns the label distribution for sentence-level evidence selection and veracity prediction yields the best performance. We compare models trained on different subsets of AmbiFC and show that models trained on the ambiguous instances perform better when faced with the identified linguistic phenomena.

Dual-space Hierarchical Learning for Goal-guided Conversational Recommendation

Proactively and naturally guiding the dialog from the non-recommendation context (e.g., Chit-chat) to the recommendation scenario (e.g., Music) is crucial for the Conversational Recommender System (CRS). Prior studies mainly focus on planning the next dialog goal (e.g., chat on a movie star) conditioned on the previous dialog. However, we find the dialog goals can be simultaneously observed at different levels, which can be utilized to improve CRS. In this paper, we propose Dual-space Hierarchical Learning (DHL) to leverage multi-level goal sequences and their hierarchical relationships for conversational recommendation. Specifically, we exploit multi-level goal sequences from both the representation space and the optimization space. In the representation space, we propose the hierarchical representation learning where a cross attention module derives mutually enhanced multi-level goal representations. In the optimization space, we devise the hierarchical weight learning to reweight lower-level goal sequences, and introduce bi-level optimization for stable update. Additionally, we propose a soft labeling strategy to guide optimization gradually. Experiments on two real-world datasets verify the effectiveness of our approach. Code and data are available here.