Wrong Labels Research Articles

A feasible procedure to detect widespread wrong labeling of Ashwagandha root extracts in dietary supplements

A feasible procedure to detect widespread wrong labeling of Ashwagandha root extracts in dietary supplements

Proxy-based robust deep metric learning in the presence of label noise

Real-world datasets contain label noise data that can deteriorate the performance of a deep learning model. Cleaning annotations manually requires substantial efforts from experts and is not practical in large datasets. Therefore, many methods are proposed to enhance the robustness of deep models in the presence of label noise. However, these methods are primarily developed for classification tasks and cannot be directly applied to Deep Metric Learning (DML) applications. To bridge this gap, we present a proxy-based deep metric learning method to detect wrong labels through the estimation of the posterior distribution for observed and true labels. Specifically, we model the distribution of the observed annotations as a mixture of two components: one component represents the clean label distribution and the other denotes the noisy labels. Besides, we present an EM-like algorithm that precisely identifies label noise data jointly with the similarity learning method. We also exploit information of the identified noisy labeled data by utilizing state-of-the-art semi-supervised learning (SSL) techniques. The results of the extensive experiments on datasets with real or synthetic label noise indicate that our method consistently outperforms the state-of-the-art methods by a large margin. Moreover, the proposed method accurately detects noisy labeled data and generates correct pseudo labels for them after only a few epochs.

Explainable Reverse Verification of Goodness of Classification of MRI Images by Clinical Experts.

Radiology offers a presumptive diagnosis. The etiology of radiological errors are prevalent, recurrent, and multi-factorial. The pseudo-diagnostic conclusions can arise from varying factors such as, poor technique, failures of visual perception, lack of knowledge, and misjudgments. This retrospective and interpretive errors can influence and alter the Ground Truth (GT) of Magnetic Resonance (MR) imaging which in turn result in faulty class labeling. Wrong class labels can lead to erroneous training and illogical classification outcomes for Computer Aided Diagnosis (CAD) systems. This work aims at verifying and authenticating the accuracy and exactness of the GT of biomedical datasets which are extensively used in binary classification frameworks. Generally such datasets are labeled by only one radiologist. Our article adheres a hypothetical approach to generate few faulty iterations. An iteration here considers simulation of faulty radiologist's perspective in MR image labeling. To achieve this, we try to simulate radiologists who are subjected to human error while taking decision regarding the class labels. In this context, we swap the class labels randomly and force them to be faulty. The experiments are carried out on some iterations (with varying number of brain images) randomly created from the brain MR datasets. The experiments are carried out on two benchmark datasets DS-75 and DS-160 collected from Harvard Medical School website and one larger input pool of self-collected dataset NITR-DHH. To validate our work, average classification parameter values of faulty iterations are compared with that of original dataset. It is presumed that, the presented approach provides a potential solution to verify the genuineness and reliability of the GT of the MR datasets. This approach can be utilized as a standard technique to validate the correctness of any biomedical dataset.

Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classification.

The development of accurate methods for multi-label classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. The MLC methods based on convolutional neural networks (CNNs) have shown strong performance gains in RS. However, they usually require a high number of reliable training images annotated with multiple land-cover class labels. Collecting such data is time-consuming and costly. To address this problem, the publicly available thematic products, which can include noisy labels, can be used to annotate RS images with zero-labeling cost. However, multi-label noise (which can be associated with wrong and missing label annotations) can distort the learning process of the MLC methods. To address this problem, we propose a novel multi-label noise robust collaborative learning (RCML) method to alleviate the negative effects of multi-label noise during the training phase of a CNN model. RCML identifies, ranks, and excludes noisy multi-labels in RS images based on three main modules: 1) the discrepancy module; 2) the group lasso module; and 3) the swap module. The discrepancy module ensures that the two networks learn diverse features, while producing the same predictions. The task of the group lasso module is to detect the potentially noisy labels assigned to multi-labeled training images, while the swap module is devoted to exchange the ranking information between two networks. Unlike the existing methods that make assumptions about noise distribution, our proposed RCML does not make any prior assumption about the type of noise in the training set. The experiments conducted on two multi-label RS image archives confirm the robustness of the proposed RCML under extreme multi-label noise rates. Our code is publicly available at: https://www.noisy-labels-in-rs.org.

Remote supervised relationship extraction method of clustering for knowledge graph in aviation field

In the process of building domain knowledge graph, the result of relationship extraction between entities is an important guarantee of the quality of the graph. Therefore, we propose a clustering method based on reinforcement learning for remote supervised relation extraction. For the relationship extraction of accident information in the aviation domain mapping, a clustering method combining local dense and global dissimilarity is proposed in combination with remote supervision, which can obtain a large amount of low-noise labeled data and reduce part of the wrong labeling and missing labeling due to the strong specialization in the aviation domain; meanwhile, reinforcement learning is introduced to denoise the negative instance noise in the positive sample data; Finally, we propose a two-attention segmentation (DAPCNN) relationship extraction model to mine deep semantic sentences. The experimental results show that in the civil aviation relationship extraction text constructed in this paper, the Micro_R, Micro_P and Micro_F1 values of the proposed relationship extraction method reach 83.41 %, 84.16 % and 83.96 %. In the open relationship extraction dataset DuIE, The Micro_R, Micro_P and Micro_F1 of the proposed method are up to 83.41 %, 93.58 % and 94.02 % respectively. Compared with the current advanced multi-instance and multi-label model, the proposed method can more accurately extract the relationship between aviation accident entities. At the same time, the performance of the open data set is also good, and has a certain universality.

Label noise correction for crowdsourcing using dynamic resampling

Crowdsourcing provides a cost-effective labeling solution for the acquisition of labeled training samples for machine learning by employing workers on the Internet. A common approach to improving the label quality is to employ a truth inference method to infer integrated labels for samples from their multiple noisy labels obtained from different crowd workers. Although the quality of integrated labels is significantly improved compared with that of the original noisy ones, it still cannot completely eliminate the noises inevitably existing in the integrated labels. To further improve the label quality, this paper proposes a novel label noise correction method for crowdsourcing based on dynamic resampling (DRNC). DRNC first divides the dataset with inferred labels into a clean set and a noisy set through a filter. According to a certain proportion, the clean set and the noisy set are resampled to train multiple heterogeneous classifiers, which form an ensemble classifier. Then, the dataset is divided by the ensemble classifier into a new sub-noisy set and a sub-clean set. The whole process repeats multiple rounds, generating multiple sub-clean sets. Finally, these sub-clean sets are used to train classifiers, which jointly correct the wrong labels in the dataset by voting. Experimental results on 25 simulated and 4 real-world datasets consistently show that the proposed DRNC averagely improves the quality of labels as well as the quality of learned models in the range of 1 to 10 percentage points, compared with four state-of-the-art crowdsourcing noise correction methods.

Suppressing Uncertainty in Gaze Estimation

Uncertainty in gaze estimation manifests in two aspects: 1) low-quality images caused by occlusion, blurriness, inconsistent eye movements, or even non-face images; 2) uncorrected labels resulting from the misalignment between the labeled and actual gaze points during the annotation process. Allowing these uncertainties to participate in training hinders the improvement of gaze estimation. To tackle these challenges, in this paper, we propose an effective solution, named Suppressing Uncertainty in Gaze Estimation (SUGE), which introduces a novel triplet-label consistency measurement to estimate and reduce the uncertainties. Specifically, for each training sample, we propose to estimate a novel ``neighboring label'' calculated by a linearly weighted projection from the neighbors to capture the similarity relationship between image features and their corresponding labels, which can be incorporated with the predicted pseudo label and ground-truth label for uncertainty estimation. By modeling such triplet-label consistency, we can largely reduce the negative effects of unqualified images and wrong labels through our designed sample weighting and label correction strategies. Experimental results on the gaze estimation benchmarks indicate that our proposed SUGE achieves state-of-the-art performance.

TIKP: Text-to-Image Knowledge Preservation for Continual Semantic Segmentation

Continual Semantic Segmentation (CSS) is an emerging trend, where catastrophic forgetting has been a perplexing problem. In this paper, we propose a Text-to-Image Knowledge Preservation (TIKP) framework to address this issue. TIKP applies Text-to-Image techniques to CSS by automatically generating prompts and content adaptation. It extracts associations between the labels of seen data and constructs text-level prompts based on these associations, which are preserved and maintained at each incremental step. During training, these prompts generate correlated images to mitigate the catastrophic forgetting. Particularly, as the generated images may have different distributions from the original data, TIKP transfers the knowledge by a content adaption loss, which determines the role played by the generated images in incremental training based on the similarity. In addition, for the classifier, we use the previous model from a different perspective: misclassifying new classes into old objects instead of the background. We propose a knowledge distillation loss based on wrong labels, enabling us to attribute varying weights to individual objects during the distillation process. Extensive experiments conducted in the same setting show that TIKP outperforms state-of-the-art methods by a large margin on benchmark datasets.

Fully Data-Driven Pseudo Label Estimation for Pointly-Supervised Panoptic Segmentation

The core of pointly-supervised panoptic segmentation is estimating accurate dense pseudo labels from sparse point labels to train the panoptic head. Previous works generate pseudo labels mainly based on hand-crafted rules, such as connecting multiple points into polygon masks, or assigning the label information of labeled pixels to unlabeled pixels based on the artificially defined traversing distance. The accuracy of pseudo labels is limited by the quality of the hand-crafted rules (polygon masks are rough at object contour regions, and the traversing distance error will result in wrong pseudo labels). To overcome the limitation of hand-crafted rules, we estimate pseudo labels with a fully data-driven pseudo label branch, which is optimized by point labels end-to-end and predicts more accurate pseudo labels than previous methods. We also train an auxiliary semantic branch with point labels, it assists the training of the pseudo label branch by transferring semantic segmentation knowledge through shared parameters. Experiments on Pascal VOC and MS COCO demonstrate that our approach is effective and shows state-of-the-art performance compared with related works. Codes are available at https://github.com/BraveGroup/FDD.

Deep Reconstruction Transfer Convolutional Neural Network for Rolling Bearing Fault Diagnosis.

Deep transfer learning has been widely used to improve the versatility of models. In the problem of cross-domain fault diagnosis in rolling bearings, most models require that the given data have a similar distribution, which limits the diagnostic effect and generalization of the model. This paper proposes a deep reconstruction transfer convolutional neural network (DRTCNN), which satisfies the domain adaptability of the model under cross-domain conditions. Firstly, the model uses a deep reconstruction convolutional automatic encoder for feature extraction and data reconstruction. Through sharing parameters and unsupervised training, the structural information of target domain samples is effectively used to extract domain-invariant features. Secondly, a new subdomain alignment loss function is introduced to align the subdomain distribution of the source domain and the target domain, which can improve the classification accuracy by reducing the intra-class distance and increasing the inter-class distance. In addition, a label smoothing algorithm considering the credibility of the sample is introduced to train the model classifier to avoid the impact of wrong labels on the training process. Three datasets are used to verify the versatility of the model, and the results show that the model has a high accuracy and stability.

Tolerance analysis of wrong labels based on transformer for sentiment analysis

Tolerance analysis of wrong labels based on transformer for sentiment analysis

Sparse Graph Transformer With Contrastive Learning

Information aggregation and propagation over networks via graph neural networks (GNNs) plays an important role in node or graph representation learning, which currently depend on the calculation with a fixed adjacency matrix, facing over-smoothing problem, and difficulty to stack multiple layers for high-level representations. In contrast, Transformer calculates an importance score for each node to learn its embedding via the attention mechanism and has achieved great successes in many natural language processing (NLP) and computer vision (CV) tasks. However, Transformer is inflexible to extend to graphs, as its input and output must have the same dimension. It will also become intractable to allocate attention over a large-scale graph due to distractions. Moreover, most graph Transformers are trained in supervised ways, which consume additional resources to annotate samples with potentially wrong labels and have limited generalization of representations. Therefore, this article attempts to build a new Sparse Graph Transformer with Contrastive learning for graph representation learning, called SGTC. Specifically, we first employ centrality measures to remove the redundant topological information from input graph according to the influences of nodes and edges, then disturb the pruned graph to get two different augmentation views, and learn node representations in a contrastive manner. Besides, a novel sparse attention mechanism is also proposed to capture structural features of graphs, which effectively save memory and training time. SGTC can produce low-dimensional and high-order node representations, which have better generalization for multiple tasks. The proposed model is evaluated on three downstream tasks over six networks, and experimental results confirm its superior performance against the state-of-the-art baselines.

Partial Sequence Labeling With Structured Gaussian Processes.

Existing partial sequence labeling models mainly focus on a max-margin framework that fails to provide an uncertainty estimation of the prediction. Furthermore, the unique ground-truth disambiguation strategy employed by these models may include wrong label information for parameter learning. In this article, we propose structured Gaussian processes for partial sequence labeling (SGPPSL), which encodes uncertainty in the prediction and does not need extra effort for model selection and hyperparameter learning. The model employs factor-as-piece approximation that divides the linear-chain graph structure into the set of pieces, which preserves the basic Markov random field structure and effectively avoids handling a large number of candidate output sequences generated by partially annotated data. Then, confidence measure is introduced in the model to address different contributions of candidate labels, which enables the ground-truth label information to be utilized in parameter learning. Based on the derived lower bound of the variational lower bound of the proposed model, variational parameters and confidence measures are estimated in the framework of alternating optimization. Moreover, a weighted Viterbi algorithm is proposed to incorporate confidence measures to sequence prediction, which considers label ambiguity arose from multiple annotations in the training data and thus helps improve the performance. SGPPSL is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.

Explanatory subgraph attacks against Graph Neural Networks

Graph Neural Networks (GNNs) are often viewed as black boxes due to their lack of transparency, which hinders their application in critical fields. Many explanation methods have been proposed to address the interpretability issue of GNNs. These explanation methods reveal explanatory information about graphs from different perspectives. However, the explanatory information may also pose an attack risk to GNN models.In this work, we will explore this problem from the explanatory subgraph perspective. To this end, we utilize a powerful GNN explanation method called SubgraphX and deploy it locally to obtain explanatory subgraphs from given graphs. Then we propose methods for conducting evasion attacks and backdoor attacks based on the local explainer. In evasion attacks, the attacker gets explanatory subgraphs of test graphs from the local explainer and replace their explanatory subgraphs with an explanatory subgraph of other labels, making the target model misclassify test graphs as wrong labels. In backdoor attacks, the attacker employs the local explainer to select an explanatory trigger and locate suitable injection locations. We validate the effectiveness of our proposed attacks on state-of-art GNN models and different datasets. The results also demonstrate that our proposed backdoor attack is more efficient, adaptable, and concealed than previous backdoor attacks.

Noise is the fatal poison: A Noise-aware Network for noisy dataset classification

Recently, many benchmark datasets have been found to contain noisy labels caused by unavoidable human mistakes. Many researchers propose new noise-aware loss functions to achieve robust classification performance. However, we discover that existing noise-aware loss functions cannot fully heal the damage caused by the noise. On the other hand, some methods filter out low confidence samples and train new models, whereas the filtered samples contain both noisy and hard samples that are critical for the robustness of models. Based on the above two discoveries, we devised the Noise-aware Network (NA-Net) for robust training with noisy labels. Each layer of NA-Net contains three groups of convolution kernels responsible for mix samples, clean samples, and noisy samples, termed as mix-kernels, clean-kernels, and noise-kernels, respectively. Mix-kernels are used for finding the clean samples with a newly devised noise-immune (NI) loss function; clean-kernels are targeted at learning better features without being misguided by noise; noise-kernels are trained by the remaining samples to rectify wrong labels for the next iteration. Meanwhile, for increasing the classification performance of mix-kernels, the extracted feature maps of clean-kernels without being poisoned are combined as the input of mix-kernels of the next layer. Also, the knowledge distillation strategy is adopted to distill the knowledge from clean-kernels to the noise-kernels. Extensive experiments demonstrate that the mutual promotion of three groups of kernels in NA-Net achieves state-of-the-art performance on both artificial noisy datasets and real-world datasets.

B-U-net: Holographic image segmentation of multi-scale dense particle field with noisy training dataset

Particle field characterization in reconstructed holographic images especially depicting fuel atomization in aerospace engines has been a challenge due to the multi-scale dense particles with complex structures. For the lack of standard dataset, the difficulties of extracting correct information from wrong labels created by traditional segmentation methods restrict the utilization of supervised deep learning models. This study proposes a deep neural network called B-U-net which contains Bayesian convolution layers to learn the distribution of parameters instead of the fixed values to restrain the overfitting of the noisy labels. The performance of the B-U-net is quantitatively tested in inline and off-axis reconstructed holographic images compared with other three methods. Two metrics named Positive Intersection over Union (PIoU) and adjusted Structural Similarity (SSIM) are applied for evaluation. The highest metric results of B-U-net prove the strong and robust ability in overall predicting and structural accuracy for two different kinds of images.

Robust Wafer Classification With Imperfectly Labeled Data Based on Self-Boosting Co-Teaching

Wafer classification is a critical task for semiconductor manufacturing. Most conventional algorithms require a large-scale perfectly-labeled dataset to train accurate classifiers. In practice, it is usually difficult or even impossible to collect perfect labels without errors, and the classification accuracy in the presence of imperfectly labeled data would degrade. To facilitate robust wafer classification with noisy labels, we propose a novel self-boosting co-teaching (SB-CT) approach. Specifically, we iteratively correct the wrong labels by using the predictions of two classifiers that are jointly trained with noisily labeled data. To make the proposed method of practical utility, we develop a novel method to accurately estimate the noise rate, and adopt a probability scaling technique to further improve the classification accuracy. As demonstrated by the experimental results based on two industrial datasets, the proposed SB-CT approach achieves superior accuracy over other conventional methods.

IS3L: An Integrated Self-training Semi-Supervised Learning Strategy for Underwater Acoustic Target Detection

This paper proposed IS3L (Integrated Self-training Semi-Supervised Learning), a universal underwater acoustic target detection strategy with a semi-supervised learning architecture, which can comprehensively refine weakly labeled samples and label samples automatically to boost the detection accuracy. To our best knowledge, this is the first work to combine semi-supervised learning with underwater target detection. A universal feature extraction strategy is employed to extract comprehensive features ranging from time/frequent domain to dedicated auditory parameters. An integrated self-training strategy is proposed to generate pseudo labels from unlabeled samples and meanwhile avoid wrong labeling via a voting mechanism. Then, the final detection can be made by the rear classifier, which has been fully trained with original labeled samples and generated pseudo-labeled samples. Experimental results, which are based on real sea trail data, show that the proposed algorithm can reach up to 98.13% in detection accuracy with only 50 labeled samples (7040 samples in total).

Hyperparameter Optimization for Transfer Learning-based Disease Detection in Cassava Plants

Cassava is quite possibly the most widely recognized staple food crop. It is a nutty-flavored, starchy root vegetable that is a primary energy source and carbs for individuals. During crop cultivation, cassava plant infections can influence the leaf and root, bringing about a tremendous loss to the harvest and financial market esteem. Hence, it is vital to detect diseases in cassava plants. But it requires enormous labor, longer time planning, and thorough plant-specific knowledge. If disease detection is possible at the initial stages, then actions can be taken on time. Hence, there is a need to develop automatic detection methods for monitoring different parts of cassava plants. This study evaluates the efficiency of applying transfer learning to the pre-trained models for identifying diseases in cassava plants. The pre-trained EfficientNet model detects the disorders using data augmentation, fine-tuning the hyperparameters, cross-validation, and transfer learning. The experimentation is done with the cassava dataset provided by Kaggle, which contains cassava plant leaf images belonging to five classes. An experimental investigation shows that EfficientNet with transfer learning attains up to 89% accuracy. The effect of transfer learning is significant; consider getting the results of high accuracy and less dispersion; in very few cases, the model forecasts the wrong class labels. The outcomes give a promising strength to the objective of this work, i.e., a model trained explicitly for agriculture with transfer learning can assist the farmers with highly accurate results during farming to get a high yield.

Uncertainty estimation-based adversarial attacks: a viable approach for graph neural networks

Uncertainty estimation has received momentous consideration in applied machine learning to capture model uncertainty. For instance, the Monte-Carlo dropout method (MC-dropout), an approximated Bayesian approach, has gained intensive attention in producing model uncertainty due to its simplicity and efficiency. However, MC-dropout has revealed shortcomings in capturing erroneous predictions lying in the overlapping classes. Such predictions underlie noisy data points that can neither be reduced by more training data nor detected by model uncertainty. On the other hand, Monte-Carlo based on adversarial attacks (MC-AA), an outstanding method, performs perturbations on the inputs using the adversarial attack idea to capture model uncertainty. This method admittedly mitigates the shortcomings of the previous methods by capturing wrong labels in overlapping regions. Motivated by this method that was only validated with neural networks, we sought to apply MC-AA on various graph neural network models to obtain uncertainties using two public real-world graph datasets known as Elliptic and GitHub. First, we perform binary node classifications, then we apply MC-AA and other recent uncertainty estimation methods to capture the uncertainty of the models. Uncertainty evaluation metrics are computed to evaluate and compare the performance of the uncertainty of the model. We highlight the efficacy of MC-AA in capturing uncertainties in graph neural networks wherein MC-AA outperforms other given methods.