Inaccurate Labels Research Articles

Beyond clean data: Exploring the effects of label noise on object detection performance

In recent years, the growth of large-scale datasets has significantly propelled the progress of deep learning applications. Yet, annotating these datasets remains a labor-intensive endeavor, pushing the reliance on cost-effective but less specialized data collection methods and internet data sources. This often results in noisy and inaccurate labels, compromising data quality. Traditional machine learning models assume clean data, but real-world datasets often exhibit significant label noise. This paper examines the impact of such noise on object detection performance, a pivotal aspect of computer vision. We analyze the influence of noisy labels using three renowned object detection frameworks: YOLOv5, Faster R-CNN, and the recent YOLOv8, alongside established datasets: MS COCO, VOC, and ExDARK. Additionally, experiments with the UVM dataset explore domain-specific tasks in dense object scenarios. Two new metrics — Model Health and Detection Capability — were introduced to evaluate the results. Findings indicate that models maintain over 80% of their health (a 20% decline in mAP from the baseline) with up to 40% label corruption. However, Detection Capability deteriorates more sharply under the same conditions. The research also employs the D-RISE method for model explainability, highlighting crucial image regions affecting detection outcomes. Despite the noise, critical detection areas in models remain similar to those in clean data up to the 40% corruption level, as verified by similarity metrics. This study underscores the resilience of object detection models to label noise and provides insights into maintaining performance amidst data quality challenges.

Investigation of the Role of Chemical Analysis in Causality Assessment of Herbal and Dietary Supplement-Induced Liver Injury.

The attribution of drug-induced liver injury (DILI) to specific herbal and dietary supplements (HDS) is confounded by inaccurate labels and undisclosed ingredients. The US Drug-Induced Liver Injury Network (DILIN) determines the attribution of injury to an agent through its structured expert opinion causality assessment process, but without the use of chemical analysis data of HDS. We aimed to determine the impact of chemical analysis of HDS products on prior causality assessment scores. Obtained samples of HDS consumed by DILIN-enrolled patients were analyzed by high-performance liquid chromatography-mass spectrometry (HPLC-MS). Chemical analysis data were compared to label accuracy and detect whether the product contained botanical and non-botanical compounds. A comparison of the causality scores reassessed with chemical analysis was compared with the original scores. A total of 54 previously adjudicated cases with chemical analysis available were reassessed for causality with chemical analysis data; reviewers were blinded to original causality scores. Using the chemical analysis data, 37% (n = 20) of the 54 cases were scored with a higher likelihood of DILI compared with the original causality scores; 14 of the 20 (70%) moved from probable to highly likely; 52% had no change in causality score; and 11% of cases were scored as a lower likelihood of DILI. Our study demonstrates that there is value in using HDS chemical analysis data in the causality assessment process for DILI. In more than a third of cases, chemical analysis of products led to an increased confidence in DILI attribution to HDS. These findings suggest that chemical analysis is an important tool in causality assessment for HDS agents, specifically in challenging situations, and further studies are needed to confirm its applicability in clinical practice.

Wages: The Worst Transistor Aging Analysis for Large-scale Analog Integrated Circuits via Domain Generalization

Transistor aging leads to the deterioration of analog circuit performance over time. The worst aging degradation is used to evaluate the circuit reliability. It is extremely expensive to obtain it since several circuit stimuli need to be simulated. The worst degradation collection cost reduction brings an inaccurate training dataset when a machine learning (ML) model is used to fast perform the estimation. Motivated by the fact that there are many similar subcircuits in large-scale analog circuits, in this article we propose Wages to train an ML model on an inaccurate dataset for the worst aging degradation estimation via a domain generalization technique. A sampling-based method on the feature space of the transistor and its neighborhood subcircuit is developed to replace inaccurate labels. A consistent estimation for the worst degradation is enforced to update model parameters. Label updating and model updating are performed alternately to train an ML model on the inaccurate dataset. Experimental results on the very advanced 5 nm technology node show that our Wages can significantly reduce the label collection cost with a negligible estimation error for the worst aging degradations compared to the traditional methods.

Enhanced Domain Adaptation for Foot Ulcer Segmentation Through Mixing Self-Trained Weak Labels.

Wound management requires the measurement of the wound parameters such as its shape and area. However, computerized analysis of the wound suffers the challenge of inexact segmentation of the wound images due to limited or inaccurate labels. It is a common scenario that the source domain provides an abundance of labeled data, while the target domain provides only limited labels. To overcome this, we propose a novel approach that combines self-training learning and mixup augmentation. The neural network is trained on the source domain to generate weak labels on the target domain via the self-training process. In the second stage, generated labels are mixed up with labels from the source domain to retrain the neural network and enhance generalization across diverse datasets. The efficacy of our approach was evaluated using the DFUC 2022, FUSeg, and RMIT datasets, demonstrating substantial improvements in segmentation accuracy and robustness across different data distributions. Specifically, in single-domain experiments, segmentation on the DFUC 2022 dataset scored a dice score of 0.711, while the score on the FUSeg dataset achieved 0.859. For domain adaptation, when these datasets were used as target datasets, the dice scores were 0.714 for DFUC 2022 and 0.561 for FUSeg.

The indexation of retracted literature in seven principal scholarly databases: a coverage comparison of dimensions, OpenAlex, PubMed, Scilit, Scopus, The Lens and Web of Science

In this study, the coverage and overlap of retracted publications, retraction notices and withdrawals are compared across seven significant scholarly databases, with the aim to check for discrepancies, pinpoint the causes of those discrepancies, and choose the best product to produce the most accurate picture of retracted literature. Seven scholarly databases were searched to obtain all the retracted publications, retraction notices and withdrawal from 2000. Only web search interfaces were used, excepting in OpenAlex and Scilit. The findings demonstrate that non-selective databases (Dimensions, OpenAlex, Scilit, and The Lens) index a greater amount of retracted literature than do databases that rely their indexation on venue selection (PubMed, Scopus, and WoS). The key factors explaining these discrepancies are the indexation of withdrawals and proceeding articles. Additionally, the high coverage of OpenAlex and Scilit could be explained by the inaccurate labeling of retracted documents in Scopus, Dimensions, and The Lens. 99% of the sample is jointly covered by OpenAlex, Scilit and WoS. The study suggests that research on retracted literature would require querying more than one source and that it should be advisable to accurately identify and label this literature in academic databases.

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network.

Despite recent notable advancements in highlight image restoration techniques, the dearth of annotated data and the lightweight deployment of highlight removal networks pose significant impediments to further advancements in the field. In this paper, to the best of our knowledge, we first propose a semi-supervised learning paradigm for highlight removal, merging the fusion version of a teacher-student model and a generative adversarial network, featuring a lightweight network architecture. Initially, we establish a dependable repository to house optimal predictions as pseudo ground truth through empirical analyses guided by the most reliable No-Reference Image Quality Assessment (NR-IQA) method. This method serves to assess rigorously the quality of model predictions. Subsequently, addressing concerns regarding confirmation bias, we integrate contrastive regularization into the framework to curtail the risk of overfitting on inaccurate labels. Finally, we introduce a comprehensive feature aggregation module and an extensive attention mechanism within the generative network, considering a balance between network performance and computational efficiency. Our experimental evaluations encompass comprehensive assessments on both full-reference and non-reference highlight benchmarks. The results demonstrate conclusively the substantive quantitative and qualitative enhancements achieved by our proposed algorithm in comparison to state-of-the-art methodologies.

Electrical Sensor Calibration by Fuzzy Clustering with Mandatory Constraint.

Electrical tomography sensors have been widely used for pipeline parameter detection and estimation. Before they can be used in formal applications, the sensors must be calibrated using enough labeled data. However, due to the high complexity of actual measuring environments, the calibrated sensors are inaccurate since the labeling data may be uncertain, inconsistent, incomplete, or even invalid. Alternatively, it is always possible to obtain partial data with accurate labels, which can form mandatory constraints to correct errors in other labeling data. In this paper, a semi-supervised fuzzy clustering algorithm is proposed, and the fuzzy membership degree in the algorithm leads to a set of mandatory constraints to correct these inaccurate labels. Experiments in a dredger validate the proposed algorithm in terms of its accuracy and stability. This new fuzzy clustering algorithm can generally decrease the error of labeling data in any sensor calibration process.

Multi-source domain adaptation handling inaccurate label spaces

Domain adaptation with inaccurate label is a challenging and interesting topic in transfer learning, dealing with source and target domains with shift label spaces. Most existing domain adaptation methods assume aware label distributions among source and target domains. However, this cannot always be guaranteed in reality. Furthermore, existing multi-domain adaptation methods rarely deal with label heterogeneity among source domains. Thus, in this paper, we propose a multi-source domain adaptation method handling Inaccurate label (IncLabDA) during transfer. The proposed method designs a module that can transfer knowledge from multi-source domains with both homogeneous and heterogeneous label spaces in universal scenario. Anchors are generated from pre-trained model to build data-matching via a contrastive method avoiding to referring original data. In addition, class center consistency combined with clustering strategy considering both global and local confidences is adopted to recognize out-of-distribution samples. By removing source private classes and target unknown samples, highly confident target samples are collected to self-supervise the adaptation. At the same time, constraints enlarging the distance among target known classes and between the known and unknown samples are applied to enhance the performance of the proposed model. Experiments on real-world datasets validate the superiority of the IncLabDA model.

Label Accuracy of Weight Loss Dietary Supplements Marketed Online With Military Discounts

Dietary supplements for weight loss, among the most popular supplement products on the market, are promoted not only for losing weight and shedding fat, but also for added benefits of energy and performance, all packed into 1 capsule with multiple combinations of ingredients. Fraudulent marketing of weight loss supplements, some with exaggerated claims, some that are potentially dangerous, and some that contain illegal ingredients, is ever present, especially through online sources, where multiple manufacturers target service members by offering military discounts. To examine whether select dietary supplements marketed online for weight loss from companies advertising military discounts are accurately labeled according to the Supplement Facts listed ingredients, whether they contain any ingredients prohibited for use in the military, and to qualitatively describe the products' label claims. In this case series, 30 dietary supplement products marketed for weight loss were selected and purchased in June 2023 from 12 online companies advertising military discounts. Data were analyzed from July to August 2023. Liquid chromatography-mass spectrometry was used to verify whether products were accurately labeled according to the Supplement Facts listed ingredients and whether they contained any substances on the DoD Prohibited Dietary Supplement Ingredients List. A separate analysis was conducted to describe product label claims by using the Operation Supplement Safety (OPSS) Risk Assessment Scorecard. Of the 30 products tested, analysis showed that 25 had inaccurate labels. Of these, 24 had ingredients listed on the label that were not detected (misbranded); 7 had hidden components not present on the label, some of which would be considered adulterated; and 10 had substances on the DoD Prohibited Dietary Supplement Ingredients List either on or hidden from the label. All products were rated as risky when applying the OPSS Scorecard. In this case series study, the majority of products had inaccurate labels. Some were misbranded, others would be considered adulterated with ingredients not allowed in dietary supplements, and some contained ingredients prohibited for use in the military.

Improved training framework in a neural network model for disruption prediction and its application on EXL-50

A neural network model with a classical annotation method has been used on the EXL-50 tokamak to predict impending disruption. However, the results revealed issues of overfitting and overconfidence in predictions caused by inaccurate labeling. To mitigate these issues, an improved training framework has been proposed. In this approach, soft labels from previous training serve as teachers to supervise the further learning process; this has lead to a significant improvement in predictive model performance. Notably, this enhancement is primarily attributed to the coupling effect of the soft labels and correction mechanism. This improved training framework introduces an instance-specific label smoothing method, which reflects a more nuanced model assessment on the likelihood of a disruption. It presents a possible solution to effectively address the challenges associated with accurate labeling across different machines.

Generative Calibration of Inaccurate Annotation for Label Distribution Learning

Label distribution learning (LDL) is an effective learning paradigm for handling label ambiguity. When applying LDL, it typically requires datasets annotated with label distributions. However, obtaining supervised data for LDL is a challenging task. Due to the randomness of label annotation, the annotator can produce inaccurate annotation results for the instance, affecting the accuracy and generalization ability of the LDL model. To address this problem, we propose a generative approach to calibrate the inaccurate annotation for LDL using variational inference techniques. Specifically, we assume that instances with similar features share latent similar label distributions. The feature vectors and label distributions are generated by Gaussian mixture and Dirichlet mixture, respectively. The relationship between them is established through a shared categorical variable, which effectively utilizes the label distribution of instances with similar features, and achieves a more accurate label distribution through the generative approach. Furthermore, we use a confusion matrix to model the factors that contribute to the inaccuracy during the annotation process, which captures the relationship between label distributions and inaccurate label distributions. Finally, the label distribution is used to calibrate the available information in the noisy dataset to obtain the ground-truth label distribution.

Evaluation of dispensaries’ cannabis flowers for accuracy of labeling of cannabinoids content

BackgroundCannabis policies have changed drastically over the last few years with many states enacting medical cannabis laws, and some authorizing recreational use; all against federal laws. As a result, cannabis products are marketed in dispensaries in different forms, most abundantly as flowers intended for smoking and sometimes vaping. All samples used in this study were obtained directly from law enforcement. The sample collection process was facilitated and funded by the National Marijuana Initiative (NMI), part of the High-Intensity Drug Trafficking Area (HIDTA) program. This initial report focuses on cannabis flowers. Similar studies with other cannabis products will be the subject of a future report.MethodsA total of 107 Δ9-THC cannabis flower samples were collected by law enforcement from adult commercial use cannabis dispensaries, located in three different states (Colorado, Oregon, and California) and analyzed in this study for cannabinoid concentration. Samples were analyzed by GC-FID following our previously published procedure.DiscussionThe label claims for total Δ9-THC content ranged from 12.04 to 58.20% w/w, while GC-FID results showed a concentration ranging from 12.95 to 36.55% w/w. Of the evaluated 107 products, only 32 samples have Δ9-THC content within ± 20% of the labeled content. However, the remaining 75 samples were found to be out of the ± 20% acceptance criteria. The degree of agreement for the tested samples using ± 20% tolerance with label claims was only 30%. The results of this study indicate that there is a need for more stringent regulations to ensure that product labeling is accurate, as 70% of the evaluated products did not meet the ± 20% acceptance criteria. This highlights the importance of healthcare professionals and patients being vigilant about the Δ9-THC content, as inaccurate labeling of cannabis products could potentially result in adverse health effects. Furthermore, there is a pressing need for more rigorous regulation of commercial cannabis products in the United States.

Multi-class center dynamic contrastive learning for unsupervised domain adaptation person re-identification

Unsupervised domain adaptation person re-identification (UDA Re-ID) aims to leverage the pedestrian knowledge learned from labeled source domain to assist in learning the pedestrian knowledge in the unlabeled target domain. Most of existing investigations typically utilize single-class center clustering algorithms to group unlabeled target domain instances. Unfortunately, single-class center clustering algorithms tend to cluster pedestrian pictures from different identities into the same cluster, leading to inaccurate labels. Training with these noisy labels can undesirably deteriorate the accuracy of UDA Re-ID. Responding to the problem, we propose a multi-class center dynamic contrastive learning (MCC-DCL) for UDA Re-ID, which includes three main parts: multi-center clustering (MCC), dynamic pseudo-labeling (DPL), and dynamic contrastive learning (DCL). In order to reduce noisy labels generated during clustering, we introduce MCC method to generates reliable pseudo-labels for instances. Furthermore, to fully utilize the knowledge learned by the network during each iteration, we propose DPL method to optimizes the pseudo-labels of instances. Finally, for improving the discriminative performance of model and its tolerance to noisy labels, we propose DCL method that utilizes dynamic pseudo-labels and dynamic contrastive loss for supervised training. Comprehensive experiments and analyses demonstrate that MCC-DCL significantly outperforms existing approaches in UDA Re-ID tasks.

Attention-guided jaw bone lesion diagnosis in panoramic radiography using minimal labeling effort

Developing a deep-learning-based diagnostic model demands extensive labor for medical image labeling. Attempts to reduce the labor often lead to incomplete or inaccurate labeling, limiting the diagnostic performance of models. This paper (i) constructs an attention-guiding framework that enhances the diagnostic performance of jaw bone pathology by utilizing attention information with partially labeled data; (ii) introduces an additional loss to minimize the discrepancy between network attention and its label; (iii) introduces a trapezoid augmentation method to maximize the utility of minimally labeled data. The dataset includes 716 panoramic radiograph data for jaw bone lesions and normal cases collected and labeled by two radiologists from January 2019 to February 2021. Experiments show that guiding network attention with even 5% of attention-labeled data can enhance the diagnostic accuracy for pathology from 92.41 to 96.57%. Furthermore, ablation studies reveal that the proposed augmentation methods outperform prior preprocessing and augmentation combinations, achieving an accuracy of 99.17%. The results affirm the capability of the proposed framework in fine-grained diagnosis using minimally labeled data, offering a practical solution to the challenges of medical image analysis.

Benchmarking informatics approaches for virus discovery: caution is needed when combining in silico identification methods.

The identification of viruses from environmental metagenomes using informatics tools has offered critical insights in microbial ecology. However, it remains difficult for researchers to know which tools optimize viral recovery for their specific study. In an attempt to recover more viruses, studies are increasingly combining the outputs from multiple tools without validating this approach. After benchmarking combinations of six viral identification tools against mock metagenomes and environmental samples, we found that these tools should only be combined cautiously. Two to four tool combinations maximized viral recovery and minimized non-viral contamination compared with either the single-tool or the five- to six-tool ones. By providing a rigorous overview of the behavior of in silico viral identification strategies and a pipeline to replicate our process, our findings guide the use of existing viral identification tools and offer a blueprint for feature engineering of new tools that will lead to higher-confidence viral discovery in microbiome studies.

Multi-Identity Recognition of Darknet Vendors Based on Metric Learning

Dark web vendor identification can be seen as an authorship aliasing problem, aiming to determine whether different accounts on different markets belong to the same real-world vendor, in order to locate cybercriminals involved in dark web market transactions. Existing open-source datasets for dark web marketplaces are outdated and cannot simulate real-world situations, while data labeling methods are difficult and suffer from issues such as inaccurate labeling and limited cross-market research. The problem of identifying vendors’ multiple identities on the dark web involves a large number of categories and a limited number of samples, making it difficult to use traditional multiclass classification models. To address these issues, this paper proposes a metric learning-based method for dark web vendor identification, collecting product data from 21 currently active English dark web marketplaces and using a multi-dimensional feature extraction method based on product titles, descriptions, and images. Using pseudo-labeling technology combined with manual labeling improves data labeling accuracy compared to previous labeling methods. The proposed method uses a Siamese neural network with metric learning to learn the similarity between vendors and achieve the recognition of vendors’ multiple identities. This method achieved better performance with an average F1-score of 0.889 and an accuracy rate of 97.535% on the constructed dataset. The contributions of this paper lie in the proposed method for collecting and labeling data for dark web marketplaces and overcoming the limitations of traditional multiclass classifiers to achieve effective recognition of vendors’ multiple identities.

ATR-FTIR spectroscopy and LDA: a rapid, non-destructive and cost effective strategy to trace the geographical origin of Cannabis sativa L

Abstract Cannabis recreational and/or medicinal use have been legalized in the past years in many states and countries. As a consequence, many Cannabis growers and product developers have emerged in a new market throughout the world; at the same time, issues regarding questionable quality control have also risen, as several reports on Cannabis users’ health-related problems caused by inaccurate labeling content in Cannabis-based medicines, edibles or other derivatives are being published and brought out to the public’s attention. These facts make traceability methodologies crucial whether for forensic use, such as drug trafficking eradication, or for quality control purposes of legal Cannabis and of products derived from it. Hence, the objective of this study was to analyze Cannabis by means of Attenuated Total Reflection Spectroscopy (ATR-FTIR) in order to assess the capability of this technique to trace the geographical origin of Cannabis cultivated in Brazil and in Colorado, United States of America. Forty-seven samples from Brazil and 18 samples from Colorado were analyzed by ATR-FTIR. Linear Discriminant Analysis (LDA) was employed to source the samples. The combination of ATR-FTIR and LDA achieved up to 95.23% accuracy in assigning Cannabis samples to their geographical locations of origin in Brazil and up to 100% in Colorado.

Relieving the burden of intensive labeling for stress monitoring in the wild by using semi-supervised learning.

Stress, a natural process affecting individuals' wellbeing, has a profound impact on overall quality of life. Researchers from diverse fields employ various technologies and methodologies to investigate it and alleviate the negative effects of this phenomenon. Wearable devices, such as smart bands, capture physiological data, including heart rate variability, motions, and electrodermal activity, enabling stress level monitoring through machine learning models. However, labeling data for model accuracy assessment poses a significant challenge in stress-related research due to incomplete or inaccurate labels provided by individuals in their daily lives. To address this labeling predicament, our study proposes implementing Semi-Supervised Learning (SSL) models. Through comparisons with deep learning-based supervised models and clustering-based unsupervised models, we evaluate the performance of our SSL models. Our experiments show that our SSL models achieve 77% accuracy with a classifier trained on an augmented dataset prepared using the label propagation (LP) algorithm. Additionally, our deep autoencoder network achieves 76% accuracy. These results highlight the superiority of SSL models over unsupervised learning techniques and their comparable performance to supervised learning models, even with limited labeled data. By relieving the burden of labeling in daily life stress recognition, our study advances stress-related research, recognizing stress as a natural process rather than a disease. This facilitates the development of more efficient and accurate stress monitoring methods in the wild.

Inaccurate Label Distribution Learning

Inaccurate Label Distribution Learning

Current State of Milk, Dairy Products, Meat and Meat Products, Eggs, Fish and Fishery Products Authentication and Chemometrics.

Food fraud is a matter of major concern as many foods and beverages do not follow their labelling. Because of economic interests, as well as consumers' health protection, the related topics, food adulteration, counterfeiting, substitution and inaccurate labelling, have become top issues and priorities in food safety and quality. In addition, globalized and complex food supply chains have increased rapidly and contribute to a growing problem affecting local, regional and global food systems. Animal origin food products such as milk, dairy products, meat and meat products, eggs and fish and fishery products are included in the most commonly adulterated food items. In order to prevent unfair competition and protect the rights of consumers, it is vital to detect any kind of adulteration to them. Geographical origin, production methods and farming systems, species identification, processing treatments and the detection of adulterants are among the important authenticity problems for these foods. The existence of accurate and automated analytical techniques in combination with available chemometric tools provides reliable information about adulteration and fraud. Therefore, the purpose of this review is to present the advances made through recent studies in terms of the analytical techniques and chemometric approaches that have been developed to address the authenticity issues in animal origin food products.