Label Ambiguity Research Articles

Large Margin Weighted k-Nearest Neighbors Label Distribution Learning for Classification.

Label distribution learning (LDL) helps solve label ambiguity and has found wide applications. However, it may suffer from the challenge of objective inconsistency when adopted to classification problems because the learning objective of LDL is inconsistent with that of classification. Some LDL algorithms have been proposed to solve this issue, but they presume that label distribution can be represented by the maximum entropy model, which may not hold in many real-world problems. In this article, we design two novel LDL methods based on the k -nearest neighbors ( k NNs) approach without assuming any form of label distribution. First, we propose the large margin weighted k NN LDL (LW- k NNLDL). It learns a weight vector for the k NN algorithm to learn label distribution and implement a large margin to address the objective inconsistency. Second, we put forward the large margin distance-weighted k NN LDL (LD k NN-LDL) that learns distance-dependent weight vectors to consider the difference in the neighborhoods of different instances. Theoretical results show that our methods can learn any general-form label distribution. Moreover, extensive experimental studies validate that our methods significantly outperform state-of-the-art LDL approaches.

Augmenting biomedical named entity recognition with general-domain resources

ObjectiveTraining a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. MethodsWe proposed GERBERA, a simple-yet-effective method that utilized general-domain NER datasets for training. We performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. ResultsWe systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with additional BioNER datasets. Specifically, our models consistently outperformed the baseline models in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight entities. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset. ConclusionThis study introduces a new training method that leverages cost-effective general-domain NER datasets to augment BioNER models. This approach significantly improves BioNER model performance, making it a valuable asset for scenarios with scarce or costly biomedical datasets. We make data, codes, and models publicly available via https://github.com/qingyu-qc/bioner_gerbera.

Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification

Abstract Hierarchical text classification (HTC) is an important task with broad applications, and few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.

Decoupling Implantation Prediction and Embryo Ranking in Machine Learning: The Impact of Clinical Data and Discarded Embryos

Automated live embryo imaging has transformed in vitro fertilization (IVF) into a data‐intensive field. Unlike clinicians who rank embryos from the same IVF cycle cohort based on the embryos visual quality and determine how many embryos to transfer based on clinical factors, machine learning solutions usually combine these steps by optimizing for implantation prediction and using the same model for ranking the embryos within a cohort. Herein, it is established that this strategy can lead to suboptimal selection of embryos. It is revealed that despite enhancing implantation prediction, inclusion of clinical properties hampers ranking. Moreover, it is found that ambiguous labels of failed implantations, due to either low‐quality embryos or poor clinical factors, confound both the optimal ranking and even implantation prediction. To overcome these limitations, conceptual and practical steps are proposed to enhance machine learning‐driven IVF solutions. These consist of separating the optimizing of implantation from ranking by focusing on visual properties for ranking and reducing label ambiguity.

Adaptive ambiguity-aware weighting for multi-label recognition with limited annotations

In multi-label recognition, effectively addressing the challenge of partial labels is crucial for reducing annotation costs and enhancing model generalization. Existing methods exhibit limitations by relying on unrealistic simulations with uniformly dropped labels, overlooking how ambiguous instances and instance-level factors impacts label ambiguity in real-world datasets. To address this deficiency, our paper introduces a realistic partial label setting grounded in instance ambiguity, complemented by Reliable Ambiguity-Aware Instance Weighting (R-AAIW)—a strategy that utilizes importance weighting to adapt dynamically to the inherent ambiguity of multi-label instances. The strategy leverages an ambiguity score to prioritize learning from clearer instances. As proficiency of the model improves, the weights are dynamically modulated to gradually shift focus towards more ambiguous instances. By employing an adaptive re-weighting method that adjusts to the complexity of each instance, our approach not only enhances the model’s capability to detect subtle variations among labels but also ensures comprehensive learning without excluding difficult instances. Extensive experimentation across various benchmarks highlights our approach’s superiority over existing methods, showcasing its ability to provide a more accurate and adaptable framework for multi-label recognition tasks.

Feature Selection for Handling Label Ambiguity Using Weighted Label-Fuzzy Relevancy and Redundancy

Feature Selection for Handling Label Ambiguity Using Weighted Label-Fuzzy Relevancy and Redundancy

Incomplete label distribution learning via label correlation decomposition

Label distribution learning (LDL) has garnered increased attention in recent studies on label ambiguity. However, collecting complete annotations for LDL tasks is often time-consuming and labor-intensive compared to traditional learning paradigms. Therefore, designing effective incomplete LDL algorithms is crucial to broaden LDL’s application scope. In this paper, we propose a novel LDL algorithm, called Incomplete Label Distribution Learning via Label Correlation Decomposition (IncomLDL-LCD), which simultaneously learns label distributions and recovers missing description degrees of labels through label correlations. Specifically, we decompose the label correlation into sparse local label correlation and low-rank global label correlation using a soft-thresholding operator and a singular value thresholding operator, respectively. The former is utilized to capture the related label subsets necessary for reconstructing each possible label, while the latter focuses on extracting the coarse-grained semantic concepts from all labels and exploring the groupings of labels. Additionally, we develop an alternating solution with the accelerated proximal gradient descent method for optimization. Extensive experiments on 16 real-world data sets with varying degrees of missing annotations validate that our algorithm effectively handles incomplete LDL tasks and outperforms state-of-the-art algorithms.

Label distribution feature selection based on hierarchical structure and neighborhood granularity

Label Distribution Learning (LDL) addresses label ambiguity in datasets but struggles with high-dimensional data due to irrelevant features. Label Distribution Feature Selection (LDFS) methods can effectively unravel the issues, but they often overlook the advantages of utilizing hierarchical relationships among data, which can improve feature discriminability. Furthermore, these methods inadequately consider the granulation process, directly affecting the important features’ identification. To overcome these challenges, this study proposes a novel LDFS approach incorporating hierarchical structures and neighborhood granularity. Our algorithm proceeds in three stages: initially, it forms a multi-granular representation of data to reveal hierarchical relationships; subsequently, in the granulation process, it employs a variable precision rough set model, leveraging neighborhood granularity for a nuanced feature relevance assessment; and finally, it synthesizes these findings via a fusion strategy, culminating in a hierarchical feature ranking. Extensive experiments are conducted on thirteen benchmark datasets against five different algorithms in terms of six evaluation metrics. The results show that our method outperforms competitors in about 80% of the cases, demonstrating its effectiveness and generalization.

Repeat and learn: Self-supervised visual representations learning by Repeated Scene Localization

Large labeled datasets are crucial for video understanding progress. However, the labeling process is time-consuming, expensive, and tiresome. To overcome this impediment, various pretexts use the temporal coherence in videos to learn visual representations in a self-supervised manner. However, these pretexts (order verification and sequence sorting) struggle when encountering cyclic actions due to the label ambiguity problem. To overcome these limitations, we present a novel temporal pretext task to address self-supervised learning of visual representations from unlabeled videos. Repeated Scene Localization (RSL) is a multi-class classification pretext that involves changing the temporal order of the frames in a video by repeating a scene. Then, the network is trained to identify the modified video, localize the location of the repeated scene, and identify the unmodified original videos that do not have repeated scenes. We evaluated the proposed pretext on two benchmark datasets, UCF-101 and HMDB-51. The experimental results show that the proposed pretext achieves state-of-the-art results in action recognition and video retrieval tasks. In action recognition, our S3D model achieves 88.15% and 56.86% on UCF-101 and HMDB-51, respectively. It outperforms the current state-of-the-art by 1.05% and 3.26%. Our R(2+1)D-Adjacent model achieves 83.52% and 54.50% on UCF-101 and HMDB-51, respectively. It outperforms the single pretext tasks by 8.7% and 13.9%. In video retrieval, our R(2+1)D-Offset model outperforms the single pretext tasks by 4.68% and 1.1% Top 1 accuracies on UCF-101 and HMDB-51, respectively. The source code and the trained models are publicly available at https://github.com/Hussein-A-Hassan/RSL-Pretext.

Stream label distribution learning processing via broad learning system

Label distribution learning (LDL) is designed for label ambiguity problem which widely exists in tasks like image classification and sentiment analysis. Nevertheless, most existing LDL methods adopt a batch-wise processing strategy that is not applicable to stream data issues, like video semantic segmentation and object tracking in auto-driving applications. For these scenarios where dynamic new data is generated continuously, stream LDL processing can be applied to reflect the changes in time and improve the accuracy of online prediction. This paper proposes a novel broad learning system (BLS) based online label distribution learning framework (SLD_BLS). Broad learning system was adopted as the baseline model because its incremental weight updating scheme, excellent efficiency, and outstanding performance can process streaming data in dynamically changing environments. However, there are several challenges to adapt BLS for LDL: 1) BLS's update rule cannot process LDL data; 2) BLS cannot show the mapping relationship between feature space and label space; 3) BLS neglects the inter-label relationships. To tackle these challenges, 1) a new weight update rule for LDL is designed; 2) a manifold regularization is applied to exploit the feature space manifolds; 3) an explicit label collaborating approach is introduced to positively bootstrapping the model training. Extensive experiments on 13 label distribution datasets indicated the performance of SLD_BLS outperforms 9 state-of-the-art LDL models on most datasets with significant reductions in training time across 6 metrics.

HE-Mind: A model for automatically predicting hematoma expansion after spontaneous intracerebral hemorrhage

PurposeTo develop and validate an end-to-end model for automatically predicting hematoma expansion (HE) after spontaneous intracerebral hemorrhage (sICH) using a novel deep learning framework. MethodsThis multicenter retrospective study collected cranial noncontrast computed tomography (NCCT) images of 490 patients with sICH at admission for model training (n = 236), internal testing (n = 60), and external testing (n = 194). A HE-Mind model was designed to predict HE, which consists of a densely connected U-net for segmentation process, a multi-instance learning strategy for resolving label ambiguity and a Siamese network for classification process. Two radiomics models based on support vector machine or logistic regression and two deep learning models based on residual network or Swin transformer were developed for performance comparison. Reader experiments including physician diagnosis mode and artificial intelligence mode were conducted for efficiency comparison. ResultsThe HE-Mind model showed better performance compared to the comparative models in predicting HE, with areas under the curve of 0.849 and 0.809 in the internal and external test sets respectively. With the assistance of the HE-Mind model, the predictive accuracy and work efficiency of the emergency physician, junior radiologist, and senior radiologist were significantly improved, with accuracies of 0.768, 0.789, and 0.809 respectively, and reporting times of 7.26 s, 5.08 s, and 3.99 s respectively. ConclusionsThe HE-Mind model could rapidly and automatically process the NCCT data and predict HE after sICH within three seconds, indicating its potential to assist physicians in the clinical diagnosis workflow of HE.

High levels of mislabelling of shark flesh in Australian fish markets and seafood shops

High levels of mislabelling of shark flesh in Australian fish markets and seafood shops

Label disambiguation-based feature selection for partial label learning via fuzzy dependency and feature discernibility

Partial label learning is a multi-class classification issue in which each training instance is associated with a set of candidate labels. Feature selection is an effective method to improve the performance of the learning model, and at the same time, feature selection being a challenging problem in partial label learning due to the label ambiguity of partial labels. To tackle this challenge, this paper proposes a label disambiguation-based feature selection for partial label learning via fuzzy dependency and feature discernibility. Specifically, considering the high sensitivity of the fuzzy rough sets model to pseudo labels, a instance distribution-based label disambiguation method is presented to reduce the noise from the candidate labels. Based on this, a weighted fuzzy rough sets model is constructed in accordance with the distribution information of labels, and the function of fuzzy dependency is redefined. Then, the evaluation function of feature significance is obtained by fusing fuzzy dependency and feature discernibility for identifying the critical features. Finally, extensive experiments have confirmed the feasibility and effectiveness of the proposed method. Compared to other feature selection algorithms, the proposed method exhibits superior performance and enhances the generalization of partial label learning models. Furthermore, the feasibility of the proposed label disambiguation method is demonstrated through comparison with state-of-the-art label disambiguation methods.

Learning from feature and label spaces’ bias for uncertainty-adaptive facial emotion recognition

Developing an accurate deep model for facial emotion recognition is a long-term challenge. It is because the uncertainty of emotions, stemming from the ambiguity of different emotional categories and the difference of subjective annotations, can ruin the ability of model to achieve the desired optimization. This paper constructs two distinct datasets, namely original sample set and ambiguous sample set, to explore an effective ambiguous knowledge transfer method to realize the adaptive awareness of uncertainty in facial emotion recognition. The original sample set is the weakly-augmented data with relatively low uncertainty, as most emotions are clean in reality. Meanwhile, the ambiguous sample set is strongly-augmented data that introduces feature and label bias with regard to emotion, which are with relatively high uncertainty. The proposed framework consists of two sub-nets, which are trained using the original set and the ambiguous set respectively. To achieve uncertainty-adaptive learning for two sub-nets, we introduce two modules. One is the cross-space attention consistency learning module that performs attention coupling across original and ambiguous feature spaces, achieving uncertainty-aware representation learning in feature granularity. The other is the soft-label learning module that models and utilizes uncertainty in label granularity, through aligning the posterior distributions between original label space and ambiguous label space. Experimental studies on public datasets indicate that our method is competitive with the state-of-the-art.

Retrospective and prospective study designs

The labels "retrospective" and "prospective" strongly connote study quality, often favouring prospective studies. However, three definitions of these terms exist, each suggesting distinct methodological limitations. In this review, we summarize and evaluate these definitions. Caution is warranted when labeling a study "retrospective": This label should only be used when implying a risk of recall bias, which can only occur in retrospective data collection. Generally, assessing random and systematic errors is necessary to appraise study quality rather than relying on ambiguous labels.

Generative Calibration of Inaccurate Annotation for Label Distribution Learning

Label distribution learning (LDL) is an effective learning paradigm for handling label ambiguity. When applying LDL, it typically requires datasets annotated with label distributions. However, obtaining supervised data for LDL is a challenging task. Due to the randomness of label annotation, the annotator can produce inaccurate annotation results for the instance, affecting the accuracy and generalization ability of the LDL model. To address this problem, we propose a generative approach to calibrate the inaccurate annotation for LDL using variational inference techniques. Specifically, we assume that instances with similar features share latent similar label distributions. The feature vectors and label distributions are generated by Gaussian mixture and Dirichlet mixture, respectively. The relationship between them is established through a shared categorical variable, which effectively utilizes the label distribution of instances with similar features, and achieves a more accurate label distribution through the generative approach. Furthermore, we use a confusion matrix to model the factors that contribute to the inaccuracy during the annotation process, which captures the relationship between label distributions and inaccurate label distributions. Finally, the label distribution is used to calibrate the available information in the noisy dataset to obtain the ground-truth label distribution.

Partial Sequence Labeling With Structured Gaussian Processes.

Existing partial sequence labeling models mainly focus on a max-margin framework that fails to provide an uncertainty estimation of the prediction. Furthermore, the unique ground-truth disambiguation strategy employed by these models may include wrong label information for parameter learning. In this article, we propose structured Gaussian processes for partial sequence labeling (SGPPSL), which encodes uncertainty in the prediction and does not need extra effort for model selection and hyperparameter learning. The model employs factor-as-piece approximation that divides the linear-chain graph structure into the set of pieces, which preserves the basic Markov random field structure and effectively avoids handling a large number of candidate output sequences generated by partially annotated data. Then, confidence measure is introduced in the model to address different contributions of candidate labels, which enables the ground-truth label information to be utilized in parameter learning. Based on the derived lower bound of the variational lower bound of the proposed model, variational parameters and confidence measures are estimated in the framework of alternating optimization. Moreover, a weighted Viterbi algorithm is proposed to incorporate confidence measures to sequence prediction, which considers label ambiguity arose from multiple annotations in the training data and thus helps improve the performance. SGPPSL is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.

The third way: object reordering as ambiguous labeling resolution

Abstract This paper examines free object order alternation in ditransitives, focusing on Slovenian. It is shown that neither a scrambling nor a base generation analysis is fully satisfactory. A new analysis based on the Labeling Algorithm is proposed, where it is argued that the introduction of a second object creates an ambiguous labeling scenario ({NP,VP}), which has two equivalent resolutions: (i) movement of the VP with the first object inside, or (ii) movement of the second object. This is shown to derive both free object order in the general case and the restrictions on object order in select contexts, as due to the specifics of the VP-movement analysis it is possible for selectional restrictions to filter out either derivation (i) or (ii).

RGBT Tracking via Challenge-Based Appearance Disentanglement and Interaction.

RGB and thermal source data suffer from both shared and specific challenges, and how to explore and exploit them plays a critical role in representing the target appearance in RGBT tracking. In this paper, we propose a novel approach, which performs target appearance representation disentanglement and interaction via both modality-shared and modality-specific challenge attributes, for robust RGBT tracking. In particular, we disentangle the target appearance representations via five challenge-based branches with different structures according to their properties, including three parameter-shared branches to model modality-shared challenges and two parameter-independent branches to model modality-specific challenges. Considering the complementary advantages between modality-specific cues, we propose a guidance interaction module to transfer discriminative features from one modality to another one to enhance the discriminative ability of weak modality. Moreover, we design an aggregation interaction module to combine all challenge-based target representations, which could form more discriminative target representations and fit the challenge-agnostic tracking process. These challenge-based branches are able to model the target appearance under certain challenges so that the target representations can be learned by a few parameters even in the situation of insufficient training data. In addition, to relieve labor costs and avoid label ambiguity, we design a generation strategy to generate training data with different challenge attributes. Comprehensive experiments demonstrate the superiority of the proposed tracker against the state-of-the-art methods on four benchmark datasets.

Modeling Uncertainty for Low-Resolution Facial Expression Recognition

Recently, facial expression recognition techniques have made significant progress on high-resolution web images. However, in real-world applications, the obtained images are often with low resolution since they are mostly captured in a wide range of public spaces. As a result, the ambiguity of the expression labels hinders recognition performance due to not only subjective emotion annotations but also ambiguous images. Existing approaches tend to perform poorly when the resolution of face images decreases. In this work, we aim to model the aleatoric uncertainty induced by low-image-resolution and label ambiguity for robust facial expression recognition. We propose probabilistic data uncertainty learning to capture the ambiguity induced by poor image resolution. Additionally, we introduce the emotion wheel to learn the label-uncertainty-aware embedding. Moreover, we exploit the ambiguous nature of neutrality and propose a neutral expression constraint to learn more robust features for facial expression recognition. To the best of our knowledge, this is the first work utilizing the intrinsic nature of neutrality as a regularization to benefit model training. Extensive experimental results show the effectiveness and robustness of our approach. Under low-resolution conditions, our proposed method outperforms the state-of-the-art approaches by 3.02% and 3.16% in terms of accuracy on RAF-DB and FERPlus, respectively.