Distribution Of Label Research Articles

Distance-guided domain adaptation for bearing fault diagnosis under variable operating conditions

Transfer learning (TL) has demonstrated effective application in diagnosing bearing faults under varying operating conditions. The current TL methods achieve domain alignment by minimizing the variation in the marginal distribution of data between the source and target domains in the feature space. However, this approach yields incomplete similarity, leading to domain shift and a decrease in diagnostic performance. To overcome this problem, this paper proposes a new distance-guided domain adaptation method that consists of two modules: deep domain adaptive correlation alignment (Deep CORAL) combined with joint maximum mean discrepancy (JMMD) for guided domain adaptation. Deep CORAL employs nonlinear transformations to synchronize second-order statistical correlations across source and target domains, thus ensuring feature-level alignment between these domains. JMMD is utilized to align the joint distribution of input features and output labels within the activation layer in the deep network, thereby bolstering domain alignment. Building on this, we propose a network structure that merges ResNet and bidirectional long short-term memory, powered by wavelet kernels, serving as a feature extractor. This structure is designed to learn domain-invariant features and incorporates attention mechanisms to amplify important information while diminishing the impact of redundant data. An analysis of bearing experiments is used to demonstrate the effectiveness of this method, and the proposed method significantly outperforms several popular methods in diagnostic performance.

Multi-source domain adaptation handling inaccurate label spaces

Domain adaptation with inaccurate label is a challenging and interesting topic in transfer learning, dealing with source and target domains with shift label spaces. Most existing domain adaptation methods assume aware label distributions among source and target domains. However, this cannot always be guaranteed in reality. Furthermore, existing multi-domain adaptation methods rarely deal with label heterogeneity among source domains. Thus, in this paper, we propose a multi-source domain adaptation method handling Inaccurate label (IncLabDA) during transfer. The proposed method designs a module that can transfer knowledge from multi-source domains with both homogeneous and heterogeneous label spaces in universal scenario. Anchors are generated from pre-trained model to build data-matching via a contrastive method avoiding to referring original data. In addition, class center consistency combined with clustering strategy considering both global and local confidences is adopted to recognize out-of-distribution samples. By removing source private classes and target unknown samples, highly confident target samples are collected to self-supervise the adaptation. At the same time, constraints enlarging the distance among target known classes and between the known and unknown samples are applied to enhance the performance of the proposed model. Experiments on real-world datasets validate the superiority of the IncLabDA model.

Instance redistribution-based label integration for crowdsourcing

Crowdsourcing provides an inexpensive solution to employ crowd workers labeling instances, and hence each instance is labeled with a multiple noisy label set instead of its true label. Label integration aims to infer a true label from its multiple noisy label set. However, existing fine-tuned methods increase the model complexity while enhancing the performance of label integration. Therefore, it is a challenge to search for a simple but efficient label integration method. Meeting this challenge, this paper proposes instance redistribution-based label integration (IRLI), a novel method for label integration. Firstly, from the label view, we estimate the probability of an instance belonging to each class based on its multiple noisy label set and obtain a multiple noisy label distribution vector. Secondly, from the attribute view, we reestimate the probability of an instance belonging to each class based on its attribute set and obtain an instance redistribution vector. Finally, we combine the multiple noisy label distribution vector and the instance redistribution vector of each instance to infer its integrated label. Comprehensive experimental results on both simulated and real-world crowdsourced datasets validate the superiority of IRLI.

Sequential Label Enhancement.

Label distribution learning (LDL) is a novel machine learning paradigm for solving ambiguous tasks, where the degree to which each label describing the instance is ambiguous. However, obtaining the label distribution is high cost and the description degree is difficult to quantify. Most existing research works focus on designing an objective function to obtain the whole description degrees at once but seldom care about the sequentiality in the process of recovering the label distribution. In this article, we formulate the label distribution recovering task as a sequential decision process called sequential label enhancement (Seq_LE), which is more consistent with the process of annotating the label distribution in human brains. Specifically, the discrete label and its description degree are serially mapped by the reinforcement learning (RL) agent. Besides, we carefully design a joint reward function to drive the agent to fully learn the optimal decision policy. Extensive experiments on 16 LDL datasets are conducted under various evaluation metrics. The experimental results demonstrate convincingly that the proposed sequential label enhancement (LE) leads to better performance over the state-of-the-art methods.

Decentralized and robust privacy-preserving model using blockchain-enabled Federated Deep Learning in intelligent enterprises

In Federated Deep Learning (FDL), multiple local enterprises are allowed to train a model jointly. Then, they submit their local updates to the central server, and the server aggregates the updates to create a global model. However, trained models usually perform worse than centralized models, especially when the training data distribution is non-independent and identically distributed (non-IID). Because non-IID data harms the accuracy and performance of the model. Second, due to the centrality of federated learning (FL) and the untrustworthiness of enterprises, traditional FL solutions are vulnerable to security and privacy attacks. Therefore, to tackle this issue, we propose FEDANIL, a secure blockchain-enabled Federated Deep LeArning Model that improves enterprise models’ decentralized, performance, and tamper-proof properties, including two main phases. The first phase is proposed to address the non-IID challenge (label and feature distribution skew). In this phase, local models with similar data distributions are grouped into homogeneous clusters using the cosine similarity (CS) and affinity propagation (AP) techniques. Then, for each homogeneous cluster, Wasserstein Generative Adversarial Networks (WGAN) are used to deal with label and feature distribution skew. The second phase was adopted to address security and privacy concerns against poisoning and inference attacks via three steps. In the first step, data poisoning attacks are prevented by using CS. Then, in the second step, collude attacks were prevented by randomly selecting enterprises in the consortium blockchain. Finally, in the third step, model poisoning, membership inference, and reconstruction attacks have been prevented using the CKKS Fully Homomorphic Encryption (CKKS-FHE) technique and consortium blockchain. Extensive experiments were conducted using the Sent140, Fashion-MNIST, FEMNIST, and CIFAR-10 new real-world datasets to evaluate FedAnil’s robustness and performance. The simulation results demonstrate that FEDANIL satisfies FDL privacy-preserving requirements. In terms of convergence analysis, the model parameter obtained with the FEDANIL converges to the optimum of the model parameter. In addition, it performs better in terms of accuracy (more than 11, 15, and 24%) and computation overhead (less than 8, 10, and 15%) compared with baseline approaches, namely SHIELDFL, RVPFL, and RFA, respectively. The FEDANIL source code can be found on GitHub.11Code available on GitHub Repository: https://github.com/rezafotohi/FedAnil. For any questions about the code, please contact Fotohi.reza@gmail.com.

Multi-Label Sampling based on Label Imbalance Rate and Neighborhood Distribution

Existing multi-label classification algorithms often assume that label distribution in the training set is balanced, but practical datasets frequently exhibit significant label imbalance. This imbalance affects the learning and generalization performance of the classifiers. To address the problem of label imbalance in multi-label classification, this paper proposes a new synthetic oversampling algorithm, named Multi-Label Synthetic Oversampling based on Label Imbalance Rate and Neighborhood Distribution (MLSIN). This algorithm synthesizes new samples by considering both the imbalance rate of labels and the distribution of samples in their neighborhood, aiming to improve the classifiers performance on minority labels. The rest of this chapter first introduces the evaluation metrics for multi-label classification effectiveness. Then it defines and computes the degree of label imbalance, describes the calculation of imbalance weights, and proposes a sample type correction penalty strategy, detailing the algorithm's process for selecting base and auxiliary samples., and validates the proposed method on public datasets and summarizes the experimental results.

Business text classification with imbalanced data and moderately large label spaces for digital transformation

Digital transformation refers to an organization’s use of digital technology to improve its products, services, and operations, aligning them with evolving business requirements. To demonstrate this transformative process, we present a real-life case study where a company seeks to automate the classification of their textual data rather than relying on manual methods. Transitioning to automated classification involves deploying machine learning models, which rely on pre-labeled datasets for training and making predictions on new data. However, upon receiving the dataset from the company, we faced challenges due to the imbalanced distribution of labels and moderately large label spaces. To tackle text classification with such a business dataset, we evaluated four distinct methods for multi-label text classification: fine-tuned Bidirectional Encoder Representations from Transformers (BERT), Binary Relevance, Classifier Chains, and Label Powerset. The results revealed that fine-tuned BERT significantly outperformed the other methods across key metrics like Accuracy, F1-score, Precision, and Recall. Binary Relevance also displayed competence in handling the dataset effectively, while Classifier Chains and Label Powerset exhibited comparatively less impressive performance. These findings highlight the remarkable effectiveness of fine-tuned BERT model and the Binary Relevance classifier in multi-label text classification tasks, particularly when dealing with imbalanced training datasets and moderately large label spaces. This positions them as valuable assets for businesses aiming to automate data classification in the digital transformation era.

A New Method for Bearing Fault Diagnosis across Machines Based on Envelope Spectrum and Conditional Metric Learning.

In recent years, most research on bearing fault diagnosis has assumed that the source domain and target domain data come from the same machine. The differences in equipment lead to a decrease in diagnostic accuracy. To address this issue, unsupervised domain adaptation techniques have been introduced. However, most cross-device fault diagnosis models overlook the discriminative information under the marginal distribution, which restricts the performance of the models. In this paper, we propose a bearing fault diagnosis method based on envelope spectrum and conditional metric learning. First, envelope spectral analysis is used to extract frequency domain features. Then, to fully utilize the discriminative information from the label distribution, we construct a deep Siamese convolutional neural network based on conditional metric learning to eliminate the data distribution differences and extract common features from the source and target domain data. Finally, dynamic weighting factors are employed to improve the convergence performance of the model and optimize the training process. Experimental analysis is conducted on 12 cross-device tasks and compared with other relevant methods. The results show that the proposed method achieves the best performance on all three evaluation metrics.

Learning label smoothing for text classification.

Training with soft labels instead of hard labels can effectively improve the robustness and generalization of deep learning models. Label smoothing often provides uniformly distributed soft labels during the training process, whereas it does not take the semantic difference of labels into account. This article introduces discrimination-aware label smoothing, an adaptive label smoothing approach that learns appropriate distributions of labels for iterative optimization objectives. In this approach, positive and negative samples are employed to provide experience from both sides, and the performances of regularization and model calibration are improved through an iterative learning method. Experiments on five text classification datasets demonstrate the effectiveness of the proposed method.

Estimating Uncertainty in Labeled Changes by SZZ Tools on Just-In-Time Defect Prediction

The aim of Just-In-Time (JIT) defect prediction is to predict software changes that are prone to defects in a project in a timely manner, thereby improving the efficiency of software development and ensuring software quality. Identifying changes that introduce bugs is a critical task in just-in-time defect prediction, and researchers have introduced the SZZ approach and its variants to label these changes. However, it has been shown that different SZZ algorithms introduce noise to the dataset to a certain extent, which may reduce the predictive performance of the model. To address this limitation, we propose the Confident Learning Imbalance (CLI) model. The model identifies and excludes samples whose labels may be corrupted by estimating the joint distribution of noisy labels and true labels, and mitigates the impact of noisy data on the performance of the prediction model. The CLI consists of two components: identifying noisy data (Confident Learning Component) and generating a predicted probability matrix for imbalanced data (Imbalanced Data Probabilistic Prediction Component). The IDPP component generates precise predicted probabilities for each instance in the training set, while the CL component uses the generated predicted probability matrix and noise labels to clean up the noise and build a classification model. We evaluate the performance of our model through extensive experiments on a total of 126,526 changes from ten Apache open source projects, and the results show that our model outperforms the baseline methods.

Transfer learning in near infrared spectroscopy for stingless bee honey quality prediction across different months

Near infrared spectroscopy (NIRS) coupled with machine learning has demonstrated its capability as an efficient secondary measurement method in various applications. However, a NIRS predictive model might be unable to maintain its accuracy when it is applied to samples that were acquired from different periods. Particularly, the qualities of stingless bee honey are affected by various uncontrolled factors, e.g. ecosystem, origins, and weather. Since it is unrealistic to have a NIRS dataset that can represent unforeseen future changes, an algorithm that can adapt existing data for new samples is worth to be investigated. Thus, this study aims to evaluate the feasibility of homogenous transfer learning approaches to overcome data constraints in developing NIRS predictive models of stingless bee honey qualities across different months. First, the near infrared transmittance spectra were acquired across honey samples prior to their respective conventional quality measurements in four different analysis months. Then, data from different analysis months were used as source and target datasets. Next, joint distribution adaptation based partial least square (JDA-PLS) and transfer component analysis based PLS (TCA-PLS) were implemented to establish NIRS predictive models of moisture, hydroxymethylfurfural (HMF), and glucose quality. Results show that the proposed TCA-PLS outperformed JDA-PLS and PLS by achieving the lowest RMSEP of 3.401 % and 0.804 mg/kg in predicting the moisture and glucose of stingless bee honey, respectively, across different months. Lastly, finding shows that both JDA and TCA were inefficacy when the label distributions of the calibration and validation datasets were similar.

Broad, strong, and soft: Using geospatial analysis to understand folk-linguistic terminology

Abstract This study uses a modified online version of the “draw-a-map” task and Garrett, Williams, and Evans’ (2005b) “keywords” methodology to explore the geospatial distribution of different accent and dialect labels and descriptors in Greater Manchester, UK. Specifically, we consider the distribution of the three most frequent labels related to “accentedness”: Broad, Strong, and Soft, as provided by 349 Greater Manchester residents. This analysis finds that these descriptors were clustered in separate areas of Greater Manchester, suggesting that they were being used to describe perceptually distinct varieties of English. In order to uncover the nuances in these folk-linguistic terms, we consider how they correlate with other concepts emerging from the dataset, finding that they are being used to differentiate between varieties with contrasting social associations. By combining innovative approaches, this study demonstrates how the subtleties of folk-linguistic modes of awareness can be uncovered through in-depth analysis of the terminology employed to describe linguistic variation on a very local scale. In so doing, it paves the way for further development of draw-a-map techniques that will enable similarly nuanced analysis in different regions, thus pushing the sub-discipline forward.

Dynamic Feature Selection Based on F-fuzzy Rough Set for Label Distribution Learning

Dynamic Feature Selection Based on F-fuzzy Rough Set for Label Distribution Learning

Calibrated Aleatoric Uncertainty-Based Adaptive Label Distribution Learning for Pose Estimation of Sichuan Peppers

Calibrated Aleatoric Uncertainty-Based Adaptive Label Distribution Learning for Pose Estimation of Sichuan Peppers

Data-balanced transformer for accelerated ionizable lipid nanoparticles screening in mRNA delivery.

Despite the widespread use of ionizable lipid nanoparticles (LNPs) in clinical applications for messenger RNA (mRNA) delivery, the mRNA drug delivery system faces an efficient challenge in the screening of LNPs. Traditional screening methods often require a substantial amount of experimental time and incur high research and development costs. To accelerate the early development stage of LNPs, we propose TransLNP, a transformer-based transfection prediction model designed to aid in the selection of LNPs for mRNA drug delivery systems. TransLNP uses two types of molecular information to perceive the relationship between structure and transfection efficiency: coarse-grained atomic sequence information and fine-grained atomic spatial relationship information. Due to the scarcity of existing LNPs experimental data, we find that pretraining the molecular model is crucial for better understanding the task of predicting LNPs properties, which is achieved through reconstructing atomic 3D coordinates and masking atom predictions. In addition, the issue of data imbalance is particularly prominent in the real-world exploration of LNPs. We introduce the BalMol block to solve this problem by smoothing the distribution of labels and molecular features. Our approach outperforms state-of-the-art works in transfection property prediction under both random and scaffold data splitting. Additionally, we establish a relationship between molecular structural similarity and transfection differences, selecting 4267 pairs of molecular transfection cliffs, which are pairs of molecules that exhibit high structural similarity but significant differences in transfection efficiency, thereby revealing the primary source of prediction errors. The code, model and data are made publicly available at https://github.com/wklix/TransLNP.

Generative Calibration of Inaccurate Annotation for Label Distribution Learning

Label distribution learning (LDL) is an effective learning paradigm for handling label ambiguity. When applying LDL, it typically requires datasets annotated with label distributions. However, obtaining supervised data for LDL is a challenging task. Due to the randomness of label annotation, the annotator can produce inaccurate annotation results for the instance, affecting the accuracy and generalization ability of the LDL model. To address this problem, we propose a generative approach to calibrate the inaccurate annotation for LDL using variational inference techniques. Specifically, we assume that instances with similar features share latent similar label distributions. The feature vectors and label distributions are generated by Gaussian mixture and Dirichlet mixture, respectively. The relationship between them is established through a shared categorical variable, which effectively utilizes the label distribution of instances with similar features, and achieves a more accurate label distribution through the generative approach. Furthermore, we use a confusion matrix to model the factors that contribute to the inaccuracy during the annotation process, which captures the relationship between label distributions and inaccurate label distributions. Finally, the label distribution is used to calibrate the available information in the noisy dataset to obtain the ground-truth label distribution.

GLDL: Graph Label Distribution Learning

Label Distribution Learning (LDL), as a more general learning setting than generic single-label and multi-label learning, has been commonly used in computer vision and many other applications. To date, existing LDL approaches are designed and applied to data without considering the interdependence between instances. In this paper, we propose a Graph Label Distribution Learning (GLDL) framework, which explicitly models three types of relationships: instance-instance, label-label, and instance-label, to learn the label distribution for networked data. A label-label network is learned to capture label-to-label correlation, through which GLDL can accurately learn label distributions for nodes. Dual graph convolution network (GCN) Co-training with heterogeneous message passing ensures two GCNs, one focusing on instance-instance relationship and the other one targeting label-label correlation, are jointly trained such that instance-instance relationship can help induce label-label correlation and vice versa. Our theoretical study derives the error bound of GLDL. For verification, four benchmark datasets with label distributions for nodes are created using common graph benchmarks. The experiments show that considering dependency helps learn better label distributions for networked data, compared to state-of-the-art LDL baseline. In addition, GLDL not only outperforms simple GCN and graph attention networks (GAT) using distribution loss but is also superior to its variant considering label-label relationship as a static network. GLDL and its benchmarks are the first research endeavors to address LDL for graphs. Code and benchmark data are released for public access.

Label enhancement via manifold approximation and projection with graph convolutional network

Label enhancement (LE) aims to enrich logical labels into their corresponding label distributions. But existing LE algorithms fail to fully leverage the structural information in the feature space to improve LE learning. To address this key issue, we first apply manifold learning to map the relatedness between low-dimensional feature samples to the label space. Based on the smoothness assumption of manifolds, the implicit correlation between low-dimensional feature and label spaces effectively promotes the LE process, enabling the learning model to accurately capture the mapping relationship between feature and label manifolds. This leads to an LE based on feature representation (LEFR) algorithm. We also propose an LE algorithm based on graph convolutional network (GCN), called LE-GCN. Inspired by the relationship between threshold connections and label connections, we extend GCN to the LE field for the first time to fully exploit the hidden relationships between nodes and labels. By enhancing node information with threshold connections and label connections, the label learning accuracy reaches a new level. Experiments on real-world datasets show that our LEFR and LE-GCN outperform several state-of-the-art LE algorithms.

LAMM: Label Alignment for Multi-Modal Prompt Learning

With the success of pre-trained visual-language (VL) models such as CLIP in visual representation tasks, transferring pre-trained models to downstream tasks has become a crucial paradigm. Recently, the prompt tuning paradigm, which draws inspiration from natural language processing (NLP), has made significant progress in VL field. However, preceding methods mainly focus on constructing prompt templates for text and visual inputs, neglecting the gap in class label representations between the VL models and downstream tasks. To address this challenge, we introduce an innovative label alignment method named \textbf{LAMM}, which can dynamically adjust the category embeddings of downstream datasets through end-to-end training. Moreover, to achieve a more appropriate label distribution, we propose a hierarchical loss, encompassing the alignment of the parameter space, feature space, and logits space. We conduct experiments on 11 downstream vision datasets and demonstrate that our method significantly improves the performance of existing multi-modal prompt learning models in few-shot scenarios, exhibiting an average accuracy improvement of 2.31(\%) compared to the state-of-the-art methods on 16 shots. Moreover, our methodology exhibits the preeminence in continual learning compared to other prompt tuning methods. Importantly, our method is synergistic with existing prompt tuning methods and can boost the performance on top of them. Our code and dataset will be publicly available at https://github.com/gaojingsheng/LAMM.

FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning

Federated Learning (FL) heavily depends on label quality for its performance. However, the label distribution among individual clients is always both noisy and heterogeneous. The high loss incurred by client-specific samples in heterogeneous label noise poses challenges for distinguishing between client-specific and noisy label samples, impacting the effectiveness of existing label noise learning approaches. To tackle this issue, we propose FedFixer, where the personalized model is introduced to cooperate with the global model to effectively select clean client-specific samples. In the dual models, updating the personalized model solely at a local level can lead to overfitting on noisy data due to limited samples, consequently affecting both the local and global models’ performance. To mitigate overfitting, we address this concern from two perspectives. Firstly, we employ a confidence regularizer to alleviate the impact of unconfident predictions caused by label noise. Secondly, a distance regularizer is implemented to constrain the disparity between the personalized and global models. We validate the effectiveness of FedFixer through extensive experiments on benchmark datasets. The results demonstrate that FedFixer can perform well in filtering noisy label samples on different clients, especially in highly heterogeneous label noise scenarios.