Imbalanced Labels Research Articles

Towards value-sensitive and poisoning-proof model aggregation for federated learning on heterogeneous data

Federated Learning (FL) enables collaborative model training without sharing data, but traditional static averaging of local updates leads to poor performance on heterogeneous data. The following remedies, either by scheduling data distribution or mitigating local discrepancies, predominately fail to handle fine-grained heterogeneity (e.g., local imbalanced labels). To commence, we reveal that static averaging leads to the global model suffering from the mean fallacy. That is, the averaging process favors the local model with large parameters numerically rather than knowledge. To tackle this, we introduce FedVSA, a simple-yet-effective model aggregation framework sensitive to heterogeneous local data merits. Specifically, we invent a new global loss function for FL by prioritizing the valuable local updates, facilitating efficient convergence. We deduce a softmax-based aggregation rule and prove its convergence property via rigorous theoretical analysis. Additionally, we expose poisoning threats of model replacement that utilize the mean fallacy for attacks. To mitigate this threat, we propose a two-step mechanism involving auditing historic local training statistics and analyzing the Shapley Value. Through extensive experiments, we show that FedVSA achieves faster convergence (~1.52×) and higher accuracy (~1.6%) compared to the baselines. It also effectively mitigates poisoning attacks by agilely recovering and returning to normal aggregation.

Predicting Multiple Outcomes Associated with Frailty based on Imbalanced Multi-label Classification.

Frailty syndrome is prevalent among the elderly, often linked to chronic diseases and resulting in various adverse health outcomes. Existing research has predominantly focused on predicting individual frailty-related outcomes. However, this paper takes a novel approach by framing frailty as a multi-label learning problem, aiming to predict multiple adverse outcomes simultaneously. In the context of multi-label classification, dealing with imbalanced label distribution poses inherent challenges to multi-label prediction. To address this issue, our study proposes a hybrid resampling approach tailored for handling imbalance problems in the multi-label scenario. The proposed resampling technique and prediction tasks were applied to a high-dimensional real-life medical dataset comprising individuals aged 65years and above. Several multi-label algorithms were employed in the experiment, and their performance was evaluated using multi-label metrics. The results obtained through our proposed approach revealed that the best-performing prediction model achieved an average precision score of 83%. These findings underscore the effectiveness of our method in predicting multiple frailty outcomes from a complex and imbalanced multi-label dataset.

A Multitask Dynamic Graph Attention Autoencoder for Imbalanced Multilabel Time Series Classification.

Graph learning is widely applied to process various complex data structures (e.g., time series) in different domains. Due to multidimensional observations and the requirement for accurate data representation, time series are usually represented in the form of multilabels. Accurately classifying multilabel time series can provide support for personalized predictions and risk assessments. It requires effectively capturing complex label relevance and overcoming imbalanced label distributions of multilabel time series. However, the existing methods are unable to model label relevance for multilabel time series or fail to fully exploit it. In addition, the existing multilabel classification balancing strategies suffer from limitations, such as disregarding label relevance, information loss, and sampling bias. This article proposes a dynamic graph attention autoencoder-based multitask (DGAAE-MT) learning framework for multilabel time series classification. It can fully and accurately model label relevance for each instance by using a dynamic graph attention-based graph autoencoder to improve multilabel classification accuracy. DGAAE-MT employs a dual-sampling strategy and cooperative training approach to improve the classification accuracy of low-frequency classes while maintaining the classification accuracy of high-frequency and mid-frequency classes. It avoids information loss and sampling bias. DGAAE-MT achieves a mean average precision (mAP) of 0.955 and an F1 score of 0.978 on a mixed medical time series dataset. It outperforms state-of-the-art works in the past two years.

Oversampling multi-label data based on natural neighbor and label correlation

In multi-label learning, addressing the class imbalance issue is of paramount importance. Oversampling methods are preferred as they offer a more general solution independent of the model choice, i.e., they alleviate the imbalance of datasets by augmenting instances in the pre-processing step. Existing neighbor-based oversampling methods employ an empirical number of neighbors (k=5) to identify the local region for new instances creation. However, a single fixed k value cannot fit all labels, because every label usually has its own distinct distribution and complexity. Furthermore, the label assignment for synthetic instances usually depends on the statistics of individual labels within the corresponding neighborhood, ignoring the informative correlation among labels. To overcome these limitations, we propose an oversampling method called Multi-Label Oversampling with Natural neighbor and label Correlation (MLONC). Our approach offers three main advantages: (1) the adaptive number of neighbors for each label related to the data complexity is obtained via natural neighbor detection; (2) it encourages generating more instances proximate to the decision boundary of highly imbalanced labels, and diminishes the impact of outliers; (3) exploitation of label correlation in label assignment enhances the quality of the synthetic instances. Experimental results demonstrate the effectiveness of MLONC under various base classifiers.

Remotely sensed estimates of long-term biochemical oxygen demand over Hong Kong marine waters using machine learning enhanced by imbalanced label optimisation

In many coastal cities around the world, continuing water degradation threatens the living environment of humans and aquatic organisms. To assess and control the water pollution situation, this study estimated the Biochemical Oxygen Demand (BOD) concentration of Hong Kong's marine waters using remote sensing and an improved machine learning (ML) method. The scheme was derived from four ML algorithms (RBF, SVR, RF, XGB) and calibrated using a large amount (N > 1000) of in-situ BOD5 data. Based on labeled datasets with different preprocessing, i.e., the original BOD5, the log10(BOD5), and label distribution smoothing (LDS), three types of models were trained and evaluated. The results highlight the superior potential of the LDS-based model to improve BOD5 estimate by dealing with imbalanced training dataset. Additionally, XGB and RF outperformed RBF and SVR when the model was developed using log10(BOD5) or LDS(BOD5). Over two decades, the BOD5 concentration of Hong Kong marine waters in the autumn (Sep. to Nov.) shows a downward trend, with significant decreases in Deep Bay, Western Buffer, Victoria Harbour, Eastern Buffer, Junk Bay, Port Shelter, and the Tolo Harbour and Channel. Principal component analysis revealed that nutrient levels emerged as the predominant factor in Victoria Harbour and the interior of Deep Bay, while chlorophyll-related and physical parameters were dominant in Southern, Mirs Bay, Northwestern, and the outlet of Deep Bay. LDS provides a new perspective to improve ML-based water quality estimation by alleviating the imbalance in the labeled dataset. Overall, the remotely sensed BOD5 can offer insight into the spatial-temporal distribution of organic matter in Hong Kong coastal waters and valuable guidance for the pollution control.

IRDA: Implicit data augmentation for deep imbalanced regression

Imbalanced data distributions are prevalent in real-world classification and regression tasks. Data augmentation is a commonly employed technique to mitigate this issue, with implicit methods gaining attention for their effectiveness and efficiency. However, implicit data augmentation methods have not been extensively explored in the context of regression tasks. To address this gap, we introduce IRDA, a novel learning method for regression that incorporates implicit data augmentation. Our approach includes developing a new augmentation strategy specifically tailored for deep imbalanced regression tasks, and a regression loss function that is suitable for real-world data with imbalanced label distributions and non-uniformly distributed features. We derive an easily computable surrogate loss and propose two implicit data augmentation algorithms, one incorporating meta-learning and one without. Additionally, we provide a regularization perspective to offer a deeper understanding of IRDA. We evaluate IRDA on five datasets, including a large-scale dataset, demonstrating its effectiveness in mitigating the adverse effects of imbalanced data distribution and its adaptability to various regression tasks.

Federated variational generative learning for heterogeneous data in distributed environments

Distributedly training models across diverse clients with heterogeneous data samples can significantly impact the convergence of federated learning. Various novel federated learning methods address these challenges but often require significant communication resources and local computational capacity, leading to reduced global inference accuracy in scenarios with imbalanced label data distribution and quantity skew. To tackle these challenges, we propose FedVGL, a Federated Variational Generative Learning method that directly trains a local generative model to learn the distribution of local features and improve global target model inference accuracy during aggregation, particularly under conditions of severe data heterogeneity. FedVGL facilitates distributed learning by sharing generators and latent vectors with the global server, aiding in global target model training from mapping local data distribution to the variational latent space for feature reconstruction. Additionally, FedVGL implements anonymization and encryption techniques to bolster privacy during generative model transmission and aggregation. In comparison to vanilla federated learning, FedVGL minimizes communication overhead, demonstrating superior accuracy even with minimal communication rounds. It effectively mitigates model drift in scenarios with heterogeneous data, delivering improved target model training outcomes. Empirical results establish FedVGL's superiority over baseline federated learning methods under severe label imbalance and data skew condition. In a Label-based Dirichlet Distribution setting with α=0.01 and 10 clients using the MNIST dataset, FedVGL achieved an exceptional accuracy over 97% with the VGG-9 target model.

Toward Robustness in Multi-Label Classification: A Data Augmentation Strategy against Imbalance and Noise

Multi-label classification poses challenges due to imbalanced and noisy labels in training data. In this paper, we propose a unified data augmentation method, named BalanceMix, to address these challenges. Our approach includes two samplers for imbalanced labels, generating minority-augmented instances with high diversity. It also refines multi-labels at the label-wise granularity, categorizing noisy labels as clean, re-labeled, or ambiguous for robust optimization. Extensive experiments on three benchmark datasets demonstrate that BalanceMix outperforms existing state-of-the-art methods. We release the code at https://github.com/DISL-Lab/BalanceMix.

Enhancing Predictive Models: An In-depth Analysis of Feature Selection Techniques Coupled with Boosting Algorithms

This research addresses the critical need to enhance predictive models for fetal health classification using Cardiotocography (CTG) data. The literature review underscores challenges in imbalanced labels, feature selection, and efficient data handling. This paper aims to enhance predictive models for fetal health classification using Cardiotocography (CTG) data by addressing challenges related to imbalanced labels, feature selection, and efficient data handling. The study uses Recursive Feature Elimination (RFE) and boosting algorithms (XGBoost, AdaBoost, LightGBM, CATBoost, and Histogram-Based Boosting) to refine model performance. The results reveal notable variations in precision, Recall, F1-Score, accuracy, and AUC across different algorithms and RFE applications. Notably, Random Forest with XGBoost exhibits superior performance in precision (0.940), Recall (0.890), F1-Score (0.920), accuracy (0.950), and AUC (0.960). Conversely, Logistic Regression with AdaBoost demonstrates lower performance. The absence of RFE also impacts model effectiveness. In conclusion, the study successfully employs RFE and boosting algorithms to enhance fetal health classification models, contributing valuable insights for improved prenatal diagnosis.

A heuristic method for discovering multi-class classification rules from multi-source data in cloud–edge system

The integration of diverse devices has made the establishment of multi-class classification models for multi-source data a primary concern for data mining in cloud–edge system. Developing rule-based classifiers is essential because they present results that express the reasons for classification. However, existing rule learning methods are not compatible with multi-source tabular data containing mixed features, imbalanced labels and missing values, making it difficult to build cross-data source and cross-device data mining models. We developed a heuristic multi-class rule learning method that can handle complex tabular datasets without relying too much on cumbersome preprocessing techniques. We abstract the training process of the classifier into a multi-objective optimization problem and design a novel hybrid evolutionary algorithm to obtain Pareto-optimal solutions as rule-based classifier. Compared with the existing explainable classification methods, this method has obvious advantages in the classification performance of tabular data.

Multi-label feature selection with global and local label correlation

In various application domains, high-dimensional multi-label data has become more prevalent, presenting two significant challenges: instances with high-dimensional features and a large number of labels. In the context of multi-label feature selection, the objective is to choose a subset of features from a given set that is highly pertinent for predicting multiple labels or categories associated with each instance. However, certain characteristics of multi-label classification, such as label dependencies and imbalanced label distribution, have often been overlooked although they hold valuable insights for designing effective multi-label feature selection algorithms. In this paper, we propose a feature selection model which exploits explicit global and local label correlations to select discriminative features across multiple labels. In addition, by representing the feature matrix and label matrix in a shared latent space, the model aims to capture the underlying correlations between features and labels. The shared representation can reveal common patterns or relationships that exist across multiple labels and features. An objective function involving L2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. The proposed method was evaluated on 14 real-world multi-label datasets using six evaluation metrics, through comprehensive experiments. The results indicate its effectiveness, surpassing that of several representative methods.

Nonlocal Hybrid Network for Long-tailed Image Classification

It is a significant issue to deal with long-tailed data when classifying images. A nonlocal hybrid network (NHN) that takes into account both feature learning and classifier learning is proposed. The NHN can capture the existence of dependencies between two locations that are far away from each other as well as alleviate the impact of long-tailed data on the model to some extent. The dependency relationship between distant pixels is obtained first through a nonlocal module to extract richer feature representations. Then, a learnable soft class center is proposed to balance the supervised contrastive loss and reduce the impact of long-tailed data on feature learning. For efficiency, a logit adjustment strategy is adopted to correct the bias caused by the different label distributions between the training and test sets and obtain a classifier that is more suitable for long-tailed data. Finally, extensive experiments are conducted on two benchmark datasets, the long-tailed CIFAR and the large-scale real-world iNaturalist 2018, both of which have imbalanced label distributions. The experimental results show that the proposed NHN model is efficient and promising.

Large-Scale Object Detection in the Wild With Imbalanced Data Distribution, and Multi-Labels.

Training with more data has always been the most stable and effective way of improving performance in the deep learning era. The Open Images dataset, the largest object detection dataset, presents significant opportunities and challenges for general and sophisticated scenarios. However, its semi-automatic collection and labeling process, designed to manage the huge data scale, leads to label-related problems, including explicit or implicit multiple labels per object and highly imbalanced label distribution. In this work, we quantitatively analyze the major problems in large-scale object detection and provide a detailed yet comprehensive demonstration of our solutions. First, we design a concurrent softmax to handle the multi-label problems in object detection and propose a soft-balance sampling method with a hybrid training scheduler to address the label imbalance. This approach yields a notable improvement of 3.34 points, achieving the best single-model performance with a mAP of 60.90% on the public object detection test set of Open Images. Then, we introduce a well-designed ensemble mechanism that substantially enhances the performance of the single model, achieving an overall mAP of 67.17%, which is 4.29 points higher than the best result from the Open Images public test 2018. Our result is published on https://www.kaggle.com/c/open-images-2019-object-detection/leaderboard.

Transfer Learning for Anomaly Detection Using Bearings' Vibration Signals

Predictive maintenance is becoming increasingly important in the industry. Despite considerable advances in data collection and data-driven models, there are still limitations when deploying models in practice. One of the main limitations is the large datasets required to train these models. As a potential solution, transfer learning can be used to reuse knowledge acquired from large datasets for similar tasks under different conditions. This paper investigates the transferability of the specific anomaly detection scenario, an unsupervised learning task with heavily imbalanced label distribution. This paper uses a dataset generated from a bearing test platform in which bearings are run until failure under different operating conditions. A lightweight deep learning model, MobileNetV2, is employed to create a baseline model capable of detecting anomalies for a specific working condition. The model is then adapted using transfer learning to identify anomalies under new operating conditions with limited data accurately. The results show that the data for new conditions is insufficient to train an adequate model and that transfer learning can overcome this limitation. The adapted models can detect anomalies before the expert's knowledge reference value. Although this shows that transfer learning can detect anomalies earlier, the results must be evaluated carefully to avoid false positives. While anomaly detection aims to identify changes in feature distributions, transfer learning aims to align different feature distributions. Transfer learning for unsupervised learning has rarely been explored. To the best of our knowledge, this is one of the few works addressing it in the context of predictive maintenance for anomaly detection.

Clustered FedStack: Intermediate Global Models with Bayesian Information Criterion

Federated Learning (FL) is currently one of the most popular technologies in the field of Artificial Intelligence (AI) due to its collaborative learning and ability to preserve client privacy. However, it faces challenges such as non-identically and non-independently distributed (non-IID) data with imbalanced labels among local clients. To address these limitations, the research community has explored various approaches such as using local model parameters, federated generative adversarial learning, and federated representation learning. In our study, we propose a novel Clustered FedStack framework based on the previously published Stacked Federated Learning (FedStack) framework. Here, the local clients send their model predictions and output layer weights to a server, which then builds a robust global model. This global model clusters the local clients based on their output layer weights using a clustering mechanism. We adopt three clustering mechanisms, namely K-Means, Agglomerative, and Gaussian Mixture Models, into the framework and evaluate their performance. Bayesian Information Criterion (BIC) is used with the maximum likelihood function to determine the number of clusters. Our results show that Clustered FedStack models outperform baseline models with clustering mechanisms. To estimate the convergence of our proposed framework, we use Cyclical learning rates.

Hyperbolic graph convolutional neural network with contrastive learning for automated ICD coding

The International Classification of Diseases (ICD) is a widely used criterion for disease classification, health monitoring, and medical data analysis. Deep learning-based automated ICD coding has gained attention due to the time-consuming and costly nature of manual coding. The main challenges of automated ICD coding include imbalanced label distribution, code hierarchy and noisy texts. Recent works have considered using code hierarchy or description for better label representation to solve the problem of imbalanced label distribution. However, these methods are still ineffective and redundant since they only interact with a constant label representation. In this work, we introduce a novel Hyperbolic Graph Convolutional Network with Contrastive Learning (HGCN-CL) to solve the above problems and the shortcomings of the previous methods. We adopt a Hyperbolic graph convolutional network on ICD coding to capture the hierarchical structure of codes, which can solve the problem of large distortions when embedding hierarchical structure with graph convolutional network. Besides, we introduce contrastive learning for automatic ICD coding by injecting code features into text encoder to generate hierarchical-aware positive samples to solve the problem of interacting with constant code features. We conduct experiments on the public MIMIC-III and MIMIC-II datasets. The results on MIMIC III show that HGCN-CL outperforms previous state-of-art methods for automatic ICD coding, which achieves a 2.7% and 3.6% improvement respectively compared to previous best results (Hypercore). We also provide ablation experiments and hierarchy visualization to verify the effectiveness of components in our model.

Prediction of traditional Chinese medicine prescriptions based on multi-label resampling

PurposeTraditional Chinese medicine (TCM) prescriptions have always relied on the experience of TCM doctors, and machine learning(ML) provides a technical means for learning these experiences and intelligently assists in prescribing. However, in TCM prescription, there are the main (Jun) herb and the auxiliary (Chen, Zuo and Shi) herb collocations. In a prescription, the types of auxiliary herbs are often more than the main herb and the auxiliary herbs often appear in other prescriptions. This leads to different frequencies of different herbs in prescriptions, namely, imbalanced labels (herbs). As a result, the existing ML algorithms are biased, and it is difficult to predict the main herb with less frequency in the actual prediction and poor performance. In order to solve the impact of this problem, this paper proposes a framework for multi-label traditional Chinese medicine (ML-TCM) based on multi-label resampling.Design/methodology/approachIn this work, a multi-label learning framework is proposed that adopts and compares the multi-label random resampling (MLROS), multi-label synthesized resampling (MLSMOTE) and multi-label synthesized resampling based on local label imbalance (MLSOL), three multi-label oversampling techniques to rebalance the TCM data.FindingsThe experimental results show that after resampling, the less frequent but important herbs can be predicted more accurately. The MLSOL method is shown to be the best with over 10% improvements on average because it balances the data by considering both features and labels when resampling.Originality/valueThe authors first systematically analyzed the label imbalance problem of different sampling methods in the field of TCM and provide a solution. And through the experimental results analysis, the authors proved the feasibility of this method, which can improve the performance by 10%−30% compared with the state-of-the-art methods.

A Multi-Label Learning Framework for Predicting Chemical Classes and Biological Activities of Natural Products from Biosynthetic Gene Clusters.

Natural products (NP) or secondary metabolites, as a class of small chemical molecules that are naturally synthesized by chromosomally clustered biosynthesis genes (also called biosynthetic gene clusters, BGCs) encoded enzymes or enzyme complexes, mediates the bioecological interactions between host and microbiota and provides a natural reservoir for screening drug-like therapeutic pharmaceuticals. In this work, we propose a multi-label learning framework to functionally annotate natural products or secondary metabolites solely from their catalytical biosynthetic gene clusters without experimentally conducting NP structural resolutions. All chemical classes and bioactivities constitute the label space, and the sequence domains of biosynthetic gene clusters that catalyse the biosynthesis of natural products constitute the feature space. In this multi-label learning framework, a joint representation of features (BGCs domains) and labels (natural products annotations) is efficiently learnt in an integral and low-dimensional space to accurately define the inter-class boundaries and scale to the learning problem of many imbalanced labels. Computational results on experimental data show that the proposed framework achieves satisfactory multi-label learning performance, and the learnt patterns of BGCs domains are transferrable across bacteria, or even across kingdom, for instance, from bacteria to Arabidopsis thaliana. Lastly, take Arabidopsis thaliana and its rhizosphere microbiome for example, we propose a pipeline combining existing BGCs identification tools and this proposed framework to find and functionally annotate novel natural products for downstream bioecological studies in terms of plant-microbiota-soil interactions and plant environmental adaption.

Curriculum label distribution learning for imbalanced medical image segmentation.

Label distribution learning (LDL) has the potential to resolve boundary ambiguity in semantic segmentation tasks. However, existing LDL-based segmentation methods suffer from severe label distribution imbalance: the ambiguous label distributions contain a small fraction of the data, while the unambiguous label distributions occupy the majority of the data. The imbalanced label distributions induce model-biased distribution learning and make it challenging to accurately predict ambiguous pixels. In this paper, we propose a curriculum label distribution learning (CLDL) framework to address the above data imbalance problem by performing a novel task-oriented curriculum learning strategy. Firstly, the region label distribution learning (R-LDL) is proposed to construct more balanced label distributions and improves the imbalanced model learning. Secondly, a novel learning curriculum (TCL) is proposed to enable easy-to-hard learning in LDL-based segmentation by decomposing the segmentation task into multiple label distribution estimation tasks. Thirdly, the prior perceiving module (PPM) is proposed to effectively connect easy and hard learning stages based on the priors generated from easier stages. Benefiting from the balanced label distribution construction and prior perception, the proposed CLDL effectively conducts a curriculum learning-based LDL and significantly improves the imbalanced learning. We evaluated the proposed CLDL using the publicly available BRATS2018 and MM-WHS2017 datasets. The experimental results demonstrate that our method significantly improves different segmentation metrics compared to many state-of-the-art methods. The code will be available.1.

LPN: Label-Enhanced Prototypical Network for Legal Judgment Prediction.

As one of the most critical tasks in legal artificial intelligence, legal judgment prediction (LJP) has garnered growing attention, especially in the civil law system. However, current methods often overlook the challenge of imbalanced label distributions, treating each label with equal importance, which can lead the model to be biased toward labels with high frequency. In this paper, we propose a label-enhanced prototypical network (LPN) suitable for LJP, that adopts a strategy of uniform encoding and separate decoding. Specifically, LPN adopts a multi-scale convolutional neural network to uniformly encode case factual description to capture long-distance features of the document. At the decoding end, a prototypical network incorporating label semantic features is used to guide the learning of prototype representations of high-frequency and low-frequency labels, respectively. At the same time, we also propose a prototype-prototype loss to optimize the prototypical representation. We conduct extensive experiments on two real datasets and show that our proposed method effectively improves the performance of LJP, with an average F1 of 1.23% and 1.13% higher than the state-of-the-art model on two subtasks, respectively.