Imbalanced Class Distribution Research Articles

FedGA: A greedy approach to enhance federated learning with Non-IID data

In the context of federated learning, smart terminal devices inherently exhibit various factors such as personalized user behaviors, regional differences, and heterogeneous hardware configurations due to their unique data capturing capabilities. Consequently, they inevitably present non-independent and identically distributed (Non-IID) data during the training process, making it challenging for traditional federated learning methods to achieve desired model performance and convergence effects when dealing with such complex data distributions. To address this challenge, researchers have explored data augmentation techniques, adaptive optimization algorithms, and improved model aggregation rules. However, these methods often fail to deliver satisfactory performance when confronted with Non-IID problems. In this paper, we propose a federated learning framework based on a greedy algorithm (FedGA). Unlike traditional approaches that rely on global parameter averaging aggregation, FedGA progressively searches for partially optimal models and aggregates them to obtain the global model. We first gather client data information and finely classify all clients, employing two strategies to optimize client data distribution: (1) for clients with imbalanced quantities, we utilize data sampling methods to balance client data; (2) for clients with imbalanced class distributions, we introduce a complementary federated meta-learning method, called FedComMeta, to improve the data distribution of client groups. Experimental results demonstrate that in scenarios involving Non-IID data, FedGA achieves significant improvements in model performance compared to existing methods such as FedAvg, FedProx, and Astraea, validating the effectiveness and superiority of our approach in handling Non-IID data distributions.

Complex Relation Embedding for Scene Graph Generation.

Given an input image, scene graph generation (SGG) aims to generate comprehensive visual relationships between objects in the form of graphs. Recently, more attention to the design of complex networks and complicated strategies has been paid to the long tail issue caused by the imbalanced class distribution. However, most existing methods adopt the concatenated features of two objects in real space as the final relation representation for a given triplet. We mainly argue that such a simple concatenation may neglect the importance of complex interactions between objects, which results in the diversity of visual relations. In addition, the representation learning in real space is also inadequate to express this property. To alleviate these issues, we seamlessly incorporate Hermitian inner product into existing models to facilitate the generation of scene graphs by learning Relation Embedding in Complex space (CoRE). More specifically, we first introduce the concept of complex-valued representations for entities and then formulate the relation triplets with Hermitian inner product in complex space. Finally, we investigate the effect of utilizing only real component or both of Hermitian inner product on inferring more reasonable interaction between objects for scene graphs. Comprehensive experiments on two widely used benchmark datasets, Visual Genome (VG) and Open Image, demonstrate our effectiveness, superiority, and generalization on various metrics for biased or unbiased inference.

A survey on imbalanced learning: latest research, applications and future directions

Imbalanced learning constitutes one of the most formidable challenges within data mining and machine learning. Despite continuous research advancement over the past decades, learning from data with an imbalanced class distribution remains a compelling research area. Imbalanced class distributions commonly constrain the practical utility of machine learning and even deep learning models in tangible applications. Numerous recent studies have made substantial progress in the field of imbalanced learning, deepening our understanding of its nature while concurrently unearthing new challenges. Given the field’s rapid evolution, this paper aims to encapsulate the recent breakthroughs in imbalanced learning by providing an in-depth review of extant strategies to confront this issue. Unlike most surveys that primarily address classification tasks in machine learning, we also delve into techniques addressing regression tasks and facets of deep long-tail learning. Furthermore, we explore real-world applications of imbalanced learning, devising a broad spectrum of research applications from management science to engineering, and lastly, discuss newly-emerging issues and challenges necessitating further exploration in the realm of imbalanced learning.

A novel ensemble learning framework based on a genetic algorithm for the classification of pneumonia

Pneumonia is a disease that can be detected by the opacity changes in chest X-rays and can lead to fatal consequences. Medical image analysis has several challenges, such as limited labeled datasets, imbalanced class distribution, image noise, and overfitting, so individual Convolutional Neural Networks (CNNs) are insufficient to detect pneumonia accurately. Although ensemble CNN models have been used in previous studies, the literature lacks guidance on identifying the optimal CNN models and weight ratio to combine them. In this study, we propose a novel ensemble CNN framework to accurately detect pneumonia, with optimum weights set by a Genetic Algorithm (GA). Firstly, a noise outside the lung was removed, and the model performance was enhanced by performing lung segmentation on Chest X-ray. The performances of several CNN models were analyzed by hyperparameter optimization. The framework combines the three models that give the best accuracy and the two models that provide the lowest false-negative value with the ensemble method in the ratio of the most appropriate weights. The proposed framework provided the best performance on the public test dataset with an accuracy of 97.23% and an F1-score of 97.45% compared to state-of-the-art methods. The study's main contributions are determining suitable models and their optimal weights of the ensemble method based on the GA. The proposed framework enables a rapid and effective diagnostic process, less costly healthcare services, and more efficient use of resources. The demo-link is https://www.youtube.com/watch?v=KZ50K3HL70U.

Review Paper on An Early-Stage Autism Spectrum Detection System

The Early-Stage Autism Detection System presents a breakthrough approach to identifying Autism Spectrum Disorder (ASD) in its initial stages, particularly focusing on early childhood diagnosis. Leveraging machine learning (ML) techniques such as Random Forest and Support Vector Machines, the system meticulously analyses behavioural patterns and social interactions to pinpoint potential indicators of ASD, even in toddlers. It adeptly tackles challenges like imbalanced class distributions by employing random oversampling and adopts feature scaling and selection methods to heighten prediction accuracy. Through extensive experimentation on diverse ASD datasets, the system discerns crucial features pivotal for precise diagnosis. Its implementation promises timely intervention and improved outcomes by enabling the early detection and support of individuals with ASD from the outset of development

MT4MTL-KD: A Multi-Teacher Knowledge Distillation Framework for Triplet Recognition.

The recognition of surgical triplets plays a critical role in the practical application of surgical videos. It involves the sub-tasks of recognizing instruments, verbs, and targets, while establishing precise associations between them. Existing methods face two significant challenges in triplet recognition: 1) the imbalanced class distribution of surgical triplets may lead to spurious task association learning, and 2) the feature extractors cannot reconcile local and global context modeling. To overcome these challenges, this paper presents a novel multi-teacher knowledge distillation framework for multi-task triplet learning, known as MT4MTL-KD. MT4MTL-KD leverages teacher models trained on less imbalanced sub-tasks to assist multi-task student learning for triplet recognition. Moreover, we adopt different categories of backbones for the teacher and student models, facilitating the integration of local and global context modeling. To further align the semantic knowledge between the triplet task and its sub-tasks, we propose a novel feature attention module (FAM). This module utilizes attention mechanisms to assign multi-task features to specific sub-tasks. We evaluate the performance of MT4MTL-KD on both the 5-fold cross-validation and the CholecTriplet challenge splits of the CholecT45 dataset. The experimental results consistently demonstrate the superiority of our framework over state-of-the-art methods, achieving significant improvements of up to 6.4% on the cross-validation split.

FormulAI: Designing Rule-Based Datasets for Interpretable and Challenging Machine Learning Tasks

In an era marked by the transformative impact of machine learning algorithms across various disciplines, challenges in achieving model interpretability persist. Existing evaluation datasets often lack transparency, thereby obscuring the decision-making process of machine learning models, particularly in complex deep learning architectures. This opacity raises concerns across sectors like healthcare, emphasizing the pivotal role of explainability in fostering trust and adhering to non-supervisory norms. While progress has been made through the development of interpretable models, the absence of formalized, interpretable datasets hampers the validation and comparison of techniques. Rule-based datasets, distinct from general synthetic datasets, provide an avenue to simulate real-world challenges while maintaining interpretability. This paper introduces FormulAI, a framework for generating comprehensive rule-grounded datasets encompassing categorical and continuous features, calibrated noise, and imbalanced class distribution. Emphasizing scalability and reproducibility, these datasets serve as a robust standard, fostering exploration in interpretability and robustness.

A post-processing framework for class-imbalanced learning in a transductive setting

Traditional classification tasks suffer from the class-imbalanced problem, where some classes far outnumber others. To address this issue, existing class-imbalanced learning (CIL) methods either preprocess class-imbalanced datasets or adapt traditional classification algorithms to the imbalanced class distribution. Inspired by the idea of transductive learning, we propose a post-processing framework called PPF for CIL. Distinct from existing CIL methods, PPF directly adjusts the predicted labels of test data to fit the imbalanced class distribution. Specifically, we relabel some test data according to their prediction probabilities so that the class proportion of test data is close to that of training data. The underlying assumption is that training and test data, drawn independently from one data space, should obey the same class distribution. Furthermore, we propose a Compact Prototype-based Nearest Neighbor (CPNN) algorithm to assist the original classifier with the adjustment. Instead of training a classifier, CPNN classifies test data according to their distances to a set of prototypes estimated on labeled data. Thus, it is computationally simple and relatively robust to class imbalance. As a general framework, PPF can be easily applied to both traditional classification and CIL algorithms. To validate the effectiveness of the proposed method, we conducted extensive experiments on a variety of class-imbalanced datasets, using SVM and C4.5 as the original classifiers, respectively. Measured by F-measure, G-mean, and AUC, both PPF-SVM and PPF-C4.5 outperform 10 state-of-the-art CIL algorithms. Additionally, PPF further improved their performances when applied to 10 CIL algorithms.

IOFM: Using the Interpolation Technique on the Over-Fitted Models to Identify Clean-Annotated Samples

Most recent state-of-the-art algorithms for handling noisy label problems are based on the memorization effect, which is a phenomenon that deep neural networks (DNNs) memorize clean data before noisy ones. While the memorization effect can be a powerful tool, there are several cases where memorization effect does not occur. Examples are imbalanced class distributions and heavy contamination on labels. To address this limitation, we introduce a whole new approach called the interpolation with the over-fitted model (IOFM), which leverages over-fitted deep neural networks. The IOFM utilizes a new finding of over-fitted DNNs: for a given training sample, its neighborhoods chosen from the feature space are distributed differently on the original input space depending on the cleanness of the target sample. The IOFM has notable features in two aspects: 1) it yields superior results even when the training data are imbalanced or heavily noisy, 2) since we utilize over-fitted deep neural networks, a fine-tuning procedure to select the optimal training epoch, which is an essential yet sensitive factor for the success of the memorization effect, is not required, and thus, the IOFM can be used for non-experts. Through extensive experiments, we show that our method can serve as a promising alternative to existing solutions dealing with noisy labels, offering improved performance even in challenging situations.

Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning

We address the challenging problem of Long-Tailed Semi-Supervised Learning (LTSSL) where labeled data exhibit imbalanced class distribution and unlabeled data follow an unknown distribution. Unlike in balanced SSL, the generated pseudo-labels are skewed towards head classes, intensifying the training bias. Such a phenomenon is even amplified as more unlabeled data will be mislabeled as head classes when the class distribution of labeled and unlabeled datasets are mismatched. To solve this problem, we propose a novel method named ComPlementary Experts (CPE). Specifically, we train multiple experts to model various class distributions, each of them yielding high-quality pseudo-labels within one form of class distribution. Besides, we introduce Classwise Batch Normalization for CPE to avoid performance degradation caused by feature distribution mismatch between head and non-head classes. CPE achieves state-of-the-art performances on CIFAR-10-LT, CIFAR-100-LT, and STL-10-LT dataset benchmarks. For instance, on CIFAR-10-LT, CPE improves test accuracy by over >2.22% compared to baselines. Code is available at https://github.com/machengcheng2016/CPE-LTSSL.

Deep reinforcement learning based on balanced stratified prioritized experience replay for customer credit scoring in peer-to-peer lending

In recent years, deep reinforcement learning (DRL) models have been successfully utilised to solve various classification problems. However, these models have never been applied to customer credit scoring in peer-to-peer (P2P) lending. Moreover, the imbalanced class distribution in experience replay, which may affect the performance of DRL models, has rarely been considered. Therefore, this article proposes a novel DRL model, namely a deep Q-network based on a balanced stratified prioritized experience replay (DQN-BSPER) model, for customer credit scoring in P2P lending. Firstly, customer credit scoring is formulated as a discrete-time finite-Markov decision process. Subsequently, a balanced stratified prioritized experience replay technology is presented to optimize the loss function of the deep Q-network model. This technology can not only balance the numbers of minority and majority experience samples in the mini-batch by using stratified sampling technology but also select more important experience samples for replay based on the priority principle. To verify the model performance, four evaluation measures are introduced for the empirical analysis of two real-world customer credit scoring datasets in P2P lending. The experimental results show that the DQN-BSPER model can outperform four benchmark DRL models and seven traditional benchmark classification models. In addition, the DQN-BSPER model with a discount factor γ of 0.1 has excellent credit scoring performance.

ASPP+-LANet: A Multi-Scale Context Extraction Network for Semantic Segmentation of High-Resolution Remote Sensing Images

Semantic segmentation of remote sensing (RS) images is a pivotal branch in the realm of RS image processing, which plays a significant role in urban planning, building extraction, vegetation extraction, etc. With the continuous advancement of remote sensing technology, the spatial resolution of remote sensing images is progressively improving. This escalation in resolution gives rise to challenges like imbalanced class distributions among ground objects in RS images, the significant variations of ground object scales, as well as the presence of redundant information and noise interference. In this paper, we propose a multi-scale context extraction network, ASPP+-LANet, based on the LANet for semantic segmentation of high-resolution RS images. Firstly, we design an ASPP+ module, expanding upon the ASPP module by incorporating an additional feature extraction channel, redesigning the dilation rates, and introducing the Coordinate Attention (CA) mechanism so that it can effectively improve the segmentation performance of ground object targets at different scales. Secondly, we introduce the Funnel ReLU (FReLU) activation function for enhancing the segmentation effect of slender ground object targets and refining the segmentation edges. The experimental results show that our network model demonstrates superior segmentation performance on both Potsdam and Vaihingen datasets, outperforming other state-of-the-art (SOTA) methods.

Measuring cyclists’ subjective perceptions of the street riding environment using K-means SMOTE-RF model and street view imagery

Cyclists' willingness to ride is usually influenced by their subjective perception of the street riding environment. Measuring this perception is crucial for enhancing residents' willingness to ride. We propose an SSB framework (Public Security, Traffic Safety, Scenic Beauty) to quantify cyclists' subjective perception using street view imagery (SVI) and volunteer rating data. To address the issue of imbalanced class distribution in the volunteer rating data and enhance the model's ability to distinguish between positive and negative perception scenes, we employed a combination of the Kmeans Synthetic Minority Over-Sampling Technique (Kmeans-SMOTE) and the Random Forest (RF) classifier. The Kmeans SMOTE-RF model improved Area Under the Curve (AUC) by 0.327 for public safety, 0.2 for traffic safety, and 0.209 for scenic beauty compared to the RF model. Additionally, we incorporated Shapley Additive Explanations (SHAP) to examine how the visual features of SVI impact cyclists' subjective perception. Trees had a positive impact on all dimensions. Fence and sidewalk were key features for enhancing traffic safety perception, while roads positively affected public security and scenic beauty. These insights support urban planners in understanding the relationship between SVI features and cyclists' perceptions, aiding the design of cyclist-friendly street environments.

MolFeSCue: enhancing molecular property prediction in data-limited and imbalanced contexts using few-shot and contrastive learning.

Predicting molecular properties is a pivotal task in various scientific domains, including drug discovery, material science, and computational chemistry. This problem is often hindered by the lack of annotated data and imbalanced class distributions, which pose significant challenges in developing accurate and robust predictive models. This study tackles these issues by employing pretrained molecular models within a few-shot learning framework. A novel dynamic contrastive loss function is utilized to further improve model performance in the situation of class imbalance. The proposed MolFeSCue framework not only facilitates rapid generalization from minimal samples, but also employs a contrastive loss function to extract meaningful molecular representations from imbalanced datasets. Extensive evaluations and comparisons of MolFeSCue and state-of-the-art algorithms have been conducted on multiple benchmark datasets, and the experimental data demonstrate our algorithm's effectiveness in molecular representations and its broad applicability across various pretrained models. Our findings underscore MolFeSCues potential to accelerate advancements in drug discovery. We have made all the source code utilized in this study publicly accessible via GitHub at http://www.healthinformaticslab.org/supp/ or https://github.com/zhangruochi/MolFeSCue. The code (MolFeSCue-v1-00) is also available as the supplementary file of this paper.

Automatic Optimization of Deep Learning Training through Feature-Aware-Based Dataset Splitting

The proliferation of classification-capable artificial intelligence (AI) across a wide range of domains (e.g., agriculture, construction, etc.) has been allowed to optimize and complement several tasks, typically operationalized by humans. The computational training that allows providing such support is frequently hindered by various challenges related to datasets, including the scarcity of examples and imbalanced class distributions, which have detrimental effects on the production of accurate models. For a proper approach to these challenges, strategies smarter than the traditional brute force-based K-fold cross-validation or the naivety of hold-out are required, with the following main goals in mind: (1) carrying out one-shot, close-to-optimal data arrangements, accelerating conventional training optimization; and (2) aiming at maximizing the capacity of inference models to its fullest extent while relieving computational burden. To that end, in this paper, two image-based feature-aware dataset splitting approaches are proposed, hypothesizing a contribution towards attaining classification models that are closer to their full inference potential. Both rely on strategic image harvesting: while one of them hinges on weighted random selection out of a feature-based clusters set, the other involves a balanced picking process from a sorted list that stores data features’ distances to the centroid of a whole feature space. Comparative tests on datasets related to grapevine leaves phenotyping and bridge defects showcase promising results, highlighting a viable alternative to K-fold cross-validation and hold-out methods.

FCM-CSMOTE: Fuzzy C-Means Center-SMOTE

Imbalanced class distributions in machine learning, where the minority class is often under-represented, pose a substantial challenge. Synthetic Minority Over-sampling Technique (SMOTE) has been widely employed to address this issue by generating synthetic minority samples through interpolation. Despite its popularity, SMOTE exhibits certain drawbacks caused by the implementation of random interpolation samples. In this paper, we introduce a new data level technique for oversampling, called Fuzzy C-Means Center-SMOTE (FCM-CSMOTE), which generates synthetic samples in each cluster using its center considered as the memory of the main data components. We demonstrate that the proposed selective strategy has a very low probability to generate noise. The experimental results demonstrate that the proposed method performs better than the state-of-the-art approaches on 21 real unbalanced data sets (regular and large size data set) in terms of several metrics, including Geometric Mean (GM), F-Measure (FM), Area Under the Curve (AUC), and Accuracy.

ResU-Former: Advancing Remote Sensing Image Segmentation with Swin Residual Transformer for Precise Global–Local Feature Recognition and Visual–Semantic Space Learning

In the field of remote sensing image segmentation, achieving high accuracy and efficiency in diverse and complex environments remains a challenge. Additionally, there is a notable imbalance between the underlying features and the high-level semantic information embedded within remote sensing images, and both global and local recognition improvements are also limited by the multi-scale remote sensing scenery and imbalanced class distribution. These challenges are further compounded by inaccurate local localization segmentation and the oversight of small-scale features. To achieve balance between visual space and semantic space, to increase both global and local recognition accuracy, and to enhance the flexibility of input scale features while supplementing global contextual information, in this paper, we propose a U-shaped hierarchical structure called ResU-Former. The incorporation of the Swin Residual Transformer block allows for the efficient segmentation of objects of varying sizes against complex backgrounds, a common scenario in remote sensing datasets. With the specially designed Swin Residual Transformer block as its fundamental unit, ResU-Former accomplishes the full utilization and evolution of information, and the maximum optimization of semantic segmentation in complex remote sensing scenarios. The standard experimental results on benchmark datasets such as Vaihingen, Overall Accuracy of 81.5%, etc., show the ResU-Former’s potential to improve segmentation tasks across various remote sensing applications.

Oversampling and undersampling for intrusion detection system in the supervisory control and data acquisition IEC 60870‐5‐104

AbstractSupervisory control and data acquisition systems are critical in Industry 4.0 for controlling and monitoring industrial processes. However, these systems are vulnerable to various attacks, and therefore, intelligent and robust intrusion detection systems as security tools are necessary for ensuring security. Machine learning‐based intrusion detection systems require datasets with balanced class distribution, but in practice, imbalanced class distribution is unavoidable. A dataset created by running a supervisory control and data acquisition IEC 60870‐5‐104 (IEC 104) protocol on a testbed network is presented. The dataset includes normal and attacks traffic data such as port scan, brute force, and Denial of service attacks. Various types of Denial of service attacks are generated to create a robust and specific dataset for training the intrusion detection system model. Three popular techniques for handling class imbalance, that is, random over‐sampling, random under‐sampling, and synthetic minority oversampling, are implemented to select the best dataset for the experiment. Gradient boosting, decision tree, and random forest algorithms are used as classifiers for the intrusion detection system models. Experimental results indicate that the intrusion detection system model using decision tree and random forest classifiers using random under‐sampling achieved the highest accuracy of 99.05%. The intrusion detection system model's performance is verified using various metrics such as recall, precision, F1‐Score, receiver operating characteristics curves, and area under the curve. Additionally, 10‐fold cross‐validation shows no indication of overfitting in the created intrusion detection system model.

SSC: The novel self-stack ensemble model for thyroid disease prediction.

Thyroid disease presents a significant health risk, lowering the quality of life and increasing treatment costs. The diagnosis of thyroid disease can be challenging, especially for inexperienced practitioners. Machine learning has been established as one of the methods for disease diagnosis based on previous studies. This research introduces a novel and more effective technique for predicting thyroid disease by utilizing machine learning methodologies, surpassing the performance of previous studies in this field. This study utilizes the UCI thyroid disease dataset, which consists of 9172 samples and 30 features, and exhibits a highly imbalanced target class distribution. However, machine learning algorithms trained on imbalanced thyroid disease data face challenges in reliably detecting minority data and disease. To address this issue, re-sampling is employed, which modifies the ratio between target classes to balance the data. In this study, the down-sampling approach is utilized to achieve a balanced distribution of target classes. A novel RF-based self-stacking classifier is presented in this research for efficient thyroid disease detection. The proposed approach demonstrates the ability to diagnose primary hypothyroidism, increased binding protein, compensated hypothyroidism, and concurrent non-thyroidal illness with an accuracy of 99.5%. The recommended model exhibits state-of-the-art performance, achieving 100% macro precision, 100% macro recall, and 100% macro F1-score. A thorough comparative assessment is conducted to demonstrate the viability of the proposed approach, including several machine learning classifiers, deep neural networks, and ensemble voting classifiers. The results of K-fold cross-validation provide further support for the efficacy of the proposed self-stacking classifier.

Leveraging on Synthetic Data Generation Techniques to Train Machine Learning Models for Tenaga Nasional Berhad Stock Price Movement Prediction

This study employs machine learning models to explore stock price prediction for Tenaga Nasional Berhad (TNB), Malaysia’s primary electricity provider. It addresses the limitations of previous studies by incorporating various input variables, including the stock market, technical, financial, and economic data. This study also tackles the issue of imbalanced class distribution due to small datasets of stock market data by generating synthetic data using Synthetic Minority Over-Sampling Technique (SMOTE) and Generative Adversarial Network-Synthetic Minority Over-Sampling Technique (GAN-SMOTE) techniques. The performance of four classifier models (random forest, Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and Artificial Neural Network (ANN)) is evaluated without any synthetic data and with synthetic data generated. The SMOTE-ANN model is the best-performing model, exhibiting superior accuracy of 93%, F1-Score of 92%, precision of 90%, recall of 94%, and specificity of 92%. Overall, this research provides valuable insights into TNB stock price movements, offers a solution for imbalanced class distribution, and identifies the top-performing model for predicting TNB stock price movement. These findings are relevant to investors, analysts, and organisations in the utility sector.