Handling Class Imbalance Research Articles

Research on Land Use and Land Cover Information Extraction Methods for Remote Sensing Images Based on Improved Convolutional Neural Networks

To address the challenges that convolutional neural networks (CNNs) face in extracting small objects and handling class imbalance in remote sensing imagery, this paper proposes a novel spatial contextual information and multiscale feature fusion encoding–decoding network, SCIMF-Net. Firstly, SCIMF-Net employs an improved ResNeXt-101 deep backbone network, significantly enhancing the extraction capability of small object features. Next, a novel PMFF module is designed to effectively promote the fusion of features at different scales, deepening the model’s understanding of global and local spatial contextual information. Finally, introducing a weighted joint loss function improves the SCIMF-Net model’s performance in extracting LULC information under class imbalance conditions. Experimental results show that compared to other CNNs such as Res-FCN, U-Net, SE-U-Net, and U-Net++, SCIMF-Net improves PA by 0.68%, 0.54%, 1.61%, and 3.39%, respectively; MPA by 2.96%, 4.51%, 2.37%, and 3.45%, respectively; and MIOU by 3.27%, 4.89%, 4.2%, and 5.68%, respectively. Detailed comparisons of locally visualized LULC information extraction results indicate that SCIMF-Net can accurately extract information from imbalanced classes and small objects.

Public Sentiment Analysis About Neuralink from Twitter Using Naïve Bayes: Multinomial, Gaussian and Complement

Elon Musk owns the business Neuralink, which attempts to build brain-machine interfaces. This study categorizes public opinion towards the use of Neuralink goods, including whether people agree (positive), disagree (negative), or feel neither way. Without accessing the Twitter API, the Twint Python Libraries were utilised to retrieve a dataset of 3000 using the keyword “neuralink”. What datasets are included in positive, neutral, or negative categories are designated using RoBERTa. Term Frequency Inverse Document Frequency (TF-IDF) is utilized for feature extraction, while Synthetic Minority Over-sampling Technique (SMOTE) is employed to handle class imbalance. Complement Naive Bayes, achieved accuracy of 81%, followed by Multinomial Naive Bayes, which achieved accuracy of 80%, and Gaussian Naive Bayes, which achieved accuracy of 75%. The model Complement Naïve Bayes was used in this study to attain the maximum accuracy, and accuracy increases when employing SMOTE compared to other Naïve bayes variants.

Prediction of Stock Industry Sectors Listed on the Indonesia Stock Exchange (IDX) based on Financial Statements with the Random Forest Method

This research aims to predict the stock industry sector listed on the Indonesia Stock Exchange (BEI) based on financial reports using the Random Forest method. The dataset used in this research includes financial data from companies listed on the IDX in the period 2010 to 2022. The data processing process includes data cleaning, handling class imbalance with oversampling techniques using SMOTE, and feature scaling using StandardScaler. The Random Forest model is used to classify companies into appropriate industry sectors. The eval_uation results show that the model has good performance with an overall accuracy of 80.21%. Several classes showed very good performance, such as the Financials class with precision of 95.24%, recall of 100%, and F-1 score of 97.56%. However, there are also classes that show lower performance, such as the Healthcare class with a precision of 51.61% and an F-1 score of 61.54%. The confusion matrix indicates that the model is able to identify most classes accurately, although there are several classes with prediction errors.

Predicting stroke occurrences: a stacked machine learning approach with feature selection and data preprocessing

Stroke prediction remains a critical area of research in healthcare, aiming to enhance early intervention and patient care strategies. This study investigates the efficacy of machine learning techniques, particularly principal component analysis (PCA) and a stacking ensemble method, for predicting stroke occurrences based on demographic, clinical, and lifestyle factors. We systematically varied PCA components and implemented a stacking model comprising random forest, decision tree, and K-nearest neighbors (KNN).Our findings demonstrate that setting PCA components to 16 optimally enhanced predictive accuracy, achieving a remarkable 98.6% accuracy in stroke prediction. Evaluation metrics underscored the robustness of our approach in handling class imbalance and improving model performance, also comparative analyses against traditional machine learning algorithms such as SVM, logistic regression, and Naive Bayes highlighted the superiority of our proposed method.

BENN: Balanced Ensemble Neural Network for Handling Class Imbalance in Big Data

ABSTRACTClass imbalance is a critical challenge in big data analytics, often leading to biased predictive models. This imbalance can lead to biased models that perform well on the majority class but poorly on the minority class. Many machine learning models tend to be biased towards the majority class because they aim to minimise overall error, often leading to poor performance on the minority class. This paper presents the balanced ensemble neural network, a novel solution to effectively address class imbalance in big data. Balanced ensemble neural network combines the robust capabilities of neural networks with the power of ensemble learning, incorporating class balancing strategies to ensure fair representation of minority classes. The methodology involves integrating multiple neural networks, each trained on balanced subsets of data using techniques like Synthetic Minority Over‐sampling Technique and Random Undersampling. This integration aims to leverage the strengths of individual networks while reducing their inherent biases. Our extensive experiments across various datasets reveal that BENN achieves an AUC‐ROC score of 0.94, surpassing other models such as random forest (0.88), support vector (0.84) and single neural net (0.80). It was also observed that BENN's performance is better compared to traditional neural network models and standard ensemble methods in key metrics like accuracy, precision, recall, F1‐score and AUC‐ROC. The results specifically highlight BENN's effectiveness in accurately classifying instances of minority classes, a notable challenge in many existing models. These findings underscore BENN's potential as a substantial advancement in handling class imbalance within big data environments, offering a promising direction for future research and application in machine learning.

Ensemble of vision transformer architectures for efficient Alzheimer’s Disease classification

Transformers have dominated the landscape of Natural Language Processing (NLP) and revolutionalized generative AI applications. Vision Transformers (VT) have recently become a new state-of-the-art for computer vision applications. Motivated by the success of VTs in capturing short and long-range dependencies and their ability to handle class imbalance, this paper proposes an ensemble framework of VTs for the efficient classification of Alzheimer’s Disease (AD). The framework consists of four vanilla VTs, and ensembles formed using hard and soft-voting approaches. The proposed model was tested using two popular AD datasets: OASIS and ADNI. The ADNI dataset was employed to assess the models’ efficacy under imbalanced and data-scarce conditions. The ensemble of VT saw an improvement of around 2% compared to individual models. Furthermore, the results are compared with state-of-the-art and custom-built Convolutional Neural Network (CNN) architectures and Machine Learning (ML) models under varying data conditions. The experimental results demonstrated an overall performance gain of 4.14% and 4.72% accuracy over the ML and CNN algorithms, respectively. The study has also identified specific limitations and proposes avenues for future research. The codes used in the study are made publicly available.

HIDIM: A novel framework of network intrusion detection for hierarchical dependency and class imbalance

Deep learning-based network intrusion detection has been extensively explored as a data-driven approach. Therefore, paying attention to the data’s characteristics is essential. By analyzing the attribute dependence and sample distribution of intrusion data, there are the following problems: “hierarchical dependency omission” and “decision boundary discontinuity.” The former means the previous attribute embedding models failed to incorporate network protocol hierarchy. The latter indicates that the small disjuncts distribution leads to sub-concept fragmentation, exacerbating the difficulty in handling class imbalance. To address these problems, we propose a novel detection framework for Hierarchical Dependency and Class Imbalance (HIDIM). First, we treat semantic attributes as words and introduce the protocol hierarchy of attributes into a paragraph embedding model. Second, we design a synthetic oversampling method. It adopts a mutual nearest neighbour approach to determine the boundaries of each disjunct. Then, it synthesizes high-quality samples within those boundary areas by crossing or mutating features based on their importance. The experimental results on multiple real-world datasets demonstrate that the proposed framework is superior to other state-of-the-art models in terms of accuracy, F1-score, and false negative rate by 2.23%, 2.12%, and 1.43% on average, respectively.

Breast cancer prognosis through the use of multi-modal classifiers: current state of the art and the way forward.

We present a survey of the current state-of-the-art in breast cancer detection and prognosis. We analyze the evolution of Artificial Intelligence-based approaches from using just uni-modal information to multi-modality for detection and how such paradigm shift facilitates the efficacy of detection, consistent with clinical observations. We conclude that interpretable AI-based predictions and ability to handle class imbalance should be considered priority.

Effect of Random Under sampling, Oversampling, and SMOTE on the Performance of Cardiovascular Disease Prediction Models

Cardiovascular Disease (CVD) or commonly known as Heart Disease is a leading cause of mortality globally, prompting extensive research into predictive models to assess individual risk and plan preventive measures. Machine learning approaches such as Random Forest, Support Vector Machine (SVM), and LASSO Logistic Regression have showed promise. Recent studies have indicated that traditional resampling methods like Random Oversampling, Random Undersampling, and SMOTE may not significantly improve model discrimination. This study aims to evaluate the impact of these techniques on the performance of Cardiovascular Disease (CVD) prediction models, utilizing data from the UCI Machine Learning Heart Disease database. By employing LASSO Logistic Regression, Random Forest, and Support Vector Machine (SVM) with resampling techniques, including Random Oversampling, Random Undersampling, and SMOTE. This research seeks to enhance understanding of model performance in addressing class imbalances within the dataset and contribute to refining cardiovascular disease (CVD) prediction strategies. This study demonstrates that the use of the SMOTE technique significantly enhances the performance of cardiovascular disease (CVD) prediction models. Specifically, when combined with the Random Forest algorithm, SMOTE achieves the best performance in terms of accuracy, sensitivity, and specificity. This highlights the importance of selecting appropriate resampling techniques to handle class imbalance in datasets. Consequently, this research contributes to refining CVD prediction strategies and provides new insights into improving prediction accuracy in imbalanced medical data.

StAlK: Structural Alignment based Self Knowledge distillation for Medical Image Classification

In the realm of medical image analysis, where challenges like high class imbalance, inter-class similarity, and intra-class variance are prevalent, knowledge distillation has emerged as a powerful mechanism for model compression and regularization. Existing methodologies, including label smoothening, contrastive learning, and relational knowledge transfer, aim to address these challenges but exhibit limitations in effectively managing either class imbalance or intricate inter and intra-class relations within input samples. In response, this paper introduces StAlK (Structural Alignment based Self Knowledge distillation) for Medical Image Classification, a novel approach which leverages the alignment of complex high-order discriminative features from a mean teacher model. This alignment enhances the student model’s ability to distinguish examples across different classes. StAlK demonstrates superior performance in scenarios involving both inter and intra-class relationships and proves significantly more robust in handling class imbalance compared to baseline methods. Extensive investigations across multiple benchmark datasets reveal that StAlK achieves a substantial improvement of 6%–7% in top-1 accuracy compared to various state-of-the-art baselines. The code is available at: https://github.com/philsaurabh/StAlK_KBS.

An efficient diagnosis of heart disease using optimized cross-layer Densenet121 pyramid mutual attention network

Heart disease significantly impacts global mortality rates, underscoring the need for improved early-stage diagnosis through deep learning algorithms to enhance prognosis accuracy and prevent mortality. However, challenges such as missing values, class imbalance, redundant features and faulty predictions persist in heart disease diagnosis. To address these issues, this research proposes an Optimized Cross-layer Densenet121 Pyramid mutual attention Network (OCDPN) to improve prediction accuracy. Initially, a Hybrid Bag-boost K-means Synthetic Minority Oversampling TEchnique (HBK-SMOTE) is employed to eliminate missing values and handle class imbalance issues. Subsequently, a Two-layer Angle Kernel Linear Discriminant Extreme Learning (TK-LDE) method extracts features and the Hybrid Binary Aquila Beluga whale Optimization (HBABO)selects the most significant features. Finally, the selected features are classified using Fick's law algorithm to achieve high accuracy. The OCDPN model, comprising several dense layers and a softmax classifier, with Fick's law optimization, enhances feature understanding and classification performance. The model's efficiency is assessed using the Framingham Heart Study (FHS), Statlog (Heart), Cleveland Heart Disease (CHD) and Z-Alizadeh Sani datasets. The introduced technique demonstrates outstanding performance, achieving accuracies of 98.2%, 99.8%, 99.2% and 98.5%, respectively, for the considered datasets. These outcomes highlight the effectiveness of the introduced method in predicting heart diseases.

Class imbalanced data handling with cyberattack classification using Hybrid Salp Swarm Algorithm with deep learning approach

Cyberattack classification involves applying deep learning (DL) and machine learning (ML) models to categorize digital threats based on their features and behaviors. These models examine system logs, network traffic, or other associated data patterns to discriminate between standard activities and malicious actions. Efficient cyberattack classification is vital for on-time threat detection and response, permitting cybersecurity specialists to categorize and reduce potential risks to a system. Handling class-imbalanced data in cyberattack classification using DL is critical for achieving exact and robust models. In cybersecurity databases, instances of normal behavior frequently significantly outnumber instances of cyberattacks, foremost due to biased methods that may complete poorly on minority classes. To address this issue approaches such as oversampling the lesser class, undersampling the popular class, or using more advanced systems can be used. These plans defend that the DL technique is more complex when determining cyberattacks, so it increases complete performance and adapts the effect of the imbalance class on the classification results. This study presents a novel Hybrid Salp Swarm Algorithm with a DL Approach for Cyberattack Classification (HSSADL-CAC) technique. The HSSADL-CAC method intends to resolve class imbalance data handling with an optimum DL model for the recognition of cyberattacks. At first, the HSSADL-CAC method experiences data normalization as a pre-processing stage. The HSSADL-CAC technique uses the ADASYN approach to handle class imbalance problems. In addition, the HSSADL-CAC technique applies an HSSA-based feature selection approach. The HSSADL-CAC technique detects cyberattacks using a deep extreme learning machine (DELM) model. Finally, the hyperparameter tuning of the ELM model takes place by utilizing the beluga whale optimization (BWO) model. The performance analysis of the HSSADL-CAC technique employs a benchmark database. The comprehensive comparison research indicates the superior performance of the HSSADL-CAC technique in the cyberattack detection procedure.

Early warning systems for financial distress: A machine learning approach to corporate risk mitigation

This research explores the development of an early warning system for corporate financial distress using machine learning techniques to address key challenges in corporate risk mitigation. The main objective is to enhance predictive accuracy by integrating financial and non-financial data, addressing class imbalance, and ensuring model interpretability. The research design involves the formulation of a new machine learning model, leveraging cost-sensitive learning and feature selection, and is tested with a numerical example using logistic regression. Methodologically, the study adopts a data-driven approach that incorporates diverse financial ratios, macroeconomic variables, and market sentiment indicators to predict corporate distress. The numerical results from a basic logistic regression model demonstrate poor performance, especially in handling class imbalance, revealing limitations in traditional statistical models. However, the research suggests that machine learning methods, particularly ensemble learning with cost-sensitive algorithms, offer superior predictive accuracy and practical applicability. The study concludes that integrating advanced techniques and diverse datasets leads to more reliable early warning systems, with significant implications for corporate governance and financial risk management. Future research should explore more sophisticated machine learning models and extend real-world applications across various industries and economic conditions.

Enhancing Machine Learning Model Performance in Addressing Class Imbalance

This research aims to investigate methods for handling class imbalance in machine learning models, with a focus on the Support Vector Machine (SVM) algorithm. We apply oversampling (SMOTE) and undersampling techniques to a dataset with class imbalance and evaluate the performance of SVM using these methods. Experiments are conducted using data from Twitter social media regarding the 2024 general electionsThe findings indicate that incorporating SMOTE effectively enhances the performance of SVM models, particularly within the SVM Polynomial variant. However, the use of undersampling shows limited impact on improving SVM model performance. This study provides valuable insights for researchers and practitioners in choosing the appropriate strategy for handling class imbalance in machine learning models.

DBOS_US: a density-based graph under-sampling method to handle class imbalance and class overlap issues in software fault prediction

DBOS_US: a density-based graph under-sampling method to handle class imbalance and class overlap issues in software fault prediction

Handling Class Imbalance for Indonesian Twitter Sentiment Analysis A Comparative Study of Algorithms

Handling Class Imbalance for Indonesian Twitter Sentiment Analysis A Comparative Study of Algorithms

Traffic Sign Recognition using HIICNN Stack Ensemble Method

Abstract: This project introduces a novel deep convolutional neural network (CNN) model aimed at enhancing traffic sign recognition for autonomous vehicle technology and road safety. Employing a stacking ensemble approach, we combine multiple CNNs with diverse improvement techniques, including preprocessing for adverse weather conditions and shooting errors, and data augmentation to handle class imbalance. Through adjustments in learning rates during model training, we mitigate overfitting. Our ensemble model achieves an exceptional 99.4 percent test accuracy on the German Traffic Sign Recognition Benchmark (GTSRB) dataset, surpassing prior studies. We utilize gradient-weighted Class Activation Mapping (Grad-CAM) for model explain- ability and an evidential deep learning approach to quantify classification uncertainty. Our framework underscores the efficacy of combining preprocessing, ensemble learning, and transfer learning for superior perfor- mance and reliability in traffic sign recognition systems, contributing significantly to autonomous driving and road safety.

Systematic Review of Models Usedto Handle Class Imbalance in Anomaly Detection for Energy Consumption

The widespread integration of Smart technologies into energy consumption systems has brought about a transformative shift in monitoring and managing electricity usage. The imbalanced nature of anomaly data often results in suboptimal performance in detecting rare anomalies. This literature review analyzes models designed to address this challenge. The methodology involves a systematic literature review based on the five-step framework proposed by Khan, encompassing framing research questions, identifying relevant literature, assessing article quality, conducting a critical review, and interpreting results. The findings show that classical machine learning models like Support Vector Machines (SVM) and Random Forests (RF) are commonly used. In conclusion, classical machine learning models like SVM and RF struggle to recognize rare anomalies, while deep learning models, notably Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM), show promise for automatically learning elaborate representations and improving performance while dealing with class imbalance.

CIIR: an approach to handle class imbalance using a novel feature selection technique

CIIR: an approach to handle class imbalance using a novel feature selection technique

Enhanced Classification of Imbalanced Medical Datasets using Hybrid Data-Level, Cost-Sensitive and Ensemble Methods

Addressing the class imbalance in classification problems is particularly challenging, especially in the context of medical datasets where misclassifying minority class samples can have significant repercussions. This study is dedicated to mitigating class imbalance in medical datasets by employing a hybrid approach that combines data-level, cost-sensitive, and ensemble methods. Through an assessment of the performance, measured by AUC-ROC values, Sensitivity, F1-Score, and G-Mean of 20 data-level and four cost-sensitive models on seventeen medical datasets - 12 small and five large, a hybridized model, SMOTE-RF-CS-LR has been devised. This model integrates the Synthetic Minority Oversampling Technique (SMOTE), the ensemble classifier Random Forest (RF), and the Cost-Sensitive Logistic Regression (CS-LR). Upon testing the hybridized model on diverse imbalanced ratios, it demonstrated remarkable performance, achieving outstanding performance values on the majority of the datasets. Further examination of the model's training duration and time complexity revealed its efficiency, taking less than a second to train on each small dataset. Consequently, the proposed hybridized model not only proves to be time-efficient but also exhibits robust capabilities in handling class imbalance, yielding outstanding classification results in the context of medical datasets.