Oversampling Method Research Articles

Geoacoustic and geophysical data‐driven seafloor sediment classification through machine learning algorithms with property‐centered oversampling techniques

AbstractThis study aims to classify seafloor sediments using physics‐inspired and data‐driven soil models combined with machine learning algorithms and oversampling techniques. The field data used for the input variables include porosity, S‐ and P‐wave velocities and depth. The soil information reported in the original literature and the “six reference sediments” and effective stress‐versus‐depth models proposed by the previous study confirm the sediment type across all of the input variables. We use three machine learning algorithms and four oversampling methods to enhance the performance accuracy and overcome data imbalance in the minority class. The results show that the averaged accuracy of sediment classification with original data corresponds to 0.88 for porosity, 0.61 for S‐wave velocity, and 0.97 for P‐wave velocity. In particular, the enhanced accuracy with oversampled input variables becomes more pronounced when the depth data are considered in a dataset. The class‐oriented grouping method newly proposed in this study appears to be a robust approach to enhancing performance. Surprisingly, model‐based input variables lead to the best performance in all cases. The proposed analyses conducted using machine learning algorithms and oversampling techniques within the physics‐inspired models could be extended to obtain a first‐order assessment of marine sediment properties.

A novel two-way rebalancing strategy for identifying carbonylation sites

BackgroundAs an irreversible post-translational modification, protein carbonylation is closely related to many diseases and aging. Protein carbonylation prediction for related patients is significant, which can help clinicians make appropriate therapeutic schemes. Because carbonylation sites can be used to indicate change or loss of protein function, integrating these protein carbonylation site data has been a promising method in prediction. Based on these protein carbonylation site data, some protein carbonylation prediction methods have been proposed. However, most data is highly class imbalanced, and the number of un-carbonylation sites greatly exceeds that of carbonylation sites. Unfortunately, existing methods have not addressed this issue adequately.ResultsIn this work, we propose a novel two-way rebalancing strategy based on the attention technique and generative adversarial network (Carsite_AGan) for identifying protein carbonylation sites. Specifically, Carsite_AGan proposes a novel undersampling method based on attention technology that allows sites with high importance value to be selected from un-carbonylation sites. The attention technique can obtain the value of each sample’s importance. In the meanwhile, Carsite_AGan designs a generative adversarial network-based oversampling method to generate high-feasibility carbonylation sites. The generative adversarial network can generate high-feasibility samples through its generator and discriminator. Finally, we use a classifier like a nonlinear support vector machine to identify protein carbonylation sites.ConclusionsExperimental results demonstrate that our approach significantly outperforms other resampling methods. Using our approach to resampling carbonylation data can significantly improve the effect of identifying protein carbonylation sites.

Advancing prognostic precision in pulmonary embolism: A clinical and laboratory-based artificial intelligence approach for enhanced early mortality risk stratification

BackgroundAcute pulmonary embolism (PE) is a critical medical emergency that necessitates prompt identification and intervention. Accurate prognostication of early mortality is vital for recognizing patients at elevated risk for unfavourable outcomes and administering suitable therapy. Machine learning (ML) algorithms hold promise for enhancing the precision of early mortality prediction in PE patients. ObjectiveTo devise an ML algorithm for early mortality prediction in PE patients by employing clinical and laboratory variables. MethodsThis study utilized diverse oversampling techniques to improve the performance of various machine learning models including ANN, SVM, DT, RF, and AdaBoost for early mortality prediction. Appropriate oversampling methods were chosen for each model based on algorithm characteristics and dataset properties. Predictor variables included four lab tests, eight physiological time series indicators, and two general descriptors. Evaluation used metrics like accuracy, F1_score, precision, recall, Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) curves, providing a comprehensive view of models' predictive abilities. ResultsThe findings indicated that the RF model with random oversampling exhibited superior performance among the five models assessed, achieving elevated accuracy and precision alongside high recall for predicting the death class. The oversampling approaches effectively equalized the sample distribution among the classes and enhanced the models' performance. ConclusionsThe suggested ML technique can efficiently prognosticate mortality in patients afflicted with acute PE. The RF model with random oversampling can aid healthcare professionals in making well-informed decisions regarding the treatment of patients with acute PE. The study underscores the significance of oversampling methods in managing imbalanced data and emphasizes the potential of ML algorithms in refining early mortality prediction for PE patients.

Deep learning based on Transformer architecture for power system short-term voltage stability assessment with class imbalance

Most existing data-driven power system short-term voltage stability assessment (STVSA) approaches presume class-balanced input data. However, in practical applications, the occurrence of short-term voltage instability following a disturbance is minimal, leading to a significant class imbalance problem and a consequent decline in classifier performance. This work proposes a Transformer-based STVSA method to address this challenge. By utilizing the basic Transformer architecture, a stability assessment Transformer (StaaT) is developed as a classification model to reflect the correlation between the operational states of the system and the resulting stability outcomes. To combat the negative impact of imbalanced datasets, this work employs a conditional Wasserstein generative adversarial network with gradient penalty (CWGAN-GP) for synthetic data generation, aiding in the creation of a balanced, representative training set for the classifier. Semi-supervised clustering learning is implemented to enhance clustering quality, addressing the lack of a unified quantitative criterion for short-term voltage stability. Numerical tests on the IEEE 39-bus test system extensively demonstrate that the proposed method exhibits robust performance under class imbalances up to 100:1 and noisy environments, and maintains consistent effectiveness even with an increased penetration of renewable energy. Comparative results reveal that the CWGAN-GP generates more balanced datasets than traditional oversampling methods and that the StaaT outperforms other deep learning algorithms. This study presents a compelling solution for real-world STVSA applications that often face class imbalance and data noise challenges.

A density-based oversampling approach for class imbalance and data overlap

In data mining classification, class imbalance is characterized that different classes have an obvious difference in the number of samples. Most classifiers typically assume a balanced class distribution or assign equal classification error costs to different classes. Therefore, directly using imbalanced class will worsen the classification performance. The oversampling algorithms can achieve the balance by synthesizing new samples, but the uncontrollable positions of the synthetic samples may aggravate the data overlap and further deteriorate the classification performance. To tackle this challenge, an improved synthetic minority oversampling technique based on kernel density estimation and neighbor density selection (KDENDS_SMOTE) is proposed in this paper. First, each sample is mapped into a high-dimensional space to avoid the choice of the window width and to overcome the nonlinear separable limitation. Kernel density estimation is then used to derive the density ratio, which serves as a measure of the degree of data overlap. Subsequently, the stability degree of the density ratio is calculated using neighbor information, and a scoring mechanism combining the density ratio and its stability degree is proposed to assess the fitness of selected samples. Furthermore, the neighbor density selection based on the above scoring mechanism can guide SMOTE to generate new samples within a safe and stable region, away from areas with data overlap. Finally, compared with six advanced oversampling methods on fifteen real-world datasets, the KDENDS_SMOTE can effectively mitigate the data overlap and improve the classification performance.

Quantitative image signature and machine learning-based prediction of outcomes in cerebral cavernous malformations

PurposeThere is increasing interest in novel prognostic tools and predictive biomarkers to help identify, with more certainty, cerebral cavernous malformations (CCM) susceptible of bleeding if left untreated. We developed explainable quantitative-based machine learning models from magnetic resonance imaging (MRI) in a large CCM cohort to demonstrate the value of artificial intelligence and radiomics in complementing natural history studies for hemorrhage and functional outcome prediction. Materials and MethodsOne-hundred-eighty-one patients from a prospectively registered cohort of 366 adults with CCM were included. Fluid attenuated inversion recovery (FLAIR) T2-weighted brain images were preprocessed, and CCM and surrounding edema were segmented before radiomic feature computation. Minority class oversampling, dimensionality reduction and feature selection methods were applied. With prospective hemorrhage as primary outcome, machine learning models were built, cross-validated, and compared using clinico-radiologic, radiomic, and combined features. SHapley Additive exPlanations (SHAP) was used for interpretation to determine the radiomic features with most contribution to hemorrhage prediction. ResultsThe highest performances in hemorrhage predictions on the test set were combining radiomic and clinico-radiological features with an area under the curve (AUC) of 83% using linear regression and selected features, and an F1 score of 61% and 85% sensitivity using K-nearest neighbors with principal component analysis (PCA). Multilayer perceptron had the best performance predicting modified Rankin Scale ≥ 2 with an AUC of 74% using PCA derived features. For interpretation of the selected radiomic signature XGBoost model, Shapley additive explanations highlighted 6 radiomic features contributing the most to hemorrhage prediction. ConclusionQuantitative image-based modeling using machine learning has the potential to highlight novel imaging biomarkers that predict hemorrhagic and functional outcomes, ensuring more precise and personalized care for CCM patients.

Comparing Different Oversampling Methods in Predicting Multi-Class Educational Datasets Using Machine Learning Techniques

Abstract Predicting students’ academic performance is a critical research area, yet imbalanced educational datasets, characterized by unequal academic-level representation, present challenges for classifiers. While prior research has addressed the imbalance in binary-class datasets, this study focuses on multi-class datasets. A comparison of ten resampling methods (SMOTE, Adasyn, Distance SMOTE, BorderLineSMOTE, KmeansSMOTE, SVMSMOTE, LN SMOTE, MWSMOTE, Safe Level SMOTE, and SMOTETomek) is conducted alongside nine classification models: K-Nearest Neighbors (KNN), Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Support Vector Machine (SVM), Logistic Regression (LR), Extra Tree (ET), Random Forest (RT), Extreme Gradient Boosting (XGB), and Ada Boost (AdaB). Following a rigorous evaluation, including hyperparameter tuning and 10 fold cross-validations, KNN with SmoteTomek attains the highest accuracy of 83.7%, as demonstrated through an ablation study. These results emphasize SMOTETomek’s effectiveness in mitigating class imbalance in educational datasets and highlight KNN’s potential as an educational data mining classifier.

Video Super-Resolution Based on Inter-Frame Information Utilization for Intelligent Transportation

Intelligent transportation infrastructure is essential to intelligent transportation system (ITS). With the continuous development of Internet of Things (IoT) technology, remote monitoring has become a critical part of ITS. However, due to the limitations of network transmission, production cost, and other factors, some video monitoring can obtain only low-resolution (LR) video. LR video features are seriously lost, thus affecting the performance of ITS. In this paper, based on the research of existing super-resolution algorithms, we focus on improving the reconstruction quality of video frame sequences by aiming at the insufficient utilization of inter-frame information and low reconstruction quality of existing video super-resolution algorithms. This paper proposes a video super-resolution algorithm based on inter-frame information utilization, which can effectively improve the performance of ITS. First, a novel U-shaped feature extractor is designed to fully extract the feature expression of video frame sequences. Second, a deformable inter-frame alignment module based on residual learning is constructed to make the inter-frame alignment more accurate and thus promote the mutual utilization of inter-frame information. Finally, an up and down sampling residual block is proposed to extract features that better match the upsampling reconstruction requirements. The experimental results show that the method has better reconstruction quality for monitoring video and is advanced and applicable compared to mainstream video oversampling methods.

An oversampling method based on differential evolution and natural neighbors

The classification problem of imbalanced data is a research focus in machine learning. An effective method for solving the class-imbalance problem is to generate synthetic samples for minority class data. Popular methods for synthesize samples include SMOTE, variants of SMOTE, and oversampling methods applying metaheuristic algorithms. Recently, an oversampling method DEBOHID based on differential evolutionary algorithm has been proposed. It uses nearest neighbors of the minority class to synthesize samples for balancing the data set. However, samples synthesized by nearest neighbors are easily interfered by noisy samples. Therefore, we proposed an oversampling method based on natural neighbors, called NaN-DEBOHID. In NaN-DEBOHID, minority samples are divided into dense samples and outliers according to the number of natural neighbors. Samples of different types are processed by different methods. The main advantages of NaN-DEBOHID are concluded as follows: (a) it finds samples that better represent the minority class distribution by applying natural neighbors; (b) it creates synthetic samples that showed better consistency with the surrounding samples by the DEBOHID method; (c) it removes noise samples of the minority class to enhance the classification performance. For experiments, Support Vector Machine (SVM) and k-Nearest Neighbor (kNN) are used as classifiers. The results indicated that NaN-DEBOHID performed competitively in terms of Accuracy, F-measure and AUC.

A New Phase Classifier with an Optimized Feature Set in ML-Based Phase Prediction of High-Entropy Alloys

The phases of high-entropy alloys (HEAs) are closely related to their properties. However, phase prediction bears a significant challenge due to the extensive search space and complex formation mechanisms of HEAs. This study demonstrates a precise and timely methodology for predicting alloy phases. It first developed a machine learning classifier using 145 features and a dataset with 1009 samples to differentiate the four types of alloy phases. Feature selection was performed on the feature set using an Embedded algorithm and a genetic algorithm, resulting in the selection of nine features. The Light GBM algorithm was chosen to train the machine learning model. Finally, the implementation of oversampling and cost-sensitive methods enables LightGBM to tackle the problem of insufficient accuracy in BCC+FCC phase classification. The resulting accuracy of the alloy phase prediction model, evaluated through ten-fold cross-validation, stands at 0.9544.

An efficient approach for data-imbalanced hate speech detection in Arabic social media

The use of hateful language in public debates and forums is becoming more common. However, this might result in antagonism and conflicts among individuals, which is undesirable in an online environment. Countries, businesses, and educational institutions are exerting their greatest efforts to develop effective solutions to manage this issue. In addition, recognizing such content is difficult, particularly in Arabic, due to a variety of challenges and constraints. Long-tailed data distribution is often one of the most significant issues in actual Arabic hate speech datasets. Pre-trained models, such as bidirectional encoder representations from transformers (BERT) and generative pre-trained transformers (GPT), have become more popular in numerous natural language processing (NLP) applications in recent years. We conduct extensive experiments to address data imbalance issues by utilizing oversampling methods and a focal loss function in addition to traditional loss functions. Quasi-recurrent neural networks (QRNN) are employed to fine-tune the cutting-edge transformer-based models, MARBERTv2, MARBERTv1, and ARBERT. In this context, we suggest a new approach using ensemble learning that incorporates best-performing models for both original and oversampled datasets. Experiments proved that our proposed approach achieves superior performance compared to the most advanced methods described in the literature.

A clustered borderline synthetic minority over-sampling technique for balancing quick access recorder data

Most of the flight accident data have uneven distribution of categories. When the traditional classifier is applied to this data, it will pay less attention to the minority class data. Synthetic Minority Over-sampling Technique (SMOTE), and its improvements are well-known methods to address this imbalance problem at the data level. However, traditional algorithms still have the problems in blurring the boundary of positive and negative classes and changing the distribution of original data. In order to overcome these problems and accurately predict flight accidents, a new Clustered Biased Borderline SMOTE(CBB-SMOTE) is proposed for Quick Access Recorder (QAR) Go-Around data. It generates more obvious positive and negative class boundaries by using K-means for boundary minority class data and safety minority class data respectively, and maintains the original data distribution to the greatest extent through a biased oversampling method. Experiments were carried out on a group of QAR Go-Around data. The data set is balanced by CBB-SMOTE, SMOTE, Cluster-SMOTE algorithm respectively, and the random forest algorithm is used to predict the new data set. The experimental results show that CBB-SMOTE outperforms the SMOTE in terms of G-means value, Recall and AUC.

IRS-BAG-Integrated Radius-SMOTE Algorithm with Bagging Ensemble Learning Model for Imbalanced Data Set Classification

Imbalanced learning problems are a challenge faced by classifiers when data samples have an unbalanced distribution among classes. The Synthetic Minority Over-Sampling Technique (SMOTE) is one of the most well-known data pre-processing methods. Problems that arise when oversampling with SMOTE are the phenomenon of noise, small disjunct samples, and overfitting due to a high imbalance ratio in a dataset. A high level of imbalance ratio and low variance conditions cause the results of synthetic data generation to be collected in narrow areas and conflicting regions among classes and make them susceptible to overfitting during the learning process by machine learning methods. Therefore, this research proposes a combination between Radius-SMOTE and Bagging Algorithm called the IRS-BAG Model. For each sub-sample generated by bootstrapping, oversampling was done using Radius SMOTE. Oversampling on the sub-sample was likely to overcome overfitting problems that might occur. Experiments were carried out by comparing the performance of the IRS-BAG model with various previous oversampling methods using the imbalanced public dataset. The experiment results using three different classifiers proved that all classifiers had gained a notable improvement when combined with the proposed IRS-BAG model compared with the previous state-of-the-art oversampling methods. Doi: 10.28991/ESJ-2023-07-05-04 Full Text: PDF

Predictive Modeling of Student Dropout in MOOCs and Self-Regulated Learning

The primary objective of this study is to examine the factors that contribute to the early prediction of Massive Open Online Courses (MOOCs) dropouts in order to identify and support at-risk students. We utilize MOOC data of specific duration, with a guided study pace. The dataset exhibits class imbalance, and we apply oversampling techniques to ensure data balancing and unbiased prediction. We examine the predictive performance of five classic classification machine learning (ML) algorithms under four different oversampling techniques and various evaluation metrics. Additionally, we explore the influence of self-reported self-regulated learning (SRL) data provided by students and various other prominent features of MOOCs as potential indicators of early stage dropout prediction. The research questions focus on (1) the performance of the classic classification ML models using various evaluation metrics before and after different methods of oversampling, (2) which self-reported data may constitute crucial predictors for dropout propensity, and (3) the effect of the SRL factor on the dropout prediction performance. The main conclusions are: (1) prominent predictors, including employment status, frequency of chat tool usage, prior subject-related experiences, gender, education, and willingness to participate, exhibit remarkable efficacy in achieving high to excellent recall performance, particularly when specific combinations of algorithms and oversampling methods are applied, (2) self-reported SRL factor, combined with easily provided/self-reported features, performed well as a predictor in terms of recall when LR and SVM algorithms were employed, (3) it is crucial to test diverse machine learning algorithms and oversampling methods in predictive modeling.

Combination of Smote and Random Forest Methods for Lung Cancer Classification

Lung cancer is a network of cells that grow abnormally in the lungs. Lung cancer has four severity levels, namely stages 1 to 4. If lung cancer is not treated quickly, it is at risk of causing death. This research aimed to combine Synthetic Minority Over-sampling (Smote) and Random Forest methods for lung cancer classification. The method used was a combination of Smote and Random Forest. Smote was used to balance the data, while Random Forest was used to classify lung cancer data. The results showed that the combination of Smote and Random Forest methods obtained an accuracy of 94.1%, sensitivity of 94.5, and specificity of 93.7%. Meanwhile, without Smote, the accuracy is 89.1%, sensitivity is 55%, and specificity is 94.5%. The use of Smote can improve the performance of the Random Forest classification method based on accuracy and sensitivity. There was an increase of 5% in accuracy and a 39% increase in sensitivity.

Short-Term Electricity Price Forecast Using Frequency Analysis and Price Spikes Oversampling

This paper proposes a new short-term electricity price forecast framework using frequency analysis and price spikes oversampling. To perform normal price forecast, firstly, the variational mode decomposition (VMD) method is adopted to decompose price data into multiple frequency band-limited modes where each mode has a center frequency. Then, the extended discrete Fourier transform (EDFT) method is utilized to transform the decomposed price modes from time domain into frequency domain to predict the price. In addition, to facilitate price spike prediction, over-sampling methods including the enhanced structure preserving oversampling (ESPO) and the synthetic minority oversampling technique (SMOTE) for regression, are adopted to synthesize price spike cases. Multiple electricity markets are studied and numerous comparison results illustrate that the proposed forecast framework is capable to accurately predict both normal electricity prices and price spikes.

Imbalanced Data Over-Sampling Method Based on ISODATA Clustering

Imbalanced Data Over-Sampling Method Based on ISODATA Clustering

Label correlation guided borderline oversampling for imbalanced multi-label data learning

Multi-label data classification has received much attention due to its wide range of application domains. Unfortunately, a class imbalance problem often occurs in multi-label datasets, causing challenges for classification algorithms. Oversampling is one of the most important approaches, as it generates minority label instances to balance the class distribution. However, existing oversampling methods ignore existing label correlations, resulting in the generation of inappropriate synthetic minority samples and making multi-label data classification tasks harder. In this work, we propose an oversampling method that considers label correlations and identifies two critical boundary regions for generating synthetic minority samples. Moreover, we propose a weighting strategy to assign weights to these instances based on their distance information. To evaluate the performance of our proposed method, we conducted experiments on sixteen public datasets. The results show that our approach outperforms the state-of-the-art approaches in terms of various assessment metrics, such as Macro F1 and Macro AUC. The code is available at https://github.com/IntelliDAL/Multi-label/tree/main/LCOS.

A new boundary-degree-based oversampling method for imbalanced data

A new boundary-degree-based oversampling method for imbalanced data

Comparación de Métodos de Extracción de Características Espectrales y Dispersas para Clasificación de Sonidos Cardíacos

Cardiovascular diseases (CVDs) remain the leading cause of morbidity worldwide. The heart sound signal or phonocardiogram (PCG) is the most simple, low-cost, and effective tool to assist physicians in diagnosing CVDs. Advances in signal processing and machine learning have motivated the design of computer-aided systems for heart illness detection based only on the PCG. The objective of this work is to compare the effects of using spectral and sparse features for a classification scheme to detect the presence/absence of a pathological state in a heart sound signal, more specifically, sparse representations using Matching Pursuit with multiscale Gabor time-frequency dictionaries, linear prediction coding, and Mel-frequency cepstral coefficients. This work compares the performance of PCGs classification applying features as a result of averaging the samples or the features for each PCG sound event when feeding a random forest (RF) classifier. For data balancing, random under-sampling and synthetic minority oversampling (SMOTE) methods were applied. Furthermore, we compare the Correlation Feature Selection (CFS) and Information Gain (IG) for the dimensionality reduction. The findings show a SE=93.17 %, SP=84.32 % and ACC=85.9 % when joining MP+LPC+MFCC features set with an AUC=0.969 showing that these features are promising to be used in heart sounds anomaly detection schemes.