Oversampling Method Research Articles

Evaluation of Oversampling Methods (OVER, SMOTE, and ROSE) in Classifying Soil Liquefaction Dataset based on SVM, RF, and Naïve Bayes

Class imbalanced datasets are prevalent in real-world applications, including engineering, medical domain, financial sector, and others. Machine learning (ML)-based prediction models have successfully demonstrated the applicability of various algorithms for the solution of different problems. However, their application for the soil liquefaction issue considering the class imbalance situation is limited. This paper presents the prediction results of random forest (RF), support vector machine (SVM), and naïve bayes (NB) algorithms with different training sample sizes for soil liquefaction. The effect of oversampling methods, namely simple oversampling (OVER), random oversampling examples (ROSE), and synthetic minority oversampling technique (SMOTE), on the prediction performance of classification algorithms is also investigated. Performance results are evaluated by means of some metrics, including Accuracy, Kappa, Precision, Recall, and F-measure. The results concluded the effectiveness of applying oversampling methods on imbalanced data before the modeling phase. All of the oversampling methods helped to enhance the overall performances of the classification models. It is also observed that the SMOTE exhibited slightly better performance than other considered oversampling methods. Furthermore, the SVM model outperformed compared to RF and NB models when all algorithms were trained by the SMOTE algorithm.

A probabilistic data analytics methodology based on Bayesian Belief network for predicting and understanding breast cancer survival

Understanding breast cancer survival has proven to be a challenging problem for practitioners and researchers. Identifying the factors affecting cancer progression, their interrelationships, and their influence on patients’ long-term survival helps make timely treatment decisions. The current study addresses this problem by proposing a Tree-Augmented Bayesian Belief Network (TAN)-based data analytics methodology comprising of four steps: data acquisition and preprocessing, variable selection via Genetic Algorithm (GA), data balancing with synthetic minority over-sampling and random under-sampling methods, and finally the development of the TAN model to determine the probabilistic inter-conditional dependency structure among breast cancer-related variables along with the posterior survival probabilities The proposed model is compared to well-known machine learning models. A what-if analysis has also been conducted to verify the associations among the variables in the TAN model. The relative importance of each variable has been investigated via sensitivity analysis. Finally, a decision support tool is developed to further explore the conditional dependency structure among the cancer-related factors. The results produced by the proposed methodology, namely the patient-specific posterior survival probabilities and the conditional relationships among the variables, can be used by healthcare professionals and physicians to improve the decision-making process in planning and managing breast cancer treatments. Our generic methodology can also accommodate other types of cancer and be applied to manage various medical procedures.

A Classification-Based Machine Learning Approach to the Prediction of Cyanobacterial Blooms in Chilgok Weir, South Korea

Cyanobacterial blooms appear by complex causes such as water quality, climate, and hydrological factors. This study aims to present the machine learning models to predict occurrences of these complicated cyanobacterial blooms efficiently and effectively. The dataset was classified into groups consisting of two, three, or four classes based on cyanobacterial cell density after a week, which was used as the target variable. We developed 96 machine learning models for Chilgok weir using four classification algorithms: k-Nearest Neighbor, Decision Tree, Logistic Regression, and Support Vector Machine. In the modeling methodology, we first selected input features by applying ANOVA (Analysis of Variance) and solving a multi-collinearity problem as a process of feature selection, which is a method of removing irrelevant features to a target variable. Next, we adopted an oversampling method to resolve the problem of having an imbalanced dataset. Consequently, the best performance was achieved for models using datasets divided into two classes, with an accuracy of 80% or more. Comparatively, we confirmed low accuracy of approximately 60% for models using datasets divided into three classes. Moreover, while we produced models with overall high accuracy when using logCyano (logarithm of cyanobacterial cell density) as a feature, several models in combination with air temperature and NO3-N (nitrate nitrogen) using two classes also demonstrated more than 80% accuracy. It can be concluded that it is possible to develop very accurate classification-based machine learning models with two features related to cyanobacterial blooms. This proved that we could make efficient and effective models with a low number of inputs.

Deep Learning-Based Imbalanced Classification With Fuzzy Support Vector Machine.

Imbalanced classification is widespread in the fields of medical diagnosis, biomedicine, smart city and Internet of Things. The imbalance of data distribution makes traditional classification methods more biased towards majority classes and ignores the importance of minority class. It makes the traditional classification methods ineffective in imbalanced classification. In this paper, a novel imbalance classification method based on deep learning and fuzzy support vector machine is proposed and named as DFSVM. DFSVM first uses a deep neural network to obtain an embedding representation of the data. This deep neural network is trained by using triplet loss to enhance similarities within classes and differences between classes. To alleviate the effects of imbalanced data distribution, oversampling is performed in the embedding space of the data. In this paper, we use an oversampling method based on feature and center distance, which can obtain more diverse new samples and prevent overfitting. To enhance the impact of minority class, we use a fuzzy support vector machine (FSVM) based on cost-sensitive learning as the final classifier. FSVM assigns a higher misclassification cost to minority class samples to improve the classification quality. Experiments were performed on multiple biological datasets and real-world datasets. The experimental results show that DFSVM has achieved promising classification performance.

Capacitance-to-voltage converter employing parallel-series passive charge integrator for low noise power efficient MEMS capacitive sensor

Parallel-series passive charge integrator (PSPI) based capacitance-to-voltage converter (CVC) is proposed in this paper to reduce the noise floor of readout circuit for MEMS capacitive sensor with high power efficiency in terms of the figure of merit (FoM). For MEMS capacitive accelerometer, open-loop structure is preferred for its low power consumption. However, in the open-loop structure, the capacitance variation of sensing element is limited to femto-farad level in order to overcome nonlinearity. As a result, the thermal noise from parasitic capacitance becomes significant, which leads to low SNR. Traditional oversampling method can reduce the parasitic-induced noise, but leads to low power efficiency. In this work, the PSPI-CVC is proposed to reduce the parasitic-induced noise with high power efficiency. The PSPI-CVC is designed by a commercial 0.18 μm CMOS process. The transistor-level simulation results show that, compared to the traditional CVC, the PSPI-CVC can improve the SNR by 24 dB without increase of the power consumption. Compared with the other similar works, the proposed readout circuit achieves competitive FoM1 (FoM1 =20.2pJ)and the best FoM2 (FoM2).

An explainable machine learning framework for lung cancer hospital length of stay prediction

This work introduces a predictive Length of Stay (LOS) framework for lung cancer patients using machine learning (ML) models. The framework proposed to deal with imbalanced datasets for classification-based approaches using electronic healthcare records (EHR). We have utilized supervised ML methods to predict lung cancer inpatients LOS during ICU hospitalization using the MIMIC-III dataset. Random Forest (RF) Model outperformed other models and achieved predicted results during the three framework phases. With clinical significance features selection, over-sampling methods (SMOTE and ADASYN) achieved the highest AUC results (98% with CI 95%: 95.3–100%, and 100% respectively). The combination of Over-sampling and under-sampling achieved the second-highest AUC results (98%, with CI 95%: 95.3–100%, and 97%, CI 95%: 93.7–100% SMOTE-Tomek, and SMOTE-ENN respectively). Under-sampling methods reported the least important AUC results (50%, with CI 95%: 40.2–59.8%) for both (ENN and Tomek- Links). Using ML explainable technique called SHAP, we explained the outcome of the predictive model (RF) with SMOTE class balancing technique to understand the most significant clinical features that contributed to predicting lung cancer LOS with the RF model. Our promising framework allows us to employ ML techniques in-hospital clinical information systems to predict lung cancer admissions into ICU.

A Robust Oversampling Approach for Class Imbalance Problem with Small Disjuncts

Class imbalance is one of the important challenges for machine learning because of it's learning to bias toward the majority classes. The oversampling method is a fundamental imbalance-learning technique with many real-world applications. However, when the small disjuncts problem occurs, how to effectively avoiding the negative oversampling results rather than using clusters previously, remains a challenging task. Thus, this study introduces a disjuncts-robust oversampling (DROS) method. The novel method shows that the data filling of new synthetic samples to the minority class areas in data space can be thought of as the searchlight illuminating with light cones to the restricted areas in real life. In the first step, DROS computes a series of light-cone structures that is first started from the inner minority class area, then passes through the boundary minority class area, last is stopped by the majority class area. In the second step, DROS generates new synthetic samples in those light-cone structures. Experiments considering both real-world and 2D emulational datasets demonstrate that our method outperforms the current state-of-the-art oversampling methods and suggest that our method is able to deal with the small disjuncts.

Stop Oversampling for Class Imbalance Learning: A Review

For the last two decades, oversampling has been employed to overcome the challenge of learning from imbalanced datasets. Many approaches to solving this challenge have been offered in the literature. Oversampling, on the other hand, is a concern. That is, models trained on fictitious data may fail spectacularly when put to real-world problems. The fundamental difficulty with oversampling approaches is that, given a real-life population, the synthesized samples may not truly belong to the minority class. As a result, training a classifier on these samples while pretending they represent minority may result in incorrect predictions when the model is used in the real world. We analyzed a large number of oversampling methods in this paper and devised a new oversampling evaluation system based on hiding a number of majority examples and comparing them to those generated by the oversampling process. Based on our evaluation system, we ranked all these methods based on their incorrectly generated examples for comparison. Our experiments using more than 70 oversampling methods and nine imbalanced real-world datasets reveal that all oversampling methods studied generate minority samples that are most likely to be majority. Given data and methods in hand, we argue that oversampling in its current forms and methodologies is unreliable for learning from class imbalanced data and should be avoided in real-world applications.

Single-Point Crossover and Jellyfish Optimization for Handling Imbalanced Data Classification Problem

The imbalanced datasets and their classification has pulled in as a hot research topic over the years. It is used in different fields, for example, security, finance, health, and many others. The imbalanced datasets are balanced by applying resampling and various solutions are designed to tackle such datasets that mainly focus on class distribution issues. The imbalanced data is rebalanced using these methods. This paper introduces a technique for balancing data through two stages: first, oversampling methods are utilized in the process of rebalancing such imbalanced dataset using the single-point crossover to generate the new data of minority classes, second, it searches for an optimal subset of the imbalanced and balanced datasets by Jellyfish Search (JS) which is an optimization method. Experiments are performed on 18 real imbalanced datasets, and results are compared with famous oversampling methods and the recently published ACOR (Ant Colony Optimization Resampling) method in terms of different appraisal measurements. Higher performance is recorded by the proposed method and comparability with well-known and recent techniques.

Optimised feature selection and cervical cancer prediction using Machine learning classification

Background: Screening and early detection play a key role in cervical cancer prevention. The present study predicts the outcome of various diagnostic tests used to diagnose cervical cancer using machine learning algorithms. Methods: The present study ran various cervical cancer risk factors on a machine learning (ML) classifier to predict outcomes of Hinselmann, Schiller, cytology and biopsy. The dataset is publicly available on the Machine Learning Repository website of the University of California Irvine. The imbalanced dataset was pre-processed using oversampling methods. The significantly varied features between the two levels of a response variable were used to train the machine learning classifiers on MATLAB. The classifiers used were Decision Trees, Support Vector Machine, K-Nearest Neighbours and Ensemble learning classifiers. The performance metrics of the classifiers were expressed as accuracy, the area under the receiver operator characteristic (AU-ROC) curve, sensitivity and specificity. Results: The Fine Gaussian SVM classifier was the best to classify Hinselmann, cytology and biopsy with the accuracy of 97.5 %, 62.5 % and 98 %, respectively. However, Boosted trees performed best in the classification of Schiller with 81.3 % accuracy. Conclusion: The present study selected optimised features among multiple risk factors to train various ML classifiers to predict cervical cancer.

Novel K-Medoids Based SMOTE Integrated With Locality Preserving Projections for Fault Diagnosis

In the field of the fault diagnosis of industrial processes, there are many problems in process data, such as missing critical fault data, high repeatability of normal state data, and poor representation of faults data, which may reduce the accuracy of fault diagnosis. In this paper, a novel K-medoids based synthetic minority over-sampling technique that combines locality preserving projections (KMS-LPP), is proposed for fault diagnosis. First, the synthetic minority sampling technology (SMOTE) is designed based on K-medoid to generate minority fault samples to address the imbalanced problem of data. Second, to extract the key fault-relevant features and reserve the local structure information at the same time, the manifold learning (ML) approach locality preserving projections (LPP) is performed to reduce the dimensionality of data. Finally, the Adaboost. M2, as an ensemble classifier, is conducted for fault classification. Simulations of the Tennessee Eastman process (TEP) are performed for validation of the performance of the presented KMS-LPP method. The obtained results show that KMS-LPP has enhanced the performance of fault diagnosis due to its higher accuracy compared with traditional oversampling and feature extraction methods, which indicates the effectiveness of KMS-LPP.

Generative Adversarial Minority Enlargement–A Local Linear Over-Sampling Synthetic Method

Generative Adversarial Minority Enlargement–A Local Linear Over-Sampling Synthetic Method

Supervised learning of COVID-19 patients' characteristics to discover symptom patterns and improve patient outcome prediction

The world today faces a new challenge that is unprecedented in the last 100 years. The emergence of a new coronavirus has led to a human catastrophe. Scientists in various sciences have been looking for solutions to this problem so far. In addition to general vaccination, maintaining social distance and adherence to government guidelines on safety precaution measures are the most well-known strategies to prevent COVID-19 infection. In this research, we tried to examine the symptoms of COVID-19 cases through different supervised machine learning methods. We solved the class imbalance problem using the synthetic minority over-sampling (SMOTE) method and then developed some classification models to predict the outcome of COVID-19 cases (recovery or death). Besides, we implemented a rule-based technique to identify different combinations of variables with specific ranges of their values that together affect disease severity. Our results showed that the random forest model with 95.6% accuracy, 97.1% sensitivity, 94.0% specification, 94.4% precision, 95.8% F-score, and 99.3% AUC-score outperforms state-of-the-art classification models. Finally, we identified the most significant rules that state various combinations of 6 features in certain ranges of their values lead to patients’ recovery with a confidence value of 90%. In conclusion, the classification results in this study show better performance than recent studies, and the extracted rules help physicians consider other important factors to improve health services and medical decision-making for different groups of COVID-19 patients.

Construction of Mitochondrial Protection and Monitoring Model of Lon Protease Based on Machine Learning under Myocardial Ischemia Environment.

The localization of a protein's submitochondrial structure is important for therapeutic design of associated disorders caused by mitochondrial abnormalities because many human diseases are directly tied to mitochondria. When Lon protease expression changes, glycolysis replaces respiratory metabolism in the cell, which is a common occurrence in cancer cells. The fact that protein formation is a dynamic research object makes it impossible to reproduce the unique living environment of proteins in an experimental setting, which surely makes it more challenging to determine protein function through experiments. This research suggests a model of Lon protease-based mitochondrial protection under myocardial ischemia based on ML (machine learning). To ensure the balance of all submitochondrial proteins, the data set is processed using a random oversampling method, each overlapping fixed-length subsequence that is created from the protein sequence functions as a channel in the convolution layer. The results demonstrate that applying the oversampling strategy increases the ROC value by 17.6%-21.3%. Our prediction method is successful as evidenced by the fact that ML prediction outperforms the predictions of other conventional classifiers.

Synthetic Minority Oversampling Technique Based on Adaptive Local Mean Vectors and Improved Differential Evolution

SMOTE is a classical oversampling method and aims to improve imbalanced classification by creating synthetic minority class samples. Overgeneralization is a great challenge in SMOTE and its improvements. Multiple variations of SMOTE are proposed against imbalances between classes and overgeneralization. However, they still have the following issues: a) most methods depend on too many parameters; b) most methods fail to detect suspicious noise effectively and modify them; c) interpolation of almost all methods is susceptible to abnormal samples. To overcome the above issues, a new synthetic minority oversampling technique based on adaptive local mean vectors and improved differential evolution (SMOTE-LMVDE) is proposed. First, a new noise detection technique based on the defined adaptive local mean vectors (NDALMV) is proposed to find suspicious noise. Second, an improved differential evolution is proposed to modify and improve detected suspicious noise. Finally, a new interpolation based on the defined adaptive local mean vectors is proposed to create synthetic minority class samples. Experiments prove that the proposed method superior to 7 popular oversampling approaches on extensive data sets in the training nearest neighbor classifier and the decision tree classifier.

A Deep Learning Approach for Viral DNA Sequence Classification using Genetic Algorithm

DNA sequence classification is one of the major challenges in biological data processing. The identification and classification of novel viral genome sequences drastically help in reducing the dangers of a viral outbreak like COVID-19. The more accurate the classification of these viruses, the faster a vaccine can be produced to counter them. Thus, more accurate methods should be utilized to classify the viral DNA. This research proposes a hybrid deep learning model for efficient viral DNA sequence classification. A genetic algorithm (GA) was utilized for weight optimization with Convolutional Neural Networks (CNN) architecture. Furthermore, Long Short-Term Memory (LSTM) as well as Bidirectional CNN-LSTM model architectures are employed. Encoding methods are needed to transform the DNA into numeric format for the proposed model. Three different encoding methods to represent DNA sequences as input to the proposed model were experimented: k-mer, label encoding, and one hot vector encoding. Furthermore, an efficient oversampling method was applied to overcome the imbalanced dataset issues. The performance of the proposed GA optimized CNN hybrid model using label encoding achieved the highest classification accuracy of 94.88% compared with other encoding methods.

Real-Time Pedestrian Conflict Prediction Model at the Signal Cycle Level Using Machine Learning Models

Compared with traditional traffic studies, real-time safety analyses can be better incorporated into proactive traffic management strategies to improve traffic safety. However, few studies have investigated the real-time pedestrian safety model. Intersections usually have mixed traffic conditions with more pedestrian-vehicle interactions. This paper uses conflict indicators, PET (Post Encroachment Time) and TTC (Time to Collision) to identify pedestrians’ conflicts from CCTV (closed-circuit television) videos. The high-resolution traffic data from the Automated Traffic Signal Performance Measures (ATSPM) system are used to derive traffic flow-related variables. The pedestrian exposure is also estimated. Pedestrians’ conflicts are predicted using multiple machine learning models and Logistic Regression. The resampling methods, random over-sampling, and random under-sampling are compared. The best model, Extreme Gradient Boosting (XGBT) with random over-sampling method can achieve AUC (area under the ROC curve) value of 0.841 and recall value of 0.739 on the test data set. The proposed model can predict pedestrians’ conflicts one cycle ahead, which can be 2–3 min. The proposed model has the potential to be implemented in the Connected and Automated Vehicles (CAV) environment to adjust signal timing accordingly and enhance traffic safety.

Improving Detection of False Data Injection Attacks Using Machine Learning with Feature Selection and Oversampling

Critical infrastructures have recently been integrated with digital controls to support intelligent decision making. Although this integration provides various benefits and improvements, it also exposes the system to new cyberattacks. In particular, the injection of false data and commands into communication is one of the most common and fatal cyberattacks in critical infrastructures. Hence, in this paper, we investigate the effectiveness of machine-learning algorithms in detecting False Data Injection Attacks (FDIAs). In particular, we focus on two of the most widely used critical infrastructures, namely power systems and water treatment plants. This study focuses on tackling two key technical issues: (1) finding the set of best features under a different combination of techniques and (2) resolving the class imbalance problem using oversampling methods. We evaluate the performance of each algorithm in terms of time complexity and detection accuracy to meet the time-critical requirements of critical infrastructures. Moreover, we address the inherent skewed distribution problem and the data imbalance problem commonly found in many critical infrastructure datasets. Our results show that the considered minority oversampling techniques can improve the Area Under Curve (AUC) of GradientBoosting, AdaBoost, and kNN by 10–12%.

An Improved Deep Learning Model: S-TextBLCNN for Traditional Chinese Medicine Formula Classification.

Purpose: This study proposes an S-TextBLCNN model for the efficacy of traditional Chinese medicine (TCM) formula classification. This model uses deep learning to analyze the relationship between herb efficacy and formula efficacy, which is helpful in further exploring the internal rules of formula combination. Methods: First, for the TCM herbs extracted from Chinese Pharmacopoeia, natural language processing (NLP) is used to learn and realize the quantitative expression of different TCM herbs. Three features of herb name, herb properties, and herb efficacy are selected to encode herbs and to construct formula-vector and herb-vector. Then, based on 2,664 formulae for stroke collected in TCM literature and 19 formula efficacy categories extracted from Yifang Jijie, an improved deep learning model TextBLCNN consists of a bidirectional long short-term memory (Bi-LSTM) neural network and a convolutional neural network (CNN) is proposed. Based on 19 formula efficacy categories, binary classifiers are established to classify the TCM formulae. Finally, aiming at the imbalance problem of formula data, the over-sampling method SMOTE is used to solve it and the S-TextBLCNN model is proposed. Results: The formula-vector composed of herb efficacy has the best effect on the classification model, so it can be inferred that there is a strong relationship between herb efficacy and formula efficacy. The TextBLCNN model has an accuracy of 0.858 and an F1-score of 0.762, both higher than the logistic regression (acc = 0.561, F1-score = 0.567), SVM (acc = 0.703, F1-score = 0.591), LSTM (acc = 0.723, F1-score = 0.621), and TextCNN (acc = 0.745, F1-score = 0.644) models. In addition, the over-sampling method SMOTE is used in our model to tackle data imbalance, and the F1-score is greatly improved by an average of 47.1% in 19 models. Conclusion: The combination of formula feature representation and the S-TextBLCNN model improve the accuracy in formula efficacy classification. It provides a new research idea for the study of TCM formula compatibility.

Machine Learning-Based Predictive Models of Behavioral and Psychological Symptoms of Dementia

Models predicting the occurrence of specific types of behavioral and psychological symptoms of dementia (BPSD) can be highly beneficial for its early intervention and individualized care planning. Using a machine learning approach, this study developed and validated predictive models of the occurrence of BPSD, categorized into seven subsyndromes, among community-dwelling older adults with dementia in South Korea. BPSD dairy was used to measure BPSD and the state of unmet needs daily. We measured sleep and activity levels using actigraphy, and stress and fatigue using a portable heart rate variability analyzer. We developed predictive models and conducted cross-validation using training data that consisted of the first two wave dataset, and then validated the models using wave 3 test data. To deal with imbalanced datasets, we used Synthetic Minority Oversampling Technique (SMOTE), an over-sampling method. Categorical variables were pre-processed using target encoding. We then compared the machine-learning models with logistic regression. The area under the receiver operating characteristic curve (AUC) scores of the support vector machine (SVM) models for the wave 3 test data showed a similar or greater value than logistic regression models across all BPSD subsyndromes. The SVM model (AUC = 0.899) had an AUC value greater than that of the logistic regression model (AUC = 0.717), particularly for hyperactivity symptoms. Machine learning algorithms, especially SVM models, can be used to develop BPSD prediction models to help identify at-risk individuals and implement symptom-targeted individualized interventions.