SMOTE Oversampling Research Articles

Abstract 16388: Feature Selection Using Recursive Feature Elimination (RFE): Machine Learning Application in Predicting Permanent Pacemaker Implantation (PPMI) in Patients Undergoing Transcatheter Aortic Valve Replacement (TAVR)

Introduction: TAVR is increasingly being performed because of several recent large-scale clinical trials supporting its use across a range of patient sub-classes. Although it provides life-altering relief of severe aortic stenosis, adverse outcomes are not uncommon including paravalvular leak, life-threatening bleeding, acute kidney injury, stroke, and PPMI. RFE can help certain classification models like logistic regression in better predicting binary variables such as PPMI by creating a model based upon a set of predictors and then progressively eliminating variables to optimize the model. Objective: To determine whether RFE when applied to logistic regression would result in discriminatory ability in the prediction of PPM implantation in patients undergoing TAVR Methods: Pre- and postoperative data from a single institution were collected for all patients undergoing TAVR without a history of PPMI between January 2016 and December 2019. EKG data obtained included QRS duration, presence of atrioventricular block, left anterior and posterior fasicular block and right bundle branch block(RBBB). Data was imported into Python and a stratified 5 fold cross validation with SMOTE oversampling was run with RFE running at every fold to avoid overfitting. Upon completion, a rank score was tabulated for each predictor and a final logistic regression model with the highest optimized receiver under the operator curve was exported and applied to a test set. The receiver under the operator score was calculated for the training and test sets and the variables of importance were identified using RFE. Results: The total sample size for this cohort was 513 patients, with a PPMI incidence of 8.58%. The training set consisted of 40 variables and 384 patients, and the test set had 129 patients. The final optimized model on the training set had an ROC of 0.75 and utilized three features out of forty. The three features identified by RFE were: implanted TAVR valve size, QRS duration, and presence of RBBB. The model had a ROC of 0.63 on the test set. Conclusions: Our results show that presence of pre-op RBBB, prolonged QRS duration and valve size were risk factors for PPMI. Logistic regression classification had modest ability in predicting the need for PPMI after TAVR.

Comparing information extraction techniques for low-prevalence concepts: The case of insulin rejection by patients.

To comparatively evaluate a range of Natural Language Processing (NLP) approaches for Information Extraction (IE) of low-prevalence concepts in clinical notes on the example of decline of insulin therapy recommendation by patients. We evaluated the accuracy of detection of documentation of decline of insulin therapy by patients using sentence-level naïve Bayes, logistic regression and support vector machine (SVM)-based classification (with and without SMOTE oversampling), token-level sequence labelling using conditional random fields (CRFs), uni- and bi-directional recurrent neural network (RNN) models with GRU and LSTM cells, and rule-based detection using Canary platform. All models were trained using the same manually annotated 50,046-document training set and evaluated on the same 1501-document held-out set. Hyperparameter optimization was performed using 10-fold cross-validation. At the sentence level, prevalence of documentation of decline of insulin therapy by patients was 0.02% in both training and held-out sets. Naïve Bayes and logistic regression models did not achieve F1 score ≥ 0.5 on the training set and were not further evaluated. Among the other models, evaluation against the held-out test set showed that SVM identified decline of insulin therapy by patients with F1 score of 0.61, CRF with F1 of 0.51, RNN with F1 of 0.67 and Canary rule-based model with F1 of 0.97. Identification of low-prevalence concepts can present challenges in medical language processing. Rule-based systems that include the designer's background knowledge of language may be able to achieve higher accuracy under these circumstances.

Cross-project defect prediction using data sampling for class imbalance learning: an empirical study

The presence of defect data related to different projects leads to cross-project defect prediction an open issue in the field of research in software engineering. In cross-project defect prediction, the source and the target projects are different. The prediction model is trained by using the data sources of the different projects and then it is tested on the target data source. The data source from the varying projects leads to a highly imbalanced source dataset. The performance of the predictive model degrades due to this imbalance nature of the dataset. This is termed as the class imbalance problem in machine learning. This paper conducts an empirical analysis in a bi-fold manner. It evaluates whether data sampling techniques can handle the class imbalance problem and improve the performance of the predictive model for cross-project defect prediction (CPDP). Secondly, it also evaluates whether the results of CPDP after data sampling are comparable to within project defect prediction (WPDP). Ensemble learning classifiers are used as the predictive model over 12 publically available object-oriented project datasets. The experimental results infer that SMOTE oversampling can be applied to overcome the problem of class imbalance on CPDP. It also gives comparable results to WPDP with statistical significance.

Churn prediction via support vector classification: An empirical comparison

An empirical framework for customer churn prediction modeling is presented in this work. This task represents a very interesting business analytics challenge, given its highly class imbalanced nature, and the presence of noisy variables that adversely affect the prediction capabilities of classification models. In this work, two SVM-based techniques are compared: Support Vector Data Description (SVDD), and standard two-class SVMs. The proposed methodology involves the comparison of these two methods under different conditions of class imbalance and using different subsets of variables. Feature ranking is performed via the Fisher Score Criterion, while the class imbalance problem is dealt with through resampling techniques, namely random undersampling and SMOTE oversampling. Experiments on four customer churn prediction datasets show the advantages of SVDD: it outperforms standard SVM in terms of predictive performance, demonstrating the importance of techniques that take the class imbalance problem into account.

TU-AB-BRA-01: Classification of Prostate Cancer Gleason Scores Through Machine Learning From Multiparametric MRI

Purpose: To develop a machine learning-based automatic classification of Gleason Score (GS) 3+4=7 versus 4+3=7 prostate cancers (PCa) from multiparametric MRI (mpMRI) combined with first and second order texture features. Methods: 140 PCa were delineated in the peripheral and transition zones from patients undergoing mpMRI 6 months pre-prostatectomy. Lesions were contoured on T2-weighted MRI and apparent diffusion coefficient (ADC) maps by correlating the images with step-section pathology maps of the excised prostates. 114 tumors had a GS (3+4) and 26 had a GS (4+3). First order (mean, standard deviation, skewness, kurtosis) and second order texture features (energy, entropy, correlation, homogeneity, contrast) were analyzed. Due to large tumor class imbalance, we employed two different sample augmentation techniques: Gibbs and SMOTE oversampling. The samples in each class were oversampled to 200 samples. We evaluated the performance of three different classifiers, t-test support vector machine (t-test SVM), recursive feature elimination SVM (RFE-SVM), and adaptive boosting (AdaBoost), using 10-fold crossvalidation. Results: RFE-SVM achieved the best classification performance (accuracy: 97%; Youden Index: 0.93) when trained using samples generated by the SMOTE method, followed by t-test SVM (78%,0.56) and AdaBoost (73%,0.46). When trained using samples generated from Gibbs oversampling, the performance of the classifiers were: RFE-SVM (90%,0.80), t-test SVM(71%,0.41), and AdaBoost (76%,0.52). In comparison, the classification performance of the same methods without sample augmentation was RFE-SVM (83%,0.11), t-test SVM(81%,0.00), and AdaBoost(79%,0.41). Classification with only mean ADC and mean T2 resulted at best in an accuracy of (67%,0.34) using Gibbs sampling. Conclusion: We developed a fully automated method for classifying GS of PCa from multi-parametric images using texture features combined with sample augmentation. Our method enables to boost classifier performance despite highly unbalanced datasets. Furthermore, our results show that texture features enable the classifiers to achieve a much higher accuracy in comparison to not using any of them.