A new machine learning technique for predicting river water quality using AVOA-RNN

In supervised machine learning, with an imbalanced dataset, achieving better classification in minority classes is a major challenge. In such situation, machine learning model shows biasness toward majority classes, which result into poor performance in another set of classes. This paper examined how Synthetic Minority Over-Sampling Technique (SMOTE) techniques help in multinomial text classification on the imbalanced dataset. The performance of SMOTE was examined with Naive Bayes (NB) and Extreme Gradient Boosting (XGBoost) algorithms. 701 questions were collected from college students residing in Mumbai, related to their lifestyle problems. Results showed that XGBoost with SMOTE (XGBoost + SMOTE) technique worked better on an imbalanced dataset in comparison with NB with SMOTE (NB + SMOTE), NB without SMOTE (NB-SMOTE), and XGBoost without SMOTE (XGBoost-SMOTE) techniques. The average classification accuracy for Naive Bayes (with and without SMOTE) was 68.0% while the average accuracy for XGBoost was 71.0%. In the selection of XGBoost and NB, researcher can opt for XGBoost with SMOTE technique to work on the multinomial imbalanced dataset.KeywordsXGBoostImbalanced dataSMOTENaïve BayesTfidfVectorizer

In the semiconductor manufacturing industry, Automatic Defect Classification (ADC) plays an important role in maintaining high wafer inspection quality and reducing yield loss. ADC performance has benefitted from using machine learning (ML) algorithms; however, performance is negatively affected by the data imbalance and limited amounts of training data. Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique to adjust the skewed class distribution of a dataset so that the bias of the majority class is reduced. This paper shows that applying SMOTE achieved higher accuracy and purity on two imbalanced datasets, consisting of scanning electron microscopy (SEM) images collected with ASML-HMI eP™ and eScan® series inspection tools. The ML models are also less sensitive to the selection of hyperparameters when SMOTE is applied. We also show that better classification results can be obtained with less training samples with SMOTE; we conducted an experiment where a ML model trained on only 25% of samples with SMOTE achieved a higher ADC accuracy and purity performance compared to the same ML model trained on all samples but without SMOTE. In another experiment using a highly imbalanced SEM dataset with very few counts of the defect-of- interest (DOI), the combination of SMOTE and random undersampling of the majority class improves the accuracy by up to 5x while maintaining the same level of purity.

An accurate and trustworthy prediction model is essential for supporting policy decisions in environmental management concerning water quality prediction. Nonetheless, imbalanced datasets are prevalent in this discipline and hinder identifying crucial ecological factors accurately. This study proposed a novel SMOTE-PCADBSCAN model to enhance the categorisation of water quality data by employing three key components: (i) synthetic minority over-sampling technique (SMOTE), (ii) principal component analysis (PCA), and (iii) density-based spatial clustering of applications with noise (DBSCAN). The minority class was initially augmented using SMOTE, which PCA then decreased the dimensionality. Subsequently, DBSCAN was utilised to generate superior-quality synthetic data by detecting and eliminating extraneous data points. A Malaysia-based multi-class water quality dataset was employed to determine the efficiency of this model. Four different versions of the dataset (Original, SMOTE, SMOTE-DBSCAN, and SMOTE-PCADBSCAN) also utilised five classifier types for the analysis process: (i) decision tree, (ii) random forest, (iii) gradient boosting method, (iv) adaptive boosting, and (v) extreme gradient boosting. Although the original datasets exhibited great accuracy, class imbalance occurred when detecting minority classes. Among the datasets, the metric performances of SMOTE-DBSCAN and SMOTE-PCADBSCAN-based synthetic datasets were superior. The highest accuracy and optimal F1 scores were also demonstrated by RF using the SMOTE-PCADBSCAN approach, which presented excellent water quality classification and imbalanced data management. Consequently, the classification accuracy of imbalanced environmental datasets could be enhanced by employing advanced oversampling techniques and ensemble approaches.

The classification of imbalanced datasets is a prominent task in text mining and machine learning. The number of samples in each class is not uniformly distributed; one class contains a large number of samples while the other has a small number. Overfitting of the model occurs as a result of imbalanced datasets, resulting in poor performance. In this study, we compare different oversampling techniques like synthetic minority oversampling technique (SMOTE), support vector machine SMOTE (SVM-SMOTE), Border-line SMOTE, K-means SMOTE, and adaptive synthetic (ADASYN) oversampling to address the issue of imbalanced datasets and enhance the performance of machine learning models. Preprocessing significantly enhances the quality of input data by reducing noise, redundant data, and unnecessary data. This enables the machines to identify crucial patterns that facilitate the extraction of significant and pertinent information from the preprocessed data. This study preprocesses the data using various top-level preprocessing steps. Furthermore, two imbalanced Twitter datasets are used to compare the performance of oversampling techniques with six machine learning models including random forest (RF), SVM, K-nearest neighbor (KNN), AdaBoost (ADA), logistic regression (LR), and decision tree (DT). In addition, the bag of words (BoW) and term frequency and inverse document frequency (TF-IDF) features extraction approaches are used to extract features from the tweets. The experiments indicate that SMOTE and ADASYN perform much better than other techniques thus providing higher accuracy. Additionally, overall results show that SVM with ’linear’ kernel tends to attain the highest accuracy and recall score of 99.67% and 1.00% on ADASYN oversampled datasets and 99.57% accuracy on SMOTE oversampled dataset with TF-IDF features. The SVM model using 10-fold cross-validation experiments achieved 97.40 mean accuracy with a 0.008 standard deviation. Our approach achieved 2.62% greater accuracy as compared to other current methods.

Classification of imbalanced travel mode choice dataset with SMOTE and prediction using interpretable machine learning

Reservoir evaluation method based on explainable machine learning with small samples

Machine learning (ML) has emerged as a powerful tool in transformer condition assessment, enabling more accurate diagnostics by leveraging historical test data. However, imbalanced datasets, often characterized by limited samples in poor transformer conditions, pose significant challenges to model performance. This study investigates the application of oversampling techniques to enhance ML model accuracy in predicting the Health Index of transformers. A dataset comprising 3850 transformer tests collected from utilities across Indonesia was used. Key parameters, including oil quality, dissolved gas analysis, and paper condition factors, were employed as inputs for ML modeling. To address the class imbalance, various oversampling methods, such as the Synthetic Minority Oversampling Technique (SMOTE), Borderline-SMOTE, SMOTE-Tomek, and SMOTE-ENN, were implemented and compared. This study explores the impact of these techniques on model performance, focusing on classification accuracy, precision, recall, and F1-score. The results reveal that all SMOTE-based methods improved model performance, with SMOTE-ENN yielding the best outcomes. It significantly reduced classification errors, particularly for minority classes, ensuring better predictive reliability. These findings underscore the importance of advanced oversampling techniques in improving transformer diagnostics. By effectively addressing the challenges posed by imbalanced datasets, this research provides a robust framework for applying ML in transformer condition monitoring and other domains with similar data constraints.

The Synthetic Minority Oversampling TEchnique (SMOTE) is widely-used for the analysis of imbalanced datasets. It is known that SMOTE frequently over-generalizes the minority class, leading to misclassifications for the majority class, and effecting the overall balance of the model. In this article, we present an approach that overcomes this limitation of SMOTE, employing Localized Random Affine Shadowsampling (LoRAS) to oversample from an approximated data manifold of the minority class. We benchmarked our algorithm with 14 publicly available imbalanced datasets using three different Machine Learning (ML) algorithms and compared the performance of LoRAS, SMOTE and several SMOTE extensions that share the concept of using convex combinations of minority class data points for oversampling with LoRAS. We observed that LoRAS, on average generates better ML models in terms of F1-Score and Balanced accuracy. Another key observation is that while most of the extensions of SMOTE we have tested, improve the F1-Score with respect to SMOTE on an average, they compromise on the Balanced accuracy of a classification model. LoRAS on the contrary, improves both F1 Score and the Balanced accuracy thus produces better classification models. Moreover, to explain the success of the algorithm, we have constructed a mathematical framework to prove that LoRAS oversampling technique provides a better estimate for the mean of the underlying local data distribution of the minority class data space.

The Impact of Augmentation and SMOTE Implementation on the Classification Models Performance: A Case Study on Student Academic Performance Dataset

This study investigates the use of machine learning (ML) models combined with a Synthetic Minority Over-sampling Technique (SMOTE) and its variants to predict perioperative pressure injuries (PIs) in an imbalanced dataset. PIs are a significant healthcare problem, often leading to prolonged hospitalization and increased medical costs. Conventional risk assessment scales are limited in their ability to predict PIs accurately, prompting the exploration of ML techniques to address this challenge.We utilized data from 7,292 patients admitted to a tertiary care hospital in Shanghai between May 2017 and July 2023, with a final dataset of 2,972 patients, including 158 with PIs. Seven ML algorithms-Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Extra Trees (ET), K-Nearest Neighbors (KNN), and Decision Trees (DT)-were used in conjunction with SMOTE, SMOTE+ENN, Borderline-SMOTE, ADASYN, and GAN to balance the dataset and improve model performance.Results revealed significant improvements in model performance when SMOTE and its variants were used. For instance, the XGBoost model hadan AUC of 0.996 with SMOTE, compared to 0.800 on raw data. SMOTE+ENN and Borderline-SMOTE further enhanced the models' ability to identify minority classes. External validation indicatedthat XGBoost, RF, and ET exhibited the highest stability and accuracy, with XGBoost having an AUC of 0.977. SHAP analysis revealed that factors such as anesthesia grade, age, and serum albumin levels significantly influenced model predictions.In conclusion, integrating SMOTE with ML algorithms effectively addressed a data imbalance and improved the prediction of perioperative PIs. Future work should focus on refining SMOTE techniques and exploring their application to larger, multi-center datasets to enhance the generalizability of these findings, and especially for diseaseswith a lowincidence.

The accurate classification of diabetes remains a significant challenge in medical diagnostics, particularly when dealing with imbalanced datasets. This study examines the impact of the Synthetic Minority Over-sampling Technique (SMOTE) on the performance of three machine learning algorithms: Support Vector Machine (SVM), Logsistic Regression, and Naïve Bayes. Initially, the models were evaluated on the original imbalanced dataset, with AUC values of 0.598 for SVM, 0.622 for Naïve Bayes, and 0.613 for Logistic Regression, reflecting the challenges posed by class imbalance. To address this issue, SMOTE was applied to balance the dataset by oversampling the minority class. After applying SMOTE, the models showed significant improvements in performance, with AUC values increasing to 0.991 for SVM, 0.987 for Logistic Regression, and 0.986 for Naïve Bayes. These results demonstrate SMOTE's effectiveness in enhancing model performance by mitigating the impact of imbalanced data, especially in the context of diabetes classification. Although the improvements in AUC were notable, it’s important to recognize the influence of data balancing on the models' behavior. These findings emphasize the need for careful handling of imbalanced datasets when developing machine learning models for medical diagnosis. This study provides valuable insights into the role of oversampling techniques like SMOTE in improving predictive accuracy for diabetes, a critical step toward more reliable medical diagnostics.

With the help of machine learning (ML) techniques, the possible errors made by the pathologists and physicians, such as those caused by inexperience, fatigue, stress and so on can be avoided, and the medical data can be examined in a shorter time and in a more detailed manner. However, while the conventional ML techniques, such as classification, achieved excellent performance in classification accuracy when applied in medical diagnoses, they have a fatal shortcoming of poor performance since the imbalanced dataset, especially for the detection of the minority category. To tackle the shortcomings of conventional classification approaches, this study proposes a novel ensemble learning paradigm for medical diagnosis with imbalanced data, which consists of three phases: data pre-processing, training base classifier and final ensemble. In the first data pre-processing phase, we introduce the extension of Synthetic Minority Oversampling Technique (SMOTE) by integrating it with cross-validated committees filter (CVCF) technique, which can not only synthesize the minority sample and thereby balance the input instances, but also filter the noisy examples so as to perform well in the process of classification. In the classification phase, we introduce ensemble support vector machine (ESVM) classification technique, which were constructed by multiple diversity structures of SVM classifiers and thus has the advantages of strong generalization performance and classification precision. Additionally, in the last phase of the final ensemble strategy, we introduce the weighted majority voting strategy and introduce simulated annealing genetic algorithm (SAGA) to optimize the weight vector and thereby enhance the overall classification performance. The efficiency of our proposed ensemble learning method was tested on nine imbalanced medical datasets and the experimental results clearly indicate that the proposed ensemble learning paradigm outperforms other state-of-the-art classification models. Promisingly, our proposed ensemble learning paradigm can effectively facilitate medical decision making for physicians.

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

Gender-based violence can include sexual, physical, mental, and economic harm inflicted in public or in private. This violence also has a direct psychological effect, physical and financial consequences, and it has multiple underlying reasons, such as social, economic, cultural, political, and religious aspects. By applying multiple resampling techniques, this study aims to improve the precision and accuracy of supervised machine learning classifications of gender-based violence (GBV) using the SDHS dataset. The class imbalance between GBV-positive and GBV-negative instances makes it very challenging to produce reliable classification machine learning models. To address this issue, oversampling machine learning approaches, including synthetic minority over-sampling technique (SMOTE), adaptive synthetic (ADASYN), and random over-sampling (ROS), were employed to classify the GBV data in Somalia. The logistic regression (LR), decision tree (CART), random forest (RF), naïve Bayes (NB), k-nearest Neighbors (KNN), and support vector machine (SVM) methods were trained and evaluated. In addition, oversampling techniques were employed for improving the imbalanced datasets. Receiver operating characteristic curve (ROC) and the area under the curve (AUC) were used to assess each machine learning classifier and to compare performance on the original GBV dataset. Among the resampling techniques, SMOTE (RF = 0.992, CART = 0.969, and KNN = 0.957) outperformed ADASYN (RF = 0.912, CART = 0.910, and KNN = 0.876) and ROS (RF = 0.920, CART = 0.919, and KNN = 0.880) across almost all evaluation metrics. The classifiers that performed the best were random forest (RF) and classification and regression trees (CART), then k-nearest Neighbors. After resampling the imbalanced dataset, we may therefore conclude that the random forest (AUC = 0.972), CART (AUC = 0.969) and KNN (AUC = 0.957) machine learning classifiers are better at accurately classifying the k-nearest Neighbors dataset. In addition, compared to the other oversampling techniques, SMOTE was used to the machine learning classifiers to balance the imbalanced class distributions in favour of the minority class. In addition, SMOTE with the Mathews correlation coefficient (MCC) outperformed ADASYN and ROS resampling techniques. The MCC values for SMOTE reached their highest values (RF = 0.86, CART = 0.85, and KNN = 0.80), indicating strong overall predictive reliability of the machine learning models. Therefore, the findings of this analysis will assist government and non-government organizations in making policy decisions to GBV risks.

Surface water quality is of utmost significance to ensure public health and facilitate sustainable economic development. Traditional water quality assessment methods are typically time-consuming and labor-intensive and require numerous field measurements and laboratory analyses, which are costly and impractical to implement in large-scale water quality monitoring. Recent advances in machine learning (ML) have brought new approaches to predicting water quality index (WQI) and classifying water quality in real time to enhance decision-making in environmental management. In this study, we propose a novel gated liquid neural network (gated-LNN) that can predict WQI and classify water quality with high accuracy. As opposed to typical ML models, the proposed gated-LNN includes a gating mechanism that enhances temporal learning and noise robustness, making it well-suited for dynamic environmental data. For ascertaining the effectiveness of the proposed approach, we conducted rigorous experiments on a publicly available water quality dataset with 1897 examples collected from varied water bodies of India between the years 2005 and 2014. The dataset comprises seven most significant parameters of water quality, i.e., dissolved oxygen, pH, conductivity, biological oxygen demand, nitrate, fecal coliform, and total coliform. The proposed gated-LNN model achieved a high R² of 0.9995 for WQI prediction and 99.74% accuracy for three-class water quality classification into "Good," "Poor," and "Unsuitable" classes, outperforming state-of-the-art models in both regression and classification tasks. While these results highlight the model’s potential as a highly accurate and efficient tool for real-time water quality assessment, its generalizability to different regions remains an important consideration. Future work will focus on enhancing computational efficiency and conducting generalization tests on datasets from diverse geographic regions and time periods to evaluate adaptability.