Leveraging Modern Machine Learning to Improve Early Warning Systems and Reduce Chronic Absenteeism in Early Childhood

Musculoskeletal disorders (MSDs) are a major health concern for children. Traditional assessment methods, which are based on subjective assessments, may be inaccurate. The main objective of this research is to evaluate Synthetic Minority Over-sampling Technique(SMOTE)-based machine learning algorithms for predicting MSDs in elementary school students with an unbalanced dataset. This study is the first to use these algorithms to increase the accuracy of MSD prediction in this age group. This cross-sectional study was conducted in 2024 on 438 primary school students (boys and girls, grades 1 to 6) in Hamedan, Iran. Random sampling was performed from 12 public and private schools. The dependent variable was the presence or absence of MSD, assessed using the Cornell questionnaire. Given the imbalanced nature of the data, SMOTE-based techniques were applied. Finally, the performance of six machine learning algorithms, including Random Forest (RF), Naive Bayes (NB), Artificial Neural Network (ANN), Decision Tree (DT), Extreme Gradient Boosting (XGBoost), and Support Vector Machine (SVM) was compared based on five SMOTE family techniques, including SVM-SMOTE, SMOTE-NC, ADASYN, Borderline SMOTE, and the SMOTE method, for predicting musculoskeletal disorders in primary school students. This study analyzed 438 students, with 28 (6.39%) having musculoskeletal disorders (MSD). Machine learning models were tested on both imbalanced and SMOTE-balanced data. Before SMOTE, sensitivity was low (e.g., DT: 18%), but after applying SMOTE, it improved significantly (DT: 85%). Area Under the Curve (AUC) for RF and XGBoost increased from 65 to 99%. Among SMOTE-based techniques, SMOTE-NC achieved the highest accuracy for XGBoost (93.65%), RF (93.41%), ANN (92.80%), SVM (90.24%), and DT (86.58%). Borderline-SMOTE yielded the highest accuracy (96%) for XGBoost and RF. RF and XGBoost performed best overall. Key predictors of MSD were regional facilities, BMI, and gender. The findings of this study demonstrate the effectiveness of SMOTE in addressing class imbalance and improving the accuracy of MSD prediction among students. Overall, RF and XGBoost outperformed other algorithms. Additionally, key factors influencing MSD included regional facilities, body mass index (BMI), and gender, highlighting the significance of environmental and physiological factors in the development of this disorder.

This study employs machine learning models to explore stock price prediction for Tenaga Nasional Berhad (TNB), Malaysia’s primary electricity provider. It addresses the limitations of previous studies by incorporating various input variables, including the stock market, technical, financial, and economic data. This study also tackles the issue of imbalanced class distribution due to small datasets of stock market data by generating synthetic data using Synthetic Minority Over-Sampling Technique (SMOTE) and Generative Adversarial Network-Synthetic Minority Over-Sampling Technique (GAN-SMOTE) techniques. The performance of four classifier models (random forest, Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and Artificial Neural Network (ANN)) is evaluated without any synthetic data and with synthetic data generated. The SMOTE-ANN model is the best-performing model, exhibiting superior accuracy of 93%, F1-Score of 92%, precision of 90%, recall of 94%, and specificity of 92%. Overall, this research provides valuable insights into TNB stock price movements, offers a solution for imbalanced class distribution, and identifies the top-performing model for predicting TNB stock price movement. These findings are relevant to investors, analysts, and organisations in the utility sector.

In supervised machine learning, with an imbalanced dataset, achieving better classification in minority classes is a major challenge. In such situation, machine learning model shows biasness toward majority classes, which result into poor performance in another set of classes. This paper examined how Synthetic Minority Over-Sampling Technique (SMOTE) techniques help in multinomial text classification on the imbalanced dataset. The performance of SMOTE was examined with Naive Bayes (NB) and Extreme Gradient Boosting (XGBoost) algorithms. 701 questions were collected from college students residing in Mumbai, related to their lifestyle problems. Results showed that XGBoost with SMOTE (XGBoost + SMOTE) technique worked better on an imbalanced dataset in comparison with NB with SMOTE (NB + SMOTE), NB without SMOTE (NB-SMOTE), and XGBoost without SMOTE (XGBoost-SMOTE) techniques. The average classification accuracy for Naive Bayes (with and without SMOTE) was 68.0% while the average accuracy for XGBoost was 71.0%. In the selection of XGBoost and NB, researcher can opt for XGBoost with SMOTE technique to work on the multinomial imbalanced dataset.KeywordsXGBoostImbalanced dataSMOTENaïve BayesTfidfVectorizer

This study investigates the use of machine learning (ML) models combined with a Synthetic Minority Over-sampling Technique (SMOTE) and its variants to predict perioperative pressure injuries (PIs) in an imbalanced dataset. PIs are a significant healthcare problem, often leading to prolonged hospitalization and increased medical costs. Conventional risk assessment scales are limited in their ability to predict PIs accurately, prompting the exploration of ML techniques to address this challenge.We utilized data from 7,292 patients admitted to a tertiary care hospital in Shanghai between May 2017 and July 2023, with a final dataset of 2,972 patients, including 158 with PIs. Seven ML algorithms-Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Extra Trees (ET), K-Nearest Neighbors (KNN), and Decision Trees (DT)-were used in conjunction with SMOTE, SMOTE+ENN, Borderline-SMOTE, ADASYN, and GAN to balance the dataset and improve model performance.Results revealed significant improvements in model performance when SMOTE and its variants were used. For instance, the XGBoost model hadan AUC of 0.996 with SMOTE, compared to 0.800 on raw data. SMOTE+ENN and Borderline-SMOTE further enhanced the models' ability to identify minority classes. External validation indicatedthat XGBoost, RF, and ET exhibited the highest stability and accuracy, with XGBoost having an AUC of 0.977. SHAP analysis revealed that factors such as anesthesia grade, age, and serum albumin levels significantly influenced model predictions.In conclusion, integrating SMOTE with ML algorithms effectively addressed a data imbalance and improved the prediction of perioperative PIs. Future work should focus on refining SMOTE techniques and exploring their application to larger, multi-center datasets to enhance the generalizability of these findings, and especially for diseaseswith a lowincidence.

Shale lithology identification is the basis of geological research and reservoir characterization, and is an essential task for oil exploration. Recently, several machine learning algorithms have been applied to improve the accuracy of lithology identification. However, stacking model for lithology identification has been less used in existing studies, and less consideration was given to imbalanced lithologies problem. In this study, we build a stacking model based on random forest (RF), extreme gradient boosting (XGBoost) and linear regression (LR), and use synthetic minority oversampling technique (SMOTE) to improve the imbalanced lithologies problem, and then compare the stacking model with support vector machine (SVM), RF and XGBoost models after adjusting model parameters using grid search and fivefold cross-validation. The authors prepared a dataset consisting of logging data and core data from 13 wells in a depression in Junggar basin, China, including a total of 2352 sample points marked with lithologic labels. The lithologies identified in this study are mudstone (MS), dolomitic mudstone (DM), siltstone (S), dolomitic siltstone (DS) and micritic dolomite (MD). The results show that (1) the overall identification performance of the stacking model is better than that of the SVM, RF and XGBoost models. (2) SMOTE algorithm can effectively improve the identification performance of the minority lithologies. (3) Density log is the most important factor in identifying lithologies. The stacking model combined with SMOTE proposed in this paper has high lithology identification performance, which renders it practicable for lithology identification.

Utilization of synthetic minority oversampling technique for improving potato yield prediction using remote sensing data and machine learning algorithms with small sample size of yield data

Effective spam detection is essential for data security, user experience, and organizational trust. However, outliers and class imbalance can impact machine learning models for spam classification. Previous studies focused on feature selection and ensemble learning but have not explicitly examined their combined effects. This study evaluates the performance of random forest (RF), gradient boosting (GB), and extreme gradient boosting (XGBoost) under four experimental scenarios: (i) without synthetic minority over-sampling technique (SMOTE) and outliers, (ii) without SMOTE but with outliers, (iii) with SMOTE and without outliers, and (iv) with SMOTE and with outliers. Results show that XGBoost achieves the highest accuracy (96%), an area under the curve-receiver operating characteristic (AUCROC) of 0.9928, and the fastest computation time (0.6184 seconds) under the SMOTE and outlier-free scenario. Additionally, RF attained an AUCROC of 0.9920, while GB achieved 0.9876 but required more processing time. These findings emphasize the need to address class imbalance and outliers in spam detection models. This study contributes to developing more robust spam filtering techniques and provides a benchmark for future improvements. By systematically evaluating these factors, it lays a foundation for designing more effective spam detection frameworks adaptable to real-world imbalanced and noisy data conditions.

Predicting credit default risk is important to financial institutions, as accurately predicting the likelihood of a borrower defaulting on their loans will help to reduce financial losses, thereby maintaining profitability and stability. Although machine learning models have been used in assessing large applications with complex attributes for these predictions, there is still a need to identify the most effective techniques for the model development process, including the technique to address the issue of data imbalance. In this research, we conducted a comparative analysis of random forest, decision tree, SVMs (Support Vector Machines), XGBoost (Extreme Gradient Boosting), ADABoost (Adaptive Boosting) and the multi-layered perceptron, to predict credit defaults using loan data from LendingClub. Additionally, XGBoost was used as a framework for testing and evaluating various techniques. Moreover, we applied this XGBoost framework to handle the issue of class imbalance observed, by testing various resampling methods such as Random Over-Sampling (ROS), the Synthetic Minority Over-Sampling Technique (SMOTE), Adaptive Synthetic Sampling (ADASYN), Random Under-Sampling (RUS), and hybrid approaches like the SMOTE with Tomek Links and the SMOTE with Edited Nearest Neighbours (SMOTE + ENNs). The results showed that balanced datasets significantly outperformed the imbalanced dataset, with the SMOTE + ENNs delivering the best overall performance, achieving an accuracy of 90.49%, a precision of 94.61% and a recall of 92.02%. Furthermore, ensemble methods such as voting and stacking were employed to enhance performance further. Our proposed model achieved an accuracy of 93.7%, a precision of 95.6% and a recall of 95.5%, which shows the potential of ensemble methods in improving credit default predictions and can provide lending platforms with the tool to reduce default rates and financial losses. In conclusion, the findings from this study have broader implications for financial institutions, offering a robust approach to risk assessment beyond the LendingClub dataset.

Classification of imbalanced travel mode choice dataset with SMOTE and prediction using interpretable machine learning

One of the prominent uses of Predictive Analytics is Health care for more accurate predictions based on proper analysis of cumulative datasets. Often times the datasets are quite imbalanced and sampling techniques like Synthetic Minority Oversampling Technique (SMOTE) give only moderate accuracy in such cases. To overcome this problem, a two-step approach has been proposed. In the first step, SMOTE is modified to reduce the class imbalance in terms of Distance-based SMOTE (D-SMOTE) and Bi-phasic SMOTE (BP-SMOTE) which were then coupled with selective classifiers for prediction. An increase in accuracy is noted for both BP-SMOTE and D-SMOTE compared to basic SMOTE. In the second step, Machine learning, Deep Learning and Ensemble algorithms were used to develop a Stacking Ensemble Framework which showed a significant increase in accuracy for Stacking compared to individual machine learning algorithms like Decision Tree, Naïve Bayes, Neural Networks and Ensemble techniques like Voting, Bagging and Boosting. Two different methods have been developed by combing Deep learning with Stacking approach namely Stacked CNN and Stacked RNN which yielded significantly higher accuracy of 96–97% compared to individual algorithms. Framingham dataset is used for data sampling, Wisconsin Hospital data of Breast Cancer study is used for Stacked CNN and Novel Coronavirus 2019 dataset relating to forecasting COVID-19 cases, is used for Stacked RNN.

Closely associated with metabolic disorders, non-alcoholic fatty liver disease (NAFLD) substantially increases the risk of hepatocellular carcinoma. This study aimed to apply machine learning (ML) algorithms to a community-based cohort in southern Taiwan to identify key risk factors for NAFLD and to develop predictive models with clinical applicability. Data were derived from community health examinations, and eighteen clinical and demographic features were analyzed. Five ML algorithms were evaluated: logistic regression (LR), random forest (RF), K-nearest neighbors (KNN), adaptive boosting (AdaBoost), and extreme gradient boosting (XGBoost). Model performance was assessed using accuracy, precision, recall, F1 score, and area under the receiver operating characteristic curve (AUROC). A total of 7,510 participants were included (38.8% male; mean age 50.9 ± 15.0 years). The dataset was randomly divided into training (80%) and testing (20%) subsets, with no significant differences observed between groups in most independent variables. The Synthetic Minority Over-sampling Technique (SMOTE) was employed to balance NAFLD and non-NAFLD groups in the training dataset. Among all models, XGBoost achieved the highest performance, with an accuracy of 83.48%, precision of 84.31%, recall of 81.21%, F1 score of 82.72%, and AUROC of 92.85%. Feature importance analysis identified low-density lipoprotein cholesterol (LDL-C), body mass index (BMI), waist circumference, fasting plasma glucose (FPG), and triglycerides (TG) as the most influential predictors of NAFLD. ML algorithms, particularly XGBoost, demonstrated high accuracy in predicting NAFLD and effectively identified key clinical predictors. These findings may enhance early diagnosis and facilitate the development of targeted intervention strategies in the management of NAFLD.

In the semiconductor manufacturing industry, Automatic Defect Classification (ADC) plays an important role in maintaining high wafer inspection quality and reducing yield loss. ADC performance has benefitted from using machine learning (ML) algorithms; however, performance is negatively affected by the data imbalance and limited amounts of training data. Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique to adjust the skewed class distribution of a dataset so that the bias of the majority class is reduced. This paper shows that applying SMOTE achieved higher accuracy and purity on two imbalanced datasets, consisting of scanning electron microscopy (SEM) images collected with ASML-HMI eP™ and eScan® series inspection tools. The ML models are also less sensitive to the selection of hyperparameters when SMOTE is applied. We also show that better classification results can be obtained with less training samples with SMOTE; we conducted an experiment where a ML model trained on only 25% of samples with SMOTE achieved a higher ADC accuracy and purity performance compared to the same ML model trained on all samples but without SMOTE. In another experiment using a highly imbalanced SEM dataset with very few counts of the defect-of- interest (DOI), the combination of SMOTE and random undersampling of the majority class improves the accuracy by up to 5x while maintaining the same level of purity.

In this study, we leveraged machine learning (ML) approach to develop and validate new assessment tools for predicting stroke and bleeding among patients with atrial fibrillation (AFib) and cancer. We conducted a retrospective cohort study including patients who were newly diagnosed with AFib with a record of cancer from the 2012–2018 Surveillance, Epidemiology, and End Results (SEER)-Medicare database. The ML algorithms were developed and validated separately for each outcome by fitting elastic net, random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and neural network models with tenfold cross-validation (train:test = 7:3). We obtained area under the curve (AUC), sensitivity, specificity, and F2 score as performance metrics. Model calibration was assessed using Brier score. In sensitivity analysis, we resampled data using Synthetic Minority Oversampling Technique (SMOTE). Among 18,388 patients with AFib and cancer, 523 (2.84%) had ischemic stroke and 221 (1.20%) had major bleeding within one year after AFib diagnosis. In prediction of ischemic stroke, RF significantly outperformed other ML models [AUC (0.916, 95% CI 0.887–0.945), sensitivity 0.868, specificity 0.801, F2 score 0.375, Brier score = 0.035]. However, the performance of ML algorithms in prediction of major bleeding was low with highest AUC achieved by RF (0.623, 95% CI 0.554–0.692). RF models performed better than CHA2DS2-VASc and HAS-BLED scores. SMOTE did not improve the performance of the ML algorithms. Our study demonstrated a promising application of ML in stroke prediction among patients with AFib and cancer. This tool may be leveraged in assisting clinicians to identify patients at high risk of stroke and optimize treatment decisions.

Diabetes represents a persistent and financially demanding challenge that increases the susceptibility of individuals to develop depression. Consequently, effective management of the co-occurrence of diabetes and depression is likely to significantly improve the quality of patient care. This research study presents a comprehensive comparative analysis of eight distinct machine learning (ML) algorithms to predict depression among individuals with diabetes. The algorithms evaluated include logistic regression (LR), k-nearest neighbors (KNN), decision tree (DT), random forest (RF), Adaptive Boosting (AdaBoost), support vector machine (SVM), Extreme Gradient Boosting (XGBoost), and Categorical Boosting (CatBoost). The study utilizes a dataset from Morocco, specifically curated for this purpose, and employs the Synthetic Minority Over-sampling Technique (SMOTE) to balance the dataset. Feature selection is performed using ExtraTreesClassifier, while hyperparameter tuning is accomplished through a grid search approach. The obtained results showcase promising performance of the ML algorithms in predicting depression among individuals with diabetes. Notably, the RF and CatBoost classifiers emerge as top performers, achieving an impressive accuracy rate of 82%. These findings hold significant implications for further research, aiming to refine prediction models in this context. This study underscores the considerable potential of ML algorithms in effectively predicting depression disorders in individuals living with diabetes. The remarkable accuracy demonstrated by the RF and CatBoost classifiers highlights their potential viability as reliable tools, particularly in clinical settings. Ongoing investigations are essential to validate and extend these results, exploring the real-world applicability of these models. Ultimately, this research contributes to advancing the management and overall well-being of individuals with diabetes and coexisting depressive disorders.

A stroke is a sudden disruption in the blood supply to the brain, affecting one or more blood vessels that nourish the brain. This results in a disturbance or deficiency in the brain’s oxygen supply, causing damage or impairment to brain cells. In some cases, determining the timing and severity of a stroke can be challenging. This study proposes an EMS (Elastic Net – MLP – SMOTE) model built on artificial intelligence, specifically utilizing two machine learning algorithms, Elastic Net and multilayer perceptron (MLP) by using Synthetic Minority Over-sampling Technique (SMOTE). The Elastic Net algorithm was employed for feature selection to identify crucial features, followed by prediction using the MLP algorithm. The Elastic Net algorithm was used due to its incorporation of both L2 and L1 regularization, providing good results in discerning influential features in model performance. The MLP algorithm was employed for its reliance on deep learning techniques, which yield promising results in such cases. This algorithm classified data from a comprehensive dataset containing essential features related to stroke. SMOTE is used to increase the performance of the model. Notably, no previous research study has integrated these three techniques together (Elastic Net – MLP – SMOTE). EMS achieved a prediction accuracy of 95% and MSE = 0.05. This model facilitates predicting the occurrence of stroke by relying on the patient’s historical data, mitigating the sudden onset of this serious disease. Received: 15 January 2024 | Revised: 22 February 2024 | Accepted: 28 March 2024 Conflicts of Interest The author declares that he has no conflicts of interest to this work. Data Availability Statement The data that support this work are available upon reasonable request to the corresponding author.