Comparative analysis of machine learning approaches in Kazakh banknote classification

A chemistry-informed hybrid machine learning approach to predict metal adsorption onto mineral surfaces

Vortex core detection remains an unsolved problem in the field of experimental and computational fluid dynamics. Available methods such as the Q, delta, and swirling strength criterion are based on a decomposed velocity gradient tensor but detect spurious vortices (false positives and false negatives), making these methods less robust. To overcome this, we propose a new hybrid machine learning approach in which we use a convolutional neural network to detect vortex regions within surface streamline plots and an additional deep neural network to detect vortex cores within identified vortex regions. Furthermore, we propose an automatic labeling approach based on K-means clustering to preprocess our input images. We show results for two classical test cases in fluid mechanics: the Taylor–Green vortex problem and two rotating blades. We show that our hybrid approach is up to 2.6 times faster than a pure deep neural network-based approach and furthermore show that our automatic K-means clustering labeling approach achieves within 0.45% mean square error of the more labour-intensive, manual labeling approach. At the same time, by using a sufficient number of samples, we show that we are able to reduce false positives and negatives entirely and thus show that our hybrid machine learning approach is a viable alternative to currently used vortex detection tools in fluid mechanics applications.

This study investigates the buckling behavior of columns with variable cross-sections using analytical, numerical, and hybrid machine learning (ML) approaches. Initially, the power series method is employed to calculate the buckling loads of columns with both constant and varying cross-sections under diverse boundary conditions. Then a finite element (FE) analyses of the columns are performed to obtain the buckling loads and the results are validate by comparing them with results from power series method. Once validated, the FE model is used to generate a large dataset encompassing a wide range of cross-sections, lengths, and material properties, as per the samples obtained through the Sobol sampling method. A hybrid ML model is then developed by integrating the XGBoost algorithm with the particle swarm optimization (PSO) technique for hyperparameter tuning. This hybrid PSO-XGBoost model is trained to predict the buckling loads of columns with varying cross-sections. Its performance for input parameters outside the training dataset is evaluated using statistical metrics and scatter plots. The results demonstrate excellent agreement between the FE analysis and the power series method, confirming the reliability of both approaches. The PSO-XGBoost model achieved remarkable predictive accuracy, with R 2 values of 0.999 and 0.996 for the training and testing datasets, respectively. Furthermore, SHapley Additive exPlanations (SHAP) analysis is conducted to explore the influence and interactions of input parameters on buckling loads, providing valuable insights into the model’s interpretability and the underlying mechanics of column buckling.

Cardiovascular diseases (CVD) are a leading cause of mortality globally, necessitating the development of efficient diagnostic tools. Machine learning (ML) and metaheuristic algorithms have become prevalent in addressing these challenges, providing promising solutions in medical diagnostics. However, traditional ML approaches often need to be improved in feature selection and optimization, leading to suboptimal performance in complex diagnostic tasks. To overcome these limitations, this study introduces a new hybrid method called CSA-DE-LR, which combines the clonal selection algorithm (CSA) and differential evolution (DE) with logistic regression. This integration is designed to optimize logistic regression weights efficiently for the accurate classification of CVD. The methodology employs three optimization strategies based on the F1 score, the Matthews correlation coefficient (MCC), and the mean absolute error (MAE). Extensive evaluations on benchmark datasets, namely Cleveland and Statlog, reveal that CSA-DE-LR outperforms state-of-the-art ML methods. In addition, generalization is evaluated using the Breast Cancer Wisconsin Original (WBCO) and Breast Cancer Wisconsin Diagnostic (WBCD) datasets. Significantly, the proposed model demonstrates superior efficacy compared to previous research studies in this domain. This study's findings highlight the potential of hybrid machine learning approaches for improving diagnostic accuracy, offering a significant advancement in the fields of medical data analysis and CVD diagnosis.

Ultra-high-performance concrete (UHPC) is a sustainable construction material; it can be applied as a substitute for cement concrete. Artificial intelligence methods have been used to evaluate concrete composites to reduce time and money in the construction industries. So, this study applied machine learning (ML) and hybrid ML approaches to predict the compressive and flexural strength of UHPC. A dataset of 626 compressive strength and 317 flexural strength data points was collected from the published research articles, where fourteen important variables were selected as input parameters for the analysis of hybrid ML and ML algorithms. This research used XGBoost, LightGBM, and hybrid XGBoost- LightGBM algorithms to predict UHPC materials. Grid search (GS) techniques were used to adjust model hyper-parameters in search of improved high accuracy and efficiency. ML and hybrid ML models were train, and the test stage utilized statistical assessments such as coefficient of determination (R-square), root mean square error (RMSE), mean absolute error (MAE), and coefficient of efficiency (CE). The results presented hybrid ML algorithm was superior to the XGBoost and LightGBM algorithms in terms of R-square and RMSE values for both compressive and flexural strength prediction. A hybrid ML model and two ML models showed CS considerable R-square values above 0.94 at the testing stages and just over 0.97 at the training phase. Hybrid ML model performance accuracy for CS prediction R-square value found that almost 0.996 for training and 0.963 for testing phases. At the same time, the FS prediction result showed that the R-square value of the Hybrid ML model and two traditional ML models were found at almost 0.95 for the training phase and around 0.81 for the testing phase. But among them, the hybrid XGB-LGB model prediction performance was high accuracy and lowest error for CS and FS of UHPC trained and its hyperparameters optimized. Additionally, the SHAP investigation reveals the impact and relationship of the input variables with the output variables. SHAP analysis outcome reveals that curing age and steel fiber content input parameter had the highest positive impact on compressive strength and flexural strength of UHPC.

The storage system in large scale data centers is typically built upon thousands or even millions of disks, where disk failures constantly happen. A disk failure could lead to serious data loss and thus system unavailability or even catastrophic consequences if the lost data cannot be recovered. While replication and erasure coding techniques have been widely deployed to guarantee storage availability and reliability, disk failure prediction is gaining popularity as it has the potential to prevent disk failures from occurring in the first place. Recent trends have turned toward applying machine learning approaches based on disk SMART attributes for disk failure predictions. However, traditional machine learning (ML) approaches require a large set of training data in order to deliver good predictive performance. In large-scale storage systems, new disks enter gradually to augment the storage capacity or to replace failed disks, leading storage systems to consist of small amounts of new disks from different vendors and/or different models from the same vendor as time goes on. We refer to this relatively small amount of disks as minority disks. Due to the lack of sufficient training data, traditional ML approaches fail to deliver satisfactory predictive performance in evolving storage systems which consist of heterogeneous minority disks. To address this challenge and improve the predictive performance for minority disks in large data centers, we propose a minority disk failure prediction model named TLDFP based on a transfer learning approach. Our evaluation results on two realistic datasets have demonstrated that TLDFP can deliver much more precise results, compared to four popular prediction models based on traditional ML algorithms and two state-of-the-art transfer learning methods.

The storage system in large scale data centers is typically built upon thousands or even millions of disks, where disk failures constantly happen. A disk failure could lead to serious data loss and thus system unavailability or even catastrophic consequences if the lost data cannot be recovered. While replication and erasure coding techniques have been widely deployed to guarantee storage availability and reliability, disk failure prediction is gaining popularity as it has the potential to prevent disk failures from occurring in the first place. Recent trends have turned toward applying machine learning approaches based on disk SMART attributes for disk failure predictions. However, traditional machine learning (ML) approaches require a large set of training data in order to deliver good predictive performance. In large-scale storage systems, new disks enter gradually to augment the storage capacity or to replace failed disks, leading storage systems to consist of small amounts of new disks from different vendors and/or different models from the same vendor as time goes on. We refer to this relatively small amount of disks as minority disks. Due to the lack of sufficient training data, traditional ML approaches fail to deliver satisfactory predictive performance in evolving storage systems which consist of heterogeneous minority disks. To address this challenge and improve the predictive performance for minority disks in large data centers, we propose a minority disk failure prediction model named TLDFP based on a transfer learning approach. Our evaluation results in two realistic datasets have demonstrated that TLDFP can deliver much more precise results and lower additional maintenance cost, compared to four popular prediction models based on traditional ML algorithms and two state-of-the-art transfer learning methods.

Machine learning approaches have a valuable contribution in improving competency in automated decision systems. Several machine learning approaches have been developed in the past studies in individual disease diagnosis prediction. The present study aims to develop a hybrid machine learning approach for diagnosis predictions of multiple diseases based on the combination of efficient feature generation, selection, and classification methods. Specifically, the combination of latent semantic analysis, ranker search, and fuzzy-rough-k-nearest neighbor has been proposed and validated in the diagnosis prediction of the primary tumor, post-operative, breast cancer, lymphography, audiology, fertility, immunotherapy, and COVID-19, etc. The performance of the proposed approach is compared with single and other hybrid machine learning approaches in terms of accuracy, analysis time, precision, recall, F-measure, the area under ROC, and the Kappa coefficient. The proposed hybrid approach performs better than single and other hybrid approaches in the diagnosis prediction of each of the selected diseases. Precisely, the suggested approach achieved the maximum recognition accuracy of 99.12%of the primary tumor, 96.45%of breast cancer Wisconsin, 94.44%of cryotherapy, 93.81%of audiology, and significant improvement in the classification accuracy and other evaluation metrics in the recognition of the rest of the selected diseases. Besides, it handles the missing values in the dataset effectively.

Predicting housing prices is particularly of interest to many scholars and policymakers. However, housing prices are highly volatile and difficult to predict. This study used both traditional and advanced machine learning (ML) approaches to address the issue of housing price prediction. This study involves and compares the predictive power between advanced ML models, including random forest, gradient boosting, k-nearest neighbors (KNN), bagged classification and regression trees (CART), and traditional ML models based on linear regression and its modifications. Notably, in this study, we employed both performance metrics, including the mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2), and k-fold cross-validation (CV) procedure in order to investigate the predictive performance of each model. Empirically, based on a dataset comprising 78,704 real estate sales in Hanoi, Vietnam, we find that advanced ML approaches outperform traditional approaches. Specifically, advanced ML models enhance the accuracy of house price prediction and the decision-making process related to housing buying and selling activities. Our findings also reveal that among advanced ML algorithms, the random forest algorithm performs better than the other models in predicting housing prices.

The growing complexity and size of healthcare systems have rendered fraud detection increasingly challenging; however, the current literature lacks a holistic view of the latest machine learning (ML) techniques with practical implementation concerns. The present study addresses this gap by highlighting the importance of machine learning (ML) in preventing and mitigating healthcare fraud, evaluating recent advancements, investigating implementation barriers, and exploring future research dimensions. To further address the limited research on the evaluation of machine learning (ML) and hybrid approaches, this study considers a broad spectrum of ML techniques, including supervised ML, unsupervised ML, deep learning, and hybrid ML approaches such as SMOTE-ENN, explainable AI, federated learning, and ensemble learning. The study also explored their potential use in enhancing fraud detection in imbalanced and multidimensional datasets. A significant finding of the study was the identification of commonly employed datasets, such as Medicare, the List of Excluded Individuals and Entities (LEIE), and Kaggle datasets, which serve as a baseline for evaluating machine learning (ML) models. The study’s findings comprehensively identify the challenges of employing machine learning (ML) in healthcare systems, including data quality, system scalability, regulatory compliance, and resource constraints. The study provides actionable insights, such as model interpretability to enable regulatory compliance and federated learning for confidential data sharing, which is particularly relevant for policymakers, healthcare providers, and insurance companies that intend to deploy a robust, scalable, and secure fraud detection infrastructure. The study presents a comprehensive framework for enhancing real-time healthcare fraud detection through self-learning, interpretable, and safe machine learning (ML) infrastructures, integrating theoretical advancements with practical application needs.

Classification of SAR images has been an interesting task considering its major role in environmental and natural research areas. Existing studies proposed for Land use/land cover (LU/LC) classification using SAR data can be grouped into two categories: traditional Machine Learning (ML) approaches and approaches that are based on deep Convolutional Neural Networks (CNNs). Traditional ML approaches are generally based on discovering powerful features in terms of discrimination and utilizing different combinations of them. On the other hand, in the latter group, the feature extraction and classification topologies are combined in a single learning framework and they are optimized together. Major drawbacks of the deep CNN methods are that they require a significant amount of annotated data to achieve the desired generalization capability and special hardware, i.e., GPUs for the training and usually inference as well. To address these limitations, employing compact CNNs for the SAR classification is proposed in several existing studies. In this chapter, we investigate the performance of compact CNNs that aim for minimum computational complexity and limited annotated data for the SAR classification. The provided analysis will cover commonly used SAR benchmark datasets consisting of four fully polarimetric, one dual- and one single-polarized SAR data including both spaceborne and airborne sensors.

Tropical Cyclones (TCs) are among the most impactful weather phenomena, with climate change intensifying their duration and strength, posing significant risks to ecosystems and human life. Accurate TC detection, encompassing localization and tracking of TC centers, has become a critical focus for the climate science community. Traditional methods often rely on subjective threshold tuning and might require several input variables, thus making the tracking computationally expensive. We propose a cost-effective hybrid Machine Learning (ML) approach consisting in splitting the TC detection into two separate sub-tasks: localization and tracking. The TC task localization is fully data-driven: multiple Deep Neural Networks (DNNs) architectures have been explored to localize TC centers using a different set of input fields related to the cyclo-genesis, aiming also at reducing the number of input drivers required for detection. A neighborhood matching algorithm is then applied to join previously localized TC center estimates into potential trajectories over time. We train the DNNs on 40 years of ERA5 reanalysis data and International Best Track Archive for Climate Stewardship (IBTrACS) records across the East and West North Pacific basins. The hybrid approach is then compared with four state-of-the-art deterministic trackers (namely OWZ, TRACK, CNRM and UZ), reporting comparable or even better results in terms of Probability of Detection and False Alarm Rate, additionally capturing the interannual variability and spatial distribution of TCs in the target domain. The resulting hybrid ML model represents the core component of a Digital Twin (DT) application implemented in the context of the EU-funded interTwin project.

It's getting harder for 21st-century citizens to effectively detect sarcasm using sentiment analysis in a world full of sarcastic people and identifying sarcasm aids in understanding the unpleasant truth hidden beneath polite language. While sarcasm in text is frequently identified, very little research has been done on text sarcasm recognition in memes. This study uses a hybrid machine learning strategy to increase accuracy in identifying sarcasm text in sentiment analysis. It also compares the hybrid approach to existing approaches, like Random Forest, Logistic Regression, Naive Bayes, Stochastic Gradient Descent, and Decision Tree. The effectiveness of several methods is assessed in this study using recall, precision, and f-measure. The results showed that the suggested strategy (0.8004%) received the highest score when the prediction accuracy of several machine learning approaches was compared. The proposed hybrid approach performs much better in terms of enhancing accuracy.

Wireless Body Area Networks (WBANs) consist of small, low-power sensors placed on or inside the human body for real-time health monitoring. One of the primary challenges in WBANs is ensuring energy efficiency, given the limited power capacity of biosensor nodes. Recent advancements in the field, including sophisticated techniques such as clustering, Particle Swarm Optimization (PSO), reinforcement learning, and hybrid Machine Learning (ML) approaches, have demonstrated significant improvements over traditional routing methods. This paper investigates energy-aware routing in WBANs using ML models, specifically Decision Tree (DT), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Linear Regression (LinReg). The objective of this study is to predict energy consumption per transmission and analyze the resulting impact on network lifetime. The study visualizes results through energy depletion curves and alive node count plots. The findings demonstrate that ML models can effectively predict energy consumption, enabling optimized packet transmission and extended network lifespan. Moreover, the analysis identifies the most efficient ML-based routing strategy for WBANs and demonstrates that ML approaches outperform existing methods in the literature.

This study develops a hybrid machine learning (ML) algorithm integrated with IoT technology to improve the accuracy and efficiency of soil monitoring and tomato crop disease prediction in Anakapalle, a south Indian station. An IoT device collected one-minute and critical soil parameters—humidity, temperature, pH values, nitrogen (N), phosphorus (P), and potassium (K), during the vegetative growth stage, which are essential for assessing soil health and optimizing crop growth. Kendall’s correlations were computed to rank these parameters for utilization in hybrid ML techniques. Various ML algorithms including K-nearest neighbors (KNN), support vector machines (SVM), decision tree (DT), random forest (RF), and logistic regression (LR) were evaluated. A novel hybrid algorithm, ‘Bayesian optimization with KNN’, was introduced to combine multiple ML techniques and enhance predictive performance. The hybrid algorithm demonstrated superior results with 95% accuracy, precision, and recall, and an F1 score of 94%, while individual ML algorithms achieved varying results: KNN (80% accuracy), SVM (82%), DT (77%), RF (80%), and LR (81%) with differing precision, recall, and F1 scores. This hybrid ML approach proved highly effective in predicting tomato crop diseases in natural environments, underscoring the synergistic benefits of IoT and advanced ML techniques in optimizing agricultural practices.