Ensemble Machine Learning Algorithms Research Articles

Non-destructive Machine Vision System based Rice Classification using Ensemble Machine Learning Algorithms

Aims and Background: Agriculture plays a major role in the global economy, providing food, raw materials, and jobs to billions of people and driving economic growth and poverty reduction. Rice is the most widely consumed crop domestically, making it a particularly important crop for rural populations. The exact number of rice varieties worldwide is difficult to determine as new varieties are constantly being developed and marketed. Objectives: The most common method of rice variety identification is a comparison of its physical and chemical properties to a reference collection of known types. Methodology: This is a relatively quick and cost-effective approach that can be used to accurately differentiate between distinct varieties. In some cases, genetic testing may be used to confirm the identity of a variety, although this technique is more expensive and time-consuming. However, we can also utilize efficient, precise, and cost-effective digital image processing and machine vision techniques. Results: This study describes different types of ensemble methods, such as bagging (Decision Tree, Random Forest, Extra Tree), boosting (AdaBoost, Gradient Boost, and XGBoost), and voting classifiers to classify five different varieties of rice. Extreme Gradient Boosting (XGBoost) has achieved the highest average classification accuracy of 99.60% among all the algorithms. Conclusion: The findings of the performance measurement indicated that the proposed model was successful in classifying the various varieties of rice.

Bagging Vs. Boosting in Ensemble Machine Learning? An Integrated Application to Fraud Risk Analysis in the Insurance Sector

ABSTRACT Addressing the pressing challenge of insurance fraud, which significantly impacts financial losses and trust within the insurance industry, this study introduces an innovative automated detection system utilizing ensemble machine learning (EML) algorithms. The approach encompasses four strategic phases: 1) Tackling data imbalance through diverse re-sampling methods (Over-sampling, Under-sampling, and Hybrid); 2) Optimizing feature selection (Filtering, Wrapping, and Embedding) to enhance model accuracy; 3) employing binary classification techniques (Bagging and Boosting) for effective fraud identification; and 4) applying explanatory model analysis (Shapley Additive Explanations, Break-down plot, and variable-importance Measure) to evaluate the influence of individual features on model performance. Our comprehensive analysis reveals that while not every re-sampling technique improves model performance, all feature selection methods markedly bolster predictive accuracy. Notably, the combination of the Gradient Boosting Machine (GBM) algorithm with NCR re-sampling and GBMVI feature selection emerges as the most effective configuration, offering superior fraud detection capabilities. This study not only advances the theoretical framework for combating insurance fraud through AI but also provides a practical blueprint for insurance companies aiming to incorporate advanced AI strategies into their fraud detection arsenals, thereby mitigating financial risks and fostering trust systems.

Development of Random Forest Model for Stroke Prediction

Stroke is a significant cause of mortality and morbidity worldwide, and early detection and prevention of stroke are essential for improving patient outcomes. Machine learning algorithms have been used in recent years to predict the risk of stroke by leveraging large amounts of clinical and demographic data. The development of a stroke prediction system using Random Forest machine learning algorithm is the main objective of this thesis. The primary goal of the project is to increase the accuracy of stroke detection while addressing the shortcomings of the current system, which include real- time deployment and interpretability issues with logistic regression. The development and use of an ensemble machine learning-based stroke prediction system, performance optimization through the use of ensemble machine learning algorithms, performance assessment, and real-time model deployment through the use of Python Django are among the goals of the research. The study's potential to improve public health by lessening the severity and consequences of strokes through early diagnosis and treatment makes it significant. Data collection, preprocessing, model selection, evaluation, and real-time deployment using Python Django are all part of the research technique. Our dataset consists of 5110 rows of tuples and columns with total size of 69kg. The performance of our stroke prediction algorithm was evaluated using confusion metrics-consisting of accuracy, precision, recall and F1-score. At the end of the research, Random Forest model gave an accuracy of 98.5% compared to the existing model logistic regression which has 86% accuracy.

DDoS SourceTracer: An Intelligent Application for DDoS Attack Mitigation in SDN

Software-Defined Networking (SDN) enhances network management and efficiency and is particularly effective in defending against Distributed Denial of Service (DDoS) attack through its centralized structure. Our proposed DDoS SourceTracer Application utilizes SDN to efficiently identify and mitigate DDoS attack by employing tracebacking and clustering techniques. This application uses supervised and ensemble machine learning algorithms for attack detection, and feature selection methods like the Chi-square test, ANOVA (Analysis of Variance) F-test, Correlation matrix, and Extra tree classifier to optimize the feature set. Our results show that the clustering approach outperforms traditional methods like rate limiting and blocking and effectively mitigates the attack in just 3.5 s. We used the sFlow-RT tool on the Zoo topology to perform real-time analysis and validate our application’s effectiveness during attack and normal traffic. This tool analyzes how attack traffic is impacted when using clustering and tracebacking methods to mitigate DDoS attack.

EDT-STACK: A stacking ensemble-based decision trees algorithm for tire tread depth condition classification

Tires serve as the critical interface between a vehicle and the road. Their condition significantly impacts road safety, vehicle performance, fuel efficiency, and operational costs. Developing a machine learning model for accurately and efficiently classifying tire conditions using tread depth measurements poses a critical challenge in enhancing automotive safety and maintenance. This research aims to address this challenge by leveraging machine learning techniques to categorize and assess tire conditions based on tread depth, thereby contributing to developing a robust and automated system for tire safety condition monitoring in diverse vehicle fleets. This article presents a novel ensemble method-based machine learning algorithm called EDT-STACK (Stacking Ensembles-Based Decision Tree) that combines several decision tree-based methods using a stacking ensemble technique. The model is trained and tested on a dataset of 500 tires (new and disposed) with different tread depths, The model achieves 99 % accuracy in all classes, demonstrating its efficacy in classifying tire conditions from tread depth data. The study also compares the performance of EDT-STACK with other algorithms. Its superior performance outshines other decision tree algorithms, such as XGBoost, AdaBoost, Random Forest, ID3, and Histogram-based Decision Tree, which achieve accuracies of 98 %, 78 %, 85 %, 98 %, and 98 %, respectively. This study concludes that EDT-STACK offers an innovative solution for tire condition assessment, which can enhance road safety, reduce environmental impacts, and optimize vehicle maintenance.

Air leaks fault detection in maintenance using machine learning

PurposeThe objective of this paper is to develop a condition-based maintenance (CBM) scheme for pneumatic cylinders. The CBM scheme will detect two common types of air leaking failure modes and identify the leaky/faulty cylinder. The successful implementation of the proposed scheme will reduce energy consumption, scrap and rework, and time to repair.Design/methodology/approachEffective implementation of maintenance is important to reduce operation cost, improve productivity and enhance quality performance at the same time. Condition-based monitoring is an effective maintenance scheme where maintenance is triggered based on the condition of the equipment monitored either real time or at certain intervals. Pneumatic air systems are commonly used in many industries for packaging, sorting and powering air tools among others. A common failure mode of pneumatic cylinders is air leaks which is difficult to detect for complex systems with many connections. The proposed method consists of monitoring the stroke speed profile of the piston inside the pneumatic cylinder using hall effect sensors. Statistical features are extracted from the speed profiles and used to develop a fault detection machine learning model. The proposed method is demonstrated using a real-life case of tea packaging machines.FindingsBased on the limited data collected, the ensemble machine learning algorithm resulted in 88.4% accuracy. The algorithm can detect failures as soon as they occur based on majority vote rule of three machine learning models.Practical implicationsEarly air leak detection will improve quality of packaged tea bags and provide annual savings due to time to repair and energy waste reduction. The average annual estimated savings due to the implementation of the new CBM method is $229,200 with a payback period of less than two years.Originality/valueTo the best of the authors’ knowledge, this paper is the first in terms of proposing a CBM for pneumatic systems air leaks using piston speed. Majority, if not all, current detection methods rely on expensive equipment such as infrared or ultrasonic sensors. This paper also contributes to the research gap of economic justification of using CBM.

Cluster-based wireless sensor network framework for denial-of-service attack detection based on variable selection ensemble machine learning algorithms

A Cluster-Based Wireless Sensor Network (CBWSN) is a system designed to remotely control and monitor specific events or phenomena in areas such as smart grids, intelligent healthcare, circular economies in smart cities, and underwater surveillance. The wide range of applications of technology in almost every field of human activity exposes it to various security threats from cybercriminals. One of the pressing concerns that requires immediate attention is the risk of security breaches, such as intrusions in wireless sensor network traffic. Poor detection of denial-of-service (DoS) attacks, such as Grayhole, Blackhole, Flooding, and Scheduling attacks, can deplete the energy of sensor nodes. This can cause certain sensor nodes to fail, leading to a degradation in network coverage or lifetime. The detection of such attacks has resulted in significant computational complexity in the related works. As new threats arise, security attacks get more sophisticated, focusing on the target system's vulnerabilities. This paper proposed the development of Cluster-Based Wireless Sensor Network and Variable Selection Ensemble Machine Learning Algorithms (CBWSN_VSEMLA) as a security threats detection system framework for DoS attack detection. The CBWSN model is designed using a Fuzzy C-Means (FCM) clustering technique, whereas VSEMLA is a detection system comprised of Principal Component Analysis (PCA) for feature selection and various ensemble machine learning algorithms (Bagging, LogitBoost, and RandomForest) for the detection of grayhole attacks, blackhole attacks, flooding attacks, and scheduling attacks. The experimental results of the model performance and complexity comparison for DoS attack evaluation using the WSN-DS dataset show that the PCA_RandomForest IDS model outperforms with 99.999 % accuracy, followed by the PCA_Bagging IDS model with 99.78 % accuracy and the PCA_LogitBoost model with 98.88 % accuracy. However, the PCA_RandomForest model has a high computational complexity, taking 231.64 s to train, followed by the PCA_LogitBoost model, which takes 57.44 s to train, and the PCA_Bagging model, which takes 0.91 s to train to be the best in terms of model computational complexity. Thus, the models surpassed all baseline models in terms of model detection accuracy on flooding, scheduling, grayhole, and blackhole attacks.

Optimizing machine learning for agricultural productivity: A novel approach with RScv and remote sensing data over Europe

CONTEXTAccurate estimating of crop yield is crucial for developing effective global food security strategies which can lead to reduce of hunger and more sustainable development. However, predicting crop yields is a complex task as it requires frequent monitoring of many weather and socio-economic factors over an extended period. Satellite remote sensing products have become a reliable source for climate-based variables. They are easier to obtain and provide detailed spatial and temporal coverage. OBJECTIVEThe aim of this study is to assess the effectiveness of implement a novel optimization algorithm, called Randomized Search cross validation (RScv), on various machine learning algorithms and measure the prediction accuracy enhancement. METHODSAnnual yields of four crops (Barley, Oats, Rye, and Wheat) were predicted across 20 European countries for 20 years (2000–2019). Two NASA missions, namely GPCP and GLDAS satellites, provided us with climate- and soil-based input variables. Those variables were employed as the input of four ensemble Machine Learning (ML) algorithms (Ada-Boost (AB), Gradient Boost (GB), Random Forest (RF) and Extra Tree (ET)) which are faster and more adoptable compare to classic AI algorithms. RESULTS AND CONCLUSIONSMain results show that applying RScv improves the prediction ability of all ML models over the four crops. In particular, the RScv-AB reaches the overall highest accuracy for predicting yields (Rmax2=0.9). Spatial evaluation of predicting errors depicts that the proposed models were more shifted toward underestimation. An uncertainty analysis was also carried out which shows that applying ML algorithms creates higher and lowers uncertainty in Barley and Wheat respectively. SIGNIFICANCEConsidering the robustness of the optimised ML models and the global coverage of remote sensing data, our current methodology demonstrates great transferability and can be applied in other regions across the globe with higher temporal extents. In addition, this tool could be beneficial to decision makers in various sectors to improve the water allocations, deal with climate change effects and keep sustainable agricultural development.

Hyperspectral Imaging and Machine Learning as a Nondestructive Method for Proso Millet Seed Detection and Classification.

Millet is a small-seeded cereal crop with big potential. There are many different cultivars of proso millet (Panicum miliaceum L.) with different characteristics, bringing forth the issue of sorting which are important for growers, processors, and consumers. Current methods of grain cultivar detection and classification are subjective, destructive, and time-consuming. Therefore, there is a need to develop nondestructive methods for sorting the cultivars of proso millet. In this study, the feasibility of using near-infrared (NIR) hyperspectral imaging (900-1700 nm) to discriminate between different cultivars of proso millet seeds was evaluated. A total of 5000 proso millet seeds were randomly obtained and investigated from the ten most popular cultivars in the United States, namely Cerise, Cope, Earlybird, Huntsman, Minco, Plateau, Rise, Snowbird, Sunrise, and Sunup. To reduce the large dimensionality of the hyperspectral imaging, principal component analysis (PCA) was applied, and the first two principal components were used as spectral features for building the classification models because they had the largest variance. The classification performance showed prediction accuracy rates as high as 99% for classifying the different cultivars of proso millet using a Gradient tree boosting ensemble machine learning algorithm. Moreover, the classification was successfully performed using only 15 and 5 selected spectral features (wavelengths), with an accuracy of 98.14% and 97.6%, respectively. The overall results indicate that NIR hyperspectral imaging could be used as a rapid and nondestructive method for the classification of proso millet seeds.

Ensemble Machine Learning Algorithm for Diabetes Prediction in Maiduguri, Borno State

Ensemble Machine Learning Algorithm for Diabetes Prediction in Maiduguri, Borno State

Landslide susceptibility assessment along the Karakoram highway, Gilgit Baltistan, Pakistan: A comparative study between ensemble and neighbor-based machine learning algorithms

This study addressed the complex challenges associated with landslide detection along the Karakoram Highway (KKH), where tectonic events and data availability limitations posed significant obstacles. To overcome these hurdles, the research framework encompassed several critical components. First, it tackled the issue of multicollinearity through the application of statistical measures such as Variable Inflation Factor (VIF) and Information Gain (IG). Secondly, the study emphasized the importance of selecting a study area that would comprehensively represent the multivariate landscape, with KKH serving as an illustrative example. In striving for an equilibrium between implementation ease and algorithmic performance, the research favored the adoption of Random Forest (RF) and Extremely Randomized Trees (EXT) over XGBoost. Lastly, to fine-tune the algorithms and optimize their parameters, the study employed Particle Swarm Optimization (PSO) and evaluated their performance using metrics like the Area Under the Curve (AUC). Remarkably, this comprehensive approach yielded accuracy rates exceeding 90% for all algorithms tested (RF, EXT, and K-Nearest Neighbor (KNN)), with specific AUC values of 0.967, 0.968, and 0.914, respectively. These findings offer invaluable insights into enhancing disaster prevention strategies and informing land-use planning efforts along the KKH highway.

Fire Prediction Analysis Based on Ensemble Machine Learning Algorithms

Fire Prediction Analysis Based on Ensemble Machine Learning Algorithms

Credit Risk Prediction by using Ensemble Machine Learning Algorithms

Credit Risk Prediction by using Ensemble Machine Learning Algorithms

Ensemble Machine Learning for Predicting 90-Day Outcomes and Analyzing Risk Factors in Acute Kidney Injury Requiring Dialysis.

Our objectives were to (1) employ ensemble machine learning algorithms utilizing real-world clinical data to predict 90-day prognosis, including dialysis dependence and mortality, following the first hospitalized dialysis and (2) identify the significant factors associated with overall outcomes. We identified hospitalized patients with Acute kidney injury requiring dialysis (AKI-D) from a dataset of the Taipei Medical University Clinical Research Database (TMUCRD) from January 2008 to December 2020. The extracted data comprise demographics, comorbidities, medications, and laboratory parameters. Ensemble machine learning models were developed utilizing real-world clinical data through the Google Cloud Platform. The Study Analyzed 1080 Patients in the Dialysis-Dependent Module, Out of Which 616 Received Regular Dialysis After 90 Days. Our Ensemble Model, Consisting of 25 Feedforward Neural Network Models, Demonstrated the Best Performance with an Auroc of 0.846. We Identified the Baseline Creatinine Value, Assessed at Least 90 Days Before the Initial Dialysis, as the Most Crucial Factor. We selected 2358 patients, 984 of whom were deceased after 90 days, for the survival module. The ensemble model, comprising 15 feedforward neural network models and 10 gradient-boosted decision tree models, achieved superior performance with an AUROC of 0.865. The pre-dialysis creatinine value, tested within 90 days prior to the initial dialysis, was identified as the most significant factor. Ensemble machine learning models outperform logistic regression models in predicting outcomes of AKI-D, compared to existing literature. Our study, which includes a large sample size from three different hospitals, supports the significance of the creatinine value tested before the first hospitalized dialysis in determining overall prognosis. Healthcare providers could benefit from utilizing our validated prediction model to improve clinical decision-making and enhance patient care for the high-risk population.

A Multi-Scale Forest Above-Ground Biomass Mapping Approach: Employing a Step-by-Step Spatial Downscaling Method with Bias-Corrected Ensemble Machine Learning

The accurate estimation of forest above-ground biomass (AGB) is vital for monitoring changes in forest carbon sinks. However, the spatial heterogeneity of AGB, coupled with inherent uncertainties, poses challenges in acquiring high-quality AGBs. This study introduced a bias-corrected ensemble machine learning (ML) algorithm for AGB downscaling that integrated a ML for AGB mapping with another for residual mapping. The accuracies of six bias-corrected ensemble ML algorithms were evaluated at resolutions of 0.05°, 0.025°, and 0.01°. Moreover, a step-by-step downscaling (SBSD) method was introduced, utilizing bias-corrected ensemble ML algorithms to downscale AGB from 0.1° to 0.05°, 0.025°, and 0.01° resolutions and was compared with the direct downscaling (DD) at three scales. A comparative analysis was conducted in the Daxing’anling Mountains and Xiaoxing’anling Mountains. AGB and corresponding uncertainty maps at three scales were generated using SBSD. The results showed that the efficacy of the XGBoost-based AGB model combined with the random forest-based residual correction model was superior. Spatial patterns in AGB maps generated by SBSD and DD were found to be similar. Notably, SBSD yielded enhanced accuracy in the Daxing’anling Mountains with complex topography, while both performed comparably in the Xiaoxing’anling Mountains with milder topography, highlighting SBSD’s advantages in high heterogeneity areas.

An Ensemble Learning Method to Predict Airline Ticket Price Using Machine Learning

In recent years airline travel has become a common mode of transportation. People often prefer travelling through airlines to reach destination as early as possible. But the ticket booking for airlines is somewhat a hectic process because the price value changes day to day based on the availability of flights. So far so many Machine Learning algorithms were used to improve the prediction of ticket prices. This paper presents a comparison of two ensemble machine learning algorithms namely Random Forest and XGBoost to detect the airplane ticket price. Detecting airline price using XGBoost given more accuracy when compared to existing system.

Optimizing precision medicine for second-step depression treatment: a machine learning approach.

Less than a third of patients with depression achieve successful remission with standard first-step antidepressant monotherapy. The process for determining appropriate second-step care is often based on clinical intuition and involves a protracted course of trial and error, resulting in substantial patient burden and unnecessary delay in the provision of optimal treatment. To address this problem, we adopt an ensemble machine learning approach to improve prediction accuracy of remission in response to second-step treatments. Data were derived from the Level 2 stage of the STAR*D dataset, which included 1439 patients who were randomized into one of seven different second-step treatment strategies after failing to achieve remission during first-step antidepressant treatment. Ensemble machine learning models, comprising several individual algorithms, were evaluated using nested cross-validation on 155 predictor variables including clinical and demographic measures. The ensemble machine learning algorithms exhibited differential classification performance in predicting remission status across the seven second-step treatments. For the full set of predictors, AUC values ranged from 0.51 to 0.82 depending on the second-step treatment type. Predicting remission was most successful for cognitive therapy (AUC = 0.82) and least successful for other medication and combined treatment options (AUCs = 0.51-0.66). Ensemble machine learning has potential to predict second-step treatment. In this study, predictive performance varied by type of treatment, with greater accuracy in predicting remission in response to behavioral treatments than to pharmacotherapy interventions. Future directions include considering more informative predictor modalities to enhance prediction of second-step treatment response.

Predicting Road Traffic Collisions Using a Two-Layer Ensemble Machine Learning Algorithm

Road traffic collisions are among the world’s critical issues, causing many casualties, deaths, and economic losses, with a disproportionate burden falling on developing countries. Existing research has been conducted to analyze this situation using different approaches and techniques at different stretches and intersections. In this paper, we propose a two-layer ensemble machine learning (ML) technique to assess and predict road traffic collisions using data from a driving simulator. The first (base) layer integrates supervised learning techniques, namely k- Nearest Neighbors (k-NN), AdaBoost, Naive Bayes (NB), and Decision Trees (DT). The second layer predicts road collisions by combining the base layer outputs by employing the stacking ensemble method, using logistic regression as a meta-classifier. In addition, the synthetic minority oversampling technique (SMOTE) was performed to handle the data imbalance before training the model. To simplify the model, the particle swarm optimization (PSO) algorithm was used to select the most important features in our dataset. The proposed two-layer ensemble model had the best outcomes with an accuracy of 88%, an F1 score of 83%, and an AUC of 86% as compared with k-NN, DT, NB, and AdaBoost. The proposed two-layer ensemble model can be used in the future for theoretical as well as practical applications, such as road safety management for improving existing conditions of the road network and formulating traffic safety policies based on evidence.

Analyzing and forecasting climate variability in Nainital district, India using non-parametric methods and ensemble machine learning algorithms

Analyzing and forecasting climate variability in Nainital district, India using non-parametric methods and ensemble machine learning algorithms

Machine learning tools in the tasks of selecting the determinants of socio-economic situation and development potential of Russian regions

Subject. The article addresses the selection of determinants that are significant for assessing the level of socio-economic situation and development potential of Russian regions. Objectives. The aim is to study machine learning algorithms for the selection of determinants – predictors of the level of socio-economic situation and development potential of Russian regions, to build models of classification of regions, according to the level of socio-economic situation, using various machine learning algorithms. Methods. To build classification models, I used data from the Federal State Statistics Service, the Institute of Scientific Communications, the RIA Novosti news agency, and the TAdviser Internet portal. Procedures for data classification, model parameter estimation, selection of significant determinants and visualization of results are performed, using the basic functions of the PyCaret library. Cohen's Kappa statistics and Matthews correlation coefficient were employed as priority metrics for evaluating the model productivity. The algorithms for selecting determinants are implemented in the Google Colab analytical environment. Results. I constructed multiclass classification models based on simple and ensemble machine learning algorithms. Simple classification algorithms, including logistic and ridge regression models, naive Bayesian algorithm, decision tree, support vector machine, and k-nearest neighbor methods are characterized by accuracy values at 77%, however, Cohen's Kappa statistics and the Matthews correlation coefficient only show a satisfactory relationship between the actual and predicted value of the region class. Ensemble algorithms, including random forest, gradient boosting and extreme gradient boosting, are characterized by a close relationship between the actual and forecast estimates of the classifier at a level of more than 70%. Conclusions. The random forest algorithm is recognized as the most effective classification model. The gross regional product and investments in fixed assets are informative determinants for measuring the socio-economic status.