Ensemble Classification Method Research Articles

Improving Driving Style in Connected Vehicles via Predicting Road Surface, Traffic, and Driving Style

This paper investigates the application of ensemble learning in improving the accuracy and reliability of predictions in connected vehicle systems, focusing on driving style, road surface quality, and traffic conditions. Our study’s central methodology is the voting classifier ensemble method, which integrates predictions from multiple machine learning models to improve overall predictive performance. Specifically, the ensemble method combines insights from random forest, decision tree, and K-nearest neighbors models, leveraging their individual strengths while compensating for their weaknesses. This approach resulted in high accuracy rates of 94.67% for driving style, 99.10% for road surface, and 98.80% for traffic predictions, demonstrating the robustness of the ensemble technique. Additionally, our research emphasizes the importance of model explanation ability, employing the tree interpreter tool to provide detailed insights into how different features influence predictions. This paper proposes a model based on the algorithm GLOSA for sharing data between connected vehicles and the algorithm CTCRA for sending road information to navigation application users. Based on prediction results using ensemble learning and similarity in driving styles, road surface conditions, and traffic conditions, an ensemble learning approach is used. This not only contributes to the predictions’ transparency and trustworthiness but also highlights the practical implications of ensemble learning in improving real-time decision-making and vehicle safety in intelligent transportation systems. The findings underscore the significant potential of advanced ensemble methods for addressing complex challenges in vehicular data analysis.

A clustering-based adaptive undersampling ensemble method for highly unbalanced data classification

The class imbalance issue is prevalent in various practical classification tasks. A high unbalanced rate will significantly decrease the classification performance of unbalanced learning. However, existing methods for highly unbalanced data classification still face two key difficulties: (1) fairly learning key information, and (2) maintaining consistency. To address these difficulties, we propose a novel majority clustering-based adaptive undersampling enhanced ensemble classification method, which integrates undersampling and ensemble techniques. In the adaptive undersampling process, we first consider the spatial distribution of majority samples to ensure distribution consistency. We then consider an adaptive sampling rate and introduce a feedback mechanism to obtain more representative majority samples from each cluster. In the classifier ensemble process, multiple ensemble iterations are introduced to achieve fair attention to key information in different classes. Finally, six kinds of experiments are conducted on 17 real highly unbalanced datasets from multiple fields. Experimental results demonstrate that the proposed method outperforms existing methods in terms of effectiveness, robustness, and adaptability.

New ensemble learning algorithm based on classification certainty and semantic correlation

A new bootstrap-aggregating (bagging) ensemble learning algorithm is proposed based on classification certainty and semantic correlation to improve the classification accuracy of ensemble learning. First, two predetermined thresholds are introduced to construct the long and short-text sample subsets, and different deep learning methods are compared to construct the optimal base classifier groups for each sample subsets. Then, the random sampling method employed in traditional bagging classification algorithms is improved, and a threshold group based random sampling method is proposed to obtain long and short training sample subsets of each iteration. Finally, the sample classification certainty of the base classifiers for different categories is defined, and the semantic correlation information is integrated with the traditional weighted voting classifier ensemble method to avoid the loss of important information during the sampling process. The experimental results on multiple datasets demonstrate that the algorithm significantly improves text classification accuracy and outperforms typical deep learning algorithms. The proposed algorithm achieves the improvements of approximately 0.082, 0.061 and 0.019 on CNews dataset when the F1 measurement is used over the traditional ensemble learning algorithms such as random forest, M_ADA_A_SMV and CNN_SVM_LR. Moreover, it achieves the best F1 values of 0.995, 0.985, and 0.989 on the datasets of Spam, CNews, and SogouCS datasets, respectively, when compared with the ensemble learning algorithms using different base classifiers.

Evaluating the Performance of Voting Classifier in Multiclass Classification of Dry Bean Varieties

This study explores the application of a voting classifier, integrating Decision Tree, Logistic Regression, and Gaussian Naive Bayes models, for the multiclass classification of dry bean varieties. Utilizing a dataset comprising 13,611 images of dry bean grains, captured through a high-resolution computer vision system, we extracted 16 features to train and test the classifier. Through a rigorous 5-fold cross-validation process, we assessed the model's performance, focusing on accuracy, precision, recall, and F1-score metrics. The results demonstrated significant variability in the classifier's performance across different data subsets, with accuracy rates fluctuating between 31.23% and 96.73%. This variability highlights the classifier's potential under specific conditions while also indicating areas for improvement. The research contributes to the agricultural informatics field by showcasing the effectiveness and challenges of using ensemble learning methods for crop variety classification, a crucial task for enhancing agricultural productivity and food security. Recommendations for future research include exploring additional features to improve model generalization, extending the dataset for broader applicability, and comparing the voting classifier's performance with other ensemble methods or advanced machine learning models. This study underscores the importance of machine learning in advancing agricultural classification tasks, paving the way for more efficient and accurate crop sorting and grading processes.

A new method of fault diagnosis for aeroengines with dispersedly clumped gas path parameters

In an aircraft fleet with multiple aeroengines in service, the gas path parameter measurement values from engines of the same type are dispersed within a certain range due to manufacturing technology, assembly tolerances, operation conditions, and maintenance. This dispersion cannot be ignored and makes it impossible to apply unified health diagnosis criteria to all engines. To address this issue, a fault diagnosis solution for aeroengines with dispersedly clumped gas path parameters based on meta aero-engine is proposed. The concept of meta aero-engine is proposed to derive an engine group with dispersedly clumped parameters. To diagnose engines, a sphenophyllum classifier is proposed, which has a unified judgement criterion. A new weak classifier ensemble method based on divide-and-conquer is proposed to enhance the classification ability of the sphenophyllum classifier. The results of the basic classifier are partitioned and then analyzed in combination with the results of other classifiers in the classifier ensemble method. The effectiveness of the proposed method is verified through turbofan engine simulation dataset. The proposed method has strong interpretability for engine fleet with dispersedly clumped gas path parameters.

A deep ensemble learning method for cherry classification

AbstractIn many agricultural products, information technologies are utilized in classification processes at the desired quality. It is undesirable to mix different types of cherries, especially in export-type cherries. In this study on cherries, one of the important export products of Turkey, the classification of cherry species was carried out with ensemble learning methods. In this study, a new dataset consisting of 3570 images of seven different cherry species grown in Isparta region was created. The generated new dataset was trained with six different deep learning models with pre-learning on the original and incremental dataset. As a result of the training with incremental data, the best result was obtained from the DenseNet169 model with an accuracy of 99.57%. The two deep learning models with the best results were transferred to ensemble learning and a 100% accuracy rate was obtained with the Maximum Voting model.

Enhancing Cancer Classification Through Ensemble Machine Learning and Gene Selection Approaches

The vast dimensionality of gene expression data and the limited number of relevant genes necessitate the adoption of gene selection techniques. Furthermore, the choice of an efficient classifier plays a pivotal role in achieving accurate results. In this study, we employ the Minimum Redundancy Maximum Relevance (mRMR) method for gene selection, coupled with ensemble classifiers and individual classifiers like K-Nearest Neighbors (KNN) and Decision Trees (DT). A comparative analysis between two ensemble classifiers and two individual classifiers is conducted, revealing the superior performance of the ensemble classifiers. Our investigation utilizes four distinct cancer gene expression datasets to showcase the efficacy of employing ensemble classifiers and gene selection methods for cancer classification. The ensemble classifier (Bagging Classifier), in conjunction with the MRMR method selecting only the top 30 genes, achieved an impressive overall accuracy of 94% across all four employed datasets.

A ROBUST DETECTION OF CYBER INCIDENTS UTILIZING MACHINE LEARNING TECHNIQUES

A reliable Cyber Attack Detection Model (CADM) is a system that works as safeguard for the users of modern technological devices and assistant for the operators of networks. The research paper aims to develop a CADM for analyzing the network data patterns to classify cyber-attacks. CADM finds out attack wise detection accuracy using ensemble classification method. LASSO has been used to extract important features. It can work with large datasets, and it has more visualization capability. Gradient Boosting and Random Forest algorithms have been used for classification of network traffic data to build an ensemble method. Gradient Boosting algorithm trains weak learning models and select the best decision trees to deliver more improved prediction accuracy and Random Forest algorithm trains each tree in parallel manner. In this research work, Jive datasets such as NSL-KDD, KDD Cup 99, UNSWNB15, URL 2016 and CICIDS 2017 are also applied to check the efficiency of the proposed model.

A ROBUST DETECTION OF CYBER INCIDENTS UTILIZING MACHINE LEARNING TECHNIQUES

A reliable Cyber Attack Detection Model (CADM) is a system that works as safeguard for the users of modern technological devices and assistant for the operators of networks. The research paper aims to develop a CADM for analyzing the network data patterns to classify cyber-attacks. CADM finds out attack wise detection accuracy using ensemble classification method. LASSO has been used to extract important features. It can work with large datasets, and it has more visualization capability. Gradient Boosting and Random Forest algorithms have been used for classification of network traffic data to build an ensemble method. Gradient Boosting algorithm trains weak learning models and select the best decision trees to deliver more improved prediction accuracy and Random Forest algorithm trains each tree in parallel manner. In this research work, Jive datasets such as NSL-KDD, KDD Cup 99, UNSWNB15, URL 2016 and CICIDS 2017 are also applied to check the efficiency of the proposed model.

Machine Learning Based Precision Agriculture using Ensemble Classification with TPE Model

Many tasks are part of smart farming, including predicting crop yields, analysing soil fertility, making crop recommendations, managing water, and many more. In order to execute smart agricultural tasks, researchers are constantly creating several Machine Learning (ML) models. In this work, we integrate ML with the Internet of Things. Either the UCI dataset or the Kaggle dataset was used to gather the data. Effective data pretreatment approaches, such as the Imputation and Outlier (IO) methods, are necessary to manage the intricacies and guarantee proper analysis when dealing with data that exhibits irregular patterns or contains little changes that can have a substantial influence on analysis and decision making. The goal of this research is to provide a more meaningful dataset by investigating data preparation approaches that are particular to processing data. Following the completion of preprocessing, the data is classified using an average approach based on the Ensemble of Adaptive Neuro-Fuzzy Inference System (ANFIS), Random Neural Network (PNN), and Clustering-Based Decision Tree (CBDT) techniques. The next step in optimising the hyperparameter tuning of the proposed ensemble classifier is to employ a new Tree-Structured Parzen Estimator (TPE). Applying the suggested TPE based Ensemble classification method resulted in a 99.4 percent boost in accuracy

Ensemble Feature Selection Method for Single Pulse Classification

Affected by a large number of radio frequency interference signals, it has become an important task for astronomical data processing to quickly and accurately identify single pulse signals from massive observation data. Designing and extracting effective data features is the key issue for efficient identification of single pulse signals using machine learning. This paper proposes an ensemble feature selection method for single pulse signal classification. The method first mixed three types of features, including the parametric features, statistical features, and abstract features of single pulse signals, and then used five individual feature selection methods to select the corresponding optimal feature set, respectively. At last, the features selected by the five individual methods are mixed, and the greedy strategy was used to select the optimal ensemble feature set. The experimental results show that the ensemble feature set can improve F1-score by a value of 1.8% at most, and can obtain higher accuracy than the features selected by individual methods. Under the background of high-speed and large-scale sky survey, the ensemble feature selection method plays an important role in reducing the number of features, improving classification performance, and speeding up data processing.

A Learning Approach for The Identification of Network Intrusions Based on Ensemble XGBoost Classifier

The limitation of signature-based Intrusion Detection systems has given rise for the popularity of Machine learning (ML) approaches r for building such intrusion detection systems (IDSs). ML is a sub-filed of Artificial Intelligence that enables algorithms to learn from data and its applications have been widely accepted and used in many domains. To achieve a promising ML-based model that can identify attacks and intrusions in networks and the cyber space, different stages of machine learning approach like pre-processing, attribute selection, model building, hyper parameter tuning can be very important. CICIDS2017 intrusion dataset was used for all the experimentations. This study focuses on building cyber threat detection model based on the ensemble feature selection and classification method. Innovative approaches were used for the analysis and pre-processing of the dataset. Thereafter, XGboost algorithm was used for selecting relevant features from the default input attributes in each of the captures. Thereafter, the reduced features were employed in the identification of cyber intrusions. The average accuracy achieved in the 8 captures of the dataset is 98% while precision is 0.98. Also, recall is 0.98, f1-score is 0.98 while AUC ROC score is 0.99. The study concluded that XGBoost-based model was able to achieve promising results based on the proper dataset encoding, feature importance-based feature selection and tuning of the algorithm for intrusion identification.

An ensemble classification method based on machine learning models for malicious Uniform Resource Locators (URL).

Web applications are important for various online businesses and operations because of their platform stability and low operation cost. The increasing usage of Internet-of-Things (IoT) devices within a network has contributed to the rise of network intrusion issues due to malicious Uniform Resource Locators (URLs). Generally, malicious URLs are initiated to promote scams, attacks, and frauds which can lead to high-risk intrusion. Several methods have been developed to detect malicious URLs in previous works. There has been a good amount of work done to detect malicious URLs using various methods such as random forest, regression, LightGBM, and more as reported in the literature. However, most of the previous works focused on the binary classification of malicious URLs and are tested on limited URL datasets. Nevertheless, the detection of malicious URLs remains a challenging task that remains open to research. Hence, this work proposed a stacking-based ensemble classifier to perform multi-class classification of malicious URLs on larger URL datasets to justify the robustness of the proposed method. This study focuses on obtaining lexical features directly from the URL to identify malicious websites. Then, the proposed stacking-based ensemble classifier is developed by integrating Random Forest, XGBoost, LightGBM, and CatBoost. In addition, hyperparameter tuning was performed using the Randomized Search method to optimize the proposed classifier. The proposed stacking-based ensemble classifier aims to take advantage of the performance of each machine learning model and aggregate the output to improve prediction accuracy. The classification accuracies of the machine learning model when applied individually are 93.6%, 95.2%, 95.7% and 94.8% for random forest, XGBoost, LightGBM, and CatBoost respectively. The proposed stacking-based ensemble classifier has shown significant results in classifying four classes of malicious URLs (phishing, malware, defacement, and benign) with an average accuracy of 96.8% when benchmarked with previous works.

Classifier Ensemble Based on Multiview Optimization for High-Dimensional Imbalanced Data Classification.

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Analysis of Fault Ride through the Improvement of PV Power Plant Based on Capacitive Bridge Fault Current Limiter Using Machine Learning

Photovoltaic power plant (PVPP) has increased importance among renewable energy sources due to their ability to be connected more easily to a modern power grid. However, the reliability and stability operation of a grid-connected PVPP system is very important to ensure even during grid faults. In this study, a capacitive bridge fault current limiter (CBFCL) using a machine learning (ML) method is applied to enhance the fault ride-through (FRT) capability of a grid-connected PVPP system. Three different protection methods called DC chopper, CBFCL, and DC chopper + CBFCL are designed to prevent the harmful effects of overcurrent that occurs during grid faults to protect the grid-connected PVPP system. The ML algorithm can be trained on historical data to predict optimum control parameters based on real-time conditions such as normal and fault operations of the grid-connected PVPP system. An ensemble classification algorithm has the best results among the four classification algorithms in machine learning methods. The ensemble classification algorithm is separately implemented into the control systems of three protection strategies. Bagged Trees and Subspace KNN classifiers in ensemble classification methods have obtained an impressive accuracy of 98% in ML classification methods. The simulation results illustrate that the DC chopper + CBFCL based ensemble provides the best protection for the grid-connected PVPP system compared to other protection systems.

Ensemble methods for testing a global null

Abstract Testing a global null is a canonical problem in statistics and has a wide range of applications. In view of the fact that no uniformly most powerful test exists, prior and/or domain knowledge are commonly used to focus on a certain class of alternatives to improve the testing power. However, it is generally challenging to develop tests that are particularly powerful against a certain class of alternatives. In this paper, motivated by the success of ensemble learning methods for prediction or classification, we propose an ensemble framework for testing that mimics the spirit of random forests to deal with the challenges. Our ensemble testing framework aggregates a collection of weak base tests to form a final ensemble test that maintains strong and robust power for global nulls. We apply the framework to four problems about global testing in different classes of alternatives arising from whole-genome sequencing (WGS) association studies. Specific ensemble tests are proposed for each of these problems, and their theoretical optimality is established in terms of Bahadur efficiency. Extensive simulations and an analysis of a real WGS dataset are conducted to demonstrate the type I error control and/or power gain of the proposed ensemble tests.

Evaluating and comparing bagging and boosting of hybrid learning for breast cancer screening

PurposeIn many fields, including bioinformatics, ensemble methods have been often employed to perform prediction tasks such as classification and regression due to their ability to improve the accuracy and robustness of machine learning models. Given that the most prevalent cancer and the primary cause of death for women is breast cancer, researchers have started using ensemble approaches to advance their studies in this area. Effectively, ensemble methods can be used to develop clinical decision support systems that can help healthcare providers make more accurate judgements. MethodsIn the present study, we construct and evaluate bagging and boosting ensemble methods for the binary classification of breast cancer screening images from the CBIS-DDSM dataset. To build the bagging ensembles, a hybrid architecture consisting of three feature extractors (Inception V3, MobileNet V2, DenseNet 201) and four classifiers (K-nearest neighbors, Multilayer perceptron, Support vector machine, Decision trees) was chosen. As for the boosting ensembles, the method consists of the same three feature extractors with a decision trees-based classifier and the four boosting methods (AdaBoost, GBM, XGboost, LightGBM). ResultsThe evaluation of both ensemble methods over the mammography dataset was made using four performance metrics (accuracy, sensitivity, F1-score, and recall), as well as the Skott–Knott statistical test and the Borda count voting system to rank all the possible model combinations. Results proved that bagging and boosting ensembles generally outperformed both their base learners and single models, and that the best performing ensemble overall was Gradient boosting with DenseNet 201 as feature extractor and 200 trees which scored an accuracy of 85.50 %. ConclusionsThe purpose of this research was to use hybrid learning based ensembles to maximize the potent performance of numerous models. It effectively illustrated the prospective potential of using ML approaches for the binary classification of mammography data used for breast cancer screening.

Ensemble Learning Based Malicious Node Detection in SDN-Based VANETs

Background: The architecture of Software Defined Networking (SDN) integrated with Vehicular Ad-hoc Networks (VANETs) is considered a practical method for handling large-scale, dynamic, heterogeneous vehicular networks, since it offers flexibility, programmability, scalability, and a global understanding. However, the integration with VANETs introduces additional security vulnerabilities due to the deployment of a logically centralized control mechanism. These security attacks are classified as internal and external based on the nature of the attacker. The method adopted in this work facilitated the detection of internal position falsification attacks. Objective: This study aimed to investigate the performance of k-NN, SVM, Naïve Bayes, Logistic Regression, and Random Forest machine learning (ML) algorithms in detecting position falsification attacks using the Vehicular Reference Misbehavior (VeReMi) dataset. It also aimed to conduct a comparative analysis of two ensemble classification models, namely voting and stacking for final decision-making. These ensemble classification methods used the ML algorithms cooperatively to achieve improved classification. Methods: The simulations and evaluations were conducted using the Python programming language. VeReMi dataset was selected since it was an application-specific dataset for VANETs environment. Performance evaluation metrics, such as accuracy, precision, recall, F-measure, and prediction time were also used in the comparative studies. Results: This experimental study showed that Random Forest ML algorithm provided the best performance in detecting attacks among the ML algorithms. Voting and stacking were both used to enhance classification accuracy and reduce time required to identify an attack through predictions generated by k-NN, SVM, Naïve Bayes, Logistic Regression, and Random Forest classifiers. Conclusion: In terms of attack detection accuracy, both methods (voting and stacking) achieved the same level of accuracy as Random Forest. However, the detection of attack using stacking could be achieved in roughly less than half the time required by voting ensemble. Keywords: Machine learning methods, Majority voting ensemble, SDN-based VANETs, Security attacks, Stacking ensemble classifiers, VANETs,

A multi-layer multi-view stacking model for credit risk assessment

Credit risk assessment plays a key role in determining the banking policies and commercial strategies of financial institutions. Ensemble learning approaches have been validated to be more competitive than individual classifiers and statistical techniques for default prediction. However, most researches focused on improving overall prediction accuracy rather than improving the identification of actual defaulted loans. In addition, model interpretability has not been paid enough attention in previous studies. To fill up these gaps, we propose a Multi-layer Multi-view Stacking Integration (MLMVS) approach to predict default risk in the P2P lending scenario. As the main innovation, our proposal explores multi-view learning and soft probability outputs to produce multi-layer integration based on stacking. An interpretable artificial intelligence tool LIME is embedded for interpreting the prediction results. We perform a comprehensive analysis of MLMVS on the Lending Club dataset and conduct comparative experiments to compare it with a number of well-known individual classifiers and ensemble classification methods, which demonstrate the superiority of MLMVS.

An Ensemble Classification Method Based on Deep Neural Networks for Breast Cancer Diagnosis

Advances in technology have led to advances in breast cancer screening by detecting symptoms that doctors have overlooked. In this paper, an automatic detection system for breast cancer cases based on Internet of Things (IoT) is proposed. First, using IoT technology, direct medical images are sent to the data repository after the suspicious person's visit through medical equipment equipped with IoT. Then, in order to help radiologists, interpret medical images as best as possible, we use four pre-trained convolutional neural network models including InceptionResNetV2, InceptionV3, VGG19 and ResNet152. These models are combined by an ensemble classifier. Also, these models are used to accurately predict cases with breast cancer, healthy people, and cases with pneumonia by using two datasets of X-RAY and CT-scan in a three-class classification. Finally, the best result obtained for CT-scan images belongs to InceptionResNetV2 architecture with 99.36% accuracy and for X-RAY images belongs to InceptionV3 architecture with 96.94% accuracy. The results show that this method leads to a reduction in daily visits to medical centers and thus reduces the pressure on the medical care system. It also helps radiologists and medical staff to detect breast cancer in its early stages.