Ensemble Classification Techniques Research Articles

A comprehensive ensemble classification techniques detecting and managing concept drift in dynamic imbalanced data streams

A comprehensive ensemble classification techniques detecting and managing concept drift in dynamic imbalanced data streams

Speech Emotion Recognition Using Deep Neural Networks, Transfer Learning, and Ensemble Classification Techniques

Speech emotion recognition (SER) is the task of determining the affective content present in speech, a promising research area of great interest in recent years, with important applications especially in the field of forensic speech and law enforcement operations, among others. In this paper, systems based on deep neural networks (DNNs) spanning five levels of complexity are proposed, developed, and tested, including systems leveraging transfer learning (TL) for the top modern image recognition deep learning models, as well as several ensemble classification techniques that lead to significant performance increases. The systems were tested on the most relevant SER datasets: EMODB, CREMAD, and IEMOCAP, in the context of: (i) classification: using the standard full sets of emotion classes, as well as additional negative emotion subsets relevant for forensic speech applications; and (ii) regression: using the continuously valued 2D arousal-valence affect space. The proposed systems achieved state-of-the-art results for the full class subset for EMODB (up to 83% accuracy) and performance comparable to other published research for the full class subsets for CREMAD and IEMOCAP (up to 55% and 62% accuracy). For the class subsets focusing only on negative affective content, the proposed solutions offered top performance vs. previously published state of the art results.

Toward improving the performance of learning by joining feature selection and ensemble classification techniques: an application for cancer diagnosis.

Breast cancer is known as the most common type of cancer in women, and this has raised the importance of its diagnosis in medical science as one of the most important issues. In addition to reducing costs, the diagnosis of benign or malignant breast cancer is very important in determining the treatment method. The purpose of this paper is to present a model based on data mining techniques including feature selection and ensemble classification that can accurately predict breast cancer patients in the early stages. The proposed breast cancer detection model is developed by joining Adaptive Differential Evolution (ADE) algorithm for feature selection and Learning Vector Quantization (LVQ) neural network for classification. Our proposed model as ADE-LVQ has the ability to automatically and quickly diagnose breast cancer patients into two classes, benign and malignant. As a new evolutionary approach, ADE performs optimal configuration for LVQ neural network in addition to selecting effective features from breast cancer data. Meanwhile, we configure an ensemble classification technique based on LVQ, which significantly improves the prediction performance. ADE-LVQ has been analyzed from different perspectives on different datasets from Wisconsin breast cancer database. We apply different approaches to handle missing values and improve data quality on this database. The results of the simulations showed that the ADE-LVQ model is more successful than the equivalent and state-of-the-art models in diagnosing breast cancer patients. Also, ADE-LVQ provides better performance with less complexity, considering feature selection and ensemble learning. In particular, ADE-LVQ improves accuracy (up to 3.4%) and runtime (up to 2.3%) on average compared to the existing best method. Combined methods based on data mining techniques for breast cancer diagnosis can help doctors in making better decisions for disease treatment.

Ensemble models based on CNN and LSTM for dropout prediction in MOOC

Massive Open Online Courses (MOOCs) have gained a lot of popularity recently. Despite the large number of students enrolled in these courses, a large percentage drop out. Due to this, predicting student dropout has taken on fundamental importance in this area. Predicting dropout early allows course organizers and educators to intervene and provide targeted support to at-risk students. They can offer additional resources, personalized assistance, or interventions tailored to address specific challenges faced by students, increasing their chances of successful course completion.This study first pre-processes the dataset to create a thirty-day correlation matrix for each learner, enabling early dropout prediction by the end of the first week. Then, six new models have been proposed using ensemble classification techniques with Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM). CNN is used for automatic feature extraction, while LSTM considers the time series aspect of the data to improve early prediction performance.As ensemble classifiers can reduce the variance of prediction errors, using ensemble classifiers in addition to neural networks can enhance accuracy and F1 score without overfitting. The application of these techniques results in more accurate week-by-week dropout prediction.The experimental results on the KDD Cup 2015 dataset (representing XuetangX, a MOOC platform in China with 39 courses, 79,186 students, and 120,542 registered students, with 8,157,277 records collected over 30 days) show that all Bagging models improve performance of their base models. In one of the proposed models (Bagging LSTM-LSTM), at the end of the fifth week, the accuracy reached 94%, and the average accuracy reached 91%. Also, precision and recall reached an average of 92%, and F1 score reached 98%, which shows a significant improvement compared to previous researches.

A metaheuristic-based ensemble feature selection framework for cyber threat detection in IoT-enabled networks

Internet of Things (IoT) enabled networks are highly vulnerable to cyber threats due to insecure wireless communication, resource constraint architecture, different types of IoT devices, and a high volume of sensor data being transported across the network. Therefore, IoT-compatible cybersecurity solutions are required. An intrusion detection system is one of the most common solutions for detecting cyber threats in IoT-enabled networks. However, most of the existing solutions for cyber threat detection suffer from many issues like poor accuracy, high learning complexity, low scalability, and high false positive rate (FPR). We propose a metaheuristic-based intelligent and novel framework for cyber threat detection using ensemble feature selection and classification approaches to overcome these issues. First, a metaheuristic-based ensemble feature selection framework is designed using Binary Gravitational Search Algorithm (BGSA) and Binary Grey Wolf Optimization (BGWO) to get an optimized set of features to avoid the curse of dimensionality for efficient learning. Next, Decision Tree and ensemble learning-based classification techniques such as AdaBoost and Random Forest (RF) are employed separately to detect and classify cyber threats. The UNSW-NB15 dataset assesses the effectiveness of the proposed framework, and its performance is evaluated against recent state-of-the-art frameworks. Based on the result analysis, it is found that the RF outperforms existing modern cyber threats detection methods due to the optimized feature subset (4 features out of 42), maximum accuracy (99.41%), maximum detection rate (99.09%), and maximum F1-score (99.33%) with the lowest FPR (0.03%).

An Effectual Model for Early Prediction of Academic Perfomance using Ensemble Classification

In the past few years, researchers are focused towards educational data mining (EDM) to improve the quality of education. Student’s academic performance prediction is a vital issue for improving the value of education. Research study conducted in the literature review mainly captivated the academic performance prediction at higher education. Though the academic performance at secondary level is infrequent, the same could be a scale for a student's performance at subsequent levels of education. Poor grades at lower levels also impact student’s future performance. In this paper, an effectual model is built with the help of significant factors that affect a student's academic performance at secondary level using single and ensemble techniques of classificification For this, both single and ensemble classification techniques are used in this paper To do the same, three single classifiers (classification algorithm) i.e., MLP, Random Forest and PART along with three well recognized ensemble algorithms Bagging (BAG), LogitBoost (LB) and Voting (VT) are applied on the datasets. For better performance of aforementioned classifiers, blended versions (single + ensemble-based classifiers) of classification models are also built. Assessment metrics i.e., accuracy, precision, recall and F-measure used to evaluate the performance of our proposed model. Evaluation results shows that Logitboost with Random Forest outperformed with 99.8% accuracy. It is clearly visible from results that the proposed model is useful for academic performance prediction to improve learning outcomes in future.

An Effectual Model for Early Prediction of Academic Perfomance using Ensemble Classification

In the past few years, researchers are focused towards educational data mining (EDM) to improve the quality of education. Student’s academic performance prediction is a vital issue for improving the value of education. Research study conducted in the literature review mainly captivated the academic performance prediction at higher education. Though the academic performance at secondary level is infrequent, the same could be a scale for a student's performance at subsequent levels of education. Poor grades at lower levels also impact student’s future performance. In this paper, an effectual model is built with the help of significant factors that affect a student's academic performance at secondary level using single and ensemble techniques of classificification For this, both single and ensemble classification techniques are used in this paper To do the same, three single classifiers (classification algorithm) i.e., MLP, Random Forest and PART along with three well recognized ensemble algorithms Bagging (BAG), LogitBoost (LB) and Voting (VT) are applied on the datasets. For better performance of aforementioned classifiers, blended versions (single + ensemble-based classifiers) of classification models are also built. Assessment metrics i.e., accuracy, precision, recall and F-measure used to evaluate the performance of our proposed model. Evaluation results shows that Logitboost with Random Forest outperformed with 99.8% accuracy. It is clearly visible from results that the proposed model is useful for academic performance prediction to improve learning outcomes in future.

Exploring Deep Ensemble Model for Insect and Pest Detection from Images

Pests and insects are the main hazards to crops, and they have a significant negative impact on both human health and wealth. Identification and classification of different types of crop insects accurately is an important task to avoid these losses. A manual examination is a quite sluggish and less efficient way of accomplishing this task while computer vision technology can significantly assist in this issue. A CNN-based deep learning model can classify insects and pests efficiently. In this paper, an ensemble-based model has been proposed using transfer learning. The experimental setup comprises pre-trained models like VGG16, VGG19, and ResNetv50 with a voting classifier ensemble technique. These pre-trained models are used to train the training dataset in a parallel pipeline model which is united with Ensemble Voting Classifier to generate the final prediction over the input sample. The benchmark IP102 dataset, having more than 75000 samples over 102 classes, is used to train and evaluate the model's performance. Results show that the proposed ensemble model achieves an accuracy of 82.5%, which is significant improvement over the recent state-of-the-art models proposed over the same selected dataset, hence strongly supporting the efficacy of the model.

A Survey on Cardiac Disease Classifying from Imbalanced Healthcare Facts Using Ensemble Classification Techniques

Abstract: Heart disease is the leading problem of death globally, and early detection is pivotal in preventing the progression of the disease. In this paper, Improved EnsembleSVM method is proposed for the prediction of heart disease risk. The technique involves randomly partitioning the dataset into smaller subsets using a mean based splitting approach. Oversampling of imbalanced data introduces the extension of Synthetic Minority Over-sampling Technique through a recent ideology, and recurrent ensemble-based noise filter called duplicative-Partitioning Filter, which can overwhelm the hindrance fashioned by noisy and frontier models in overbalanced dataset. Ensemble classifier with oversampling technique plays an accurate result for predict heart disease in efficient way.

Deep learning and natural language processing in computation for offensive language detection in online social networks by feature selection and ensemble classification techniques

Offensive communications have made their way into social media posts. Using computational algorithms to distinguish objectionable content is one of the most effective ways to deal with this problem. One of the most effective approaches to deal with this issue is to use computational methods to distinguish undesirable content. This research aims to tackle MOLD_DL (Multilingual Offensive Language Detection using deep learning) techniques and natural language processing used in feature selection and classification. Here the dataset has been collected from YouTube, Twitter and Facebook, which has been pre-processed for noise removal, filtering and removing the stop words and segmented. The feature selection has been carried out for segmented data using Fuzzy based convolutional neural network (FCNN). Then the extraction of selected features and classification has been carried out using ensemble architecture of Bi-LSTM model with Naïve Bayes architecture hybrid with Support Vector Machines (SVM). Evaluation of offensive language detection is classified automatically based on the emotions of the text. Here the experimental analysis has been carried out for YouTube, Twitter and Facebook datasets in terms of accuracy of 98%, precision of 95%, recall of 90%, F-1 score of 92.5% and RMSE of 45% with the confusion matrix in detecting offensive text of various languages.

Ensemble classification and regression techniques combined with portable near infrared spectroscopy for facile and rapid detection of water adulteration in bovine raw milk

AbstractThe objective of the present contribution is to use a portable near infrared (NIR) spectrometer (in spectral range of 900–1700 nm) as a rapid, facile, and nondestructive technique in combination with ensemble methods, for detection of water in bovine milk samples in concentration range 1% to 30% (v/v). On this matter, the pattern of the milk samples (pure and adulterated) was explored by principal component analysis (PCA). Then, random subspace discriminant ensemble (RSDE) was used for classification. The classification figures of merit for the RSDE method was evaluated in terms of sensitivity (Sen), specificity (Spe), accuracy (Acc), error rate (ER), and reliability (Rel). All the values were satisfactory for calibration, cross‐validation, and prediction sets, which showed the reliability and robustness of the developed model. Furthermore, the performance of RSDE method was compared with partial least square‐discriminant analysis (PLS‐DA) and support vector machine (SVM) as the most common classification techniques. In overall, the RSDE classification outperformed the other tested classification methods (PLS‐DA and RBF‐SVM) in terms of accuracy and reliability. Finally, boosted regression tree (BRT) was used for quantifying the level of water adulterant in milk. The performance of the ensemble regression model was evaluated using regression coefficient (R2) and root mean square error (RMSE). The values of R2 and RMSE in prediction set for BRT were 0.95 and 0.58, respectively. The performance of BRT method was also compared with partial least squares regression (PLSR) which again showed outperformance in comparison with this frequent used regression technique.

Elderly People Activity Recognition in Smart Grid Monitoring Environment

Elderly people activity recognition has become a vital necessity in many countries, because most of the elderly people live alone and are vulnerable. Thus, more research to advance in the monitoring systems used to recognize the activities of elderly people is required. Many researchers have proposed different monitoring systems for activity recognition using wired and wireless wearable sensing devices. However, the activity classification accuracy achieved so far should be improved to meet the challenges of more precise activity monitoring. Our study proposes a smart Human Activity Recognition system architecture utilizing an open source dataset generated by wireless, batteryless sensors used by 14 healthy aged persons and unsupervised and supervised machine learning algorithms. In this paper, we also propose using a smart grid for checking regularly the wearable sensing device operational status to address the well-known reliability challenges of these devices, such as wireless charging and data trustworthiness. As the data from the sensing device is very noisy, we employ the K-means++ clustering to identify outliers and use advanced ensemble classification techniques, such as the stacking classifier for which a meta model built using the random forest algorithm gave better results than all other base models considered. We also employ a bagging classifier, which is an ensemble meta-estimator fitting the prediction outputs of the base classifiers and aggregating them to produce the ensemble output. The best classification accuracy of 99.81 was achieved by the stacking classifier in training and 99.78% in testing, respectively. Comparisons for finding the best model were conducted using the recall, F1 score, and precision values.

Prediction of COVID-19 Patients Recovery using Ensemble Machine Learning and Vital Signs Data Collected by Novel Wearable Device

During the spread of a pandemic such as COVID- 19, the effort required of health institutions increases dra-matically. Generally, Health systems’ response and efficiency depend on monitoring vital signs such as blood oxygen level, heartbeat, and body temperature. At the same time, remote health monitoring and wearable health technologies have revolutionized the concept of effective healthcare provision from a distance. However, analyzing such a large amount of medical data in time to provide the decision-makers with necessary health procedures is still a challenge. In this research, a wearable device and monitoring system are developed to collect real data from more than 400 COVID-19 patients. Based on this data, three classifiers are implemented using two ensemble classification techniques (Adaptive Boosting and Adaptive Random Forest). The analysis of collected data showed a remarkable relationship between the patient’s age and chronic disease on the one hand and the speed of recovery on the other. The experimental results indicate a highly accurate performance for Adaptive Boosting classifiers, reaching 99%, while the Adaptive Random Forest got a 91% accuracy metric.

Choquet fuzzy integral-based classifier ensemble technique for COVID-19 detection

The COVID-19 outbreak has resulted in a global pandemic and led to more than a million deaths to date. COVID-19 early detection is essential for its mitigation by controlling its spread from infected patients in communities through quarantine. Although vaccination has started, it will take time to reach everyone, especially in developing nations, and computer scientists are striving to come up with competent methods using image analysis. In this work, a classifier ensemble technique is proposed, utilizing Choquet fuzzy integral, wherein convolutional neural network (CNN) based models are used as base classifiers. It classifies chest X-ray images from patients with common Pneumonia, confirmed COVID-19, and healthy lungs. Since there are few samples of COVID-19 cases for training on a standard CNN model from scratch, we use the transfer learning scheme to train the base classifiers, which are InceptionV3, DenseNet121, and VGG19. We utilize the pre-trained CNN models to extract features and classify the chest X-ray images using two dense layers and one softmax layer. After that, we combine the prediction scores of the data from individual models using Choquet fuzzy integral to get the final predicted labels, which is more accurate than the prediction by the individual models. To determine the fuzzy-membership values of each classifier for the application of Choquet fuzzy integral, we use the validation accuracy of each classifier. The proposed method is evaluated on chest X-ray images in publicly available repositories (IEEE and Kaggle datasets). It provides 99.00%, 99.00%, 99.00%, and 99.02% average recall, precision, F-score, and accuracy, respectively. We have also evaluated the performance of the proposed model on an inter-dataset experimental setup, where chest X-ray images from another dataset (CMSC-678-ML-Project GitHub dataset) are fed to our trained model and we have achieved 99.05% test accuracy on this dataset. The results are better than commonly used classifier ensemble methods as well as many state-of-the-art methods.

Detecting Multi-Decadal Changes in Seagrass Cover in Tauranga Harbour, New Zealand, Using Landsat Imagery and Boosting Ensemble Classification Techniques

Seagrass provides a wide range of essential ecosystem services, supports climate change mitigation, and contributes to blue carbon sequestration. This resource, however, is undergoing significant declines across the globe, and there is an urgent need to develop change detection techniques appropriate to the scale of loss and applicable to the complex coastal marine environment. Our work aimed to develop remote-sensing-based techniques for detection of changes between 1990 and 2019 in the area of seagrass meadows in Tauranga Harbour, New Zealand. Four state-of-the-art machine-learning models, Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boost (XGB), and CatBoost (CB), were evaluated for classification of seagrass cover (presence/absence) in a Landsat 8 image from 2019, using near-concurrent Ground-Truth Points (GTPs). We then used the most accurate one of these models, CB, with historic Landsat imagery supported by classified aerial photographs for an estimation of change in cover over time. The CB model produced the highest accuracies (precision, recall, F1 scores of 0.94, 0.96, and 0.95 respectively). We were able to use Landsat imagery to document the trajectory and spatial distribution of an approximately 50% reduction in seagrass area from 2237 ha to 1184 ha between the years 1990–2019. Our illustration of change detection of seagrass in Tauranga Harbour suggests that machine-learning techniques, coupled with historic satellite imagery, offers potential for evaluation of historic as well as ongoing seagrass dynamics.

Feature selection model for healthcare analysis and classification using classifier ensemble technique

Feature selection model for healthcare analysis and classification using classifier ensemble technique

Combining feature selection, instance selection, and ensemble classification techniques for improved financial distress prediction

Bankruptcy prediction and credit scoring are major problems in financial distress prediction. Studies have shown that prediction models can be made more effective by performing data preprocessing procedures. Moreover, classifier ensembles are likely to outperform single classifiers. Although feature selection, instance selection, and classifier ensembles are known to affect final prediction results, their combined effects on bankruptcy prediction and credit scoring problems have not been fully explored. This study compares the performance of three feature selection algorithms, three instance selection algorithms, four classification algorithms, and two ensemble learning techniques. The results obtained using five bankruptcy prediction and five credit scoring datasets indicate that by carefully considering the combination of these three factors, better prediction models can be developed than by considering only one related factor.

COVID-19 Diagnosis in Chest X-rays Using Deep Learning and Majority Voting

The COVID-19 disease has spread all over the world, representing an intriguing challenge for humanity as a whole. The efficient diagnosis of humans infected by COVID-19 still remains an increasing need worldwide. The chest X-ray imagery represents, among others, one attractive means to detect COVID-19 cases efficiently. Many studies have reported the efficiency of using deep learning classifiers in diagnosing COVID-19 from chest X-ray images. They conducted several comparisons among a subset of classifiers to identify the most accurate. In this paper, we investigate the potential of the combination of state-of-the-art classifiers in achieving the highest possible accuracy for the detection of COVID-19 from X-ray. For this purpose, we conducted a comprehensive comparison study among 16 state-of-the-art classifiers. To the best of our knowledge, this is the first study considering this number of classifiers. This paper’s innovation lies in the methodology that we followed to develop the inference system that allows us to detect COVID-19 with high accuracy. The methodology consists of three steps: (1) comprehensive comparative study between 16 state-of-the-art classifiers; (2) comparison between different ensemble classification techniques, including hard/soft majority, weighted voting, Support Vector Machine, and Random Forest; and (3) finding the combination of deep learning models and ensemble classification techniques that lead to the highest classification confidence on three classes. We found that using the Majority Voting approach is an adequate strategy to adopt in general cases for this task and may achieve an average accuracy up to 99.314%.

Efficient feature selection and classification through ensemble method for network intrusion detection on cloud computing

Cloud computing is a preferred option for organizations around the globe, it offers scalable and internet-based computing resources as a flexible service. Security is a key concern factor in any cloud solution due to its distributed nature. Security and privacy are huge obstacles faced in its success of the on-demand service as it is easily vulnerable to intruders for any kind of attack. A huge upsurge in network traffic has paved the way to security breaches which are more complicated and widespread. Tackling these attacks has become an inefficient application of traditional intrusion detection systems (IDS) environment. In this research, we developed an efficient Intrusion Detection System (IDS) for the cloud environment using ensemble feature selection and classification techniques. This proposed method was relying on the univariate ensemble feature selection technique, which is used for the selection of valuable reduced feature sets from the given intrusion datasets. While the ensemble classifiers that can competently fuse the single classifiers to produce a robust classifier using the voting technique. An ensemble based proposed method effectively classifies whether the network traffic behavior is normal or attack. The implementation of the proposed method was measured by applying various performance evaluation metrics and ROC-AUC (“area under the receiver operating characteristic curves”) across various classifiers. The results of the proposed methodology achieved a strong considerable amount of performance enhancement compared with other existing methods. Moreover, we performed a pairwise t test and proved that the performance of the proposed method was statistically significantly different from other existing approaches. Finally, the outcome of this investigation was obtained with the best accuracy and lowest false alarm rate (FAR).

TargetMM: Accurate Missense Mutation Prediction by Utilizing Local and Global Sequence Information with Classifier Ensemble.

Missense mutation (MM) may lead to various human diseases by disabling proteins. Accurate prediction of MM is important and challenging for both protein function annotation and drug design. Although several computational methods yielded acceptable success rates, there is still room for further enhancing the prediction performance of MM. In the present study, we designed a new feature extracting method, which considers the impact degree of residues in the microenvironment range to the mutation site. Stringent cross-validation and independent test on benchmark datasets were performed to evaluate the efficacy of the proposed feature extracting method. Furthermore, three heterogeneous prediction models were trained and then ensembled for the final prediction. By combining the feature representation method and classifier ensemble technique, we reported a novel MM predictor called TargetMM for identifying the pathogenic mutations from the neutral ones. Comparison outcomes based on statistical evaluation demonstrate that TargetMM outperforms the prior advanced methods on the independent test data. The source codes and benchmark datasets of TargetMM are freely available at https://github.com/sera616/TargetMM.git for academic use.