Ensemble Classification Model Research Articles

Design of Ensemble Classifier Model Based on MLP Neural Network For Breast Cancer Diagnosis

Nowadays, breast cancer is one of the leading causes of death women in the worldwide. If breast cancer is detected at the beginning stage, it can ensure long-term survival. Numerous methods have been proposed for the early prediction of this cancer, however, efforts are still ongoing given the importance of the problem. Artificial Neural Networks (ANN) have been established as some of the most dominant machine learning algorithms, where they are very popular for prediction and classification work. In this paper, an Intelligent Ensemble Classification method based on Multi-Layer Perceptron neural network (IEC-MLP) is proposed for breast cancer diagnosis. The proposed method is split into two stages, parameters optimization and ensemble classification. In the first stage, the MLP Neural Network (MLP-NN) parameters, including optimal features, hidden layers, hidden nodes and weights, are optimized with an Evolutionary Algorithm (EA) for maximize the classification accuracy. In the second stage, an ensemble classification algorithm of MLP-NN is applied to classify the patient with optimized parameters. Our proposed IEC-MLP method which can not only help to reduce the complexity of MLP-NN and effectively selection the optimal feature subset, but it can also obtain the minimum misclassification cost. The classification results were evaluated using the IEC-MLP for different breast cancer datasets and the prediction results obtained were very promising (98.74% accuracy on the WBCD dataset). Meanwhile, the proposed method outperforms the GAANN and CAFS algorithms and other state-of-the-art classifiers. In addition, IEC-MLP could also be applied to other cancer diagnosis.

An ensemble classification model for fake feedback detection using proposed labeled CloudArmor dataset

With an increase in the use of Cloud Services, Cloud Customers (CCs) have started browsing reviews about various Cloud Service Providers (CSPs) online provided by numerous CCs to gather their sentiments. This pattern has proved to be fruitful, but it can become ineffective if fake reviews are posted. This article aims at introducing an ensemble model to classify reviews as fake or genuine on the proposed labeled CloudArmour dataset. Labeling has been achieved through these aspects: Number of Reviews provided by a CC to single CSP; Number of CSPs reviewed by a CC; & Frequency of feedbacks published. Additionally, we have compared the performance of the proposed ensemble model with these supervised machine learning classification techniques: KNN; Logistic_Regression; and SVM_rbf on the basis of the following parameters: accuracy; precision, recall; and F1-Score. Ensemble model has yielded highest values as: 97.51%; 98.19%; 93.65%; & 95.86% respectively. Thus, we conclude that the ensemble model has outperformed other models.

Multiple Similarity-based Features Blending for Detecting Code Clones using Consensus-Driven Classification

Code clone detection helps to reduce the costs associated with software maintenance and bug prevention. Machine learning methods have previously suggested many ways by which to detect code clones. The majority of clone detectors are traditional in their approach, they can detect syntactic clones but are poor at detecting semantic clones. Researchers use machine learning to detect semantic clones and automatically scan the data to learn latent semantic features. In this study, we have introduced a new formal model of similarity which combines similarity measures so that method blocks can measure both the syntactic and semantic distances between method block pairs. The uniqueness of our study is in the use of different similarity measures, and similarity scores as features in machine learning, to detect code clones. We use a number of similarity measure computations to extract similarity score features, these features are then represented as vectors. Using ensemble classification models, we perform extensive comparisons and evaluations of the effectiveness of our proposed idea. The results indicate that our approach is significantly better at detecting clone types compared to contemporary code clone detectors. We achieved a 99% success rate in detecting cloned codes based on F-score, recall, and precision. Our approach achieves 98–100% accuracy in the majority of cases.

Research on Grammar Checking System Using Computer Big Data and Convolutional Neural Network Constructing Classification Model

This article proposes an automatic grammatical correction method for typos and word order errors that may occur in Chinese writing by beginners. The thesis first constructs heuristic rules and expands the corpus by analyzing the characteristics of different grammatical error types in the data set. Secondly, when the paper uses classification methods to detect grammatical errors, it extracts sentence-level binary and ternary part-of-speech combinations, n-gram models based on part-of-speech statistics and other three types of features to construct single classification and ensemble classification models, and then use convolutional neural the network constructs classification models from different angles. Finally, when the paper adopts the method based on sequence labeling for grammatical error detection, it mainly uses dependency syntax tree features, and realizes grammatical error detection by constructing a conditional random field model. This method can automatically detect grammatical errors while also identifying the sentence the location of the error. On this basis, the paper implements a simple Chinese grammatical error automatic detection system, which can provide help for the optimization of questions and answers in the question-and-answer system

An Ensemble Classification Model With Unsupervised Representation Learning for Driving Stress Recognition Using Physiological Signals

This paper presents an ensemble classification model with unsupervised feature learning for driving stress recognition under real-world driving conditions. The driving stress is detected using drivers’ different physiological signals, specifically the electromyogram, electrocardiogram, galvanic skin response, heart rate and respiration. The proposed model consists of two modules: 1) a multilayer representation learning module using autoencoder as its building block. The autoencoders are trained with a quasi-automated, non-gradient descent based unsupervised learning algorithm; 2) an ensemble classification module under the AdaBoost framework. The proposed model is completely data driven, does not require additional feature extraction and feature selection process, and can perform in an end-to-end way in which it takes the physiological signal as the input instead of the handcrafted features. Experimental results show that our proposed model can effectively recognize the driving stress with fewer physiological sensors compared with most of state of the art methods. Experiments also demonstrate that the proposed model can simplify the model structure tuning and improve the learning efficiency compared with the baseline deep learning model.

Psychophysiological Modeling of Trust In Technology

Trust as a precursor for users' acceptance of artificial intelligence (AI) technologies that operate as a conceptual extension of humans (e.g., autonomous vehicles (AVs)) is highly influenced by users' risk perception amongst other factors. Prior studies that investigated the interplay between risk and trust perception recommended the development of real-time tools for monitoring cognitive states (e.g., trust). The primary objective of this study was to investigate a feature selection method that yields feature sets that can help develop a highly optimized and stable ensemble trust classifier model. The secondary objective of this study was to investigate how varying levels of risk perception influence users' trust and overall reliance on technology. A within-subject four-condition experiment was implemented with an AV driving game. This experiment involved 25 participants, and their electroencephalogram, electrodermal activity, and facial electromyogram psychophysiological signals were acquired. We applied wrapper, filter, and hybrid feature selection methods on the 82 features extracted from the psychophysiological signals. We trained and tested five voting-based ensemble trust classifier models using training and testing datasets containing only the features identified by the feature selection methods. The results indicate the superiority of the hybrid feature selection method over other methods in terms of model performance. In addition, the self-reported trust measurement and overall reliance of participants on the technology (AV) measured with joystick movements throughout the game reveals that a reduction in risk results in an increase in trust and overall reliance on technology.

Research on an Identification Method for Gas Disaster Risk Based on the Selective Ensemble Classification Model.

To improve the accuracy of gas disaster risk identification, a selective ensemble classification model is proposed based on clustering selection and a new degree of combination fitness (CS–NDCF). First, nine base classifiers for gas disasters are constructed on the training data set, including the backpropagation (BP) neural network classifier, naive Bayes (NB) classifier, K-nearest neighbor (KNN) classifier, logistic regression (LR) classifier, decision tree (DT) classifier, support vector machine (SVM) classifier, SVM classifier with cross-validation (SVMCV), random forest (RF) classifier, and gradient boosting DT (GBDT) classifier. Second, the K-means clustering algorithm is used to cluster the base classifiers according to their classification performance. Then, the best performing classifier in each cluster is selected to compose the first selection set. Third, the degree of combination fitness is used to filter the first selection set again to obtain the optimal base classifier result set. Finally, an ensemble classification model is constructed with the optimal base classifier result set. The experimental results on actual mine monitoring data show that compared with the BP, NB, KNN, LR, DT, SVM, SVMCV, RF, and GBDT classifiers, the accuracy of CS–NDCF increases by 7.34, 34.83, 8.28, 12.94, 5.51, 11.72, 6.47, 1.31, and 1.20%, respectively, and CS–NDCF achieves the best forecasting results. Thus, CS–NDCF is an effective method for identifying gas disasters and has a good application value.

Impact of resampling methods and classification models on the imbalanced credit scoring problems

For imbalanced credit scoring, the most common solution is to balance the class distribution of the training set with a resampling method, and then train a classification model and classify the customer samples in the test set. However, it is still difficult to select the most appropriate resampling methods and classification models, and the optimal combinations of them have not been identified. Therefore, this study proposes a new benchmark models comparison framework for imbalanced credit scoring. In the framework, we introduce the index of balanced accuracy and four other evaluation measures, experimentally compare the performance of 10 benchmark resampling methods and nine benchmark classification models respectively on six credit scoring data sets, and analyze the optimal combinations of them. The experimental result shows: (1) as for benchmark resampling methods, random under-sampling (a traditional resampling method) and synthetic minority over-sampling technique combined with Wilson’s edited nearest neighbor (an intelligent resampling method) present the best performance; (2) as for benchmark classification models, logistic regression (a single classification model) and adaptive boosting (an ensemble classification model) present the best performance; (3) as for optimal combinations, random under-sampling combined with random subspace (an ensemble classification model) can obtain the most satisfactory credit scoring performance.

Machine Learning and Feature Selection Methods for EGFR Mutation Status Prediction in Lung Cancer

The evolution of personalized medicine has changed the therapeutic strategy from classical chemotherapy and radiotherapy to a genetic modification targeted therapy, and although biopsy is the traditional method to genetically characterize lung cancer tumor, it is an invasive and painful procedure for the patient. Nodule image features extracted from computed tomography (CT) scans have been used to create machine learning models that predict gene mutation status in a noninvasive, fast, and easy-to-use manner. However, recent studies have shown that radiomic features extracted from an extended region of interest (ROI) beyond the tumor, might be more relevant to predict the mutation status in lung cancer, and consequently may be used to significantly decrease the mortality rate of patients battling this condition. In this work, we investigated the relation between image phenotypes and the mutation status of Epidermal Growth Factor Receptor (EGFR), the most frequently mutated gene in lung cancer with several approved targeted-therapies, using radiomic features extracted from the lung containing the nodule. A variety of linear, nonlinear, and ensemble predictive classification models, along with several feature selection methods, were used to classify the binary outcome of wild-type or mutant EGFR mutation status. The results show that a comprehensive approach using a ROI that included the lung with nodule can capture relevant information and successfully predict the EGFR mutation status with increased performance compared to local nodule analyses. Linear Support Vector Machine, Elastic Net, and Logistic Regression, combined with the Principal Component Analysis feature selection method implemented with 70% of variance in the feature set, were the best-performing classifiers, reaching Area Under the Curve (AUC) values ranging from 0.725 to 0.737. This approach that exploits a holistic analysis indicates that information from more extensive regions of the lung containing the nodule allows a more complete lung cancer characterization and should be considered in future radiogenomic studies.

Credit Card Fraud Detection under Extreme Imbalanced Data: A Comparative Study of Data-level Algorithms

ABSTRACT Credit card fraud is one of the biggest cybercrimes faced by users. Intelligent machine learning based fraudulent transaction detection systems are very effective in real-world scenarios. However, while designing these systems, machine learning approaches suffer from the problem of imbalanced data, i.e. imbalanced class distribution. Therefore, balancing the dataset becomes an imperative sub-task. Investigation of state-of-the-art approaches reveals that there is a need for a systematic study of class imbalance handling strategies to design an intelligent and capable system to detect the fraudulent transaction. This work aims to provide a comparative study of different class imbalance handling methods. To compare the effectiveness and efficiency of different class imbalance approaches in conjunction with state-of-the-art classification approaches, we have performed an extensive experimental study. We compared these methods on many performance indicators such as Precision, Recall, K-fold Cross-validation, AUC-ROC curve and execution time. In this study, we found that the Oversampling followed by Undersampling methods performs well for ensemble classification models such as AdaBoost, XGBoost and Random Forest.

Ensemble learning with diversified base models for fault diagnosis in nuclear power plants

The performance and improvement of most currently proposed fault diagnosis models for nuclear power plants are limited due to their dependence on a single classification method. To solve this problem, this study proposes novel ensemble learning models (ELMs) obtained by integrating diversified base models through the plurality voting method and weighted voting method, respectively. Simulation data including one steady-state condition and 12 fault conditions are used to evaluate ELMs. The results show that ELMs achieve the best overall diagnostic performance and exhibit excellent performance for all faults. Compared with other common ensemble classification models, ELMs get better diagnostic performance at a dramatically lower time cost, and show significantly stronger robustness when diagnosing data with random noise. The universality of ELMs is illustrated by analyzing their performance under different input features. In summary, ensemble learning is a promising solution for fault diagnosis of nuclear power plants to improve safety and economy.

An ensemble approach of classification model for detection and classification of power quality disturbances in PV integrated microgrid network

In this study, different Power Quality Disturbances (PQDs) in Photovoltaic (PV) integrated Microgrid (MG) network have been detected and classified using a voting method of ensemble classification model along with Discrete Wavelet Transform (DWT) analysis. The proposed ensemble classification model is useful to classify the most common PQDs (voltage sag, voltage swell, and harmonics) in islanded MG network, and different PQ transients in both grid-connected and islanded MG network. For this study, a PV integrated MG model has been developed in the Matlab/Simulink software environment with introduction of different PQDs. The result obtained reveals that the performance of proposed ensemble classification model-2 (combination of Bayesian net, Multi-layer perceptron (MLP) and J48 decision tree (JDT) classifiers) attains higher classification accuracy (100%) as compared to other ensemble classification model-1 (combination of Bayes net and MLP classifiers) and base classifiers such as Bayesian net, MLP and JDT. Further, the effectiveness of classifiers has been assessed using performance indices (PI) such as Kappa statistics, Mean absolute error (MAE), Root mean square error (RMSE), Precision, Recall, F-measure, and Receiver operating characteristics (ROC). From the results of PI, it can be concluded that the proposed ensemble model-2 outperforms ensemble model-1 and other base classifiers.

A Credit Scoring Heterogeneous Ensemble Model Using Stacking and Voting

Background/Objectives: Recent studies emphasized on using ensemble models over single ones to solve credit scoring problems. The objective of this study is to build a heterogeneous ensemble classifier model with an improved classification accuracy. Methods: This study focuses on developing a heterogeneous ensemble classifier using Logistic Regression, K-nearest neighbor, Decision tree, Random Forest, Naïve Base and Support vector machine as base classifiers and Random Forest, Logistic Regression and Support vector machine as meta-classifiers. The proposed model is built using these six base classifiers for ensemble aggregation. A feature selection algorithm based on the random forest technique is used for selecting the best features. A stacking and voting method are used for building ensemble model. Findings: The ensemble classifier gives superior predictive performance than single classifiers SVM, DT, RF, NB, KNN and LR with an accuracy of 91.56% for Australian dataset and 84.35% for German dataset. Novelty: The proposed model uses stacking and majority voting method for ensemble classification. Initially, stacking is applied to the base classifiers. This is done in two levels. First the training dataset is split into 10 folds for cross validation. The output of each classifier is taken, and the dataset is updated with the meta-features. In the second level, three meta-classifiers (MC), namely LR, SVM and RF are used. Majority voting is applied to the output of these meta-classifiers for the prediction. Keywords: Credit scoring; ensemble model; SVM; DT; RF; NB; KNN; LR

Design ensemble deep learning model for pneumonia disease classification.

With the recent spread of the SARS-CoV-2 virus, computer-aided diagnosis (CAD) has received more attention. The most important CAD application is to detect and classify pneumonia diseases using X-ray images, especially, in a critical period as pandemic of covid-19 that is kind of pneumonia. In this work, we aim to evaluate the performance of single and ensemble learning models for the pneumonia disease classification. The ensembles used are mainly based on fined-tuned versions of (InceptionResNet_V2, ResNet50 and MobileNet_V2). We collected a new dataset containing 6087 chest X-ray images in which we conduct comprehensive experiments. As a result, for a single model, we found out that InceptionResNet_V2 gives 93.52% of F1 score. In addition, ensemble of 3 models (ResNet50 with MobileNet_V2 with InceptionResNet_V2) shows more accurate than other ensembles constructed (94.84% of F1 score).

CoVNet-19: A Deep Learning model for the detection and analysis of COVID-19 patients

Background:The ongoing fight with Novel Corona Virus, getting quick treatment, and rapid diagnosis reports have become an act of high priority. With millions getting infected daily and a fatality rate of 2%, we made it our motive to contribute a little to solve this real-world problem by accomplishing a significant and substantial method for diagnosing COVID-19 patients. Aim:The Exponential growth of COVID-19 cases worldwide has severely affected the health care system of highly populated countries due to proportionally a smaller number of medical practitioners, testing kits, and other resources, thus becoming essential to identify the infected people. Catering to the above problems, the purpose of this paper is to formulate an accurate, efficient, and time-saving method for detecting positive corona patients. Method:In this paper, an Ensemble Deep Convolution Neural Network model “CoVNet-19” is being proposed that can unveil important diagnostic characteristics to find COVID-19 infected patients using X-ray images chest and help radiologists and medical experts to fight this pandemic. Results:The experimental results clearly show that the overall classification accuracy obtained with the proposed approach for three-class classification among COVID-19, Pneumonia, and Normal is 98.28%, along with an average precision and Recall of 98.33% and 98.33%, respectively. Besides this, for binary classification between Non-COVID and COVID Chest X-ray images, an overall accuracy of 99.71% was obtained. Conclusion:Having a high diagnostic accuracy, our proposed ensemble Deep Learning classification model can be a productive and substantial contribution to detecting COVID-19 infected patients.

Classifier Selection and Ensemble Model for Multi-class Imbalance Learning in Education Grants Prediction

ABSTRACT Ensemble learning combines base classifiers to improve the performance of the models and obtains a higher classification accuracy than a single classifier. We propose a multi-classification method to predict the level of grant for each college student based on feature integration and ensemble learning. It extracted from expense, score, in/out dormitory, book loan conditions of 10885 students’ daily behavior data and constructed a 21-dimensional feature. The ensemble learning method integrated gradient boosting decision tree, random forest, AdaBoost, and Support Vector Machine classifiers for college grant classification. The proposed method is evaluated with 10885 students set and experiments show that the proposed method has an average accuracy of 0.954 5 and can be used as an effective means of assisting decision-making for college student grants.

An Efficient PSO Based Ensemble Classification Model on High Dimensional Datasets

An Efficient PSO Based Ensemble Classification Model on High Dimensional Datasets

Application of Ensemble Learning in EEG Signal Analysis of Fatigue Driving

Fatigue detection has now become an important direction of fatigue driving research. As a reliable physiological signal, EEG signals can be used to detect fatigue driving. This passage uses the idea of ensemble learning to establish an ensemble learning classification model of Bagged Tree, RSM Discrimination and RUSBoosted Tree, and uses the fatigue driving experiment data as the object to divide the fatigue driving object into normal state and fatigue state, using Pearson Correlation Coefficient method build a functional brain network. Six methods of Betweenness Centrality, Edge Betweenness Centrality, Clustering Coefficient, Degree, Local Efficiency, and Rich Club Coefficient are used to process and analyze six kinds of EEG signal characteristics, and then bring them into different in the ensemble learning model, the EEG signal characteristics and the ensemble learning model are matched and selected to improve the accuracy and stability of the detection model. By simulating 24 subjects, the results show that the accuracy and robustness of RSM Discrimination ensemble learning model and Degree method are the best, with an average accuracy rate of 90.35% and a standard deviation of 0.081.

A stacking ensemble classification model for detection and classification of power quality disturbances in PV integrated power network

This paper proposes a stacking ensemble classification model to classify the different Power Quality Disturbances (PQDs) in Photovoltaic (PV) integrated power network. For this study, the network is developed in Matlab-Simulink with introduction of different PQDs for analysis. In pre-processing stage, Discrete Wavelet Transform technique is used to extract the features from different PQDs. The extracted features are used to train the base classifiers (Logistic Regression (LR), Naïve Bayes, and J48 decision tree) at base level (level 0). The predictions from the base classifiers are used to learn the Meta classifier (LR) in next level (level 1) to get the final predictions. The proposed ensemble model attains higher classification accuracy than base classifiers under standard test condition (92.22%) and dynamic environmental condition of solar PV (91%), and addition of noise into the classifier (89.33%). Further, the proposed method offers superior performance than base classifiers in terms of evaluating performance indices.

Fertility level prediction in precision agriculture based on ensemble classifier model

Fertility level prediction in precision agriculture based on ensemble classifier model