Ensemble Classification Model Research Articles

Classifiers ensemble and synthetic minority oversampling techniques for academic performance prediction

The increasing need for data driven decision making recently has resulted in the application of data mining in various fields including the educational sector which is referred to as educational data mining. The need for improving the performance of data mining models has also been identified as a gap for future researcher. In Nigeria, higher educational institutions collect various students’ data, but these data are rarely used in any decision or policy making to improve the academic performance of students. This research work, attempts to improve the performance of data mining models for predicting students’ academic performance using stacking classifiers ensemble and synthetic minority over-sampling techniques. The research was conducted by adopting and evaluating the performance of J48, IBK and SMO classifiers. The individual classifiers models, standard stacking classifier ensemble model and stacking classifiers ensemble model were trained and tested on 206 students’ data set from the faculty of science federal university Dutse. Students’ specific previous academic performance records at Unified Tertiary Matriculation Examination, Senior Secondary Certificate Examination and first year Cumulative Grade Point Average of students are used as data inputs in WEKA 3.9.1 data mining tool to predict students’ graduation classes of degrees at undergraduate level. The result shows that application of synthetic minority over-sampling technique for class balancing improves all the various models performance with the proposed modified stacking classifiers ensemble model outperforming the various classifiers models in both performance accuracy and RSME values making it the best model.<strong></strong>

Ensemble Classification Model for Diabetes Prediction in Data Mining

The prediction analysis is the approach which can predict the future possibilities based on the current information. The diabetes prediction is the approach which is applied to predict the diabetes based on the various attributes. The diabetes dataset has various attributes and based on that attributes diabetes can be predicted. In the previous years approach of SVM is applied for the diabetes prediction. To improve accuracy of diabetes prediction voting based classification is applied in this paper. The proposed model is implemented in python and results are analyzed in terms of accuracy, execution time

K-complexes Detection in EEG Signals using Fractal and Frequency Features Coupled with an Ensemble Classification Model

K-complexes are important transient bio-signal waveforms in sleep stage 2. Detecting k-complexes visually requires a highly qualified expert. In this study, an efficient method for detecting k-complexes from electroencephalogram (EEG) signals based on fractal and frequency features coupled with an ensemble model of three classifiers is presented. EEG signals are first partitioned into segments, using a sliding window technique. Then, each EEG segment is decomposed using a dual-tree complex wavelet transform (DT-CWT) to a set of real and imaginary parts. A total of 10 sub-bands are used based on four levels of decomposition, and the high sub-bands are considered in this research for feature extraction. Fractal and frequency features based on DT-CWT and Higuchi’s algorithm are pulled out from each sub-band and then forwarded to an ensemble classifier to detect k-complexes. A twelve-feature set is finally used to detect the sleep EEG characteristics using the ensemble model. The ensemble model is designed using a combination of three classification techniques including a least square support vector machine (LS-SVM), k-means and Naïve Bayes. The proposed method for the detection of the k-complexes achieves an average accuracy rate of 97.3 %. The results from the ensemble classifier were compared with those by individual classifiers. Comparisons were also made with existing k-complexes detection approaches for which the same datasets were used. The results demonstrate that the proposed approach is efficient in identifying the k-complexes in EEG signals; it yields optimal results with a window size 0.5 s. It can be an effective tool for sleep stages classification and can be useful for doctors and neurologists for diagnosing sleep disorders.

Dissimilarity-based multi-instance learning using dictionary learning and sparse coding ensembles

In multi-instance learning problems, samples are represented by multisets, which are named as bags. Each bag includes a set of feature vectors called instances. This differs multi-instance learning problems from classical supervised learning problems. In this paper, to convert a multi-instance learning problem into a supervised learning problem, fixed-size feature vectors of bags are computed using a dissimilarity based method. Then, dictionary learning based bagging and random subspace ensemble classification models are proposed to exploit the underlying discriminative structure of the dissimilarity based features. Experimental results are obtained on 11 different datasets from different multi-instance learning problem domains. It is shown that the proposed random subspace based dictionary ensemble algorithm gives the best results on 8 datasets in terms of classification accuracy and area under curve.

Sentiment classification of social media reviews using an ensemble classifier

<p>These days it has become a common practice for business organizations and individuals to make use of social media for sharing the opinions about the products or the services. Consumers are also ready to share their views on certain products or commodities. Thus huge amount of unstructured social media data gets generated day by day. Gradually heap of text data will be formed in many areas like automated business, education, health care, and show business and so on. Opinion mining also referred as sentiment analysis or sentiment classification, deals with mining of the review text and classifying the opinions or the sentiments of that text as positive or negative. In this paper we propose an ensemble classifier model consisting of Support Vector Machine and Artificial Neural Network. It combines the knowledge from two feature sets for sentiment classification. The proposed model shows the acceptable performance in terms of accuracy when compared with the baseline model.</p>

In Silico Prediction of Drug-Induced Liver Injury Based on Ensemble Classifier Method.

Drug-induced liver injury (DILI) is a major factor in the development of drugs and the safety of drugs. If the DILI cannot be effectively predicted during the development of the drug, it will cause the drug to be withdrawn from markets. Therefore, DILI is crucial at the early stages of drug research. This work presents a 2-class ensemble classifier model for predicting DILI, with 2D molecular descriptors and fingerprints on a dataset of 450 compounds. The purpose of our study is to investigate which are the key molecular fingerprints that may cause DILI risk, and then to obtain a reliable ensemble model to predict DILI risk with these key factors. Experimental results suggested that 8 molecular fingerprints are very critical for predicting DILI, and also obtained the best ratio of molecular fingerprints to molecular descriptors. The result of the 5-fold cross-validation of the ensemble vote classifier method obtain an accuracy of 77.25%, and the accuracy of the test set was 81.67%. This model could be used for drug-induced liver injury prediction.

Automatic speaker recognition with enhanced swallow swarm optimization and ensemble classification model from speech signals

An Automatic Speaker Recognition (ASR) system which applies voice as a discriminatory feature is used for authentication and verification of identity of a person. A system which is able to deliver acceptable performance in an environment with multiple operating parameters is the requirement of a robust speaker recognition system. Enhancing the performance characteristics of the ASR becomes major significant and challenging issue. The key objective of this work is to introduce a new automatic speaker recognition system for speech signals. In this proposed work, preprocessing is done with the LMS adaptive filter. Features like zero-crossing rate, energy, auto-correlation function, Mel frequency cepstral coefficient features are extracted from noise removed signals to obtain better results. Using the extracted features, input samples are created and the dimensions have been reduced using enhanced swallow swarm optimization. Finally, ensemble classification model using many classification methods such as long short-term memory, improved convolutional neural network and support vector machine is used to perform speaker recognition. The proposed method has more potential to classify speech signals in comparisons with other models. The method indicates an accuracy of 95.69%. Proposed work adopts the external quality metrics accuracy, sensitivity and specificity to compare with the existing methods.

Development of a Novel Hybrid Intelligence Approach for Landslide Spatial Prediction

We proposed an innovative hybrid intelligent approach, namely, the multiboost based naïve bayes trees (MBNBT) method for the spatial prediction of landslides in the Mu Cang Chai District of Yen Bai Province, Vietnam. The MBNBT, which is an ensemble of the multiboost (MB) and naïve bayes trees (NBT) base classifier, has rarely been applied for landslide susceptibility mapping around the world. For the modeling, we selected 248 landslide locations in the hilly terrain of the study area. Fifteen landslide conditioning factors were selected for the construction of the database based on the one-R attribute evaluation (ORAE) technique. Model validation was done using statistical metrics, namely, sensitivity, specificity, accuracy, mean absolute error (MAE), root mean square error (RMSE), and the area under the receiver operating characteristics curve (AUC). Performance of the hybrid model was evaluated and compared with popular soft computing benchmark models, namely, multiple perceptron neural network (MLPN), Support Vector Machines (SVM), and single NBT. Results indicated that the proposed MBNBT (AUC = 0.824) model outperformed the popular models, namely, the MLPN (AUC = 0.804), SVM (AUC = 0.804), and NBT (AUC = 0.800) models. Analysis of the model results also suggested that the MB meta classifier ensemble model could enhance the prediction power of the NBT model. Therefore, the MBNBT is a suitable method for the assessment of landslide susceptibility in landslide prone areas.

Deep Learning Application to Ensemble Learning—The Simple, but Effective, Approach to Sentiment Classifying

Sentiment analysis is an active research area in natural language processing. The task aims at identifying, extracting, and classifying sentiments from user texts in post blogs, product reviews, or social networks. In this paper, the ensemble learning model of sentiment classification is presented, also known as CEM (classifier ensemble model). The model contains various data feature types, including language features, sentiment shifting, and statistical techniques. A deep learning model is adopted with word embedding representation to address explicit, implicit, and abstract sentiment factors in textual data. The experiments conducted based on different real datasets found that our sentiment classification system is better than traditional machine learning techniques, such as Support Vector Machines and other ensemble learning systems, as well as the deep learning model, Long Short-Term Memory network, which has shown state-of-the-art results for sentiment analysis in almost corpuses. Our model’s distinguishing point consists in its effective application to different languages and different domains.

Detecting outliers in industrial systems using a hybrid ensemble scheme

The massive growth of process data in industrial systems has promoted the development of data-driven techniques, while the presence of outliers in process data always deteriorates the effectiveness. This paper focuses on detecting outliers in industrial systems under the assumption that no labeled training data are available. Our method is on the basis of ensemble learning, and the base learners include both one-class classifiers and multi-class classifiers. The core idea is that one-class classifier ensemble model is used to address the problem of missing label, and the usage of multi-class classifier ensemble model is to further improve its performance when outlier examples are available. The essential motivation for this proposal is that results from a classifier trained using only positive data will not be as good as the results using both positive and negative data. We investigate the performance of the proposed scheme with a series of experiments. Ten benchmark data sets and two real-world industrial systems are used, and the results approve the performance of our detection scheme.

Ensemble Classification Model for Diabetes Prediction in Data Mining

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Computed tomography angiography-based analysis of high-risk intracerebral haemorrhage patients by employing a mathematical model

BackgroundHaemorrhagic stroke accounts for approximately 31.52% of all stroke cases, and the most common origin is hypertension. However, little is known about the method to identify high-risk populations of hypertensive intracerebral haemorrhage.ResultsThe results showed that the angle between the middle cerebral artery and the internal carotid artery (AMIC), the distance between the beginning of the median artery and superior trunk (DMS), and the density (CT value) of the lenticulostriate artery (CTL) were statistically significant enough to cause intracerebral haemorrhage. In addition, we chose these three potential features for the ensemble learning classification model. Our developed ensemble-learning method outperforms not only previous work but also three other classic classification methods based on accuracy measurements.ConclusionsThe developed mathematical model in the present study is efficient in predicting the probability of intracerebral haemorrhage.

A novel ensemble classification model based on neural networks and a classifier optimisation technique for imbalanced credit risk evaluation

Significant research has been performed on credit risk evaluation, with many machine learning and data mining techniques being employed for financial decision-making. The back propagation (BP) neural network has been a popular choice for credit risk evaluation problems, but many studies have found classifier ensembles to be superior to single classifiers. In this paper, a novel ensemble model based on the synthetic minority over-sampling technique (SMOTE) and a classifier optimisation technique is proposed for personal credit risk evaluation. To mitigate the negative effects of imbalanced datasets on the performance of the credit evaluation model, the SMOTE technique is used to rebalance the target training dataset. The particle swarm optimisation (PSO) algorithm is employed to search for the best-connected weights and deviations in the BP neural networks. Based on the optimised BP neural network classifiers, an ensemble model is developed that combines the AdaBoost approach with the base classifiers. To ensure that the proposed model provides accurate and stable performance, we thoroughly explore and discuss the optimal parameters for the ensemble classification model. Finally, the proposed ensemble model is tested on German and Australian real-world imbalanced datasets. The results demonstrate that this model is more effective at processing credit data problems compared to the other classification models examined in this study.

A novel ensemble support vector machine model for land cover classification

Nowadays, support vector machines are widely applied to land cover classification although this method is sensitive to parameter selection and noise samples. AdaBoost is an effective approach to fi...

Signal Processing of Multimodal Mobile Lifelogging Data Towards Detecting Stress in Real-World Driving

Stress is a negative emotion that is part of everyday life. However, frequent episodes or prolonged periods of stress can be detrimental to long-term health. Nevertheless, developing self-awareness is an important aspect of fostering effective ways to self-regulate these experiences. Mobile lifelogging systems provide an ideal platform to support self-regulation of stress by raising awareness of negative emotional states via continuous recording of psychophysiological and behavioral data. However, obtaining meaningful information from large volumes of raw data represents a significant challenge because these data must be accurately quantified and processed before stress can be detected. This work describes a set of algorithms designed to process multiple streams of lifelogging data for stress detection in the context of real world driving. Two data collection exercises have been performed where multimodal data, including raw cardiovascular activity and driving information, were collected from 21 people during daily commuter journeys. Our approach enabled us to 1) pre-process raw physiological data to calculate valid measures of heart rate variability, a significant marker of stress, 2) identify/correct artefacts in the raw physiological data, and 3) provide a comparison between several classifiers for detecting stress. Results were positive and ensemble classification models provided a maximum accuracy of 86.9 percent for binary detection of stress in the real-world.

Ensemble semi-supervised Fisher discriminant analysis model for fault classification in industrial processes

In this paper, an ensemble form of the semi-supervised Fisher Discriminant Analysis (FDA) model is developed for fault classification in industrial processes. This method uses the K Nearest Neighbor (KNN) algorithm to merge the metric level outputs, which are obtained by the sub-classifiers in the ensemble model, to get the final classification result. An adaptive form is further proposed to improve the classification performance by putting forward to a new method of weight adjustment. While semi-supervised learning can generate a better model by exploiting additional information contained in unlabeled data, ensemble learning achieves the promotion of algorithm robustness by integrating a series of weak learners. In addition, the property of diversity in ensemble learning can be boosted by incorporating different unlabeled data to different weak learners. Therefore, the combination of those two methods can achieve great generalization for the fault classification model. The performances of two proposed methods are evaluated through an industrial benchmark process.

Outlier detection based on a dynamic ensemble model: Applied to process monitoring

This paper focuses on outlier detection and its application to process monitoring. The main contribution is that we propose a dynamic ensemble detection model, of which one-class classifiers are used as base learners. Developing a dynamic ensemble model for one-class classification is challenging due to the absence of labeled training samples. To this end, we propose a procedure that can generate pseudo outliers, prior to which we transform outputs of all base classifiers to the form of probability. Then we use a probabilistic model to evaluate competence of all base classifiers. Friedman test along with Nemenyi test are used together to construct a switching mechanism. This is used for determining whether one classifier should be nominated to make the decision or a fusion method should be applied instead. Extensive experiments are carried out on 20 data sets and an industrial application to verify the effectiveness of the proposed method.

Kalman Filter-based Heuristic Ensemble (KFHE): A new perspective on multi-class ensemble classification using Kalman filters

This paper introduces a new perspective on multi-class ensemble classification that considers training an ensemble as a state estimation problem. The new perspective considers the final ensemble classifier model as a static state, which can be estimated using a Kalman filter that combines noisy estimates made by individual classifier models. A new algorithm based on this perspective, the Kalman Filter-based Heuristic Ensemble (KFHE), is also presented in this paper which shows the practical applicability of the new perspective. Experiments performed on 30 datasets compare KFHE with state-of-the-art multi-class ensemble classification algorithms and show the potential and effectiveness of the new perspective and algorithm. Existing ensemble approaches trade off classification accuracy against robustness to class label noise, but KFHE is shown to be significantly better or at least as good as the state-of-the-art algorithms for datasets both with and without class label noise.

Hybridization of PSO-ABC Based Ensemble Classification Model for High Dimensional Medical Datasets

Hybridization of PSO-ABC Based Ensemble Classification Model for High Dimensional Medical Datasets

Enhanced Classification Models for Iris Dataset

Data mining and machine learning are both useful tools in the field of data analysis. Classification algorithm is one of the most important techniques in data mining, therefore, it is of great significance to select suitable classification models with high efficiency to show superiority when solving classification problems with the use of Iris data. With this goal, a decision tree induction algorithm, namely graftedTree, is proposed to build randomized decision trees. Randomization is explicitly introduced into this algorithm, such that applying the algorithm several times on the same training data results in diversified models. An ensemble classification model is constructed using multiple randomized decision trees via majority voting. In order to show the performance of different models in classification, we propose the usage of precision, recall, F-Measure, the area under the ROC curve (AUC) and Gini coefficient as evaluation indexes of the classifying performance on the Iris dataset. The experimental results show that classification with Random Forests model has generally better performance than that with the Boosting Tree model and other three popular algorithms: KNN, SMO and Simple Cart. However, the Gini coefficient of the Random Forests model shows that it gets less pure training set than other models. The new GraftedTrees model inherits the advantages of Random Forest and further employs random mixture of two interchangeable node splitting rule inductions with the aim to obtain higher computational efficiency and better performance in terms of accuracy. With its superiority, it is expected that the new GraftedTrees model can prove to be the most powerful model with better performance in classification in the near future.