Classifier Combination Methods Research Articles

Verification of unemployment benefits’ claims using Classifier Combination method

Verification of unemployment benefits’ claims using Classifier Combination method

Facial Expression Recognition Using Convolution Neural Network Fusion and Texture Descriptors Representation

Facial expression recognition is an interesting research direction of pattern recognition and computer vision. It has been increasingly used in artificial intelligence, human–computer interaction and security monitoring. In recent years, Convolution Neural Network (CNN) as a deep learning technique and multiple classifier combination method has been applied to gain accurate results in classifying face expressions. In this paper, we propose a multimodal classification approach based on a local texture descriptor representation and a combination of CNN to recognize facial expression. Initially, in order to reduce the influence of redundant information, the preprocessing stage is performed using face detection, face image cropping and texture descriptors of Local Binary Pattern (LBP), Local Gradient Code (LGC), Local Directional Pattern (LDP) and Gradient Direction Pattern (GDP) calculation. Second, we construct a cascade CNN architecture using the multimodal data of each descriptor (CNNLBP, CNNLGC, CNNGDP and CNNLDP) to extract facial features and classify emotions. Finally, we apply aggregation techniques (sum and product rule) for each modality to combine the four multimodal outputs and thus obtain the final decision of our system. Experimental results using CK[Formula: see text] and JAFFE database show that the proposed multimodal classification system achieves superior recognition performance compared to the existing studies with classification accuracy of 97, 93% and 94, 45%, respectively.

Computer Assisted Grade Diagnosis Using Feature Fusion and Network Algorithm Flow

Diabetic retinopathy is one of the important causes of visual impairment and blindness. In order to strengthen the early prevention and treatment of diabetic retinal diseases and prevent the occurrence of serious diseases, feature fusion diagnosis model that combines traditional features and depth features was proposed. In the diagnostic model, four kinds of traditional algorithms were used to extract imaging features, and VGG16 and VGG19 networks were used to extract deep features. Then k-nearest neighbor, random forest, support vector machine and extreme learning machine were used to grade the degree of diabetic retinopathy based on the fused feature vectors. The diagnostic model adopted the depth feature extracted by VGG19 network and the traditional feature fusion method to achieve the best grade effect on the support vector machine, with the accuracy of 84.44%, which was better than comparative feature fusion and classifier combination methods, and also performed well in the precision rate and recall rate. The diagnostic model based on the fusion of traditional features and depth features provides a new detection method for the grading prediction of diabetic retinopathy, and at the same time improves the accuracy of clinical diagnosis.

Photogram Classification-Based Emotion Recognition

This paper presents a method for facial emotion recognition based on parameterized photograms and machine learning techniques. Videos of people displaying emotions are parameterized by a facial feature-based emotional category association process to determine whether a given photogram expresses emotions by comparing the facial action units displayed with findings in the literature about facial emotion. To test the proposed approach, two strategies are adopted. First, photograms displaying emotions are gathered, and then different machine learning classifiers are applied to check the goodness of the obtained set of categorized emotional photograms. Second, classifiers trained on the sets of emotional photograms were then used to emotionally classify all the videos in each database, using all the photograms with no preprocessing or photogram selection. The presented method was tested using the OpenFace parameterizer with emotional videos gathered from Multimedia Understanding Facial Expression (MUG) and Cohn-Kanade (CK+) databases. The outcomes achieved for emotional photogram classification on the sets of emotional photograms reached maximums of 99.80% and 99.63% in the MUG and CK+ databases, respectively. The videos were classified using different voting strategies regarding the outcome of each photogram in the video with all the photogram emotion recognition classifiers obtained results reflecting recognition rates of 70.71% and 66.36% for the videos in MUG and CK+ databases, and reached up to 72.55% and 88.37% when classifier combination strategies were used. The work carried out opens the door to follow-up work concerning data preprocessing and the use of different classifier combination methods in facial emotion recognition.

Shape Adaptive Neighborhood Information-Based Semi-Supervised Learning for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is an important research topic in detailed analysis of the Earth’s surface. However, the performance of the classification is often hampered by the high-dimensionality features and limited training samples of the HSIs which has fostered research about semi-supervised learning (SSL). In this paper, we propose a shape adaptive neighborhood information (SANI) based SSL (SANI-SSL) method that takes full advantage of the adaptive spatial information to select valuable unlabeled samples in order to improve the classification ability. The improvement of the classification mainly relies on two aspects: (1) the improvement of the feature discriminability, which is accomplished by exploiting spectral-spatial information, and (2) the improvement of the training samples’ representativeness which is accomplished by exploiting the SANI for both labeled and unlabeled samples. First, the SANI of labeled samples is extracted, and the breaking ties (BT) method is used in order to select valuable unlabeled samples from the labeled samples’ neighborhood. Second, the SANI of unlabeled samples are also used to find more valuable samples, with the classifier combination method being used as a strategy to ensure confidence and the adaptive interval method used as a strategy to ensure informativeness. The experimental comparison results tested on three benchmark HSI datasets have demonstrated the significantly superior performance of our proposed method.

EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications

Human Activity Recognition (HAR) can be defined as the automatic prediction of the regular human activities performed in our day-to-day life, such as walking, running, cooking, performing office work, etc. It is truly beneficial in the field of medical care services, for example, personal health care assistants, old-age care services, maintaining patient records for future help, etc. Input data to a HAR system can be (a) videos or still images capturing human activities, or (b) time-series data of human body movements while performing the activities taken from sensors in the smart devices like accelerometer, gyroscope, etc. In this work, we mainly focus on the second category of the input data. Here, we propose an ensemble of three classification models, namely CNN-Net, Encoded-Net, and CNN-LSTM, which is named as EnsemConvNet. Each of these classification models is built upon simple 1D Convolutional Neural Network (CNN) but differs in terms of the number of dense layers, kernel size used along with other key differences in the architecture. Each model accepts the time series data as a 2D matrix by taking a window of data at a time in order to infer information, which ultimately predicts the type of human activity. Classification outcome of the EnsemConvNet model is decided using various classifier combination methods that include majority voting, sum rule, product rule, and a score fusion approach called adaptive weighted approach. Three benchmark datasets, namely WISDM activity prediction, UniMiB SHAR, MobiAct, are used for evaluating our proposed model. We have compared our EnsemConvNet model with some existing deep learning models such as Multi Headed CNN, hybrid of CNN, and Long Short Term Memory (LSTM) models. The results obtained here establish the supremacy of the EnsemConvNet model over the other mentioned models.

Specific Emitter Identification via Bispectrum-Radon Transform and Hybrid Deep Model

Specific emitter identification is a technique that distinguishes different emitters using radio fingerprints. Feature extraction and classifier selection are critical factors affecting SEI performance. In this paper, we propose an SEI method using the Bispectrum-Radon transform (BRT) and a hybrid deep model. We propose BRT to characterize the unintentional modulation of pulses due to the superiority of bispectrum distributions in characterizing nonlinear features of signals. We then apply a hybrid deep model based on denoising autoencoders and a deep belief network to perform further deep feature extraction and discriminative identification. We design an automatic dependent surveillance-broadcast signal acquisition system to capture signals and to build dataset for validating our proposed SEI method. Theoretical analysis and experimental results show that the BRT feature outperformed traditional features in characterizing UMOP, and our proposed SEI method outperformed other feature and classifier combination methods.

Using Social Network Information in Community-Based Bayesian Truth Discovery

We investigate the problem of truth discovery based on opinions from multiple agents (who may be unreliable or biased) that form a social network. We consider the case where agents' reliabilities or biases are correlated if they belong to the same community, which defines a group of agents with similar opinions regarding a particular event. An agent can belong to different communities for different events, and these communities are unknown a priori. We incorporate knowledge of the agents' social network in our truth discovery framework and develop Laplace variational inference methods to estimate agents' reliabilities, communities, and the event states. We also develop a stochastic variational inference method to scale our model to large social networks. Simulations and experiments on real data suggest that when observations are sparse, our proposed methods perform better than several other inference methods, including majority voting, TruthFinder, AccuSim, the Confidence-Aware Truth Discovery method, the Bayesian classifier combination (BCC) method, and the community BCC method.

Combination of Classifiers With Optimal Weight Based on Evidential Reasoning

In pattern classification problem, different classifiers learnt using different training data can provide more or less complementary knowledge, and the combination of classifiers is expected to improve the classification accuracy. Evidential reasoning (ER) provides an efficient framework to represent and combine the imprecise and uncertain informations. In this paper, we want to focus on the weighted combination of classifiers based on ER. Because each classifier may have different performance on the given dataset, the classifiers to combine are considered with different weights. A new weighted classifier combination method is proposed based on ER to enhance the classification accuracy. The optimal weighting factors of classifiers are obtained by minimizing the distances between fusion results obtained by Dempster's rule and the target output in training data space to fully take advantage of the complementarity of the classifiers. A confusion matrix is additionally introduced to characterize the probability of the object belonging to one class but classified to another class by the fusion result. This matrix is also optimized using training data jointly with classifier weight, and it is used to modify the fusion result to make it as close as possible to truth. Moreover, the training patterns are considered with different weights for the parameter optimization in classifier fusion, and the patterns hard to classify are committed with bigger weight than the ones easy to deal with. The pattern weight and the other parameters (i.e., classifier weight and confusion matrix) are iteratively optimized for obtaining the highest classification accuracy. A cautious decision making strategy is introduced to reduce the errors, and the pattern hard to classify will be cautiously committed to a set of classes, because the partial imprecision of decision is considered better than error in certain case. The effectiveness of the proposed method is demonstrated with various real datasets from UCI repository, and its performances are compared with those of other classical methods.

Classifier Combination and Selection Methods at the Corporate Bankruptcy Prediction

The author of this article decided to investigate the efficiency of using classifier combination and selection methods at the prediction of corporate bankruptcy in Poland. The study allowed us to compare the predictions from 4 individual classifiers: linear discriminant analysis, logistic regression, artificial neural network and random forest with the predictions from 9 classifier combination and selection methods. Moreover, the author analyzed the influence of the number of variables in models on performances of the correct classification for all considered methods.

EXTRACTING URBAN GROUND OBJECT INFORMATION FROM IMAGES AND LiDAR DATA

Abstract. To deal with the problem of urban ground object information extraction, the paper proposes an object-oriented classification method using aerial image and LiDAR data. Firstly, we select the optimal segmentation scales of different ground objects and synthesize them to get accurate object boundaries. Then, this paper uses ReliefF algorithm to select the optimal feature combination and eliminate the Hughes phenomenon. Eventually, the multiple classifier combination method is applied to get the outcome of the classification. In order to validate the feasible of this method, this paper selects two experimental regions in Stuttgart and Germany (Region A and B, covers 0.21 km2 and 1.1 km2 respectively). The aim of the first experiment on the Region A is to get the optimal segmentation scales and classification features. The overall accuracy of the classification reaches to 93.3 %. The purpose of the experiment on region B is to validate the application-ability of this method for a large area, which is turned out to be reaches 88.4 % overall accuracy. In the end of this paper, the conclusion shows that the proposed method can be performed accurately and efficiently in terms of urban ground information extraction and be of high application value.

EXTRACTING URBAN GROUND OBJECT INFORMATION FROM IMAGES AND LiDAR DATA

To deal with the problem of urban ground object information extraction, the paper proposes an object-oriented classification method using aerial image and LiDAR data. Firstly, we select the optimal segmentation scales of different ground objects and synthesize them to get accurate object boundaries. Then, this paper uses ReliefF algorithm to select the optimal feature combination and eliminate the Hughes phenomenon. Eventually, the multiple classifier combination method is applied to get the outcome of the classification. In order to validate the feasible of this method, this paper selects two experimental regions in Stuttgart and Germany (Region A and B, covers 0.21 km<sup>2</sup> and 1.1 km<sup>2</sup> respectively). The aim of the first experiment on the Region A is to get the optimal segmentation scales and classification features. The overall accuracy of the classification reaches to 93.3 %. The purpose of the experiment on region B is to validate the application-ability of this method for a large area, which is turned out to be reaches 88.4 % overall accuracy. In the end of this paper, the conclusion shows that the proposed method can be performed accurately and efficiently in terms of urban ground information extraction and be of high application value.

INTEGRATION OF MULTIPLE CLASSIFIERS FOR COMPUTERIZED DETECTION OF LUNG NODULES IN CT

The purpose of this study was to investigate the usefulness of various classifier combination methods for improving the performance of a computer-aided diagnosis (CAD) system for pulmonary nodule detection in computed tomography (CT). We employed 85 CT scans with 110 nodules in the publicly available Lung Image Database Consortium (LIDC) dataset. We first applied our CAD scheme trained previously to the LIDC cases for identifying initial nodule candidates, and extracting 18 features for each nodule candidate. We used eight individual classifiers for false positives (FPs) reduction, including linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), Naïve Bayes, simple logistic, artificial neural network (ANN) and support vector machines (SVMs) with three different kernels. Five classifier combination methods were then employed to integrate the outputs of the eight individual classifiers for improving detection performance. The five combination methods included two supervised (a likelihood ratio (LR) method and a probability method based on the output scores of the eight individual classifiers) and three unsupervised ones (the sum, the product and the majority voting of the output scores from the eight individual classifiers). Leave-one-case-out approach was employed to train and test individual classifiers and supervised combination methods. At a sensitivity of 80%, the numbers of FPs per CT scan for the eight individual classifiers were 6.1 for LDA, 19.9 for QDA, 10.8 for Naïve Bayes, 8.4 for simple logistic, 8.6 for ANN, 23.7 for SVM-dot, 17.0 for SVM-poly, and 23.4 for SVM-anova; the numbers of FPs per CT scan for the five combination methods were 3.3 for the majority voting method, 5.0 for the sum, 4.6 for the product, 65.7 for the LR and 3.9 for the probability method. Compared to the best individual classifier, the majority voting method reduced 45% of FPs at 80% sensitivity. The performance of our CAD can be improved by combining multiple classifiers. The majority voting method achieved higher performance levels than other combination methods and all individual classifiers.

A Novel Semi-Supervised Land Cover Classification Technique of Remotely Sensed Images

This research article addresses the problem of land-cover classification from the multi-spectral remotely sensed images using a novel self-training based semi-supervised learning (SSL) technique. The proposed system, instead of using a single classifier, builds an ensemble of classifiers with the hope that the ensemble system will have a lesser generalization error than any of its members. Each component classifier is trained independently using the proposed self-training approach on different training sub-sets and their predictions on test samples are combined using a new instant run-off voting (IRV) based classifier combination method. Each training subset consists of a few labeled samples and a comparatively large number of unlabeled data items. The cluster-and-label method for self-training has been adopted here considering the support vector machines (SVM) as the supervised learner and minimum spanning tree based clustering. A normalized histogram intersection kernel has been proposed which has shown to outperform the state-of-the-art kernel functions in terms of the SVM generalization accuracy. A novel cluster validity index specifically for graph based clustering has also been introduced to access the quality of the clustering. The proposed training method has the advantage that it can find classes for which the labeled training data are unavailable initially. A number of multi-spectral images have been considered for the experimental evaluation and a comparison with some of the well-established techniques from the literature has confirmed the superiority of the proposed classifier system.

A New Ensemble Method with Feature Space Partitioning for High-Dimensional Data Classification

Ensemble data mining methods, also known as classifier combination, are often used to improve the performance of classification. Various classifier combination methods such as bagging, boosting, and random forest have been devised and have received considerable attention in the past. However, data dimensionality increases rapidly day by day. Such a trend poses various challenges as these methods are not suitable to directly apply to high-dimensional datasets. In this paper, we propose an ensemble method for classification of high-dimensional data, with each classifier constructed from a different set of features determined by partitioning of redundant features. In our method, the redundancy of features is considered to divide the original feature space. Then, each generated feature subset is trained by a support vector machine, and the results of each classifier are combined by majority voting. The efficiency and effectiveness of our method are demonstrated through comparisons with other ensemble techniques, and the results show that our method outperforms other methods.

Automatic Prosodic Break Detection and Feature Analysis

Automatic prosodic break detection and annotation are important for both speech understanding and natural speech synthesis. In this paper, we discuss automatic prosodic break detection and feature analysis. The contributions of the paper are two aspects. One is that we use classifier combination method to detect Mandarin and English prosodic break using acoustic, lexical and syntactic evidence. Our proposed method achieves better performance on both the Mandarin prosodic annotation corpus — Annotated Speech Corpus of Chinese Discourse and the English prosodic annotation corpus — Boston University Radio News Corpus when compared with the baseline system and other researches' experimental results. The other is the feature analysis for prosodic break detection. The functions of different features, such as duration, pitch, energy, and intensity, are analyzed and compared in Mandarin and English prosodic break detection. Based on the feature analysis, we also verify some linguistic conclusions.

Multiple classifier system for EEG signal classification with application to brain–computer interfaces

In this paper, we demonstrate the use of a multiple classifier system for classification of electroencephalogram (EEG) signals. The main purpose of this paper is to apply several approaches to classify motor imageries originating from the brain in a more robust manner. For this study, dataset II from BCI competition III was used. To extract features from the brain signal, discrete wavelet transform decomposition was used. Then, several classic classifiers were implemented to be utilized in the multiple classifier system, which outperforms the reported results of other proposed methods on the dataset. Also, a variety of classifier combination methods along with genetic algorithm feature selection were evaluated and compared in order to diminish classification error. Our results suggest that an ensemble system can be employed to boost EEG classification accuracy.

SSC: a classifier combination method based on signal strength.

We propose a new classifier combination method, the signal strength-based combining (SSC) approach, to combine the outputs of multiple classifiers to support the decision-making process in classification tasks. As ensemble learning methods have attracted growing attention from both academia and industry recently, it is critical to understand the fundamental issues of the combining rule. Motivated by the signal strength concept, our proposed SSC algorithm can effectively integrate the individual vote from different classifiers in an ensemble learning system. Comparative studies of our method with nine major existing combining rules, namely, geometric average rule, arithmetic average rule, median value rule, majority voting rule, Borda count, max and min rule, weighted average, and weighted majority voting rules, is presented. Furthermore, we also discuss the relationship of the proposed method with respect to margin-based classifiers, including the boosting method (AdaBoost.M1 and AdaBoost.M2) and support vector machines by margin analysis. Detailed analyses of margin distribution graphs are presented to discuss the characteristics of the proposed method. Simulation results for various real-world datasets illustrate the effectiveness of the proposed method.

A comparative study of classifier combination applied to NLP tasks

The paper is devoted to a comparative study of classifier combination methods, which have been successfully applied to multiple tasks including Natural Language Processing (NLP) tasks. There is variety of classifier combination techniques and the major difficulty is to choose one that is the best fit for a particular task. In our study we explored the performance of a number of combination methods such as voting, Bayesian merging, behavior knowledge space, bagging, stacking, feature sub-spacing and cascading, for the part-of-speech tagging task using nine corpora in five languages. The results show that some methods that, currently, are not very popular could demonstrate much better performance. In addition, we learned how the corpus size and quality influence the combination methods performance. We also provide the results of applying the classifier combination methods to the other NLP tasks, such as name entity recognition and chunking. We believe that our study is the most exhaustive comparison made with combination methods applied to NLP tasks so far.

Generalized multiple kernel framework for multiclass geospatial objects detection in high-resolution remote sensing images

Multiclass geospatial objects detection within complex environments is a challenging problem in remote-sensing areas. In this paper we propose a novel, generalized kernel-based learning framework for the purpose of enhanced object detection. There are two novel areas. (1) Multisource information, including shape, feature points, and appearance, was extracted to give a comprehensive representation of the objects. We improved a shape descriptor and introduced a two-level spatial pyramid to represent appearance, both global and local. Therefore, basis kernels were formed, one for each feature. (2) In order to illustrate the effect of each kind of feature on each pyramid level, a generalized and weighted combination method was first used to combine all of the levels and then the features. The weights and the classifier model are based on the support vector machine framework for obtaining balance between all basis kernels. This classifier was transformed into a powerful detector by using a sliding window. The reported results are for the detection on high-resolution remote-sensing images. This study demonstrates that the proposed generalized and weighted combination of kernels can yield better performance compared with traditional single-kernel classifier and other combination methods.