Classifier Combination Approach Research Articles

Language identification from multi-lingual scene text images: a CNN based classifier ensemble approach

Since the past two decades, detecting text regions in complex natural images has emerged as a problem of great interest for the research fraternity. This is because these regions of interest serve as source of information that can be utilized for various purposes. However, these regions may contain texts in multiple languages. Hence, identifying the corresponding language of a detected scene text becomes important for further information processing. Language identification of the text, captured in a wild, is an extremely challenging research field in the domain of scene text recognition. In this paper, a deep learning-based classifier combination approach is proposed to solve the problem of language identification from multi-lingual scene text images. In this work, a minimalist Convolutional Neural Network architecture is used as the base model. Five variants of an input image—three different channels of RGB color model (i.e. R for red, G for green and B for blue) along with RGB itself, and grayscale image are passed through the base model separately. The outcomes of these five models are combined using the classifier combination approaches based on sum rule and product rule. Performances of the proposed model have been evaluated on some standard datasets like KAIST and MLe2e as well as in-house multi-lingual scent text dataset. From the experimental results, it has been observed that the proposed model outperforms some state-of-the-art methods considered here for comparison.

A two-stage approach towards protein secondary structure classification

Protein secondary structure (PSS) describes the local folded structures which get formed inside a polypeptide due to interactions among atoms of the backbone. Generally, globular proteins are divided into four classes, namely all-α, all-β, α + β, andα/β. As nearly 90% of proteins fall into the said four classes, these are mostly considered for the purpose of computational classification of proteins. Classification of PSS is important for different biological functions that include protein fold recognition, tertiary structure prediction, prediction of DNA-binding sites, and reduction of the conformation search space among others. In this paper, we have proposed a machine learning-based model for secondary structure classification of proteins into four classes: all-α, all-β, α + β, andα/β. In doing so, we have considered both sequence-based and structure-based features. At first, mutual information (MI), a filter-based feature selection method, is used to remove the redundant features, and then these selected features are used to train three different classifiers-random forest, K-nearest neighbor (KNN), and multi-layer perceptron (MLP). After that, some standard classifier combination approaches are applied to integrate the decision made by the said classifiers and it has been found that weighted product rule performs the best among all. The overall accuracies obtained using the proposed model on the four standard datasets, namely 640, 1189, 25pdb, and fc699 are 86.89%, 92.93%, 91.38%, and 94.87% respectively. The proposed model outperforms some state-of-the-art methods considered here for comparison. Significantly high classification accuracy produced by our proposed model on four datasets is attributed to the development of a comprehensive feature set (by eliminating redundant features through feature selection technique) which is then passed through an ensemble consists of three different classifiers. Assigning different weights to the outcome of different classifiers thus proved to be useful in designing the model for predicting the secondary structure of proteins based on its sequence-based and structure-based features. Graphical abstract.

Classifier combination approach for question classification for Bengali question answering system

Question classification (QC) is a prime constituent of an automated question answering system. The work presented here demonstrates that a combination of multiple models achieves better classification performance than those obtained with existing individual models for the QC task in Bengali. We have exploited state-of-the-art multiple model combination techniques, i.e., ensemble, stacking and voting, to increase QC accuracy. Lexical, syntactic and semantic features of Bengali questions are used for four well-known classifiers, namely Naïve Bayes, kernel Naïve Bayes, Rule Induction and Decision Tree, which serve as our base learners. Single-layer question-class taxonomy with 8 coarse-grained classes is extended to two-layer taxonomy by adding 69 fine-grained classes. We carried out the experiments both on single-layer and two-layer taxonomies. Experimental results confirmed that classifier combination approaches outperform single-classifier classification approaches by 4.02% for coarse-grained question classes. Overall, the stacking approach produces the best results for fine-grained classification and achieves 87.79% of accuracy. The approach presented here could be used in other Indo-Aryan or Indic languages to develop a question answering system.

EEG based word familiarity using features and frequency bands combination

The appearance of unknown words often disturbs communication and reading. This paper focuses on detecting those words which are unfamiliar to the users while reading using Electroencephalography (EEG). In particular, a word-familiarity prediction approach based on EEG signals from the user’s brain waves has been developed. Word-familiarity refers whether a user is familiar with the word or not while reading the text. The recorded signals have been processed using Wavelet decomposition technique and four features, namely mean, standard deviation, Root Mean Square (RMS) and power have been computed from beta and gamma frequency bands. The prediction of word-familiarity has been performed using Random Forest (RF) classifier. Next, a decision fusion approach has also been used to boost the prediction performance. A dictionary based pop-up window has been developed to provide the meaning of the word when a user is found to be unfamiliar with the text. EEG dataset of 12 users has been developed while they are reading 25 words. The results show that the characteristics of brain waves at the time of unknown word perception can be detected, i.e., when the user comes across an unknown word then the pattern of his brain waves comes out to be totally different from the pattern when he is familiar with the word. Similarly, the pattern of brain waves has been found to be different for the word for which the user has confusion in his mind. An accuracy of 82% has been recorded using the proposed classifier combination approach. Finally, a comparative study with other popular classification technique is also discussed.

Feature selection based classifier combination approach for handwritten Devanagari numeral recognition

In this paper a method for the recognition of handwritten Hindi numerals is presented. The paper is reporting the effectiveness of the proposed approach, which is utilizing the feature selection based on the Information theory measures. The Multilayer Perceptron (MLP) based classifier combination is used along with feature selection using two criterion functions: (i) Maximum relevance minimum redundancy and (ii) Conditional mutual information maximization. Conditional mutual information based feature selection when driving the ensemble of classifier produces improved recognition results for most of the benchmarking datasets. The improvement is also observed with maximum relevance minimum redundancy based feature selection when used in combination with ensemble of classifiers. The main contribution of the proposed method is that, the method gives quite efficient results utilizing only 10% patterns of the available dataset.

Decision Fusion of Multiple Classifiers for Coronary Plaque Characterization from IVUS Images

Vascular tissue characterization is of great importance concerning the possibility of an Acute Cardiac Syndrome (ACS). Gray-scale intravascular ultrasound (IVUS) is a powerful tomographic modality providing a thorough visualization of coronary arteries. Among the existing methods, virtual histology (VH) is the most popular and clinically available technique for plaque component analysis, it suffers however from a poor longitudinal resolution. In order to surmount this demerit, a new image-based methodology for plaque assessment is suggested here that differentiates tissue components into four classes: calcium, necrotic core, fibrous and fibro-lipid. A rich set of five textural feature families are extracted from IVUS images, computed at different scales. The main contribution of this paper is that tissue classification is accomplished using the principles of multiple classifiers combination approach. At the first stage, an ensemble of base SVM classifiers is constructed from each feature family, separately. The fuzzy outputs of the individual classifiers are then aggregated to provide the final fused results. We investigate four efficient decision fusion schemes of the literature and the SVM fuser. Extensive experimentation is carried out to highlight the merits of the suggested schemes against single SVM classifiers that use reduced feature subsets obtained after feature selection or the entire feature space. The analysis demonstrates that the decision fusion techniques offer improved classification accuracies, compared to single SVM classifiers and existing methods in IVUS imaging. In addition, the method provides accurate assessments of plaque composition in IVUS images.

Combination of sequential class distributions from multiple channels using Markov fusion networks

The recognition of patterns in real-time scenarios has become an important trend in the field of multi-modal user interfaces in human computer interaction. Cognitive technical systems aim to improve the human computer interaction by means of recognizing the situative context, e.g. by activity recognition (Ahad et al. in IEEE, 1896–1901, 2008), or by estimating the affective state (Zeng et al., IEEE Trans Pattern Anal Mach Intell 31(1):39–58, 2009) of the human dialogue partner. Classifier systems developed for such applications must operate on multiple modalities and must integrate the available decisions over large time periods. We address this topic by introducing the Markov fusion network (MFN) which is a novel classifier combination approach, for the integration of multi-class and multi-modal decisions continuously over time. The MFN combines results while meeting real-time requirements, weighting decisions of the modalities dynamically, and dealing with sensor failures. The proposed MFN has been evaluated in two empirical studies: the recognition of objects involved in human activities, and the recognition of emotions where we successfully demonstrate its outstanding performance. Furthermore, we show how the MFN can be applied in a variety of different architectures and the several options to configure the model in order to meet the demands of a distinct problem.

Ant Colony Optimization Algorithm for Interpretable Bayesian Classifiers Combination: Application to Medical Predictions

Prediction and classification techniques have been well studied by machine learning researchers and developed for several real-word problems. However, the level of acceptance and success of prediction models are still below expectation due to some difficulties such as the low performance of prediction models when they are applied in different environments. Such a problem has been addressed by many researchers, mainly from the machine learning community. A second problem, principally raised by model users in different communities, such as managers, economists, engineers, biologists, and medical practitioners, etc., is the prediction models’ interpretability. The latter is the ability of a model to explain its predictions and exhibit the causality relationships between the inputs and the outputs. In the case of classification, a successful way to alleviate the low performance is to use ensemble classiers. It is an intuitive strategy to activate collaboration between different classifiers towards a better performance than individual classier. Unfortunately, ensemble classifiers method do not take into account the interpretability of the final classification outcome. It even worsens the original interpretability of the individual classifiers. In this paper we propose a novel implementation of classifiers combination approach that does not only promote the overall performance but also preserves the interpretability of the resulting model. We propose a solution based on Ant Colony Optimization and tailored for the case of Bayesian classifiers. We validate our proposed solution with case studies from medical domain namely, heart disease and Cardiotography-based predictions, problems where interpretability is critical to make appropriate clinical decisions.AvailabilityThe datasets, Prediction Models and software tool together with supplementary materials are available at http://faculty.uaeu.ac.ae/salahb/ACO4BC.htm.

Computer generated images vs. digital photographs: A synergetic feature and classifier combination approach

The development of powerful and low-cost hardware devices allied with great advances on content editing and authoring tools have promoted the creation of computer generated images (CG) to a degree of unrivaled realism. Differentiating a photo-realistic computer generated image from a real photograph (PG) can be a difficult task to naked eyes. Digital forensics techniques can play a significant role in this task. As a matter of fact, important research has been made by the scientific community in this regard. Most of the approaches focus on single image features aiming at detecting differences between real and computer generated images. However, with the current technology advances, there is no universal image characterization technique that completely solves this problem. In our work, we (1) present a complete study of several CG versus PG approaches; (2) create a large and heterogeneous dataset to be used as a training and validation database; (3) implement representative methods of the literature; and (4) devise automatic ways for combining the best approaches. We compared the implemented methods using the same validation environment showing their pros and cons with a common benchmark protocol. We collected approximately 4850 photographs and 4850 CGs with large diversity of image content and quality. We implemented a total of 13 methods. Results show that this set of methods can achieve up to 93% of accuracy when used without any form of machine learning fusion. The same methods, when combined through the implemented fusion schemes, can achieve an accuracy rate of 97%, representing a reduction of 57% of the classification error over the best individual result.

FACE RECOGNITION BASED ON GPPBTF AND LBP WITH CLASSIFIER FUSION

Robust face recognition is a challenging problem, due to facial appearance variations in illumination, pose, expression, aging, partial occlusions and other changes. This paper proposes a novel face recognition approach, where face images are represented by Gabor pixel-pattern-based texture feature (GPPBTF) and local binary pattern (LBP), and null pace-based kernel Fisher discriminant analysis (NKFDA) is applied to the two features independently to obtain two recognition results which are eventually combined together for a final identification. To get GPPBTF, we first transform an image into Gabor magnitude maps of different orientations and scales, and then use pixel-pattern-based texture feature to extract texture features from Gabor maps. In order to improve the final performance of the classification, this paper proposes a multiple NKFDA classifiers combination approach. Extensive experiments on FERET face database demonstrate that the proposed method not only greatly reduces the dimensionality of face representation, but also achieves more robust result and higher recognition accuracy.

Maritime ATR using Classifier Combination and High Resolution Range Profiles

A maritime automatic target recognition system is developed that performs ship classification using one-dimensional high resolution range profiles. Five physically based features are defined and are extracted from both VV and HH polarizations resulting in a 10-dimensional feature vector. A nonlinear classifier combination approach involving a neural network combiner along with three individual classifiers (Bayes, nearest neighbor, and neural network) is proposed. A decision confidence measure based on the classifier discriminants is developed using a nonparametric estimation approach. The confidence measure enables the system to reject samples that have a low decision confidence. The performance of the proposed neural network based combination is compared with individual classifiers and a number of other combination rules. The results show that this combination can provide high recognition accuracy along with a high probability of declaration. The performance in the presence of samples from not-before-seen classes is also investigated. A new nearest neighbor confidence thresholding approach is developed to aid the neural network combiner in rejecting these samples.

A Novel Multi-Source Classification Approach Based on Evidence Combination

In this paper, a novel classifiers combination approach based on evidence theory is proposed. In classifiers combination, the diversity among member classifiers is known to be a necessary condition for improving classification performance. In our implementation of classifiers combination, we generate the diversity by utilizing different feature spaces to implement member classifiers. And by using different types of classifiers selectively and dynamically according to their expert environments, the diversity can be further enlarged. Thus better combined classification performance can be achieved. In the experiments, the approach proposed shows its efficacy and rationality.

A dynamic classifier selection and combination approach to image region labelling

In this paper we propose a ‘bank of classifiers’ approach to image region labelling and evaluate dynamic classifier selection and classifier combination approaches against a baseline approach that works with a single best classifier chosen using a validation set. In this analysis, image segmentation, feature extraction, and classification are treated as three separate steps of analysis. The classifiers used are each trained with a different texture feature representation of training images. The paper proposes a new knowledge-based predictive approach based on estimating the Mahalanobis distance between test sample feature values and the corresponding probability distribution function from training data that selectively triggers classifiers. This approach is shown to perform better than probability-based classifier combination (all classifiers are triggered but their decisions are fused with combination rules), and single classifier, respectively, based on classification rates and confusion matrices. The experiments are performed on the natural scene analysis application.

Face recognition based on a group decision-making combination approach

This paper proposes a novel and real-time classifiers combination approach, group decision-making combination (GDC) approach, which can dynamically select the classifiers and perform linear combination. We also prove that the orthogonal wavelet transform can be regarded as an effective image's preprocessing tool adapted to classifiers combination. GDC has been successfully used for face recognition, which can improve on the recognition rate for the algebraic features. Experiment results also show that it is superior to the conventional combination method, majority voting method.

Combining Classifiers for word sense disambiguation

Classifier combination is an effective and broadly useful method of improving system performance. This article investigates in depth a large number of both well-established and novel classifier combination approaches for the word sense disambiguation task, studied over a diverse classifier pool which includes feature-enhanced Naïve Bayes, Cosine, Decision List, Transformation-based Learning and MMVC classifiers. Each classifier has access to the same rich feature space, comprised of distance weighted bag-of-lemmas, local ngram context and specific syntactic relations, such as Verb-Object and Noun-Modifier. This study examines several key issues in system combination for the word sense disambiguation task, ranging from algorithmic structure to parameter estimation. Experiments using the standard SENSEVAL2 lexical-sample data sets in four languages (English, Spanish, Swedish and Basque) demonstrate that the combination system obtains a significantly lower error rate when compared with other systems participating in the SENSEVAL2 exercise, yielding state-of-the-art performance on these data sets.

Classification of Text Documents

The exponential growth of the internet has led to a great deal of interest in developing useful and efficient tools and software to assist users in searching the Web. Document retrieval, categorization, routing and filtering can all be formulated as classification problems. However, the complexity of natural languages and the extremely high dimensionality of the feature space of documents have made this classification problem very difficult. We investigate four different methods for document classification: the naive Bayes classifier, the nearest neighbour classifier, decision trees and a subspace method. These were applied to seven-class Yahoo news groups (business, entertainment, health, international, politics, sports and technology) individually and in combination. We studied three classifier combination approaches: simple voting, dynamic classifier selection and adaptive classifier combination. Our experimental results indicate that the naive Bayes classifier and the subspace method outperform the other two classifiers on our data sets. Combinations of multiple classifiers did not always improve the classification accuracy compared to the best individual classifier. Among the three different combination approaches, our adaptive classifier combination method introduced here performed the best. The best classification accuracy that we are able to achieve on this seven-class problem is approximately 83%, which is comparable to the performance of other similar studies. However, the classification problem considered here is more difficult because the pattern classes used in our experiments have a large overlap of words in their corresponding documents.