Ensemble Of Binary Classifiers Research Articles

Ensembles of spectral-spatial convolutional neural network models for classifying soil types in hyperspectral images

The paper presents a study of various approaches to the classification of soil covers based on neural network algorithms using hyperspectral remote and proximal sensing of the Earth. The spectral distributions were recorded in the laboratory using an Offner imaging scanning hyperspectrometer. Spectral-spatial characteristics of nine soil samples from various parts of a farming land in the Samara region were experimentally studied. Using a method of energy dispersion microanalysis, the correspondence between the hyperspectral data and the chemical composition of the samples taken was established. Based on the data obtained, a neural network-aided classification of soil samples was implemented depending on the content of constituent elements such as carbon and calcium. A normalized spectral-spatial convolutional neural network was used as a classifier. As a result of the work, an approach to the classification of high-resolution hyper-spectral images based on the refinement of a multiclass convolutional neural network using an ensemble of binary classifiers is proposed. It is shown that the classification of soil samples by carbon and calcium content is carried out with an accuracy of 0.96.

Temporally sorting images from real-world events

• A data-driven method for chronologically sorting images of real-world events by analyzing the appearance of the scene. • The duration of the event is split into successive intervals and their relationships are modeled hierarchically. • An evaluation of several data augmentation techniques in scenarios with limited data availability. • A visualization of the concepts learned by the model to sort images offers additional insights about the problem. As smartphones become ubiquitous in modern life, every major event — from musical concerts to terrorist attempts — is massively captured by multiple devices and instantly uploaded to the Internet. Once shared through social media, the chronological order between available media pieces cannot be reliably recovered, hindering the understanding and reconstruction of that event. In this work, we propose data-driven methods for temporally sorting images originated from heterogeneous sources and captured from distinct angles, viewpoints, and moments. We model the chronological sorting task as an ensemble of binary classifiers whose answers are combined hierarchically to estimate an image’s temporal position within the duration of the event. We evaluate our method on images from the Notre-Dame Catedral fire and the Grenfell Tower fire events and discuss research challenges for analyzing data from real-world forensic events. Finally, we employ visualization techniques to understand what our models have learned, offering additional insights to the problem.

Detecting Aedes aegypti mosquitoes through audio classification with convolutional neural networks

The incidence of mosquito-borne diseases is significant in under-developed regions, mostly due to the lack of resources to implement aggressive control measurements against mosquito proliferation. A potential strategy to raise community awareness regarding mosquito proliferation is building a live map of mosquito incidences using smartphone apps and crowdsourcing. In this paper, we explore the possibility of identifying Aedes aegypti mosquitoes using machine learning techniques and audio analysis captured from commercially available smartphones. In summary, we downsampled Aedes aegypti wingbeat recordings and used them to train a convolutional neural network (CNN) through supervised learning. As a feature, we used the recording spectrogram to represent the mosquito wingbeat frequency over time visually. We trained and compared three classifiers: a binary, a multiclass, and an ensemble of binary classifiers. In our evaluation, the binary and ensemble models achieved accuracy of 97.65% (±0.55) and 94.56% (±0.77), respectively, whereas the multiclass had an accuracy of 78.12% (±2.09). The best sensitivity was observed in the ensemble approach (96.82% ± 1.62), followed by the multiclass for the particular case of Aedes aegypti (90.23% ± 3.83) and the binary (88.49% ± 6.68). The binary classifier and the multiclass classifier presented the best balance between precision and recall, with F1-measure close to 90%. Although the ensemble classifier achieved the lowest precision, thus impairing its F1-measure (79.95% ± 2.13), it was the most powerful classifier to detect Aedes aegypti in our dataset.

Combination of features through weighted ensembles for image classification

Image classification is a multi-class problem that is usually tackled with ensembles of binary classifiers. Furthermore, one of the most important challenges in this field is to find a set of highly discriminative image features for reaching a good performance in image classification. In this work we propose to use weighted ensembles as a method for feature combination. First, a set of binary classifiers are trained with a set of features and then, the scores are weighted with distances obtained from another set of feature vectors. We present two different approaches to weight the score vector: (1) directly multiplying each score by the weights and (2) fusing the scores values and the distances through a Neural Network. The experiments have shown that the proposed methodology improves classification accuracy of simple ensembles and even more it obtains similar classification accuracy than state-of-the-art methods, but using much less parameters.

An ensemble learning approach to lip-based biometric verification, with a dynamic selection of classifiers

Machine learning approaches are largely focused on pattern or object classification, where a combination of several classifier systems can be integrated to help generate an optimal or suboptimal classification decision. Multiple classification systems have been extensively developed because a committee of classifiers, also known as an ensemble, can outperform the ensemble’s individual members. In this paper, a classification method based on an ensemble of binary classifiers is proposed. Our strategy consists of two phases: (1) the competence of the base heterogeneous classifiers in a pool is determined, and (2) an ensemble is formed by combining those base classifiers with the greatest competences for the given input data.We have shown that the competence of the base classifiers can be successfully calculated even if the number of their learning examples was limited. Such a situation is particularly observed with biometric data. In this paper, we propose a new biometric data structure, the Sim coefficients, along with an efficient data processing technique involving a pool of competent classifiers chosen by dynamic selection.

Imbalanced classification of mental workload using a cost-sensitive majority weighted minority oversampling strategy

Identifying the temporal variations in mental workload level (MWL) is crucial for enhancing the safety of human---machine system operations, especially when there is cognitive overload or inattention of human operator. This paper proposed a cost-sensitive majority weighted minority oversampling strategy to address the imbalanced MWL data classification problem. Both the inter-class and intra-class imbalance problems are considered. For the former, imbalance ratio is defined to determine the number of the synthetic samples in the minority class. The latter problem is addressed by assigning different weights to borderline samples in the minority class based on the distance and density meaures of the sample distribution. Furthermore, multi-label classifier is designed based on an ensemble of binary classifiers. The results of analyzing 21 imbalanced UCI multi-class datasets showed that the proposed approach can effectively cope with the imbalanced classification problem in terms of several performance metrics including geometric mean (G-mean) and average accuracy (ACC). Moreover, the proposed approach was applied to the analysis of the EEG data of eight experimental participants subject to fluctuating levels of mental workload. The comparative results showed that the proposed method provides a competing alternative to several existing imbalanced learning algorithms and significantly outperforms the basic/referential method that ignores the imbalance nature of the dataset.

SU-F-R-02: Imaging-Genomics for Predicting GBM Molecular Subclasses and Survival

Purpose: Glioblastoma (GBM) is the most common and fatal primary intracranial neoplasm. GBM is categorized into five sub-classes (classical, G-CIMP, mesenchymal, proneural and neural) that can only be determined by an invasive brain biopsy followed by RNA and DNA methylation profiling. The goal of this study is to develop imaging features extracted from conventional MRI scans and an ensemble-classification method as a potential noninvasive method to predict the five molecular sub-classes and 12-month survival status. Methods: Tumors are segmented into 4 cardinal sub-volumes that are enhancing, non-enhancing, necrosis, and edema by BraTuma software and post-manual modification. After intensity normalization, mean, standard deviation, skewness, and kurtosis of voxel intensities and normalized volume of each of the 4 sub-volumes are calculated. The first four features are extracted from three different modalities (T1 pre & post contrast and T2 FLAIR) and the tumor texture is quantified. A feature selection phase (wrapper algorithm) is preceded the classification stage. Ensembles of binary classifiers with one versus rest strategy, along with a separate binary classifier are employed respectively for molecular classification and survival status. Two different binary classifiers are tested (SVM and K* IBL) and their confidences are used for tie-breaking technique. Results: With the select combination of features of the MRI modalities, the ROC area and accuracy of the prediction are 0.750 and 95% for classical, 1.000 and 100% for G-CIMP, 0.719 and 85% for mesenchymal, 0.875 and 95% for neural, 0.922 and 95% for proneural, and 0.990 and 95% for survival status respectively. Conclusion: The study proposes a set of three-dimensional and rotation invariant imaging features that represent texture and volumetric characteristics of GBM tumors. The results show the select features are predictive for five molecular sub-classes and survival status in GBM. These results indicate the feasibility of obviating the biopsy for obtaining genomic information of tumor.

Erotic Audio Recognition Using Heterogeneous Ensemble Classifiers

In this paper, we present a novel approach to the task of automatic adult video detection based on feature porno-sounds recognition using multiple feature vectors and an ensemble of binary classifiers. In this framework, firstly the audio track is extracted and segmented into equal segments from an unknown video. Then each segment is split into sequences of voiced and unvoiced unequal fragments. Multiple feature vectors are extracted for each voiced fragments. Components classifiers including Support Vector Machine (SVM) are trained based on such feature vectors. At the classification, the outputs provided by the component classifiers are combined through fusion rules to form a final output of the ensemble. According to the classification results of these fragments, each segment is classified as or Finally, based on these segments results, the audio track or further the unknown video is labeled erotic or natural. Online and off-line experiments show that the proposed approach yields high performance than using single classifiers.

Erotic Audio Recognition Using Heterogeneous Ensemble Classifiers

In this paper, we present a novel approach to the task of automatic adult video detection based on feature porno-sounds recognition using multiple feature vectors and an ensemble of binary classifiers. In this framework, firstly the audio track is extracted and segmented into equal segments from an unknown video. Then each segment is split into sequences of voiced and unvoiced unequal fragments. Multiple feature vectors are extracted for each voiced fragments. Components classifiers including Support Vector Machine (SVM) are trained based on such feature vectors. At the classification, the outputs provided by the component classifiers are combined through fusion rules to form a final output of the ensemble. According to the classification results of these fragments, each segment is classified as or Finally, based on these segments results, the audio track or further the unknown video is labeled erotic or natural. Online and off-line experiments show that the proposed approach yields high performance than using single classifiers.

A Novel Technique for Subpixel Image Classification Based on Support Vector Machine

This paper presents a novel support vector machine classifier designed for sub-pixel image classification (pixel/spectral unmixing). The proposed classifier generalizes the properties of SVMs to the identification and modeling of the abundances of classes in mixed pixels by using fuzzy logic. This results in the definition of a fuzzy-input fuzzy-output support vector machine (F2SVM) classifier that can: i) process fuzzy information given as input to the classification algorithm for modeling the sub-pixel information in the learning phase of the classifier, and ii) provide a fuzzy modeling of the classification results, allowing a relation many-to-one between classes and pixels. The presented binary F2SVM can address multicategory problems according to two strategies: the fuzzy one-against-all (FOAA) and the fuzzy one-against-one strategies (FOAO). These strategies generalize to the fuzzy case techniques based on ensembles of binary classifiers used for addressing multicategory problems in crisp classification problems. The effectiveness of the proposed F2SVM classifier is tested on three problems related to image classification in presence of mixed pixels having different characteristics. Experimental results confirm the validity of the proposed sub-pixel classification method.

Forests of nested dichotomies

Ensemble methods are often able to generate more accurate classifiers than the individual classifiers. In multiclass problems, it is possible to obtain an ensemble combining binary classifiers. It is sensible to use a multiclass method for constructing the binary classifiers, because the ensemble of binary classifiers can be more accurate than the individual multiclass classifier. Ensemble of nested dichotomies (END) is a method for dealing with multiclass classification problems using binary classifiers. A nested dichotomy organizes the classes in a tree, each internal node has a binary classifier. A set of classes can be organized in different ways in a nested dichotomy. An END is formed by several nested dichotomies. This paper studies the use of this method in conjunction with ensembles of decision trees (forests). Although forests methods are able to deal directly with several classes, their accuracies can be improved if they are used as base classifiers for ensembles of nested dichotomies. Moreover, the accuracies can be improved even more using forests of nested dichotomies, that is, ensemble methods that use as base classifiers a nested dichotomy of decision trees. The improvements over forests methods can be explained by the increased diversity of the base classifiers. The best overall results were obtained using MultiBoost with resampling.

Traffic sign recognition system with β -correction

Traffic sign classification represents a classical application of multi-object recognition processing in uncontrolled adverse environments. Lack of visibility, illumination changes, and partial occlusions are just a few problems. In this paper, we introduce a novel system for multi-class classification of traffic signs based on error correcting output codes (ECOC). ECOC is based on an ensemble of binary classifiers that are trained on bi-partition of classes. We classify a wide set of traffic signs types using robust error correcting codings. Moreover, we introduce the novel β-correction decoding strategy that outperforms the state-of-the-art decoding techniques, classifying a high number of classes with great success.

Spoken Language Recognition Using Ensemble Classifiers

In this paper, we study a novel approach to spoken language recognition using an ensemble of binary classifiers. In this framework, we begin by representing a speech utterance with a high-dimensional feature vector such as the phonotactic characteristics or the polynomial expansion of cepstral features. A binary classifier can be built based on such feature vectors. We adopt a distributed output coding strategy in ensemble classifier design, where we decompose a multiclass language recognition problem into many binary classification tasks, each of which addresses a language recognition subtask by using a component classifier. Then, we combine the results of the component classifiers to form an output code as a hypothesized solution to the overall language recognition problem. In this way, we effectively project high-dimensional feature vectors into a tractable low-dimensional space, yet maintaining language discriminative characteristics of the spoken utterances. By fusing the output codes from both phonotactic features and cepstral features, we achieve equal-error-rates of 1.38% and 3.20% for 30-s trials on the 2003 and 2005 NIST language recognition evaluation databases.

Accuracy and Diversity in Ensembles of Text Categorisers


 
 
 Error-Correcting Out Codes (ECOC) ensembles of binary classifiers are used in Text Cate- gorisation to improve the accuracy while benefiting from learning algorithms that only support two classes. An accurate ensemble relies on the quality of its corresponding decomposition ma- trix, which at the same time depends on the separation between the categories and the diversity of the dichotomies representing the binary classifiers. Important open questions include finding a good definition for diversity between two dichotomies and a way of combining all the pairwise diversity values into a single indicator that we call the decomposition quality. In this work we introduce a new measure to estimate the diversity between two learners and we compare it to the well-known Hamming distance. We also examine three functions to evaluate the decomposition quality. We present a set of experiments where these measures and functions are tested using two distinct document corpora with several configurations in each. The analysis of the results shows a weak relationship between the ensemble accuracy and its diversity.
 
 

CBSA: content-based soft annotation for multimodal image retrieval using bayes point machines

We propose a content-based soft annotation (CBSA) procedure for providing images with semantical labels. The annotation procedure starts with labeling a small set of training images, each with one single semantical label (e.g., forest, animal, or sky). An ensemble of binary classifiers is then trained for predicting label membership for images. The trained ensemble is applied to each individual image to give the image multiple soft labels, and each label is associated with a label membership factor. To select a base binary-classifier for CBSA, we experiment with two learning methods, support vector machines (SVMs) and Bayes point machines (BPMs), and compare their class-prediction accuracy. Our empirical study on a 116-category 25K-image set shows that the BPM-based ensemble provides better annotation quality than the SVM-based ensemble for supporting multimodal image retrievals.

Binary Classication Tree for Multiclass Classication with Observation-based Clustering

Many classification techniques are originally designed to solve a binary problem, but practically many classification problems involve more than two classes. A multiclass problem can be decomposed into binary sub-problems, each solved by a binary classifier. Aside from using one-against-one (OAO) or one-against-all (OAA) decomposition scheme, an ensemble of binary classifiers can be constructed hierarchically. In this study, we focus in multiclass classification with a binary classification tree and propose a new approach in splitting a top-down tree by grouping observations into two clusters with k-mean clustering. Unlike a traditional class-clustering approach, this observation-based algorithm allows one class to appear in two meta-classes so it can be examined in both sub-trees. A data cleaning process is also performed to avoid insignificant tree splits. The experiment shows how our proposed algorithm (BCTOB) performed on different data sets, compared with other binary classification tree algorithms. Then advantages and disadvantages of the algorithm are discussed.