Weighted Majority Voting Research Articles

A Cloud-Based Multi-Temporal Ensemble Classifier to Map Smallholder Farming Systems

Smallholder farmers cultivate more than 80% of the cropland area available in Africa. The intrinsic characteristics of such farms include complex crop-planting patterns, and small fields that are vaguely delineated. These characteristics pose challenges to mapping crops and fields from space. In this study, we evaluate the use of a cloud-based multi-temporal ensemble classifier to map smallholder farming systems in a case study for southern Mali. The ensemble combines a selection of spatial and spectral features derived from multi-spectral Worldview-2 images, field data, and five machine learning classifiers to produce a map of the most prevalent crops in our study area. Different ensemble sizes were evaluated using two combination rules, namely majority voting and weighted majority voting. Both strategies outperform any of the tested single classifiers. The ensemble based on the weighted majority voting strategy obtained the higher overall accuracy (75.9%). This means an accuracy improvement of 4.65% in comparison with the average overall accuracy of the best individual classifier tested in this study. The maximum ensemble accuracy is reached with 75 classifiers in the ensemble. This indicates that the addition of more classifiers does not help to continuously improve classification results. Our results demonstrate the potential of ensemble classifiers to map crops grown by West African smallholders. The use of ensembles demands high computational capability, but the increasing availability of cloud computing solutions allows their efficient implementation and even opens the door to the data processing needs of local organizations.

Rail crack recognition based on Adaptive Weighting Multi-classifier Fusion Decision

In order to make the full use of three-dimensional information of Magnetic Flux Leakage (MFL) signals, an Adaptive Weighting Multi-classifier Fusion Decision Algorithm is adopted for rail crack recognition. Support Vector Machine (SVM) is used to classify MFL signals from single-channel and single-direction, and then adaptive weightings of different SVMs are assigned according to entropy calculated by posterior probabilities of different SVMs. Finally, weighted majority vote strategy is used to make a comprehensive decision by fusing classification results of different channels and different directions. Effectiveness of the proposed method is testified by experiments based on measured MFL signals.

A novel spontaneous facial expression recognition using dynamically weighted majority voting based ensemble classifier

Facial Expression Recognition (FER) is inherently data driven. Spontaneous expressions are substantially different from posed expressions. Spontaneous facial expressions are more challenging and more difficult to recognize. Any facial expression can be represented in many different patterns of muscles movements. Moreover, the facial expressions show discrepancies in different cultures and ethnicities. Therefore, a FER system has to learn a huge problem/feature space. A base classifier trained on a sub-region of feature space, cannot perform equally well in other areas of feature space. Therefore, a base classifier should be assigned higher voting weight in the regions near to its training space and a lower voting weight in the regions far from its training space. In order to maintain high accuracy and robustness of a FER system in space and time, a Dynamic Weight Majority Voting (DWMV) mechanism for base classifiers is introduced. An ensemble system for FER is proposed that has the aptitude of incrementally learning and thus, can learn all possible patterns of expressions that may be generated in feature or in various cultures and ethnicities. Speeded-Up Robust Features (SURF) are used to represent the feature space. Since no work in literature is found on “which similarity measure is more appropriate in SURF descriptor domain for facial expression recognition”, therefore, different similarity measures are used and the results are compared. A vast range of experimentation is performed on posed and spontaneous databases that demonstrates promising results.

An investigation of ensemble combination schemes for genetic programming based hyper-heuristic approaches to dynamic job shop scheduling

Genetic programming based hyper-heuristic (GP-HH) approaches that evolve ensembles of dispatching rules have been effectively applied to dynamic job shop scheduling (JSS) problems. Ensemble GP-HH approaches have been shown to be more robust than existing GP-HH approaches that evolve single dispatching rules for dynamic JSS problems. For ensemble learning in classification, the design of how the members of the ensembles interact with each other, e.g., through various combination schemes, is important for developing effective ensembles for specific problems. In this paper, we investigate and carry out systematic analysis for four popular combination schemes. They are majority voting, which has been applied to dynamic JSS, followed by linear combination, weighted majority voting and weighted linear combination, which have not been applied to dynamic JSS. In addition, we propose several measures for analysing the decision making process in the ensembles evolved by GP. The results show that linear combination is generally better for the dynamic JSS problem than the other combination schemes investigated. In addition, the different combination schemes result in significantly different interactions between the members of the ensembles. Finally, the analysis based on the measures shows that the behaviours of the evolved ensembles are significantly affected by the combination schemes. Weighted majority voting has bias towards single members of the ensembles.

Ensemble Methods for Classification of Physical Activities from Wrist Accelerometry.

To investigate whether the use of ensemble learning algorithms improve physical activity recognition accuracy compared to the single classifier algorithms, and to compare the classification accuracy achieved by three conventional ensemble machine learning methods (bagging, boosting, random forest) and a custom ensemble model comprising four algorithms commonly used for activity recognition (binary decision tree, k nearest neighbor, support vector machine, and neural network). The study used three independent data sets that included wrist-worn accelerometer data. For each data set, a four-step classification framework consisting of data preprocessing, feature extraction, normalization and feature selection, and classifier training and testing was implemented. For the custom ensemble, decisions from the single classifiers were aggregated using three decision fusion methods: weighted majority vote, naïve Bayes combination, and behavior knowledge space combination. Classifiers were cross-validated using leave-one subject out cross-validation and compared on the basis of average F1 scores. In all three data sets, ensemble learning methods consistently outperformed the individual classifiers. Among the conventional ensemble methods, random forest models provided consistently high activity recognition; however, the custom ensemble model using weighted majority voting demonstrated the highest classification accuracy in two of the three data sets. Combining multiple individual classifiers using conventional or custom ensemble learning methods can improve activity recognition accuracy from wrist-worn accelerometer data.

To Save Our Species But How Do We Make Collective Decisions

Throughout human history, havoc has been wrought by just a handful of very powerful leaders. The ultimate catastrophe could also be caused by just a few individuals, by unleashing a nuclear war or by ignoring international agreements on climate change. Donald Trump, for example, may renege on the Paris Agreement and commit other ecological blunders, which could spell the end for everyone. In a nutshell, the current democratic process gives him far too much power. The rights of any one citizen or country must be tempered by the rights of all. Alas, current forms of decision-making often fail to involve everyone in the final decision, the most obvious instances relating to decisions taken by a (simple or weighted) majority vote. If instead decision-making were based on the local, national or international consensus, such exclusive decisions could not be taken so easily. It should also be pointed out that majoritarianism – majority rule based on majority voting – has been and still is problematic in numerous inter-communal conflicts. This paper therefore outlines the flaws involved in binary voting; next, as a better methodology, it proposes a more inclusive voting mechanism; and finally, it advocates a structure to ensure that agreements are implemented, with possible penalties for those individuals, organisations or countries that flaunt the international consensus.

Early outbreak identification and multifaceted, bundled interventions for outbreak control: Paths of victory

This paper generalizes a pivotal result from the PAC-Bayesian literature —the C-bound— primarily designed for binary classification to the general case of ensemble methods of voters with arbitrary outputs. We provide a generic version of the C-bound, an upper bound over the risk of models expressed as a weighted majority vote that is based on the first and second statistical moments of the vote's margin. On the one hand, this bound may advantageously be applied on more complex outputs than mere binary outputs, such as multiclass labels and multilabel, and on the other hand, it allows us to consider margin relaxations. We provide a specialization of the bound to multiclass classification together with empirical evidence that the presented theoretical result is tightly bound to the risk of the majority vote classifier. We also give insights as to how the proposed bound may be of use to characterize the risk of multilabel predictors.

A mutual information-Dempster-Shafer based decision ensemble system for land cover classification of hyperspectral data

Hyperspectral images contain extremely rich spectral information that offer great potential to discriminate between various land cover classes. However, these images are usually composed of tens or hundreds of spectrally close bands, which result in high redundancy and great amount of computation time in hyperspectral classification. Furthermore, in the presence of mixed coverage pixels, crisp classifiers produced errors, omission and commission. This paper presents a mutual information-Dempster-Shafer system through an ensemble classification approach for classification of hyperspectral data. First, mutual information is applied to split data into a few independent partitions to overcome high dimensionality. Then, a fuzzy maximum likelihood classifies each band subset. Finally, Dempster-Shafer is applied to fuse the results of the fuzzy classifiers. In order to assess the proposed method, a crisp ensemble system based on a support vector machine as the crisp classifier and weighted majority voting as the crisp fusion method are applied on hyperspectral data. Furthermore, a dimension reduction system is utilized to assess the effectiveness of mutual information band splitting of the proposed method. The proposed methodology provides interesting conclusions on the effectiveness and potentiality of mutual information-Dempster-Shafer based classification of hyperspectral data.

Consensus algorithms for biased labeling in crowdsourcing

Although it has become an accepted lay view that when labeling objects through crowdsourcing systems, non-expert annotators often exhibit biases, this argument lacks sufficient evidential observation and systematic empirical study. This paper initially analyzes eight real-world datasets from different domains whose class labels were collected from crowdsourcing systems. Our analyses show that biased labeling is a systematic tendency for binary categorization; in other words, for a large number of annotators, their labeling qualities on the negative class (supposed to be the majority) are significantly greater than are those on the positive class (minority). Therefore, the paper empirically studies the performance of four existing EM-based consensus algorithms, DS, GLAD, RY, and ZenCrowd, on these datasets. Our investigation shows that all of these state-of-the-art algorithms ignore the potential bias characteristics of datasets and perform badly although they model the complexity of the systems. To address the issue of handling biased labeling, the paper further proposes a novel consensus algorithm, namely adaptive weighted majority voting (AWMV), based on the statistical difference between the labeling qualities of the two classes. AWMV utilizes the frequency of positive labels in the multiple noisy label set of each example to obtain a bias rate and then assigns weights derived from the bias rate to negative and positive labels. Comparison results among the five consensus algorithms (AWMV and the four existing) show that the proposed AWMV algorithm has the best overall performance. Finally, this paper notes some potential related topics for future study.

Traffic sign recognition based on weighted ELM and AdaBoost

A novel multiclass AdaBoost-based extreme learning machine (ELM) ensemble algorithm is proposed, in which the weighted ELM is selected as the basic weak classifier because of its much faster learning speed and much better generalisation performance than traditional support vector machines. AdaBoost acts as an ensemble learning method of a number of weighted ELMs. Then, an ensemble strong classifier is constructed by the weighted majority vote of all the weighted ELMs. Compared with the existing ELM methods, the proposed algorithm solves the problem of how to train the weighted samples by ELM in multiclass classification directly. Experiments on the German Traffic Sign Recognition Benchmark database demonstrate that the proposed algorithm can achieve a high recognition accuracy of 99.12% with a relatively lower computational complexity than many state-of-the-art algorithms.

Risk upper bounds for general ensemble methods with an application to multiclass classification

This paper generalizes a pivotal result from the PAC-Bayesian literature —the C-bound— primarily designed for binary classification to the general case of ensemble methods of voters with arbitrary outputs. We provide a generic version of the C-bound, an upper bound over the risk of models expressed as a weighted majority vote that is based on the first and second statistical moments of the vote's margin. On the one hand, this bound may advantageously be applied on more complex outputs than mere binary outputs, such as multiclass labels and multilabel, and on the other hand, it allows us to consider margin relaxations. We provide a specialization of the bound to multiclass classification together with empirical evidence that the presented theoretical result is tightly bound to the risk of the majority vote classifier. We also give insights as to how the proposed bound may be of use to characterize the risk of multilabel predictors.

Generalizing intersection kernel support vector machines for color texture based recognition

This paper presents a novel recognition approach in which the component-adaptive color co-occurrence matrices (CACCMs) are designed to characterize color and texture cues in the images, while histogram intersection kernel support vector machines (HIKSVMs) are generalized to the version compatible to color co-occurrence matrix (CCM), called CCM intersection kernel support vector machines (CIKSVMs). An ensemble learning framework is proposed for synchronously training the optimal marginal CIKSVMs and corresponding CACCMs’ extractors. This learning architecture is applicable to an arbitrary color space employed for image coding, while we pay utmost attention to a perceptual uniform color space for the prominent potential in image proprieties’ display. For the formulation of recognition algorithm, the set of multi-channel CACCMs (CAMCMs) of per sample is utilized to get a balance between discriminative power and computational efficiency, while multiple marginal CIKSVMs are combined by weighted majority voting. The effectiveness of our approach is validated by promising results obtained from four experimental datasets.

Implementasi Sistem Crowdsourced Labelling Berbasis Web dengan Metode Weighted Majority Voting

Crowdsourced Labelling is a large scale data labelling process, solicits a large group of people to label the data, usually via Internet. This paper discusses about design and implementation of Web-based Crowdsourced Labelling. Supervised learning classification methods need labelled training data for its training phase. Unfortunately, in many cases, there aren’t any already available labelled training data. Large scale data labelling is a tedious and time consuming work. This research develops a web-based crowdsourced labelling which able to solicit a large group of people as data labeler to speed up the data labelling process. This system also allows multiple labeler for every data. The final label is calculated using Weighted Majority Voting method. We grabbed and used Facebook comments from the two candidates’ Facebook Page of 2014 Indonesian Presidential Election as testing data. Based on the testing conducted we can conclude that this system is able to handle all the labelling steps well and able to handle collision occurred when multiple labeler labelling a same data in the same time. The system successfully produces final label in CSV format, which can be processed further with many sentiment analysis tools or machine learning tools.
 Index Terms - Crowdsources labeling, web-based system, supervised learning, weighted majority voting.

A random-forests-based classifier using class association rules and its application to an intrusion detection system

With the rapid developments of network technology, devices connected to the network in a variety of fields have increased, and then, network security has become more important. Rule-based classification for intrusion detection is useful, because it is not only easily understood by humans, but also accurate for the classification of new patterns. Genetic network programming (GNP) is one of the rule-mining techniques as well as the evolutionary-optimization techniques. It can extract rules efficiently even from an enormous database, but still needs more accuracy and stability for practical use. This paper describes a classification system with random forests, employing weighted majority vote in the classification to enhance its performance. For the performance evaluation, NSL-KDD (Network Security Laboratory-Knowledge Discovery and Data Mining) data set is used and the proposed method is compared with the conventional methods, including other machine-learning techniques (Random forests, SVM, J4.8) in terms of the accuracy and false positive rate.

A Comparison of Four Character-Level String-to-String Translation Models for (OCR) Spelling Error Correction

Abstract We consider the isolated spelling error correction problem as a specific subproblem of the more general string-to-string translation problem. In this context, we investigate four general string-to-string transformation models that have been suggested in recent years and apply them within the spelling error correction paradigm. In particular, we investigate how a simple ‘k-best decoding plus dictionary lookup’ strategy performs in this context and find that such an approach can significantly outdo baselines such as edit distance, weighted edit distance, and the noisy channel Brill and Moore model to spelling error correction. We also consider elementary combination techniques for our models such as language model weighted majority voting and center string combination. Finally, we consider real-world OCR post-correction for a dataset sampled from medieval Latin texts.

A selective neural network ensemble classification for incomplete data

Neural network ensemble (NNE) is a simple and effective method to deal with incomplete data for classification. However, with the increase in the number of missing values, the number of incomplete feature combinations (feature subsets) grown rapidly which makes the NNE method very time-consuming and the accuracy is also need to be improved. In this paper, we propose a selective neural network ensemble (SNNE) classification for incomplete data. The SNNE first obtains all the available feature subsets of the incomplete dataset and then applies mutual information to measure the importance (relevance) degree of each feature subset. After that, an optimization process is applied to remove the feature subsets by satisfying the following condition: there is at least a feature subset contained in the removed feature subset and the difference of their importance degree is smaller than a given threshold δ. Finally, the rest of the feature subsets were used to train a group of neural networks and the classification for a given sample is decided by weighted majority voting of all available components in the ensemble. Experimental results show that δ = 0.05 is reasonable in our study. It can improve the efficiency of the algorithm without loss the algorithm accuracy. Experiments also show that SNNE outperforms the NNE-based algorithms compared. In addition, it can greatly reduce the running time when dealing with datasets with larger number of missing values.

A subspace ensemble framework for classification with high dimensional missing data

Real world classification tasks may involve high dimensional missing data. The traditional approach to handling the missing data is to impute the data first, and then apply the traditional classification algorithms on the imputed data. This method first assumes that there exist a distribution or feature relations among the data, and then estimates missing items with existing observed values. A reasonable assumption is a necessary guarantee for accurate imputation. The distribution or feature relations of data, however, is often complex or even impossible to be captured in high dimensional data sets, leading to inaccurate imputation. In this paper, we propose a complete-case projection subspace ensemble framework, where two alternative partition strategies, namely bootstrap subspace partition and missing pattern-sensitive subspace partition, are developed for incomplete datasets with even missing patterns and uneven missing patterns, respectively. Multiple component classifiers are then separately trained in these subspaces. After that, a final ensemble classifier is constructed by a weighted majority vote of component classifiers. In the experiments, we demonstrate the effectiveness of the proposed framework over eight high dimensional UCI datasets. Meanwhile, we apply the two proposed partition strategies over data sets with different missing patterns. As indicated, the proposed algorithm significantly outperforms existing imputation methods in most cases.

Incomplete data classification with voting based extreme learning machine

Extreme learning machine (ELM) was proposed as a new efficient learning algorithm for single-hidden layer feedforward neural networks (SLFN) in recent years. It is featured by its much faster training speed and better generalization performance over traditional SLFN learning techniques. However, ELM cannot deal directly with incomplete data which widely exists in real-world applications. In this paper, we propose a new algorithm to handle incomplete data with voting based extreme learning machine (V-ELMI). V-ELMI did not rely on any assumptions about missing values. It first obtains a group of data subsets according to the missing values of the training set. Then, it applies mutual information to measure the importance degree of each data subsets. After that, it trains a group of subclassifiers on these data subsets by applying ELM as base learning algorithm. Finally, for a given test sample with missing values, V-ELMI selects the subclassifiers whose input did not require the missing values to predict it. And final prediction is determined by weighted majority voting according to the mean value of the norms of the output weights and the importance degree of each available subclassifier. Experimental results on 15 UCI incomplete datasets and 5 UCI complete datasets have shown that, V-ELMI generally has better performance than the algorithms compared. Moreover, compared with the classification algorithms based on neural network ensemble (NNE), V-ELMI can greatly improve algorithm computational efficiency.

A new approach to human microRNA target prediction using ensemble pruning and rotation forest.

MicroRNAs (miRNAs) are small non-coding RNAs that have important functions in gene regulation. Since finding miRNA target experimentally is costly and needs spending much time, the use of machine learning methods is a growing research area for miRNA target prediction. In this paper, a new approach is proposed by using two popular ensemble strategies, i.e. Ensemble Pruning and Rotation Forest (EP-RTF), to predict human miRNA target. For EP, the approach utilizes Genetic Algorithm (GA). In other words, a subset of classifiers from the heterogeneous ensemble is first selected by GA. Next, the selected classifiers are trained based on the RTF method and then are combined using weighted majority voting. In addition to seeking a better subset of classifiers, the parameter of RTF is also optimized by GA. Findings of the present study confirm that the newly developed EP-RTF outperforms (in terms of classification accuracy, sensitivity, and specificity) the previously applied methods over four datasets in the field of human miRNA target. Diversity-error diagrams reveal that the proposed ensemble approach constructs individual classifiers which are more accurate and usually diverse than the other ensemble approaches. Given these experimental results, we highly recommend EP-RTF for improving the performance of miRNA target prediction.

Recognition of Off-Line Handwritten Devanagari Characters using Combinational Feature Extraction

Handwritten Devanagari off-line characters recognition is a challenging task due to peculiarities involved in the writing styles of different person. We are collected features from 64 dimensional feature extraction techniques and shadow features for the Devanagari character recognition in this proposed scheme. These calculated features are used for further classification. Features which are used here for recognition means histograms of direction chain code of the contour points of the characters [4]. The features are classified using neural network and selected the best result by weighted majority voting technique.