Weak Base Classifiers Research Articles

SEM: Stacking Ensemble Meta-Learning for IOT Security Framework

Despite the popularity of the applicability of Internet of Things (IoT) devices in many applications, security and anomaly detection has always been a rising concern for any domain of research. In the current smart era, the increased use of IoT and related applications in almost all domains tune us for the alarming situations of security measures. Many such insecure threats such as the denial of service, malicious control, and operations can be the real harm for any IoT devices. Compared to the initial days of developments in IoT control, many advanced techniques based on machine learning are designed for effective control of such malicious activities. In this paper, a stacked ensemble meta-learning (SEM) model has been developed to enhance the performance of the base machine learning model used for anomaly detection in IoT devices. The proposed model learns from the prediction errors of the base classifiers to build a more accurate prediction model. The proposed SEM constructs a higher-level prediction model over the predictions of weak base classifiers.

Sieve: An Ensemble Algorithm Using Global Consensus for Binary Classification

In the field of machine learning, an ensemble approach is often utilized as an effective means of improving on the accuracy of multiple weak base classifiers. A concern associated with these ensemble algorithms is that they can suffer from the Curse of Conflict, where a classifier’s true prediction is negated by another classifier’s false prediction during the consensus period. Another concern of the ensemble technique is that it cannot effectively mitigate the problem of Imbalanced Classification, where an ensemble classifier usually presents a similar magnitude of bias to the same class as its imbalanced base classifiers. We proposed an improved ensemble algorithm called “Sieve” that overcomes the aforementioned shortcomings through the establishment of the novel concept of Global Consensus. The proposed Sieve ensemble approach was benchmarked against various ensemble classifiers, and was trained using different ensemble algorithms with the same base classifiers. The results demonstrate that better accuracy and stability was achieved.

A cost-sensitive three-way combination technique for ensemble learning in sentiment classification

Deep neural networks (DNN) have achieved remarkable results in sentiment classification. Some ensemble methods of DNN models and traditional feature-based models are proposed recently. However, to the best of our knowledge, most of the works use traditional ensemble combination techniques, e.g. voting and stacking, which are designed for weak base classifiers. So far many base classifiers, e.g. DNN, have been able to achieve good results in sentiment classification tasks, so there should be a new ensemble combination technique designed for strong base classifiers. To address this issue, we proposed a cost-sensitive combination technique using sequential three-way decisions (3WD), which is named S3WC. In S3WC, base classifiers are arranged in a linear arrangement, and a gate mechanism is constructed in each step to divide the objects into three groups, i.e., positive region, negative region and boundary region, which respectively correspond to acceptance, rejection and deferment in sequential 3WD. Each object is grouped by minimizing its total cost consisting of misclassification cost and time cost. The objects in boundary region require more information to decrease the misclassification cost, so they are reclassified by the subsequent base classifiers in order to obtain more information, while the time cost increases. In the experiment, we apply S3WC to DNN models and traditional feature-based models on five benchmark datasets, and compare its performance with traditional ensemble combination techniques. The experimental results show that S3WC outperforms any of its base classifiers in terms of classification accuracy, and the total cost of S3WC is lower than that of the existing ensemble combination techniques (e.g. majority-voting, weighted-voting, meta-learning).

On random hyper-class random forest for visual classification

Random forest is an effective ensemble classifier. It has shown to be efficient with good generalization performance thanks to the randomness in training. However, for visual classification problems, where high dimensions and large number of classes exist, the impurity measurement in training the tree nodes split function not only neglects the strong conditional dependencies among visual attributes, but also leads to rather weak base classifiers, which may not reflect enough discriminative capability. Addressing this, we develop a random hyper-class random forest (RHC-RF) for visual classification tasks in this paper. During training each tree node, the entire class space is randomly partitioned into two hyper-classes and a weak learner targeting at separating the two hyper-classes is trained, where a linear classifier is learned with multiple feature attributes. This novel training scheme well preserves the randomness by random partition of the label space, and at the same time, the objective of training each node directly targets at discriminating classes in a divide-and-conquer-like manner. The proposed method is expected to benefit from the improved discrimination for visual classification compared with conventional random forests while preserving good generalization. Extensive experiments on various multi-class and high-dimensional visual classification tasks (including scene and object classification, image-based food identification, handwritten digit recognition and face recognition) demonstrate the superior accuracy, as well as its compactness and robustness achieved by the proposed method.

Multiclass Boosting for Weak Classifiers

AdaBoost.M2 is a boosting algorithm designed for multiclass problems with weak base classifiers. The algorithm is designed to minimize a very loose bound on the training error. We propose two alter...