Decisions Of Multiple Classifiers Research Articles

SeizeMaliciousURL: A novel learning approach to detect malicious URLs

Malicious websites are increasing in abundance, which brings serious web security threats to users. As a result, individuals lose their assets, values, information, etc. to unauthorized parties and become victims while visiting such websites. The research community put efforts into developing effective and efficient models for detecting malicious URLs to make available notifications about websites that users access. Perceiving this, numerous methods make use of various types of machine learning (ML) approaches. However, until now, no technique has perfectly detected malicious URLs, as they are susceptible to false positive and false negative decisions in classifying URLs in malicious and non-malicious groups. To improve this issue, this article proposes a new approach based on a machine learning technique. The proposed approach uses multiple classifiers (i.e., ensemble learning) to predict the class probabilities of URLs and then applies a threshold to filter the decisions of multiple classifiers. Next, the decisions are combined concerning its corresponding class probabilities and find the class label with the highest class probability as the final decision on unlabeled URLs. The results show that the proposed method offers better performance in terms of malicious URL detection than other methods.

Building Detection With Decision Fusion

A novel decision fusion approach to building detection problem in VHR optical satellite images is proposed. The method combines the detection results of multiple classifiers under a hierarchical architecture, called Fuzzy Stacked Generalization (FSG). After an initial segmentation and pre-processing step, a large variety of color, texture and shape features are extracted from each segment. Then, the segments, represented in K different feature spaces are classified by K different base-layer classifiers of the FSG architecture. The class membership values of the segments, which represent the decisions of different base-layer classifiers in a decision space, are aggregated to form a fusion space which is then fed to a meta-layer classifier of the FSG to label the vectors in the fusion space. The paper presents the performance results of the proposed decision fusion model by a comparison with the state of the art machine learning algorithms. The results show that fusing the decisions of multiple classifiers improves the performance, when they are ensembled under the suggested hierarchical learning architecture.

Random forest based lung nodule classification aided by clustering

An automated lung nodule detection system can help spot lung abnormalities in CT lung images. Lung nodule detection can be achieved using template-based, segmentation-based, and classification-based methods. The existing systems that include a classification component in their structures have demonstrated better performances than their counterparts. Ensemble learners combine decisions of multiple classifiers to form an integrated output. To improve the performance of automated lung nodule detection, an ensemble classification aided by clustering (CAC) method is proposed. The method takes advantage of the random forest algorithm and offers a structure for a hybrid random forest based lung nodule classification aided by clustering. Several experiments are carried out involving the proposed method as well as two other existing methods. The parameters of the classifiers are varied to identify the best performing classifiers. The experiments are conducted using lung scans of 32 patients including 5721 images within which nodule locations are marked by expert radiologists. Overall, the best sensitivity of 98.33% and specificity of 97.11% have been recorded for proposed system. Also, a high receiver operating characteristic (ROC) A z of 0.9786 has been achieved.

Analysis of a Plurality Voting-based Combination of Classifiers

In various studies, it has been demonstrated that combining the decisions of multiple classifiers can lead to better recognition result. Plurality voting is one of the most widely used combination strategies. In this paper, we both theoretically and experimentally analyze the performance of a plurality voting based ensemble classifier. Theoretical expressions for system performance are derived as a function of the model parameters: N (number of classifiers), m (number of classes), and p (probability that a single classifier is correct). Experimental results on the human face recognition problem show that the voting strategy can successfully achieve high detection and identification rates, and, simultaneously, low false acceptance rates.

Methods and algorithms of collective recognition

The collective decision making and, in particular, the collective recognition is treated as the problem of joint application of multiple classifier decisions. The decisions are made about the class of an entity, situation, image, etc. The joint decision is used to improve quality of the final decision by aggregation and coordination of different classifier decisions using a metalevel algorithm. The studies in the field of collective recognition, which were started in the middle of the 1950s, find wide application in practice during the last decade. Since they are used for solving complex large-scale applied problems, the interest of both theoretical scientists and engineers is focused on them. A new impetus for the studies was given by the recent development of embedded distributed structures involving ensembles of intellectual sensors that make decisions under uncertainties on the base of limited local information. The final decision of high quality, in particular, the decision of higher aggregation level, is made by combining local classifier decisions on the metalevel. There are dozens of recent publications proposing new ideas and new approaches and algorithms of collective recognition. Unfortunately, some papers rediscover results publushed several decades before. The goal of this review is to present the main ideas of collective recognition and to outline the status of researches basing on the original source works. The review covers the period from the 1950s, when the first ideas and methods appeared, up to present time.

Multi-agent decision fusion for motor fault diagnosis

Improvement of recognition rate is ultimate aim for fault diagnosis researchers using pattern recognition techniques. However, the unique recognition method can only recognise a limited classification capability which is insufficient for real-life application. An ongoing strategy is the decision fusion techniques. In order to avoid the shortage of single information source coupled with unique decision method, a new approach is required to obtain better results. This paper proposes a decision fusion system for fault diagnosis, which integrates data sources from different types of sensors and decisions of multiple classifiers. First, non-commensurate sensor data sets are combined using an improved sensor fusion method at a decision level by using relativity theory. The generated decision vectors are then selected based on correlation measure of classifiers in order to find an optimal sequence of classifiers fusion, which can lead to the best fusion performance. Finally, multi-agent classifiers fusion algorithm is employed as the core of the whole fault diagnosis system. The efficiency of the proposed system was demonstrated through fault diagnosis of induction motors. The experimental results show that this system can lead to super performance when compared with the best individual classifier with single-source data.

Multiple classifier decision combination strategies for character recognition: A review

Two research strands, each identifying an area of markedly increasing importance in the current development of pattern analysis technology, underlie the review covered by this paper, and are drawn together to offer both a task-oriented and a fundamentally generic perspective on the discipline of pattern recognition. The first of these is the concept of decision fusion for high-performance pattern recognition, where (often very diverse) classification technologies, each providing complementary sources of information about class membership, can be integrated to provide more accurate, robust and reliable classification decisions. The second is the rapid expansion in technology for the automated analysis of (especially) handwritten data for OCR applications including document and form processing, pen-based computing, forensic analysis, biometrics and security, and many other areas, especially those which seek to provide online or offline processing of data which is available in a human-oriented medium. Classifier combination/multiple expert processing has a long history, but the sheer volume and diversity of possible strategies now available suggest that it is timely to consider a structured review of the field. Handwritten character processing provides an ideal context for such a review, both allowing engagement with a problem area which lends itself ideally to the performance enhancements offered by multi-classifier configurations, but also allowing a clearer focus to what otherwise, because of the unlimited application horizons, would be a task of unmanageable proportions. Hence, this paper explicitly reviews the field of multiple classifier decision combination strategies for character recognition, from some of its early roots to the present day. In order to give structure and a sense of direction to the review, a new taxonomy for categorising approaches is defined and explored, and this both imposes a discipline on the presentation of the material available and helps to clarify the mechanisms by which multi-classifier configurations deliver performance enhancements. The review incorporates a discussion both of processing structures themselves and a range of important related topics which are essential to maximise an understanding of the potential of such structures. Most importantly, the paper illustrates explicitly how the principles underlying the application of multi-classifier approaches to character recognition can easily generalise to a wide variety of different task domains.

On output independence and complementariness in rank-based multiple classifier decision systems

This study presents a theoretical analysis of output independence and complementariness between classifiers in a rank-based multiple classifier decision system in the context of the partitioned observation space theory. To enable such an analysis, an information theoretic interpretation of a rank-based multiple classifier system is developed and basic concepts from information theory are applied to develop measures for output independence and complementariness. It is shown that output independence of classifiers is not a requirement for achieving complementariness between these classifiers. Namely, output independence does not imply a performance improvement by combining multiple classifiers. A condition called dominance is shown to be important instead. The information theoretic measures proposed for output independence and complementariness are justified by simulated examples.

A statistical unified framework for rank-based multiple classifier decision combination

This study presents a theoretical investigation of the rank-based multiple classifier decision combination problem, with the aim of providing a unified framework to understand a variety of such systems. The combination of the decisions of more than one classifiers with the aim of improving overall system performance is a concept of general interest in pattern recognition, as a viable alternative to designing a single sophisticated classifier. The problem of combining the classifier decisions in the raw form of candidate class rankings is formulated as a discrete optimization problem. The objective function to be maximized is selected as the overall probability of correct decision. This formulation introduces a set of observation statistics about the joint behavior of the classifiers which are to be estimated by observing the classifiers operated on a cross-validation test set. The resulting binary programming problem is shown to have a simple and global optimum solution but which also necessitates a prohibitive number of observation statistics. From the objective function expansion, the problem observation space is defined and a method based on partitioning is introduced to reduce its prohibitive dimensionality. Within this partitioning formalism called as the Partitioned Observation Space (POS) theory, the number of behavior observation statistics can be reduced to levels which are feasible to estimate from the available cross-validation test data. It is shown by examples that such specific partitionings can be defined when reasonable assumptions or prior knowledge about the classifiers are incorporated into the problem domain. It is also demonstrated that certain specific partitionings of the classifier observation space effectively lead to the highest rank, Borda count and logistic regression rank-based decision combination methods from the literature. The analysis presented is general and promises to lead to a class of algorithms for rank-based decision combination. The potential of the theory and practical issues in implementation are illustrated by applying it in a real-life phonetic discrimination problem from speech pattern classification with encouraging results.

Rank-Based Multiple Classifier Decision Combination: A Theoretical Study

This study presents a theoretical investigation of the rank-based multiple classifier decision problem for closed-set pattern identification. The problem of combining the decisions of more than one classifiers with raw outputs in the form of candidate class rankings is considered and formulated as a general discrete optimization problem with an objective function based on the total probability of correct decision. This formulation uses certain performance statistics about the joint behavior of the ensemble of classifiers, which need to be estimated from cross-validation data. An initial approach leads to an integer (binary) programming problem with a simple and global optimum solution but of prohibitive dimensionality. Therefore, we present a partitioning formalism under which this dimensionality can be reduced by incorporating our prior knowledge about the problem domain and the structure of the training data. It is also shown that formalism can effectively explain a number of successfully used combination approaches in the literature.

Recursive Partitioning Technique for Combining Multiple Classifiers

Various methods of reducing correlation between classifiers in a multiple classifier framework have been attempted. Here we propose a recursive partitioning technique for analysing feature space of multiple classifier decisions. Spectral summation of individual pattern components in intermediate feature space enables each training pattern to be rated according to its contribution to separability, measured as k-monotonic constraints. A constructive algorithm sequentially extracts maximally separable subsets of patterns, from which is derived an inconsistently classified set (ICS). Leaving out random subsets of ICS patterns from individual (base) classifier training sets is shown to improve performance of the combined classifiers. For experiments reported here on artificial and real data, the constituent classifiers are identical single hidden layer MLPs with fixed parameters.

Optimal combinations of pattern classifiers

To improve recognition results, decisions of multiple classifiers can be combined. We study the performance of combination methods that are variations of the majority vote. A Bayesian formulation and a weighted majority vote (with weights obtained through a genetic algorithm) are implemented, and the combined performances of 7 classifiers on a large set of handwritten numerals are analyzed.