Multiple Classifier Fusion Research Articles

A Neuro-Fuzzy based System for Classification of Natural Textures

A statistical approach based on the coordinated clusters representation of images is used for classification and recognition of textured images. In this paper, two issues are being addressed; one is the extraction of texture features from the fuzzy texture spectrum in the chromatic and achromatic domains from each colour component histogram of natural texture images and the second issue is the concept of a fusion of multiple classifiers. The implementation of an advanced neuro-fuzzy learning scheme has been also adopted in this paper. The results of classification tests show the high performance of the proposed method that may have industrial application for texture classification, when compared with other works.

Risk prediction of product‐harm events using rough sets and multiple classifier fusion: an experimental study of listed companies in China

AbstractWith the increasing of frequency and destructiveness of product‐harm events, study on enterprise crisis management becomes essentially important, but little literature thoroughly explores the risk prediction method of product‐harm event. In this study, an initial index system for risk prediction was built based on the analysis of the key drivers of the product‐harm event's evolution; ultimately, nine risk‐forecasting indexes were obtained using rough set attribute reduction. With the four indexes of cumulative abnormal returns as the input, fuzzy clustering was used to classify the risk level of a product‐harm event into four grades. In order to control the uncertainty and instability of single classifiers in risk prediction, multiple classifier fusion was introduced and combined with self‐organising data mining (SODM). Further, an SODM‐based multiple classifier fusion (SB‐MCF) model was presented for the risk prediction related to a product‐harm event. The experimental results based on 165 Chinese listed companies indicated that the SB‐MCF model improved the average predictive accuracy and reduced variation degree simultaneously. The statistical analysis demonstrated that the SB‐MCF model significantly outperformed six widely used single classification models (e.g. neural networks, support vector machine, and case‐based reasoning) and other six commonly used multiple classifier fusion methods (e.g. majority voting, Bayesian method, and genetic algorithm).

Hyperspectral Image Classification by Fusion of Multiple Classifiers

Hyperspectral image mostly have very large amounts of data which makes the computational cost and subsequent classification task a difficult issue. Firstly, to solve the problem of computational complexity, spectral clustering algorithm is imported to select efficient bands for subsequent classification task. Secondly, due to lack of labeled training sample points, this paper proposes a new algorithm that combines support vector machines and Bayesian classifier to create a discriminative/generative hyperspectral image classification method using the selected features. Experimental results on real hyperspectral image show that the proposed method has better performance than the other state-of-the-art methods.

Research on the Method of Fault Diagnosis Based on Multiple Classifiers Fusion

Research on the Method of Fault Diagnosis Based on Multiple Classifiers Fusion

Application of Multiple Classifier Fusion in the Discriminant Analysis of near Infrared Spectroscopy for Agricultural Products

Near infrared spectroscopy combined with chemometrics and pattern recognition has become a primary focus in the discriminant analysis of agricultural products. To date, most studies have focused on using a single classifier to discriminate the origins, varieties and grades of products. Others have focused on using multiple classifier fusion by weighted voting. Due to their attributes of continuity and internal similarity, discriminant models sometimes present poor performance. In this study, we achieved better performance by applying multiple classifier fusion models, including support vector machine (SVM), discriminant partial least squares (DPLS) and principal component and Fisher criterion (PPF). PPF showed continuity and similarity among different parts of tobacco leaves [i.e. upper (B), cutter (C) and lug (X)]. The similarities between each class and the others were quantified to values, and the sum of the similarity values of each class was defined as its similarity. SVM–DPLS–PPF fusion by voting and similarity constraint for decision resulted in better performance, with the correct discriminant rate improved on average by 14.1%, 8.2%, 17.3% and 4.6% compared with those achieved using SVM, DPLS, PPF and SVM–DPLS–PPF fusion by weighted voting for decision, respectively; in addition, the incorrect discriminant rate between B and X was reduced to zero. Therefore, we demonstrated the feasibility of using SVM–DPLS–PPF fusion by voting and similarity constraint for decision to discriminate between different parts of tobacco leaves. This technique could provide a new method for tobacco quality management, computer-aided grading and intelligent acquisition. It also provides a new discriminant method for analysing the attributes of continuity and similarity of agricultural products using near infrared spectroscopy.

A Multiple Classifier Fusion Algorithm Using Weighted Decision Templates

Fusing classifiers’ decisions can improve the performance of a pattern recognition system. Many applications areas have adopted the methods of multiple classifier fusion to increase the classification accuracy in the recognition process. From fully considering the classifier performance differences and the training sample information, a multiple classifier fusion algorithm using weighted decision templates is proposed in this paper. The algorithm uses a statistical vector to measure the classifier’s performance and makes a weighed transform on each classifier according to the reliability of its output. To make a decision, the information in the training samples around an input sample is used by thek-nearest-neighbor rule if the algorithm evaluates the sample as being highly likely to be misclassified. An experimental comparison was performed on 15 data sets from the KDD’99, UCI, and ELENA databases. The experimental results indicate that the algorithm can achieve better classification performance. Next, the algorithm was applied to cataract grading in the cataract ultrasonic phacoemulsification operation. The application result indicates that the proposed algorithm is effective and can meet the practical requirements of the operation.

Pedestrian identification based on fusion of multiple features and multiple classifiers

Finding a specific person from videos in surveillance systems is a challenging task. In the videos, different people cannot be the same in a whole body appearance. Based on this fact, this paper has proposed new methods based on fusion of textures, angle histograms and color moments to find a specific person. The human visual system can discriminate different objects quickly and efficiently. Inspired by on-center and off-center receptive fields in the visual system, a network model based on spiking neurons is proposed to extract texture features, and it has behaviors similar to Gabor filters. According to human body proportion, a person image is divided into three parts: head, torso and leg. Texture features of three parts are extracted by means of this network. Back propagation neural network, multi-class SVM and KNN are used as classifiers. For improving recognition rate, different fusion methods have been studied such as the fusion of texture features and other features in three body parts, and decision fusion using voting mechanism, probability combination etc. The experimental results for different methods are provided and the best fusion method is proposed. The technology of Compute Unified Device Architecture is applied in the experiments, which greatly reduces the running time for extraction of texture features.

Improvement in Classification Accuracy Rate Using Multiple Classifier Fusion Towards Computer Vision Detection of Malaria Parasite (Plasmodium vivax)

Background: The main method used for the laboratory confirmation of malaria is the conventional light microscopy; however, microscopy has three main disadvantages: I) it is time-consuming and labor-intensive; II) its results depend heavily on good techniques, reagents and microscopes; III) in many cases decisions about treatment are often taken without using the result of microscopy because of long delays in providing the results to the clinician. Hence, an extreme necessity of the fast automatic detection of the disease is required to diagnose and treat promptly. Objectives: Through the improvement of classification accuracy rate, this work aims to present a computer-assisted diagnosis system for malaria parasite. Materials and Methods: This study was conducted using 400 confirmed images of blood slides infected with malaria parasite. The MATLAB software was used for the implementation of computation procedures. Using five extracted features (flat texture, saturation channel histogram, color histogram, gradient, and granulometry) and six classifiers (k-Nearest Neighbors (k-NN), 1-Nearest Neighbor (1-NN), decision tree (DT), Fisher, linear discriminant analysis (LDA), and quadratic discriminant analysis (QDA)), images were classified into two classes: parasitic and nonparasitic. Then, classifier fusion was done using several algorithms: mean, min, max, stack, median, Adaboost, and bagging. Results: Using six classifiers separately, the highest accuracy was obtained 92% using the k-NN classifier. The highest accuracy of the classifiers' fusion was obtained using the Adaboost algorithm with 95.5% success rate. Conclusions: By comparing the results of classification using multiple classifier fusion with respect to using each classifier separately, it is found that the classifier fusion is more effective in enhancing the detection accuracy.

A novel intelligent method for mechanical fault diagnosis based on dual-tree complex wavelet packet transform and multiple classifier fusion

Identifying fault categories, especially for compound faults, is a challenging task in mechanical fault diagnosis. For this task, this paper proposes a novel intelligent method based on dual-tree complex wavelet packet transform (DTCWPT) and multiple classifier fusion. In this method, in order to effectively extract underlying fault characteristic information, DTCWPT, which enjoys such attractive properties as nearly shift-invariance and reduced aliasing, is introduced and performed on vibration signals, and then an original feature set including time-domain and frequency-domain features is extracted from the frequency-band signals of DTCWPT to reveal machine conditions. In order to achieve the desired performance of fault classification, a new framework for multiple classifier fusion is developed on the basis of neighborhood rough set (NRS). The original feature set is firstly granulated into different granularity levels by NRS and different sensitive feature sets (SFSs) are obtained for reflecting the machine conditions from different perspectives. After that, the SFSs are respectively input into different classification algorithms to achieve the complementary advantages and substantive fusion of different classifiers. Finally, the classification results of multiple classifiers based on different classification algorithms and input SFSs are fused using Bayesian belief method to come up with the final diagnosis result. The proposed method is applied to the fault diagnosis of gearbox and locomotive roller bearings. The diagnosis results show that the proposed method is able to reliably identify the different fault categories which include both single fault and compound faults, which has a better classification performance compared to any one of the individual classifiers. Moreover, the validity of feature selection based on NRS and the superiority of feature extraction based on DTCWPT are also demonstrated by the testing results.

A new classifier fusion method based on historical and on-line classification reliability for recognizing common CT imaging signs of lung diseases

Common CT imaging signs of lung diseases (CISL) play important roles in the diagnosis of lung diseases. This paper proposes a new method of multiple classifier fusion to recognize the CISLs, which is based on the confusion matrices of the classifiers and the classification confidence values outputted by the classifiers. The confusion matrix reflects the historical reliability of decision-making of a classifier, while the difference between the classification confidence values reflects the on-line reliability of its decision-making. The two factors are merged to determine the weights of the classifiers’ classification confidence values. Then the classifiers are fused in a weighted-sum form to make the final decision. We apply the proposed classifier fusion method to combine five types of classifiers for CISL recognition, including support vector machine (SVM), back-propagation neural network (BPNN), Naïve Bayes (NB), k-nearest neighbor (k-NN) and decision tree (DT). In the experiments on lung CT images, our method not only brought stable improvements of recognition performance, compared with single classifiers, but also outperformed two well-known methods of classifier fusion, AdaBoost and Bagging. These results show that the proposed method is effective and promising.

Decision Fusion of Multiple Classifiers for Coronary Plaque Characterization from IVUS Images

Vascular tissue characterization is of great importance concerning the possibility of an Acute Cardiac Syndrome (ACS). Gray-scale intravascular ultrasound (IVUS) is a powerful tomographic modality providing a thorough visualization of coronary arteries. Among the existing methods, virtual histology (VH) is the most popular and clinically available technique for plaque component analysis, it suffers however from a poor longitudinal resolution. In order to surmount this demerit, a new image-based methodology for plaque assessment is suggested here that differentiates tissue components into four classes: calcium, necrotic core, fibrous and fibro-lipid. A rich set of five textural feature families are extracted from IVUS images, computed at different scales. The main contribution of this paper is that tissue classification is accomplished using the principles of multiple classifiers combination approach. At the first stage, an ensemble of base SVM classifiers is constructed from each feature family, separately. The fuzzy outputs of the individual classifiers are then aggregated to provide the final fused results. We investigate four efficient decision fusion schemes of the literature and the SVM fuser. Extensive experimentation is carried out to highlight the merits of the suggested schemes against single SVM classifiers that use reduced feature subsets obtained after feature selection or the entire feature space. The analysis demonstrates that the decision fusion techniques offer improved classification accuracies, compared to single SVM classifiers and existing methods in IVUS imaging. In addition, the method provides accurate assessments of plaque composition in IVUS images.

Polarimetric-Spatial Classification of SAR Images Based on the Fusion of Multiple Classifiers

Traditional image classification methods are undertaken using the pixel as the research unit. These methods cannot use semantic information, and their classification results may not always be satisfactory. To solve this problem, objected-oriented methods have been widely investigated to classify remote sensing images. In this paper, we propose an innovative objected-oriented technique that combines pixel-based classification and a segmentation approach for the classification of polarimetric synthetic aperture radar (PolSAR) images. In the process of the pixel-based classification, a soft voting strategy is utilized to fuse multiple classifiers, which can, to some extent, overcome the drawback of majority voting. The experimental results are presented for two quad-polarimetric SAR images. The proposed classification scheme improves the classification accuracies after assembling the multiple classifiers, and provides the classification maps with more homogeneous regions by integrating the spatial information, when compared with pixel-based classification. By deploying multi-scale segmentation, we get a series of classification results, which again show that our method is superior to the conventional object-oriented methods.

Using phenological metrics and the multiple classifier fusion method to map land cover types

Feature selection and multiple classifier fusion (MCF) are effective approaches to improve land cover classification accuracy. In this study, we combined phenological metrics and the MCF method to map land cover types in Jiangsu province of China during the second crop growing season using moderate resolution imaging spectroradiometer time-series data. Eight phenological metrics were developed and calculated, and a MCF scheme was proposed by combining a simple majority vote and the measurement of posterior probabilities. The four base classifiers (i.e., the maximum likelihood classifier, the Mahalanobis distance classifier, the support vector machine classifier, and the neural networks classifier) and the MCF method were used in classifications using two spectral indices from the original satellite data (direct classification) and the computed metric data (metrics-based classification). Accuracy assessments indicated that the overall accuracies and kappa coefficients of the metrics-based classifications were all higher than those of direct classifications. The average overall accuracy and kappa coefficient of metrics-based classifications were 8.36% and 0.1 higher than that of direct classifications, respectively. Similarly, the overall accuracy and kappa coefficient of MCF generally were close to or exceeded the highest accuracy among all the base classifiers. The highest overall accuracy and kappa coefficient was achieved by classification with the MCF method based on phenological metrics (m-MCF), which were 88% and 0.85, respectively. Our results suggested that combining phenological metrics and MCF in classification is a promising method for land cover mapping in regions where strong phenological signals can be detected.

Development of an Automated Medical Diagnosis System for Classifying Thyroid Tumor Cells using Multiple Classifier Fusion.

An automated medical diagnosis system has been developed to discriminate benign and malignant thyroid nodules in multi-stained fine needle aspiration biopsy (FNAB) images using multiple classifier fusion and presented in this paper. First, thyroid cell regions are extracted from the auto-cropped sub-image by implementing mathematical morphology segmentation method. Subsequently, statistical features are extracted by two-level wavelet decomposition based on texture characteristics of the thyroid cells. After that, decision tree (DT), k-nearest neighbor (k-NN), Elman neural network (ENN) and support vector machine (SVM) classifiers are used separately to classify thyroid nodules into benign and malignant. The four individual classifier outputs are then fused together using majority voting rule and linear combination rules to improve the performance of the diagnostic system. The classification results of ENN and SVM classifiers show an overall diagnostic accuracy (DA) of 90%, sensitivity (Se) of 85% and 100%, specificity (Sp) of 90% and 90% respectively. However, the best diagnostic accuracy of 96.66% is obtained by multiple classifier fusion with majority voting rule and linear combination rules. The experimental results show that the proposed method is a useful tool for the diagnosis of thyroid cancer and can provide a second opinion for a physician's decision.

Heterogeneous Classifier Fusion for Ligand-Based Virtual Screening: Or, How Decision Making by Committee Can Be a Good Thing

The concept of data fusion - the combination of information from different sources describing the same object with the expectation to generate a more accurate representation - has found application in a very broad range of disciplines. In the context of ligand-based virtual screening (VS), data fusion has been applied to combine knowledge from either different active molecules or different fingerprints to improve similarity search performance. Machine-learning (ML) methods based on fusion of multiple homogeneous classifiers, in particular random forests, have also been widely applied in the ML literature. The heterogeneous version of classifier fusion - fusing the predictions from different model types - has been less explored. Here, we investigate heterogeneous classifier fusion for ligand-based VS using three different ML methods, RF, naïve Bayes (NB), and logistic regression (LR), with four 2D fingerprints, atom pairs, topological torsions, RDKit fingerprint, and circular fingerprint. The methods are compared using a previously developed benchmarking platform for 2D fingerprints which is extended to ML methods in this article. The original data sets are filtered for difficulty, and a new set of challenging data sets from ChEMBL is added. Data sets were also generated for a second use case: starting from a small set of related actives instead of diverse actives. The final fused model consistently outperforms the other approaches across the broad variety of targets studied, indicating that heterogeneous classifier fusion is a very promising approach for ligand-based VS. The new data sets together with the adapted source code for ML methods are provided in the Supporting Information .

Fault Diagnosis for Aircraft Engine Based on SVM Multiple Classifiers Fusion

Support vector machine (SVM) could well solve the over-learning and the low generalization ability of the neural network. But the single classifier cannot achieve satisfactory recognition rate and anti-interference ability. An aircraft engine fault diagnosis method based on support vector machine multiple classifiers is proposed in this paper. Firstly, sample characteristic information which constitutes the fault feature vectors obtained from the existing engine fault. Then, after training the SVM multiple classifier by faulty feature vectors, the SVM model of the fault diagnosis system is established; Finally, the trained SVM multiple classifier is used to recognize and classify the test faults. Applying the noise on the test samples, SVM multiple classifiers can still get a good diagnosis effect. It shows that the fault diagnosis algorithm has good robustness and can be applied to the study of aero engine fault diagnosis.

Point-of-care diagnosis of bacterial pathogens in vitro, utilising an electronic nose and wavelet neural networks

Current clinical diagnostics are based on biochemical, immunological or microbiological methods. However, these methods are operator dependent, time-consuming and expensive and require special skills, and are therefore not suitable for point-of-care testing. Developments in gas-sensing technology and pattern recognition methods make electronic nose technology an interesting alternative for medical point-of-care devices. In this paper, the potential of an electronic nose as a monitoring tool in clinical microbiology is investigated through two case studies. Initially, an electronic nose based on chemoresistive sensors has been employed to identify in vitro 13 bacterial clinical isolates, collected from patients diagnosed with pathological infections in a Public Health Laboratory environment, while in a later stage, analysis was carried out for urinary tract infection-suspected cases incubated in a volatile generation test tube system for 4---5 h. Two issues have been considered the application of an advanced wavelet neural network and the concept of the fusion of multiple classifiers dedicated to specific feature parameters. The adopted wavelet neural network incorporates a "product operation" layer between wavelet functions and output layers, while the connection weights at output layer have been replaced by a local linear model. This study has shown the potential for early and fast detection of microbial contaminants in clinical samples utilising advanced learning-based algorithms and electronic nose technology.

Multi-sensor background subtraction by fusing multiple region-based probabilistic classifiers

In the recent years, the computer vision community has shown great interest on depth-based applications thanks to the performance and flexibility of the new generation of RGB-D imagery. In this paper, we present an efficient background subtraction algorithm based on the fusion of multiple region-based classifiers that processes depth and color data provided by RGB-D cameras. Foreground objects are detected by combining a region-based foreground prediction (based on depth data) with different background models (based on a Mixture of Gaussian algorithm) providing color and depth descriptions of the scene at pixel and region level. The information given by these modules is fused in a mixture of experts fashion to improve the foreground detection accuracy. The main contributions of the paper are the region-based models of both background and foreground, built from the depth and color data. The obtained results using different database sequences demonstrate that the proposed approach leads to a higher detection accuracy with respect to existing state-of-the-art techniques.

PHOTOMETRIC NORMALIZATION FOR FACE RECOGNITION USING LOCAL DISCRETE COSINE TRANSFORM

Variations in illumination is one of major limiting factors of face recognition system performance. The effect of changes in the incident light on face images is analyzed, as well as its influence on the low frequency components of the image. Starting from this analysis, a new photometric normalization method for illumination invariant face recognition is presented. Low-frequency Discrete Cosine Transform coefficients in the logarithmic domain are used in a local way to reconstruct a slowly varying component of the face image which is caused by illumination. After smoothing, this component is subtracted from the original logarithmic image to compensate for illumination variations. Compared to other preprocessing algorithms, our method achieved a very good performance with a total error rate very similar to that produced by the best performing state-of-the-art algorithm. An in-depth analysis of the two preprocessing methods revealed notable differences in their behavior, which is exploited in a multiple classifier fusion framework to achieve further performance improvement. The superiority of the proposal is demonstrated in both face verification and identification experiments.

Online Signature Verification Based on Feature Combination and Classifier Fusion

Fusion of multiple classifiers is an area of biometrics research that has recently gained in popularity. A scheme for online signature verification using feature combination and classifier fusion based on F-Tablet was proposed. On one hand, we paid attention to the feature combination, role of writing forces, and suitability of feature sets on each kind of classifier. Performance evaluation on F-Tablet for different classifiers and feature combinations suggested the average performance of most classifiers reached the best results when full coordinate and writing forces were presented. The DTW and SVM outperformed other classifiers in terms of EER with 2.86% and 3.68% respectively. However, when coordinate or writing forces only was offered, the writing forces outperformed trajectory, which indicated a huge information loss, especially in mobile environment that could not capture the writing forces for many sensors. On the other hand, in order to design high performance online signature verification system, it was necessary to consider the fusion of classifiers. The various classifier fusion schemes were compared experimentally. The performance of the fusion system was significantly improved compared with the performance of the single classifiers, with the best EER reaching 1.04%.