High-dimensional Pattern Classification Problems Research Articles

Colorectal Cancer Diagnostic Algorithm Based on Sub-Patch Weight Color Histogram in Combination of Improved Least Squares Support Vector Machine for Pathological Image

In order to improve the diagnostic accuracy of colon cancer, a novel classification algorithm based on sub-patch weight color histogram and improved SVM is proposed, which has good approximation ability for complex pathological image. Our proposed algorithm combines wavelet kernel SVM with color histogram to classify pathological image. Firstly, the pathological image is divided into non-overlapping sub-patches, and the features of sub-patch histogram are extracted. Then, the global and local features are fused by the sub-patch weighting algorithm. Then, the RelicfF based forward selection algorithm is used to integrate color features and texture features so as to enhance the characterization capabilities of the tumor cell. Finally, Morlet wavelet kernel-based least squares support vector machine method is adopted to enhance the generalization ability of the model for small sample with non-linear and high-dimensional pattern classification problems. Experimental results show that the proposed pathological diagnostic algorithm can gain higher accuracy compared with existing comparison algorithms.

Classification of motor imagery tasks for BCI with multiresolution analysis and multiobjective feature selection.

BackgroundBrain-computer interfacing (BCI) applications based on the classification of electroencephalographic (EEG) signals require solving high-dimensional pattern classification problems with such a relatively small number of training patterns that curse of dimensionality problems usually arise. Multiresolution analysis (MRA) has useful properties for signal analysis in both temporal and spectral analysis, and has been broadly used in the BCI field. However, MRA usually increases the dimensionality of the input data. Therefore, some approaches to feature selection or feature dimensionality reduction should be considered for improving the performance of the MRA based BCI.MethodsThis paper investigates feature selection in the MRA-based frameworks for BCI. Several wrapper approaches to evolutionary multiobjective feature selection are proposed with different structures of classifiers. They are evaluated by comparing with baseline methods using sparse representation of features or without feature selection.Results and conclusionThe statistical analysis, by applying the Kolmogorov-Smirnoff and Kruskal–Wallis tests to the means of the Kappa values evaluated by using the test patterns in each approach, has demonstrated some advantages of the proposed approaches. In comparison with the baseline MRA approach used in previous studies, the proposed evolutionary multiobjective feature selection approaches provide similar or even better classification performances, with significant reduction in the number of features that need to be computed.

Noise-free representation based classification and face recognition experiments

The representation based classification has achieved promising performance in high-dimensional pattern classification problems. As we know, in real-world applications the samples are usually corrupted by noise. However, representation based classification can take only noise in the test sample into account and is not able to deal with noise in the training sample, which causes side-effect on the classification result. In order to make the representation based classification more suitable for real-world applications such as face recognition, we propose a new representation based classification method in this paper. This method can effectively and simultaneously reduce noise in the test and training samples. Moreover, the proposed method can reduce noise in both the original and virtual training samples and then exploits them to determine the label of the test sample. The virtual training sample is generated from the original face image and shows possible variation of the face in scale, facial pose and expression. The experimental results show that the proposed method performs very well in face recognition.

An ensemble method for fuzzy rule-based classification systems

Fuzzy rule-based classification systems are very useful tools in the field of machine learning as they are able to build linguistic comprehensible models. However, these systems suffer from exponential rule explosion when the number of variables increases, degrading, therefore, the accuracy of these systems as well as their interpretability. In this article, we propose to improve the comprehensibility through a supervised learning method by automatic generation of fuzzy classification rules, designated SIFCO–PAF. Our method reduces the complexity by decreasing the number of rules and of antecedent conditions, making it thus adapted to the representation and the prediction of rather high-dimensional pattern classification problems. We perform, firstly, an ensemble methodology by combining a set of simple classification models. Subsequently, each model uses a subset of the initial attributes: In this case, we propose to regroup the attributes using linear correlation search among the training set elements. Secondly, we implement an optimal fuzzy partition thanks to supervised discretization followed by an automatic membership functions construction. The SIFCO–PAF method, analyzed experimentally on various data sets, guarantees an important reduction in the number of rules and of antecedents without deteriorating the classification rates, on the contrary accuracy is even improved.

Automatic neural-based pattern classification of motion behaviors in autonomous robots

This paper addresses the problem of providing autonomous robots with a system that allows them to classify the motion behavior patterns of groups of robots present in their surroundings. It is a first step in the development of a cognitive model that can detect and understand the events occurring in the environment that are not due to the robot's own actions. The recognition of motion patterns must be achieved from the input data acquired by the robot through its camera during real time operation and, consequently, it can be addressed as a high dimensional dynamic pattern classification problem. Artificial Neural Networks (ANN) have been widely used in this type of classification problems, where a preprocessing stage is typically introduced in order to reduce dimensionality. In this stage, the processing window size and the dimensional transformation parameters must be selected according to specific domain knowledge, and they remain fixed during the ANN classification process. Such an approach is not applicable here as there is no prior information on the number of robots present or the dimensional reduction level required to describe the possible robot motion behaviors. Consequently, this work proposes a hybrid approach based on the application of a classification system called ANPAC (Automatic Neural-based Pattern Classifier) that uses a variable size ANN to perform the classification and an advisor module to adjust the preprocessing parameters and, consequently, the size of the ANN, depending on the learning results of the network. The components and operation of ANPAC are described in depth and illustrated using an example related to the recognition of behavior patterns in the motion of flocks.

Special issue: Hybrid intelligent systems in ensembles

Efforts to artificially produce intelligence via association of computing methodologies to cooperatively achieve higher performance system requirements have reached considerable maturity. In many fields, hybrid systems have succeeded to provide theoretical basis and methodological support to build real world applications. For instance, intelligent systems applications in areas such as modeling, control, optimization, classification, forecasting, information search and retrieval, image processing and recognition are common nowadays. The Fifth International Conference on Hybrid Intelligent Systems HIS’05 was a major forum that brought state of the art information to researchers and practitioners that came worldwide. Neural networks, evolutionary computation, fuzzy systems, learning, search, support vector machines, clustering, classification, swarm intelligence, agents, artificial intelligence, to mention a few, were among the techniques used in hybrid systems suggested by authors in their papers. This special issue collects contributions that nurture hybrid intelligent systems growth, emphasizing a balance between new developments and applications. The papers selected for this special issue focus on hybrid intelligent systems in ensembles. Committees or ensembles are receiving more attention from the computational intelligence community in order to improve the performance of pattern recognition systems. They produce a consensus decision that is potentially more accurate than individual models. This strategy is particularly useful when the available committee members are individually efficient and err in different regions of the feature space. After preliminary indications made by conference papers reviewers, anonymous referees did a second refereeing process. The result is a comprehensive and representative set of papers whose content mirror the quality of the conference. The first paper, entitled Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers, by Ishibuchi and Nojima, presents a hybrid system, based on evolutionary multiobjective optimization algorithms, applied to the design of fuzzy rule-based ensemble classifiers. The idea is to attain a better performance by creating an ensemble, with high diversity, in a high-dimensional pattern classification problem. To reduce the curse of dimensionality problem, that is, the exponential increase in the total number of possible rules with the increase in the number of attributes, authors proposed a two-stage fuzzy rule selection scheme, where, in the first stage, a pre-specified number of promising fuzzy rules are generated as candidate rules, and in the second stage a genetic rule selection algorithm is applied to the set of candidate rules to construct fuzzy rule-based classifiers with high accuracy and high interpretability. The proposed hybrid system was applied to six high-dimensional data sets, providing interesting results.

Minimal Structure of Self-Organizing HCMAC Neural Network Classifier

The authors previously proposed a self-organizing Hierarchical Cerebellar Model Articulation Controller (HCMAC) neural network containing a hierarchical GCMAC neural network and a self-organizing input space module to solve high-dimensional pattern classification problems. This novel neural network exhibits fast learning, a low memory requirement, automatic memory parameter determination and highly accurate high-dimensional pattern classification. However, the original architecture needs to be hierarchically expanded using a full binary tree topology to solve pattern classification problems according to the dimension of the input vectors. This approach creates many redundant GCMAC nodes when the dimension of the input vectors in the pattern classification problem does not exactly match that in the self-organizing HCMAC neural network. These redundant GCMAC nodes waste memory units and degrade the learning performance of a self-organizing HCMAC neural network. Therefore, this study presents a minimal structure of self-organizing HCMAC (MHCMAC) neural network with the same dimension of input vectors as the pattern classification problem. Additionally, this study compares the learning performance of this novel learning structure with those of the BP neural network,support vector machine (SVM), and original self-organizing HCMAC neural network in terms of ten benchmark pattern classification data sets from the UCI machine learning repository. In particular, the experimental results reveal that the self-organizing MHCMAC neural network handles high-dimensional pattern classification problems better than the BP, SVM or the original self-organizing HCMAC neural network. Moreover, the proposed self-organizing MHCMAC neural network significantly reduces the memory requirement of the original self-organizing HCMAC neural network, and has a high training speed and higher pattern classification accuracy than the original self-organizing HCMAC neural network in most testing benchmark data sets. The experimental results also show that the MHCMAC neural network learns continuous function well and is suitable for Web page classification.

進化型多目的最適化手法を用いたファジィルール選択

One advantage of evolutionary multiobjective optimization (EMO) algorithms over classical approaches is that many non-dominated solutions can be simultaneously obtained by their single run. This paper shows how this advantage can be utilized in genetic rule selection for the design of fuzzy rule-based classification systems. Our genetic rule selection is a two-stage approach. In the first stage, a pre-specified number of candidate rules are extracted from numerical data using a data mining technique. In the second stage, an EMO algorithm is used for finding non-dominated rule sets with respect to three objectives. Since one of the three objectives is to maximize a classification rate on training patterns, the evolution of rule sets tends to overfit to training patterns. The question is whether the other two objectives with respect to complexity work as a safeguard against the over-fitting. In this paper, we examine the effect of the three-objective formulation on the generalization ability of obtained rule sets through computational experiments where many non-dominated rule sets are generated using an EMO algorithm for a number of high-dimensional pattern classification problems. Finally, we demonstrate that an ensemble of generated fuzzy rule-based systems leads to high generalization ability.

Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining

This paper shows how a small number of simple fuzzy if-then rules can be selected for pattern classification problems with many continuous attributes. Our approach consists of two phases: candidate rule generation by rule evaluation measures in data mining and rule selection by multi-objective evolutionary algorithms. In our approach, first candidate fuzzy if-then rules are generated from numerical data and prescreened using two rule evaluation measures (i.e., confidence and support) in data mining. Then a small number of fuzzy if-then rules are selected from the prescreened candidate rules using multi-objective evolutionary algorithms. In rule selection, we use three objectives: maximization of the classification accuracy, minimization of the number of selected rules, and minimization of the total rule length. Thus the task of multi-objective evolutionary algorithms is to find a number of non-dominated rule sets with respect to these three objectives. The main contribution of this paper is to propose an idea of utilizing the two rule evaluation measures as prescreening criteria of candidate rules for fuzzy rule selection. An arbitrarily specified number of candidate rules can be generated from numerical data for high-dimensional pattern classification problems. Through computer simulations, we demonstrate that such a prescreening procedure improves the efficiency of our approach to fuzzy rule selection. We also extend a multi-objective genetic algorithm (MOGA) in our former studies to a multi-objective genetic local search (MOGLS) algorithm where a local search procedure adjusts the selection (i.e., inclusion or exclusion) of each candidate rule. Furthermore, a learning algorithm of rule weights (i.e., certainty factors) is combined with our MOGLS algorithm. Such extensions to our MOGA for fuzzy rule selection are another contribution of this paper.

Three-objective genetics-based machine learning for linguistic rule extraction

Abstract This paper shows how a small number of linguistically interpretable fuzzy rules can be extracted from numerical data for high-dimensional pattern classification problems. One difficulty in the handling of high-dimensional problems by fuzzy rule-based systems is the exponential increase in the number of fuzzy rules with the number of input variables. Another difficulty is the deterioration in the comprehensibility of fuzzy rules when they involve many antecedent conditions. Our task is to design comprehensible fuzzy rule-based systems with high classification ability. This task is formulated as a combinatorial optimization problem with three objectives: to maximize the number of correctly classified training patterns, to minimize the number of fuzzy rules, and to minimize the total number of antecedent conditions. We show two genetic-algorithm-based approaches. One is rule selection where a small number of linguistically interpretable fuzzy rules are selected from a large number of prespecified candidate rules. The other is fuzzy genetics-based machine learning where rule sets are evolved by genetic operations. These two approaches search for non-dominated rule sets with respect to the three objectives.

Performance evaluation of fuzzy rule-based classification systems obtained by multi-objective genetic algorithms

In this paper, we examine the classification performance of fuzzy if-then rules selected by a GA-based multi-objective rule selection method. This rule selection method can be applied to high-dimensional pattern classification problems with many continuous attributes by restricting the number of antecedent conditions of each candidate fuzzy if-then rule. As candidate rules, we only use fuzzy if-then rules with a small number of antecedent conditions. Thus it is easy for human users to understand each rule selected by our method. Our rule selection method has two objectives: to minimize the number of selected fuzzy if-then rules and to maximize the number of correctly classified patterns. In our multi-objective fuzzy rule selection problem, there exist several solutions ( i.e., several rule sets) called “non-dominated solutions” because two conflicting objectives are considered. In this paper, we examine the performance of our GA-based rule selection method by computer simulations on a real-world pattern classification problem with many continuous attributes. First we examine the classification performance of our method for training patterns by computer simulations. Next we examine the generalization ability for test patterns. We show that a fuzzy rule-based classification system with an appropriate number of rules has high generalization ability.

Correlative components analysis for pattern classification

Based upon a novel method named correlative components analysis, a simple but efficient pattern classification technique is proposed in this paper. Using this method, the relatively important components of high-dimensional pattern can be successfully identified, the original problem will be mapped onto a lower dimensional space, and therefore the complexity of a high-dimensional pattern classification problem will be substantially reduced. For comparing with the methods of sequential discriminant analysis and the principal component analysis, an example of classifying complex chemical information was used, and the results verified that the new method is reasonable in principle and successful in practice.