Pattern Classification Problems Research Articles

Feedforward Micro-Extended-Analog-Computer Array

The micro-Extended-Analog-Computater(uEAC) is a newly emergent unconventional computing model under research and development. Although, a single uEAC unit can achieve some classic pattern classification problem, the slop of the classification face generated by uEAC is a constant and moves parallely. So a feedforward array structure is proposed for the improvement of uEAC, and Particle Swarm Optimizer(PSO) is utilized to configure the uEAC array. The analysis of the mapping characteristics and numerical experiments both illustrate the effectiveness of the improvement.

Generating Singleton Fuzzy Models from Data with Interpretable Partition

This paper presents a new methodology for obtaining singleton fuzzy model from experimental data. Each input variable is partitioned into triangular membership functions so that consecutive fuzzy sets exhibit and specific overlapping of 0.5. The recursive least squares method is employed to adjust the singleton consequences and the gradient descent method is employed to update only the modal value of each triangular membership function to preserve the overlap and reducing the number of parameters to be estimated. Applications to a function approximation problem and to a pattern classification problem are illustrated.

Simulated Annealing Mechanic Based Noncoherent Signal Detection for Ultra-wideband Sensor Networks

Ultra-wideband (UWB) sensors have extensive commercial and military applications. Unfortunately, coherent signal detection in the presence of intensive multipath propagations may generally become impractical due to complicated realization algorithms and hardware requirements. In this article, we deal with noncoherent UWB signal detection within a promising biological framework, which can also be generalized to the binary-hypothesis target-detection problem, i.e. identification of the presence or absence of a target. Through the developed characteristic representations, signal detection is firstly formulated as a two-group pattern (or target) classification problem in a 2-D feature plane, in which an optimal decision bound can be numerically derived given the supervised training instances. This optimization problem is addressed by using the nature-inspired simulated annealing algorithm (SA), which essentially emulates the physical annealing process of forming the freeze state with the minimum energy. In sharp contrast to traditional optimization techniques, by probabilistically permitting search movement towards worse solutions, SA algorithm can converge to the global optimal with an asymptotical probability of 1. The numerically derived detection performance demonstrated that our present technique is much superior to the existing noncoherent schemes, which provides the appealing signal/target detection architecture for the emerging UWB sensor networks.

An ensemble method for fuzzy rule-based classification systems

Fuzzy rule-based classification systems are very useful tools in the field of machine learning as they are able to build linguistic comprehensible models. However, these systems suffer from exponential rule explosion when the number of variables increases, degrading, therefore, the accuracy of these systems as well as their interpretability. In this article, we propose to improve the comprehensibility through a supervised learning method by automatic generation of fuzzy classification rules, designated SIFCO–PAF. Our method reduces the complexity by decreasing the number of rules and of antecedent conditions, making it thus adapted to the representation and the prediction of rather high-dimensional pattern classification problems. We perform, firstly, an ensemble methodology by combining a set of simple classification models. Subsequently, each model uses a subset of the initial attributes: In this case, we propose to regroup the attributes using linear correlation search among the training set elements. Secondly, we implement an optimal fuzzy partition thanks to supervised discretization followed by an automatic membership functions construction. The SIFCO–PAF method, analyzed experimentally on various data sets, guarantees an important reduction in the number of rules and of antecedents without deteriorating the classification rates, on the contrary accuracy is even improved.

Performance Analysis of Neural Networks Training using Real Coded Genetic Algorithm

perceptrons (MLPs) are widely used for pattern classification and regression problems. Backpropagation (BP) algorithm is known technique in the training of multilayer perceptrons. However for its optimum training convergence, the learning and momentum parameters need to be tuned on trial and error method. Further, sometimes the backpropagation algorithm fails to achieve global convergence. To alleviate these problems we suggest a genetic algorithm based training for MLP network. Both binary coded and real coded genetic algorithm are used and a comparative training performance analysis has been studied. It is observed from simulation results that both the schemes outperform backpropagation algorithm and achieve global convergence. Further the real coded GA based training shows a faster convergence than binary coded GA based training. For simulation datasets are taken from UCI based machine learning repository. Hence real-coded genetic algorithm finds an alternative for back propagation based training algorithm.

Comparison of Different T-Norm Operators in Classification Problems

Fuzzy rule based classification systems are one of the most popular fuzzy modeling systems used in pattern classification problems. This paper investigates the effect of applying nine different T-norms in fuzzy rule based classification systems. In the recent researches, fuzzy versions of confidence and support merits from the field of data mining have been widely used for both rules selecting and weighting in the construction of fuzzy rule based classification systems. For calculating these merits the product has been usually used as a T-norm. In this paper different T-norms have been used for calculating the confidence and support measures. Therefore, the calculations in rule selection and rule weighting steps (in the process of constructing the fuzzy rule based classification systems) are modified by employing these T-norms. Consequently, these changes in calculation results in altering the overall accuracy of rule based classification systems. Experimental results obtained on some well-known data sets show that the best performance is produced by employing the Aczel-Alsina operator in terms of the classification accuracy, the second best operator is Dubois-Prade and the third best operator is Dombi. In experiments, we have used 12 data sets with numerical attributes from the University of California, Irvine machine learning repository (UCI).

Accelerated max-margin multiple kernel learning

Kernel machines such as Support Vector Machines (SVM) have exhibited successful performance in pattern classification problems mainly due to their exploitation of potentially nonlinear affinity str...

Application of Self-organizing Feature Maps Neural Network on Hydrographic Zones Partitioning

Hydrographic zones partitioning is the critical issue in hydrological station network planning and a hydrographic zones partition method that can divide the hydrographic zones objectively is desirable for any region. As a kind of pattern classification problem, the hydrographic zones partitioning of Shandong province is achieved by the self-organizing feature map neural network (SOFM network) which has been successfully applied to pattern classification problems. Based on 50 hydrological stations and 8 basic factors which reflect the land surface and hydroclimate characteristics, Shandong province is automatically divided into 2 hydrographic zones by SOFM network. The average watershed characteristics of each sub-zone are consistent with the local terrain and surface conditions. And based on the partitioning results of SOFM network method, the maximum peak flow and the accuracy are analyzed which are proved reasonable and achieve pass rate of 82%. Which indicate that it is an effective method to use SOFM neural network to divide the hydrographic zones.

On minimum distribution discrepancy support vector machine for domain adaptation

Domain adaptation learning (DAL) is a novel and effective technique to address pattern classification problems where the prior information for training is unavailable or insufficient. Its effectiveness depends on the discrepancy between the two distributions that respectively generate the training data for the source domain and the testing data for the target domain. However, DAL may not work so well when only the distribution mean discrepancy between source and target domains is considered and minimized. In this paper, we first construct a generalized projected maximum distribution discrepancy (GPMDD) metric for DAL on reproducing kernel Hilbert space (RKHS) based domain distributions by simultaneously considering both the projected maximum distribution mean and the projected maximum distribution scatter discrepancy between the source and the target domain. In the sequel, based on both the structure risk and the GPMDD minimization principle, we propose a novel domain adaptation kernelized support vector machine (DAKSVM) with respect to the classical SVM, and its two extensions called LS-DAKSVM and μ-DAKSVM with respect to the least-square SVM and the v-SVM, respectively. Moreover, our theoretical analysis justified that the proposed GPMDD metric could effectively measure the consistency not only between the RKHS embedding domain distributions but also between the scatter information of source and target domains. Hence, the proposed methods are distinctive in that the more consistency between the scatter information of source and target domains can be achieved by tuning the kernel bandwidth, the better the convergence of GPMDD metric minimization is and thus improving the scalability and generalization capability of the proposed methods for DAL. Experimental results on artificial and real-world problems indicate that the performance of the proposed methods is superior to or at least comparable with existing benchmarking methods.

Crossover-based local search in cooperative co-evolutionary feedforward neural networks

Cooperative coevolution has been a major approach to neuro-evolution. Memetic algorithms employ local search to selected individuals in a population. This paper presents a new cooperative coevolution framework that incorporates crossover-based local search. The proposed approach effectively makes use of local search without adding to the computational cost in the sub-populations of cooperative coevolution. The relationship between the intensity of, and interval between the local search is empirically investigated and a heuristic for the adaptation of the local search intensity during evolution is presented. The method is used for training feedforward neural networks on eight pattern classification problems. The results show an improved performance in terms of optimisation time, scalability and robustness for most of these problems.

An Automatic Approach for Learning and Tuning Gaussian Interval Type-2 Fuzzy Membership Functions Applied to Lung CAD Classification System

The potential of type-2 fuzzy sets to manage high levels of uncertainty in the subjective knowledge of experts or of numerical information has focused on control and pattern classification systems in recent years. One of the main challenges in designing a type-2 fuzzy logic system (FLS) is how to estimate the parameters of the type-2 fuzzy membership function (T2MF) and the footprint of uncertainty (FOU) from imperfect and noisy datasets. This paper presents an automatic approach to learn and tune Gaussian interval type-2 membership functions (IT2MFs) with application to multidimensional pattern classification problems. T2MFs and their FOUs are tuned according to the uncertainties in the training dataset by a combination of genetic algorithm (GA) and cross-validation techniques. In our GA-based approach, the structure of the chromosome has fewer genes than other GA methods, and chromosome initialization is more precise. The proposed approach addresses the application of the interval type-2 fuzzy logic system (IT2FLS) for the problem of nodule classification in a lung computer-aided detection system. The designed IT2FLS is compared with its type-1 fuzzy logic system (T1FLS) counterpart. The results demonstrate that the IT2FLS outperforms the T1FLS by more than 30% in terms of classification accuracy.

An Approach for IRIS Plant Classification Using Neural Network

Classification is a machine learning technique used to predict group membership for data instances. To simplify the problem of classification neural networks are being introduced. This paper focuses on IRIS plant classification using Neural Network. The problem concerns the identification of IRIS plant species on the basis of plant attribute measurements. Classification of IRIS data set would be discovering patterns from examining petal and sepal size of the IRIS plant and how the prediction was made from analyzing the pattern to form the class of IRIS plant. By using this pattern and classification, in future upcoming years the unknown data can be predicted more precisely. Artificial neural networks have been successfully applied to problems in pattern classification, function approximations, optimization, and associative memories. In this work, Multilayer feed- forward networks are trained using back propagation learning algorithm.

On the issue of separability for problem decomposition in cooperative neuro-evolution

Cooperative coevolution divides an optimisation problem into subcomponents and employs evolutionary algorithms for evolving them. Problem decomposition has been a major issue in using cooperative coevolution for neuro-evolution. Efficient problem decomposition methods group interacting variables into the same subcomponents. It is important to find out which problem decomposition methods efficiently group subcomponents and the behaviour of neural network during training in terms of the interaction among the synapses. In this paper, the interdependencies among the synapses are analysed and a problem decomposition method is introduced for feedforward neural networks on pattern classification problems. We show that the neural network training problem is partially separable and that the level of interdependencies changes during the learning process. The results confirm that the proposed problem decomposition method has improved performance compared to its counterparts.

Towards Hebbian learning of Fuzzy Cognitive Maps in pattern classification problems

A detailed comparative analysis of the Hebbian-like learning algorithms applied to train Fuzzy Cognitive Maps (FCMs) operating as pattern classifiers, is presented in this paper. These algorithms aim to find appropriate weights between the concepts of the FCM classifier so it equilibrates to a desired state (class mapping). For these purposes, six different types of Hebbian learning algorithms from the literature have been selected and studied in this work. Along with the theoretical description of these algorithms and the analysis of their performance in classifying known patterns, a sensitivity analysis of the applied classification scheme, regarding some configuration parameters have taken place. It is worth noting that the algorithms are studied in a comparative fashion, under common configurations for several benchmark pattern classification datasets, by resulting to useful conclusions about their training capabilities.

Anomaly Based IDS Using Variable Size Detector Generation in AIS: A Hybrid Approach

Artificial Immune System (AIS) is inspired by biological immune system and provides novel ways to solve complex problems including fault detection, optimization and anomaly detection. Artificial Negative Selection is one the most important branches in AIS that discriminates normal and anomalous samples based on natural immune system self/non-self discrimination mechanism. In this paper a new schema of detector generation approach for negative selection is introduced. Negative selection is typically applied to anomaly detection problems, which can be considered as a type of pattern classification problem and is typically employed as an intrusion detection technique. This new approach, hybrids ideas from Artificial Immune Negative Selection algorithms and Restricted Coulomb Energy neural networks that are specific design of hyper sphere classifiers. While generated detectors have variable radius in real-valued space. The algorithm is tested using real-world datasets, including NSL-KDD99. The experiments in this paper showed the algorithm can tightly control the number of generated detectors.

Recursive Learning of Genetic Algorithm Featuring Incremental Attribute Learning for Higher Dimensional Classification Problems

Rule-based classifiers trained by Genetic Algorithms (GAs) have been one of the most prevailing solutions for pattern classification problem. This paper introduces an algorithm named Recursive Learning of Genetic Algorithms featuring Incremental Attribute Learning (RLGA-IAL) developed from the Recursive Learning of Genetic Algorithms with Task Decomposition and Varied Rule Set (RLGA). Instead of training all the attributes in a batch, RLGA-IAL integrates the attributes sequentially. By projecting a large multidimensional search space to single-dimensional space spaces with integration, it reduces the difficulty in deriving the classification rules.Though a series of experiments, RLGA-IAL shows a successful and promising performance in classification problems with the dimension of the datasets ranging from 5 to 60.

Daily maximum load forecasting of consecutive national holidays using OSELM-based multi-agents system with weighted average strategy

In the previous research, a Multi-Agent System based on Online Sequential Extreme Learning Machine (OSELM) neural network and Bayesian Formalism (MAS-OSELM-BF) has been introduced for solving pattern classification problems. However this model is incapable of handling regression tasks. In this article, a new OSELM-based multi-agent system with weighted average strategy (MAS-OSELM-WA) is introduced for solving data regression tasks. A MAS-OSELM-WA consists of several individual OSELM (individual agent) and the final decision (parent agent). The outputs of the individual agents are sent to the parent agent for a final decision whereby the coefficients of parent agent are computed by a gradient descent method. The effectiveness of the MAS-OSELM-WA is evaluated by an electrical load forecasting problem in Malaysia for a month with consequent national holidays (i.e., during the month of Hari Raya—Malay New Year of Malaysia). The results demonstrated that the MAS-OSELM-WA is able to produce good performance as compared with the other approaches.

One-against-all ensemble for multiclass pattern classification

This paper presents a new method called one-against-all ensemble for solving multiclass pattern classification problems. The proposed method incorporates a neural network ensemble into the one-against-all method to improve the generalization performance of the classifier. The experimental results show that the proposed method can reduce the uncertainty of the decision and it is comparable to the other widely used methods.

Multi-objective hybrid evolutionary algorithms for radial basis function neural network design

This paper presents new multi-objective evolutionary hybrid algorithms for the design of Radial Basis Function Networks (RBFNs) for classification problems. The algorithms are memetic Pareto particle swarm optimization based RBFN (MPPSON), Memetic Elitist Pareto non dominated sorting genetic algorithm based RBFN (MEPGAN) and Memetic Elitist Pareto non dominated sorting differential evolution based RBFN (MEPDEN). The proposed methods integrate accuracy and structure of RBFN simultaneously. These algorithms are implemented on two-class and multiclass pattern classification problems with one complex real problem. The results reveal that the proposed methods are viable, and provide an effective means to solve multi-objective RBFNs with good generalization ability and simple network structure. The accuracy and complexity of the network obtained by the proposed algorithms are compared through statistical tests. This study shows that the proposed methods obtain RBFNs with an appropriate balance between accuracy and simplicity.

Indexing-Based Pattern Recognition

The paper discusses the mathematics of pattern indexing and its applications to recognition of visual patterns and classification of objects that are represented by objects-properties matrices. It is shown that (a) pattern indexes can be represented by collections of inverted patterns, (b) solutions to pattern classification problems can be found as intersections of inverted patterns and, thus, matching of original patterns avoided.