Pattern Classification Tasks Research Articles

Artificial Immune System-Based Classification in Extremely Imbalanced Classification Problems

This paper focuses on a special category of machine learning problems arising in cases where the set of available training instances is significantly biased towards a particular class of patterns. Our work addresses the so-called Class Imbalance Problem through the utilization of an Artificial Immune System-(AIS)based classification algorithm which encodes the inherent ability of the Adaptive Immune System to mediate the exceptionally imbalanced “self” / “non-self” discrimination process. From a computational point of view, this process constitutes an extremely imbalanced pattern classification task since the vast majority of molecular patterns pertain to the “non-self” space. Our work focuses on investigating the effect of the class imbalance problem on the AIS-based classification algorithm by assessing its relative ability to deal with extremely skewed datasets when compared against two state-of-the-art machine learning paradigms such as Support Vector Machines (SVMs) and Multi-Layer Perceptrons (MLPs). To this end, we conducted a series of experiments on a music-related dataset where a small fraction of positive samples was to be recognized against the vast volume of negative samples. The results obtained indicate that the utilized bio-inspired classifier outperforms SVMs in detecting patterns from the minority class while its performance on the same task is competently close to the one exhibited by MLPs. Our findings suggest that the AIS-based classifier relies on its intrinsic resampling and class-balancing functionality in order to address the class imbalance problem.

Realizing Two-View TSK Fuzzy Classification System by Using Collaborative Learning

In this paper, a novel Takagi-Sugeno–Kang (TSK) fuzzy classification system (FCS) is firstly presented for pattern classification tasks. It is distinguished by having the large margin criterion properly integrated into its objective function. In order to exploit the applicability of fuzzy systems in multiview scenarios, the proposed TSK-FCS is extended to a two-view version, called two-view TSK-FCS (TwoV-TSK-FCS), by using a collaborative learning mechanism. The adopted collaborative learning mechanism not only fully considers the independent information of each view, but also effectively discovers the correlation information hidden in the two views. Thus, the performance of TwoV-TSK-FCS can be enhanced accordingly. Comprehensive experiments on two-view synthetic and UCI datasets demonstrate the effectiveness of the proposed two-view FCS.

Improving the Fuzzy Min-Max neural network with a K-nearest hyperbox expansion rule for pattern classification

An improved Fuzzy Min-Max (FMM) neural network with a K-nearest hyperbox expansion rule is proposed in this paper. The aim is to reduce the FMM network complexity for undertaking pattern classification tasks. In the proposed model, a useful modification to overcome a number of identified limitations of the original FMM network and to improve its classification performance is derived. In particular, the K-nearest hyperbox expansion rule is formulated to reduce the network complexity by avoiding the creation of too many small hyperboxes within the vicinity of the winning hyperbox during the FMM learning stage. The effectiveness of the proposed model is evaluated using a number of benchmark data sets. The results compare favorably with those from various FMM variants and other existing classifiers.

Classification of Porcine Cranial Fracture Patterns Using a Fracture Printing Interface,.

Distinguishing between accidental and abusive head trauma in children can be difficult, as there is a lack of baseline data for pediatric cranial fracture patterns. A porcine head model has recently been developed and utilized in a series of studies to investigate the effects of impact energy level, surface type, and constraint condition on cranial fracture patterns. In the current study, an automated pattern recognition method, or a fracture printing interface (FPI), was developed to classify cranial fracture patterns that were associated with different impact scenarios documented in previous experiments. The FPI accurately predicted the energy level when the impact surface type was rigid. Additionally, the FPI was exceedingly successful in determining fractures caused by skulls being dropped with a high-level energy (97% accuracy). The FPI, currently developed on the porcine data, may in the future be transformed to the task of cranial fracture pattern classification for human infant skulls.

Sample diversity, representation effectiveness and robust dictionary learning for face recognition

Conventional dictionary learning algorithms suffer from the following problems when applied to face recognition. First, since in most face recognition applications there are only a limited number of original training samples, it is difficult to obtain a reliable dictionary with a large number of atoms from these samples. Second, because the face images of the same person vary with facial poses and expressions as well as illumination conditions, it is difficult to obtain a robust dictionary for face recognition. Thus, obtaining a robust and reliable dictionary is a crucial key to improve the performance of dictionary learning algorithms for face recognition. In this paper, we propose a novel dictionary learning framework to achieve this. The proposed algorithm framework takes training sample diversities of the same face image into account and tries to obtain more effective representations of face images and a more robust dictionary. It first produces virtual face images and then designs an elaborate objective function. Based on this objective function, we obtain a mathematically tractable and computationally efficient algorithm to generate a robust dictionary. Experimental results demonstrate that the proposed algorithm framework outperforms some previous state-of-the-art dictionary learning and sparse coding algorithms in face recognition. Moreover, the proposed algorithm framework can also be applied to other pattern classification tasks.

IDENTIFYING OPTIMAL NUMBER OF ORTHONORMALISATION IN LEARNING ALGORITHM USING WEATHER FORECASTING

Artificial neural networks are more powerful than any other traditional expert system in the classification of patterns, which are non linear and in performing pattern classification tasks because they learn from examples without explicitly stating the rules. Multilayered feed forward neural networks possess a number of properties, which make them particularly suited to complex problems. Their applications to some real world problems are hampered by the lack of a training algorithm which finds a globally optimal set of weights in a relatively short time. Genetic algorithms are a class of optimization procedures, which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum. In this study, conversion of random weights into orthonormal weights (orthonormalisation) in feed forward network algorithm is proposed and also optimal number of orthonormalisation of weight is evaluated. Orthonormal weight based artificial neural network algorithms not only succeed in its task but also outperforms the genetic algorithm. This success comes from orthonormal weights instead of random and genetic weights. This paper has documented the evolution and ultimate success of this algorithm using weather forecasting data.

Greedy dictionary learning for kernel sparse representation based classifier

We present a novel dictionary learning (DL) approach for sparse representation based classification in kernel feature space. These sparse representations are obtained using dictionaries, which are learned using training exemplars that are mapped into a high-dimensional feature space using the kernel trick. However, the complexity of such approaches using kernel trick is a function of the number of training exemplars. Hence, the complexity increases for large datasets, since more training exemplars are required to get good performance for most of the pattern classification tasks. To address this, we propose a hierarchical DL approach which requires the kernel matrix to update the dictionary atoms only once. Further, in contrast to the existing methods, the dictionary is learned in a linearly transformed/coefficient space involving sparse matrices, rather than the kernel space. Compared to the existing state-of-the-art methods, the proposed method has much less computational complexity, but performs similar for various pattern classification tasks.

Simultaneous dimensionality reduction and dictionary learning for sparse representation based classification

Learning dictionaries from the training data has led to promising results for pattern classification tasks. Dimensionality reduction is also an important issue for pattern classification. However, ...

Sliced Inverse Regression With Adaptive Spectral Sparsity for Dimension Reduction.

Dimension reduction is an important topic in pattern analysis and machine learning, and it has wide applications in feature representation and pattern classification. In the past two decades, sliced inverse regression (SIR) has attracted much research efforts due to its effectiveness and efficacy in dimension reduction. However, two drawbacks limit further applications of SIR. First, the computation complexity of SIR is usually high in the situation of high-dimensional data. Second, sparsity of projection subspace is not well mined for improving the feature selection and model interpretation abilities. This paper proposes to compute the SIR projection vectors in the spectral space, then an approximated regression solution can be obtained with a faster speed. Moreover, the adaptive lasso is used to attain a sparse and globally optimal solution, which is important in variable selection. To complete the robust pattern classification task with corruptions, a correntropy-based and class-wise regression model is designed in this paper. It takes a smooth penalty instead of sparsity constraint in the regression coefficients, and it can be conducted in class-wise, thus it is more flexible in practice. Extensive experiments are conducted by using some real and benchmark data sets, e.g., high-dimensional facial images and gene microarray data, to evaluate the new algorithms. The new proposals attain competitive results and are compared with other state-of-the-art methods.

Maximum margin hidden Markov models for sequence classification

Discriminative learning methods are known to work well in pattern classification tasks and often show benefits compared to generative learning. This is particularly true in case of model mismatch, i.e. the model cannot represent the true data distribution. In this paper, we derive discriminative maximum margin learning for hidden Markov models (HMMs) with emission probabilities represented by Gaussian mixture models (GMMs). The focus is on single-label sequence classification where the margin objective is specified by the probabilistic gap between the true class and the most competing class. In particular, we use the extended Baum–Welch (EBW) framework to optimize this probabilistic margin embedded in a hinge loss function. Approximations of the margin objective and the derivatives are necessary. In the experiments, we compare maximum margin HMMs to generative maximum likelihood and discriminative conditional log-likelihood (CLL) HMM training. We present results of classifying trajectories of handwritten characters, Australian sign language data, digits of speech data and UCR time-series data. Maximum margin HMMs outperform in many cases CLL-HMMs. Furthermore, maximum margin HMMs achieve a significantly better performance than generative maximum likelihood HMMs.

Discriminative Analysis Dictionary Learning

Dictionary learning (DL) has been successfully applied to various pattern classification tasks in recent years. However, analysis dictionary learning (ADL), as a major branch of DL, has not yet been fully exploited in classification due to its poor discriminability. This paper presents a novel DL method, namely Discriminative Analysis Dictionary Learning (DADL), to improve the classification performance of ADL. First, a code consistent term is integrated into the basic analysis model to improve discriminability. Second, a triplet constraint-based local topology preserving loss function is introduced to capture the discriminative geometrical structures embedded in data. Third, correntropy induced metric is employed as a robust measure to better control outliers for classification. Then, half-quadratic minimization and alternate search strategy are used to speed up the optimization process so that there exist closed-form solutions in each alternating minimization stage. Experiments on several commonly used databases show that our proposed method not only significantly improves the discriminative ability of ADL, but also outperforms state-of-the-art synthesis DL methods.

An efficient regularized K-nearest neighbor based weighted twin support vector regression

In general, pattern classification and regression tasks do not take into consideration the variation in the importance of the training samples. For twin support vector regression (TSVR), this implies that all the training samples play the same role on the bound functions. However, the number of close neighboring samples near to each training sample has an effect on the bound functions. In this paper, we formulate a regularized version of the KNN-based weighted twin support vector regression (KNNWTSVR) called RKNNWTSVR which is both efficient and effective. By introducing the regularization term and replacing 2-norm of slack variables instead of 1-norm, our RKNNWTSVR only needs to solve a simple system of linear equations with low computational cost, and at the same time, it improves the generalization performance. Particularly, we compare four implementations of RKNNWTSVR with existing approaches. Experimental results on several synthetic and benchmark datasets indicate that, comparing to SVR, WSVR, TSVR and KNNWTSVR, our proposed RKNNWTSVR has better generalization ability and requires less computational time.

Automatic classification of respiratory patterns involving missing data imputation techniques

A comparative study of the respiratory pattern classification task, involving five missing data imputation techniques and several machine learning algorithms is presented in this paper. The main goal was to find a classifier that achieves the best accuracy results using a scalable imputation method in comparison to the method used in a previous work of the authors. The results obtained show that in general, the Self-Organising Map imputation method allows non-tree based classifiers to achieve improvements over the rest of the imputation methods in terms of the classification accuracy, and that the Feedforward neural network and the Random Forest classifiers offer the best performance regardless of the imputation method used. The improvements in terms of accuracy over the previous work of the authors are limited but the Feed Forward neural network model achieves promising results.

Outlier detection using neighborhood rank difference

Presence of outliers critically affects many pattern classification tasks. In this paper, we propose a novel dynamic outlier detection method based on neighborhood rank difference. In particular, reverse and the forward nearest neighbor rank difference is employed to capture the variations in densities of a test point with respect to various training points. In the first step of our method, we determine the influence space for a given dataset. A score for outlierness is proposed in the second step using the rank difference as well as the absolute density within this influence space. Experiments on synthetic and some UCI machine learning repository datasets clearly indicate the supremacy of our method over some recently published approaches.

Regularized maximum correntropy machine

In this paper we investigate the usage of regularized correntropy framework for learning of classifiers from noisy labels. The class label predictors learned by minimizing transitional loss functions are sensitive to the noisy and outlying labels of training samples, because the transitional loss functions are equally applied to all the samples. To solve this problem, we propose to learn the class label predictors by maximizing the correntropy between the predicted labels and the true labels of the training samples, under the regularized Maximum Correntropy Criteria (MCC) framework. Moreover, we regularize the predictor parameter to control the complexity of the predictor. The learning problem is formulated by an objective function considering the parameter regularization and MCC simultaneously. By optimizing the objective function alternately, we develop a novel predictor learning algorithm. The experiments on two challenging pattern classification tasks show that it significantly outperforms the machines with transitional loss functions.

Data Stream Classification Based on the Gamma Classifier

The ever increasing data generation confronts us with the problem of handling online massive amounts of information. One of the biggest challenges is how to extract valuable information from these massive continuous data streams during single scanning. In a data stream context, data arrive continuously at high speed; therefore the algorithms developed to address this context must be efficient regarding memory and time management and capable of detecting changes over time in the underlying distribution that generated the data. This work describes a novel method for the task of pattern classification over a continuous data stream based on an associative model. The proposed method is based on the Gamma classifier, which is inspired by the Alpha-Beta associative memories, which are both supervised pattern recognition models. The proposed method is capable of handling the space and time constrain inherent to data stream scenarios. The Data Streaming Gamma classifier (DS-Gamma classifier) implements a sliding window approach to provide concept drift detection and a forgetting mechanism. In order to test the classifier, several experiments were performed using different data stream scenarios with real and synthetic data streams. The experimental results show that the method exhibits competitive performance when compared to other state-of-the-art algorithms.

Generalised Hough Transform for Speech Pattern Classification

While typical hybrid neural network architectures for automatic speech recognition (ASR) use a context window of frame-based features, this may not be the best approach to capture the wider temporal context, which contains phonetic and linguistic information that is equally important. In this paper, we introduce a system that integrates both the spectral and geometrical shape information from the acoustic spectrum, inspired by research in the field of machine vision. In particular, we focus on the Generalized Hough Transform (GHT), which is a sophisticated technique that can model the geometrical distribution of speech information over the wider temporal context. To integrate the GHT as part of a hybrid-ASR system, we propose to use a neural network, with features derived from the probabilistic Hough voting step of the GHT, to implement an improved version of the GHT where the output of the network represents the conventional target class posteriors. A major advantage of our approach is that each step of the GHT is highly interpretable, particularly compared to deep neural network (DNN) systems which are commonly treated as powerful black-box classifiers that give little insight into how the output is achieved. Experiments are carried out on two speech pattern classification tasks. The first is the TIMIT phoneme classification, which demonstrates the performance of the approach on a standard ASR task. The second is a spoken word recognition challenge, which highlights the flexibility of the approach to capture phonetic information within a longer temporal context.

Collaborative learning based on associative models: Application to pattern classification in medical datasets

This paper addresses social networking and collaborative learning in the medical domain by focusing on two main objectives: the first one concerns about social networking between computer science experts and postgraduate students, while the second concerns about collaborative learning between medical experts and less experienced physicians. The tasks of algorithms testing and performance evaluation were assigned to computer science postgraduate students. They made extensive use of social networking in order to implement associative models to perform pattern classification tasks in medical datasets and share performance results. Associative memories have a number of properties, including a rapid, compute efficient best-match and intrinsic noise tolerance that make them ideal for diagnostic hypothesis-generation processes in the medical domain. Using supervised machine learning algorithms allows less experienced physicians to compare their diagnostic results between workgroups and verify whether their knowledge is consistent with the results delivered by computational tools. Throughout the experimental phase the proposed algorithm is applied to help diagnose diseases; particularly, it is applied in the diagnosis of five different problems in the medical field. The performance of the proposed model is validated by comparing classification accuracy of DAM against the performance achieved by other twenty well known algorithms. Experimental results have shown that DAM achieved the best performance in three of the five pattern classification problems in the medical field. Similarly, it should be noted that our proposal achieved the best classification accuracy averaged over all datasets. Experimental results confirm that the proposed algorithm can be a valuable tool for promoting collaborative learning among less experienced physicians.

Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization

Nonnegative matrix factorization (NMF), a popular part-based representation technique, does not capture the intrinsic local geometric structure of the data space. Graph regularized NMF (GNMF) was recently proposed to avoid this limitation by regularizing NMF with a nearest neighbor graph constructed from the input data set. However, GNMF has two main bottlenecks. First, using the original feature space directly to construct the graph is not necessarily optimal because of the noisy and irrelevant features and nonlinear distributions of data samples. Second, one possible way to handle the nonlinear distribution of data samples is by kernel embedding. However, it is often difficult to choose the most suitable kernel. To solve these bottlenecks, we propose two novel graph-regularized NMF methods, AGNMFFS and AGNMFMK, by introducing feature selection and multiple-kernel learning to the graph regularized NMF, respectively. Instead of using a fixed graph as in GNMF, the two proposed methods learn the nearest neighbor graph that is adaptive to the selected features and learned multiple kernels, respectively. For each method, we propose a unified objective function to conduct feature selection/multi-kernel learning, NMF and adaptive graph regularization simultaneously. We further develop two iterative algorithms to solve the two optimization problems. Experimental results on two challenging pattern classification tasks demonstrate that the proposed methods significantly outperform state-of-the-art data representation methods.

On Equivalence of FIS and ELM for Interpretable Rule-Based Knowledge Representation.

This paper presents a fuzzy extreme learning machine (F-ELM) that embeds fuzzy membership functions and rules into the hidden layer of extreme learning machine (ELM). Similar to the concept of ELM that employed the random initialization technique, three parameters of F-ELM are randomly assigned. They are the standard deviation of the membership functions, matrix-C (rule-combination matrix), and matrix-D [don't care (DC) matrix]. Fuzzy if-then rules are formulated by the rule-combination Matrix of F-ELM, and a DC approach is adopted to minimize the number of input attributes in the rules. Furthermore, F-ELM utilizes the output weights of the ELM to form the target class and confidence factor for each of the rules. This is to indicate that the corresponding consequent parameters are determined analytically. The operations of F-ELM are equivalent to a fuzzy inference system. Several benchmark data sets and a real world fault detection and diagnosis problem have been used to empirically evaluate the efficacy of the proposed F-ELM in handling pattern classification tasks. The results show that the accuracy rates of F-ELM are comparable (if not superior) to ELM with distinctive ability of providing explicit knowledge in the form of interpretable rule base.