Kernel Subspace Methods Research Articles

Sparse kernel subspace method for classifying and representing patterns from data with complex structure

The subspace method based on principal component analysis (PCA) is a useful tool for classifying and representing patterns. In order to learn the dynamical nonlinear structure of data, there is currently much discussion about kernel subspace methods based on kernel PCA. By using a nonlinear transformation, we can efficiently capture the nonlinear structure of data, which leads to effective subspace construction for data having a dynamical nonlinear structure. However, the performance of a kernel subspace method becomes dramatically worse in the presence of noisy features, because the method is based on a fully dense loading matrix in the constructed subspace. This implies that the subspace for each class is constructed by using not only crucial features but also noisy features, which significantly disturb the PCA procedures. To resolve this issue, we propose a sparse kernel subspace method based on sparse PCA. In order to construct subspaces based only on crucial features, we incorporate sparsity into the eigenvector estimation procedures in the nonlinear feature space. We then construct the subspaces of nonlinear features based on the sparse eigenvectors, and classify the unknown vector by projecting it into the constructed subspace. The proposed method can effectively not only capture nonlinear structures of data but also estimate eigenvectors even in the presence of noisy features. The effectiveness of the proposed method leads to the high-precision of the classification. Monte Carlo simulations and a real data analysis show that the proposed sparse kernel subspace method provides effective results for classification.

Learning kernel subspace for face recognition

Kernel subspace methods are effective for face recognition. However, their final solutions need to take advantage of all training examples, such that their runs in real-world applications with large scale training data sets are time-consuming. This paper proposes to apply radial basis function neural network (RBFNN) to learn the feature extraction process of kernel subspace methods, specifically, Kernel Principle Component Analysis (KPCA) and two-phase Kernel Linear Discriminant Analysis (KLDA or KPCA+LDA), in order to improve the running efficiency of testing phase of kernel-based face recognition system. Experimental results based on three small face benchmark data sets including ORL, YALE and UMIST as well as two larger benchmark data sets including AR and FERET, show that the proposed method can reach approximately the recognition accuracy of the original KPCA or the two-phase KLDA by using less vectors in its final solution than those of the two original kernel methods. The proposed method can be applied to real-time or online face recognition systems.

Adaptive kernel subspace method for speeding up feature extraction

KPCA can extract nonlinear features of data set. However, its efficiency is in inverse proportion to the size of the training sample set. In this paper, we proposed an adaptive kernel subspace method to extract features efficiently. The method is methodologically consistent with KPCA, and can improve the efficiency by adaptively selecting the spanning vectors of the kernel principal components, meanwhile, not affect the accuracy much. Experiments on two-dimensional data, MNIST dataset and USPS dataset show that the feature extraction method is more efficient than that associated with KPCA and reference methods.

New face recognition algorithm based on nonlinear dimensionality reduction

A new supervised kernel subspace method for face recognition was proposed. It makes a better use of samples' label information by introducing supervision in the construction of local neighborhood so as to improve the recognition efficiency; and a positive regularization was added in the gaining of the optimal reconstruction weight matrix to make it insensitive to the noise. In experimental phase, the ATT and Yale face databases were used to test this algorithm by using the Nearest Neighborhood (NN) algorithm to construct the classifiers. Experimental results show that the proposed method is effective, robust and is superior to non-supervised KNPP and the traditional classic methods.

A new extension of kernel feature and its application for visual recognition

In this paper, we first conceive a new perception of the kernel feature. The kernel subspace methods can be regarded as two independent steps: an explicit kernel feature extraction step and a linear subspace analysis step on the extracted kernel features. The kernel feature vector of an image is composed of dot products between the image and all the training images using nonlinear dot product kernel. Then, based on this perception, we further extend the kernel feature vector of an image to a kernel feature matrix for visual recognition. This extension takes different representation cues of images into account, respectively, while only global average information is used in the traditional kernel methods. From the view of dot product as similarity, this extension means using multiple similarities to measure two images, which is more accordant to human vision. In order to efficiently deal with the problem of numerical computation, a matrix-based kernel discriminant analysis algorithm is employed to learn discriminating kernel features for visual recognition. Experiments on the FERET face database, the COIL-100 object database, and the Wang's nature image database show the advantage of the proposed method.

Multispectral image segmentation of paintings drawn with natural mineral pigments using the kernel subspace method

Multispectral image segmentation of paintings drawn with natural mineral pigments using the kernel subspace method

MULTICLASS MOLECULAR CANCER CLASSIFICATION BY KERNEL SUBSPACE METHODS WITH EFFECTIVE KERNEL PARAMETER SELECTION

Microarray techniques provide new insights into molecular classification of cancer types, which is critical for cancer treatments and diagnosis. Recently, an increasing number of supervised machine learning methods have been applied to cancer classification problems using gene expression data. Support vector machines (SVMs), in particular, have become one of the most effective and leading methods. However, there exist few studies on the application of other kernel methods in the literature. We apply a kernel subspace (KS) method to multiclass cancer classification problems, and assess its validity by comparing it with multiclass SVMs. Our comparative study using seven multiclass cancer datasets demonstrates that the KS method has high performance that is comparable to multiclass SVMs. Furthermore, we propose an effective criterion for kernel parameter selection, which is shown to be useful for the computation of the KS method.