Kernel Canonical Correlation Analysis Algorithm Research Articles

K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human–Robot Interaction

In this article, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -meansclustering-based Kernel canonical correlation analysis algorithm is proposed for multimodal emotion recognition in human–robot interaction (HRI). The multimodal features (gray pixels; time and frequency domain) extracted from facial expression and speech are fused based on Kernel canonical correlation analysis. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means clustering is used to select features from multiple modalities and reduce dimensionality. The proposed approach can improve the heterogenicity among different modalities and make multiple modalities complementary to promote multimodal emotion recognition. Experiments on two datasets, namely SAVEE and eNTERFACE‘05, are conducted to evaluate the accuracy of the proposed method. The results show that the proposed method produces good recognition rates that are higher than the ones produced by the methods without <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means clustering; more specifically, they are 2.77% higher in SAVEE and 4.7% higher in eNTERFACE‘05.

A New Feature Extraction Algorithm Based on Orthogonal Regularized Kernel CCA and Its Application

In this paper, an orthogonal regularized kernel canonical correlation analysis algorithm (ORKCCA) is proposed. ORCCA algorithm can deal with the linear relationships between two groups of random variables. But if the linear relationships between two groups of random variables do not exist, the performance of ORCCA algorithm will not work well. Linear orthogonal regularized CCA algorithm is extended to nonlinear space by introducing the kernel method into CCA. Simulation experimental results on both artificial and handwritten numerals databases show that the proposed method outperforms ORCCA for the nonlinear problems.

A new randomized Kaczmarz based kernel canonical correlation analysis algorithm with applications to information retrieval

Canonical correlation analysis (CCA) is a powerful statistical tool for detecting the linear relationship between two sets of multivariate variables. Kernel generalization of it, namely, kernel CCA is proposed to describe nonlinear relationship between two variables. Although kernel CCA can achieve dimensionality reduction results for high-dimensional data feature selection problem, it also yields the so called over-fitting phenomenon. In this paper, we consider a new kernel CCA algorithm via randomized Kaczmarz method. The main contributions of the paper are: (1) A new kernel CCA algorithm is developed, (2) theoretical convergence of the proposed algorithm is addressed by means of scaled condition number, (3) a lower bound which addresses the minimum number of iterations is presented. We test on both synthetic dataset and several real-world datasets in cross-language document retrieval and content-based image retrieval to demonstrate the effectiveness of the proposed algorithm. Numerical results imply the performance and efficiency of the new algorithm, which is competitive with several state-of-the-art kernel CCA methods.

Robust kernel canonical correlation analysis with applications to information retrieval

Canonical correlation analysis (CCA) is a powerful statistical tool quantifying correlations between two sets of multidimensional variables. CCA cannot detect nonlinear relationship, and it is costly to derive canonical variates for high-dimensional data. Kernel CCA, a nonlinear extension of the CCA method, can efficiently exploit nonlinear relations and reduce high dimensionality. However, kernel CCA yields the so called over-fitting phenomenon in the high-dimensional feature space. To handle the shortcomings of kernel CCA, this paper develops a novel robust kernel CCA algorithm (KCCA-ROB). The derived method begins with reformulating the traditional generalized eigenvalue–eigenvector problem into a new framework. Under this novel framework, we develop a stable and fast algorithm by means of singular value decomposition (SVD) method. Experimental results on both a simulated dataset and real-world datasets demonstrate the effectiveness of the developed method.

Kernel canonical correlation analysis via gradient descent

Kernel canonical correlation analysis (CCA) is a powerful statistical tool characterizing nonlinear relations between two sets of multidimensional variables. It has been widely used in many branches of science and technology, e.g. bioinfomatics, multi-media information retrieval, cross-language document retrieval, fMRI (functional magnetic resonance imaging). Previous algorithms focus on sparsity analysis of kernel CCA. In this paper, from another viewpoint, we address a new gradient descent kernel CCA algorithm, which is based on the relation between kernel CCA and linear systems of equations. Meanwhile, stability analysis of the algorithm is addressed by means of suitable error decomposition formula and compact operator theory. Theoretical analysis is elegantly investigated in terms of choices of regularization parameter and step size. Experimental results on real-world datasets demonstrate the effectiveness of the algorithm for content-based image retrieval task. The results indicate that the proposed algorithm is stable and the performance is comparable with several state-of-the-art CCA algorithms.

New feature extraction method and its application to pattern recognition

Kernel Canonical Correlation Analysis(KCCA) is a recently addressed supervised machine learning methods,which is a powerful approach of extracting nonlinear features.However,the standard KCCA algorithm may suffer from computational problem as the training set increases.To overcome the drawback,an improved KCCA was proposed.Firstly,a scheme based on geometrical consideration was proposed to select a subset of samples that were projected to feature space(Reproducing Kernel Hilbert Space).And then,an efficient algorithm was proposed to enhance the efficiency of the feature extraction,which selected the most contributive eigenvectors for training and classification,and then calculated the corresponding eigenvectors for classification.Finally,the improved KCCA was combined with a multi-class classification method based on Support Vectors Data Description(SVDD) for classification and recognition,which put forward new ideas for pattern recognition based on kernels.The experimental results on ORL face database show that the proposed method reduces the training time and the system storage without deteriorating the recognition accuracy compared with standard KCCA.

Accelerated Kernel CCA plus SVDD: A Three-stage Process for Improving Face Recognition

Kernel canonical correlation analysis (KCCA) is a recently addressed supervised machine learning methods, which shows to be a powerful approach of extracting nonlinear features for face classification and other applications. However, the standard KCCA algorithm may suffer from computational problem as the training set increase. To overcome the drawback, we propose a three-stage method to improve the performance of KCCA. Firstly, a scheme based on geometrical consideration is proposed to enhance the extraction efficiency. The algorithm can select a subset of samples whose projections in feature space (Hilbert space) are sufficient to represent all of the data in feature space. Subsequently, an improved algorithm inspired by principal component analysis (PCA) is developed. The algorithm can select the most contributive eigenvectors for training and classification instead of considering all the ones. Finally, a multi-class classification method based on support vectors data description (SVDD) is employed to further enhance the recognition performance as it can avoid the repeated use of training data. The theoretical analysis and the experiment results demonstrate the effectiveness of improvements.

Facial Expression Recognition Using Kernel Canonical Correlation Analysis (KCCA)

In this correspondence, we address the facial expression recognition problem using kernel canonical correlation analysis (KCCA). Following the method proposed by Lyons et al. and Zhang et al., we manually locate 34 landmark points from each facial image and then convert these geometric points into a labeled graph (LG) vector using the Gabor wavelet transformation method to represent the facial features. On the other hand, for each training facial image, the semantic ratings describing the basic expressions are combined into a six-dimensional semantic expression vector. Learning the correlation between the LG vector and the semantic expression vector is performed by KCCA. According to this correlation, we estimate the associated semantic expression vector of a given test image and then perform the expression classification according to this estimated semantic expression vector. Moreover, we also propose an improved KCCA algorithm to tackle the singularity problem of the Gram matrix. The experimental results on the Japanese female facial expression database and the Ekman's "Pictures of Facial Affect" database illustrate the effectiveness of the proposed method.