Instance-specific canonical correlation analysis

Abstract

Canonical Correlation Analysis (CCA) is one of the most popular statistical methods to capture the correlations between two variables. However, it has limitations as a linear and global algorithm. Although some variants have been proposed to overcome the limitations, neither of them achieves locality and nonlinearity at the same time. In this paper, we propose a novel algorithm called Instance-Specific Canonical Correlation Analysis (ISCCA), which approximates the nonlinear data by computing the instance-specific projections along the smooth curve of the manifold. First, we propose a least squares solution for CCA which will set the stage for the proposed method. Second, based on the framework of least squares regression, CCA is extended to the instance-specific case which obtains a set of locally linear smooth but globally nonlinear transformations. Third, ISCCA can be extended to semi-supervised setting by exploiting the unlabeled data to further improve the performance. The optimization problem is proved to be convex and could be solved efficiently by alternating optimization. And the globally optimal solutions could be achieved with theoretical guarantee. Moreover, for large scale applications, iterative conjugate gradient algorithm can be used to speed up the computation procedure. Experimental results demonstrate the effectiveness of our proposed method.