A regularized formulation for spectral clustering with pairwise constraints

Abstract

A regularized method to incorporate prior knowledge into spectral clustering in the form of pairwise constraints is proposed. This method is based on a weighted kernel principal component analysis (PCA) interpretation of spectral clustering with primal-dual least squares support vector machines (LS-SVM) formulations. The weighted kernel PCA framework allows incorporating pairwise constraints into the primal problem leading to a dual eigenvalue problem involving a modified kernel matrix. This modification on the metric is a regularized rank-1 downdate of the original kernel matrix. The clustering model can also be extended to out-of-sample points which becomes important for generalization, predictive purposes and large-scale data. An extension of an existing model selection criterion is also proposed. This extension introduces an additional term to the criterion measuring the constraint fit. Simulation results with toy examples and an image segmentation problem show the applicability of the proposed method.