Generalized Maximum Margin Clustering and Unsupervised Kernel Learning

Abstract

Maximum margin was proposed lately and has shown promising performance in recent studies [1, 2]. It extends the theory of support vector machine to unsupervised learning. Despite its good performance, there are three major problems with maximum margin that question its efficiency for real-world applications. First, it is computationally expensive and difficult to scale to large-scale datasets because the number of parameters in maximum margin is quadratic in the number of examples. Second, it requires data preprocessing to ensure that any boundary will pass through the origins, which makes it unsuitable for unbalanced dataset. Third, it is sensitive to the choice of kernel functions, and requires external procedure to determine the appropriate values for the parameters of kernel functions. In this paper, we propose maximum margin clustering framework that addresses the above three problems simultaneously. The new framework generalizes the maximum margin algorithm by allowing any boundaries including those not passing through the origins. It significantly improves the computational efficiency by reducing the number of parameters. Furthermore, the new framework is able to automatically determine the appropriate kernel matrix without any labeled data. Finally, we show a formal connection between maximum margin and spectral clustering. We demonstrate the efficiency of the generalized maximum margin algorithm using both synthetic datasets and real datasets from the UCI repository.