Sparsity in unions of subspaces for classification and clustering of high-dimensional data

Abstract

In many problems in signal/image processing, machine learning and computer vision, data in multiple classes lie in multiple low-dimensional subspaces of a high-dimensional ambient space. We consider the two problems of classification and clustering of data in a union subspaces using sparse representation techniques. We use the idea that the collection of data forms a self-expressive dictionary in which a new data point can write itself as a linear combination of points from the same class/subspace. First, we consider the classification problem where the training data in each class form a few groups of the dictionary and correspond to a few subspaces. We formulate the classification problem as finding a few active subspaces in the union of subspaces using two classes of convex optimization programs. We investigate conditions under which the proposed optimization programs recover the desired solution. Next, we consider the clustering problem, where the goal is to cluster the data in a union of subspaces so that data points in each cluster correspond to points in the same subspace. We propose a convex optimization program based on sparse representation and use its solution to infer the clustering of data using spectral clustering. We investigate conditions under which the proposed convex program successfully finds a sparse representation of each point as a linear combination of points from the same subspace. We demonstrate the efficacy of the proposed classification and clustering algorithms through synthetic and real experiments.