Introducing Locality and Softness in Subspace Classification

Abstract

Subspace classifiers classify a pattern based on its distance from different vector subspaces. Earlier models of subspace classification were based on the assumption that individual classes lie in unique subspaces. In later extensions, locality was introduced into subspace classification allowing for a class to be associated with more than one sub manifold. The local subspace classifier is thus a piecewise linear classifier, and is more powerful when compared to the linear classification performed by global subspace methods. We present extensions to the basic subspace method of classification based on introducing locality and softness in the classification process. Locality is introduced by (subspace) clustering the patterns into clusters, and softness is introduced by allowing a pattern to be associated with more than one cluster. Our motivation for introducing both locality and softness is based on the premise that by introducing locality, it is possible to reduce the bias though at the cost of a possible increase in variance. By introducing softness (or aggregation), the variance can be reduced. Consequently, by introducing both locality and softness, we avoid the possibility of high variance that locality typically introduces. We derive appropriate algorithms to construct a local and soft model of subspace classifiers and present results obtained with the proposed algorithm.