CLASSIFICATION-ORIENTED LOCALLY LINEAR EMBEDDING

Abstract

The locally linear embedding (LLE) algorithm is hypothetically able to find a lower dimensional space than a linear method for preserving a data manifold originally embedded in a high dimensional space. However, uneven sampling over the manifold in real-world data ultimately causes LLE to suffer from the disconnected-neighborhood problem. Consequently, the final dimensionality required for the data manifold is multiplied by the number of disjoint groups in the complete data representation. In addition, LLE as an unsupervised method is unable to suppress between-class connections. This means that samples from different classes are mixed during reconstruction. This study presents CLLE, a classification-oriented LLE method that uses class label information from training samples to guide unsupervised LLE. The criterion for neighbor selection is redesigned using class-conditional likelihood as well as Euclidean distance. This algorithm largely eliminates fractured classes and lowers the incidence of connections between classes. Also, a reconnection technique is proposed as a supporting method for ensuring a fully connected neighborhood graph, so that CLLE is able to extract the fewest features. Experiments with simulated and real data show that CLLE exceeds the performance of linear methods. Comparable classification performance can be achieved by CLLE using fewer features. In comparison with LLE, CLLE demonstrates a higher aptitude for and flexibility towards classification.