Similarity preservation in dimensionality reduction using a kernel-based cost function

Abstract

Dimensionality reduction aims to preserve, as much as possible, the significant structure of high-dimensional data in the low-dimensional space. This allows removing noise and redundant features, which is useful in exploratory data analysis, classification, and regression tasks. There are two main challenges in dimensionality reduction: (1) how to measure the manifold structures; (2) how to quantify the embedding preservation. On the one hand, previous approaches try to measure the manifold structure using variance, dot product, distance, and similarity preservation. On the other hand, usually, the embedding quality is quantified using divergence-based measures such as Kullback–Leibler and Jensen–Shannon. We propose a dimensionality reduction method that minimizes the mismatch between high and low dimensional spaces. Unlike traditional dimensionality reduction formulations, the proposed approach uses a kernel-based cost function to quantify the embedding quality. Our approach is validated on both synthetic and real-world datasets. In terms of visual inspection and quantitative evaluation of neighborhood preservation, results show that our proposal preserves global data structures in the low-dimensional representation.