Dimensionality reduction via the Johnson–Lindenstrauss Lemma: theoretical and empirical bounds on embedding dimension

Abstract

The Johnson–Lindenstrauss (JL) lemma has led to the development of tools for dealing with datasets in high dimensions. The lemma asserts that a set of high-dimensional points can be projected into lower dimensions, while approximately preserving the pairwise distance structure. Significant improvements of the JL lemma since its inception are summarized. Particular focus is placed on reproving Matousek’s versions of the lemma (Random Struct Algorithms 33(2):142–156, 2008) first using subgaussian projection coefficients and then using sparse projection coefficients. The results of the lemma are illustrated using simulated data. The simulation suggests a projection that is more effective in terms of dimensionality reduction than is borne out by the theory. This more effective projection was applied to a very large natural, rather than simulated, dataset thus further strengthening empirical evidence of the existence of a better than the proven optimal lower bound on the embedding dimension. Additionally, we provide comparisons with other commonly used data reduction and simplification techniques.