Randomized anisotropic transform for nonlinear dimensionality reduction

Abstract

An innovative method is introduced in this paper to significantly increase computational speed and to reduce memory usage, when applied to nonlinear methods and algorithms for dimensionality reduction (DR). Due to the incapability of linear DR methods in the preservation of data geometry, the need of effective nonlinear approaches has recently attracted the attention of many researchers, particularly from the Mathematics and Computer Science communities. The common theme of the current nonlinear DR approaches is formulation of certain matrices, called dimensionality reduction kernels (DRK) in terms of the data points, followed by performing spectral decomposition. Hence, for datasets of large size with data points that lie in some high dimensional space, the matrix dimension of the DRK is very large. Typical examples for the need of very high dimensional DRK arise from such applications as processing multispectral imagery data, searching desired text documentary data in the internet, and recognizing human faces from given libraries. For such and many other applications, the matrix dimension of the DRK is so large that computation for carrying out spectral decomposition of the DRK often encounters various difficulties, not only due to the need of significantly large read-only memory (ROM), but also due to computational instability. The main objective of this paper is to introduce the notion of the anisotropic transform (AT) and to develop its corresponding effective and efficient computational algorithms by integrating random embedding with the AT to formulate the randomized anisotropic transform (RAT), in order to reduce the size of the DRK significantly, while preserving local geometries of the given datasets. Illustrations with various examples will also be given in this paper to demonstrate that RAT algorithms dramatically reduce the ROM and CPU requirement, and thus allow fast processing, even on PC and potentially for handhold devices as well.