Dimension Estimation and Topological Manifold Learning

Abstract

We describe a framework which can be used to investigate the geometrical structure of datasets in high-dimensional spaces. Understanding these geometrical properties is essential in machine learning in general. An application which has received much attention recently is the investigation of adversarial examples which can be easily identified by humans but which are misleading neural networks. We will argue that these examples can be understood (and consequently avoided) by an investigation of the dimension and geometry of the classification hypersurface of the underlying neural network as a manifold using the tools introduced in this paper.We argue that a key concept in the analysis is the dimension of a manifold. Many machine learning methods use dimension reduction techniques to understand or classify the input data. We point out different definitions of a dimension of a manifold. We use tools from the theory of topological manifolds to introduce local, or intrinsic and global dimensions. The local dimension describes the local similarity between the dataset and a Euclidean space. The global dimension is the lowest dimension of a Euclidean space into which the dataset can be embedded. For a data space with a Riemannian structure the isometric embedding dimension is the lowest dimensional space into which the dataset can be embedded under the condition that the metric tensor is preserved between the two spaces.We introduce a framework for dimension estimation and topological manifold learning based on the measure ratio method to estimate the dimensions and structure of the data manifold. As an illustration we use images of handwritten digits and points of a Klein-bottle embedded in a five-dimensional space. We compare the results obtained by the measure ratio method with the well-known local-principal component analysis estimation.