Information geometry and data manifold representations

Abstract

Information geometry studies the measurements of intrinsic information based on the mathematical discipline of differential geometry. This dissertation built several new connections between information geometry and machine learning. A Riemannian geometry of a representation manifold R is studied, where each point is a pair-wise (dis-)similarity matrix corresponding to a set of objects. A Riemannian metric of R is derived in closed form based on the Fisher information metric. This metric is featured by emphasizing local information. Information geometric measurements for manifold learning are therefore defined. Statistical mixture learning of the data manifold is viewed as a multi-body problem on a statistical manifold M, where each body is a mixture component, or a point on M. Principles based on geometric compactness can lead to effective regularization for both kernel density estimation and parametric mixture learning. New insights are given on the connection between information geometry, mixture learning, and minimum description length.