Distributed Optimization and Sampling with Constraints from Machine Learning

Abstract

Optimization and sampling are fundamental techniques in machine learning. Recent development in learning theory has brought new insight to optimization and sampling on differentiable manifolds. In this paper, we focus on a special class of manifold that is the product of multiple manifolds, especially the unit spheres and hyperbolic spaces with Lorentz model. It has many real life applications, e.g., the head movement data of a robot can be represented on a torus, which is a product of two circles. On the other hand, the hyperbolic manifold, both the Poincaré disk model and Lorentz model, is widely used in word embedding. We state some basic properties of the distributed spherical gradient descent on the product of spheres and hyperbolic spaces, showing that the exponential map provides a unified framework on optimization and sampling on this special type of manifold.