On Riemannian and non-Riemannian Optimisation, and Optimisation Geometry

Abstract

Abstract Optimisation algorithms such as the Newton method were first generalised to manifolds by generalising the components of the algorithm directly: gradients were replaced by Riemannian gradients, straight lines were replaced by geodesics, and so forth. This meant having to endow the manifold with a Riemannian metric. Traditionally then, attention focused on the geometry of the underlying manifold. However, we argue the geometry of the manifold is not the right geometry to focus on because it does not take the cost function into consideration. For online optimisation problems requiring the minimisation of many different cost functions, of most relevance is the geometry of the family of cost functions as a whole: if the cost functions fit together in a “nice” way, fast optimisation algorithms can be developed even if individual cost functions are difficult to optimise. In particular, non-convex problems are not necessarily difficult problems. This paper presents a Riemannian-based homotopy algorithm for solving such Optimisation Geometry problems and briefly explains how it can be generalised to a non-Riemannian (e.g., coordinate-adapted) algorithm.