Non-convex Optimization on Stiefel Manifold and Applications to Machine Learning

Abstract

We study non-convex optimization problems which appear in machine learning and statistics. Recently, convexity became an important feature to develop high-performance learning algorithms. On the other hand, non-convex optimization is also required in machine learning. In this paper, especially we deal with non-convex optimization problems on the Stiefel manifold. The Stiefel manifold consists of rectangular matrices, and many important problems in machine learning can be expressed in this framework. We compare some non-linear optimization methods which are applicable to machine learning algorithms.