Learning on the compact Stiefel manifold by a cayley-transform-based pseudo-retraction map

Abstract

The present research takes its moves from previous contributions by the present authors on two topics, namely, neural learning on differentiable manifolds by manifold retractions and averaging over differentiable manifolds. Learning on differentiable manifolds is a general theory that allows a neural system that insists on curved smooth spaces to adapt its parameters without violating the constraints on the geometry of the parameter spaces. In particular, the present contribution focuses on learning on the compact Stiefel manifold by manifold retraction with application to averaging `tall-skinny' matrices and generalizes some contributions recently appeared in the scientific literature about such a topic.