Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Abstract

AbstractWe prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.