Convergence rate analysis for the higher order power method in best rank one approximations of tensors

Abstract

A popular and classical method for finding the best rank one approximation of a real tensor is the higher order power method (HOPM). It is known in the literature that the iterative sequence generated by HOPM converges globally, while the convergence rate can be superlinear, linear or sublinear. In this paper, we examine the convergence rate of HOPM in solving the best rank one approximation problem of real tensors. We first show that the iterative sequence of HOPM always converges globally and provide an explicit eventual sublinear convergence rate. The sublinear convergence rate estimate is in terms of the dimension and the order of the underlying tensor space. Then, we examine the concept of nondegenerate singular vector tuples and show that, if the sequence of HOPM converges to a nondegenerate singular vector tuple, then the global convergence rate is R-linear. We show that, for almost all tensors (in the sense of Lebesgue measure), all the singular vector tuples are nondegenerate, and so, the HOPM “typically” exhibits global R-linear convergence rate. Moreover, without any regularity assumption, we establish that the sequence generated by HOPM always converges globally and R-linearly for orthogonally decomposable tensors with order at least 3. We achieved this by showing that each nonzero singular vector tuple of an orthogonally decomposable tensor with order at least 3 is nondegenerate.