Inexact Generalized Gauss–Newton for Scaling the Canonical Polyadic Decomposition With Non-Least-Squares Cost Functions

Abstract

The canonical polyadic decomposition (CPD) allows one to extract compact and interpretable representations of tensors. Several optimization-based methods exist to fit the CPD of a tensor for the standard least-squares (LS) cost function. Extensions have been proposed for more general cost functions such as β-divergences as well. For these non-LS cost functions, a generalized Gauss-Newton (GGN) method has been developed. This is a second-order method that uses an approximation of the Hessian of the cost function to determine the next iterate and with this algorithm, fast convergence can be achieved close to the solution. While it is possible to construct the full Hessian approximation for small tensors, the exact GGN approach becomes too expensive for tensors with larger dimensions, as found in typical applications. In this paper, we therefore propose to use an inexact GGN method and provide several strategies to make this method scalable to large tensors. First, the approximation of the Hessian is only used implicitly and its multilinear structure is exploited during Hessian-vector products, which greatly improves the scalability of the method. Next, we show that by using a compressed instance of the GGN Hessian approximation, the computation time of the inexact GGN method can be lowered even more, with only limited influence on the convergence speed. We also propose dedicated preconditioners for the problem. Further, the maximum likelihood estimator for Rician distributed data is examined in detail as an example of an alternative cost function. This cost function is useful for the analysis of the moduli of complex data, as in functional magnetic resonance imaging, for instance. We compare the proposed method to the existing CPD methods and demonstrate the method's speed and effectiveness on synthetic and simulated real-life data. Finally, we show that the method can scale by using randomized block sampling.