Exact line and plane search for tensor optimization

Abstract

Line and plane searches are used as accelerators and globalization strategies in many optimization algorithms. We introduce a class of optimization problems called tensor optimization, which comprises applications ranging from tensor decompositions to least squares support tensor machines. We develop algorithms to efficiently compute the global minimizers of their line and plane search subproblems. Furthermore, we introduce scaled line and plane search, which compute an optimal scaling of the solution simultaneously with the optimal line or plane search step, and show that this scaling can be computed at almost no additional cost. Obtaining the global minimizers of (scaled) line and plane search problems often requires solving a bivariate or polyanalytic polynomial system. We show how to compute the isolated real solutions of bivariate polynomial systems and the isolated complex solutions of polyanalytic polynomial systems using a single generalized eigenvalue decomposition. Finally, we apply block term decompositions to the problem of blind multi-user detection-estimation in DS-CDMA communication to demonstrate that exact line and plane search can significantly reduce computation time of the workhorse tensor decomposition algorithm alternating least squares.