Tucker compression and local optima

Abstract

This paper deals with a particular Tucker compression problem. Formally, given a higher-order tensor, we are interested in its best low multilinear rank approximation. We study the local minima of the cost function that measures the quality of putative low multilinear rank approximations. This discussion is specific to higher-order tensors since, in the matrix case, the low rank approximation problem has only one local, hence global, minimum. Furthermore, in the higher-order tensor case, convergence to the solution with lowest value of the cost function cannot be guaranteed with existing algorithms even if they are initialized with the truncated higher-order singular value decomposition. We observed that the values of the cost function at different local minima can be very close to each other. Thus, for memory savings based on Tucker compression, the choice between these local minima does not really matter. On the other hand, we found that the subspaces on which the original tensor is projected can be very different. If the subspaces are of importance, different local minima may yield very different results. We provide numerical examples and indicate a relation between the number of local minima that are found and the distribution of the higher-order singular values of the tensor.