Low Multilinear Rank Approximation Research Articles

RA-HOOI: Rank-adaptive higher-order orthogonal iteration for the fixed-accuracy low multilinear-rank approximation of tensors

In this paper, we propose a novel rank-adaptive higher-order orthogonal iteration (RA-HOOI) algorithm to solve the fixed-accuracy low multilinear-rank approximation of tensors. On the one hand, RA-HOOI relies on a greedy strategy to expand the subspace, which avoids computing the full SVD of the matricization of the input tensor. On the other hand, the new rank-adaptive strategy introduced in the RA-HOOI algorithm enables the obtained truncation to be more accurate. A series of numerical experiments related to synthetic and real-world tensors are carried out to show that the proposed RA-HOOI algorithm is comparable to state-of-the-art methods in terms of both accuracy and efficiency and performs better in certain situations.

Low tensor train and low multilinear rank approximations of 3D tensors for compression and de-speckling of optical coherence tomography images

Objective. Many methods for compression and/or de-speckling of 3D optical coherence tomography (OCT) images operate on a slice-by-slice basis and, consequently, ignore spatial relations between the B-scans. Thus, we develop compression ratio (CR)-constrained low tensor train (TT)—and low multilinear (ML) rank approximations of 3D tensors for compression and de-speckling of 3D OCT images. Due to inherent denoising mechanism of low-rank approximation, compressed image is often even of better quality than the raw image it is based on. Approach. We formulate CR-constrained low rank approximations of 3D tensor as parallel non-convex non-smooth optimization problems implemented by alternating direction method of multipliers of unfolded tensors. In contrast to patch- and sparsity-based OCT image compression methods, proposed approach does not require clean images for dictionary learning, enables CR as high as 60:1, and it is fast. In contrast to deep networks based OCT image compression, proposed approach is training free and does not require any supervised data pre-processing. Main results. Proposed methodology is evaluated on twenty four images of a retina acquired on Topcon 3D OCT-1000 scanner, and twenty images of a retina acquired on Big Vision BV1000 3D OCT scanner. For the first dataset, statistical significance analysis shows that for CR ≤ 35, all low ML rank approximations and Schatten-0 (S 0) norm constrained low TT rank approximation can be useful for machine learning-based diagnostics by using segmented retina layers. Also for CR ≤ 35, S 0-constrained ML rank approximation and S 0-constrained low TT rank approximation can be useful for visual inspection-based diagnostics. For the second dataset, statistical significance analysis shows that for CR ≤ 60 all low ML rank approximations as well as S 0 and S 1/2 low TT ranks approximations can be useful for machine learning-based diagnostics by using segmented retina layers. Also, for CR ≤ 60, low ML rank approximations constrained with S p , p ∊ {0, 1/2, 2/3} and one surrogate of S 0 can be useful for visual inspection-based diagnostics. That is also true for low TT rank approximations constrained with S p , p ∊ {0, 1/2, 2/3} for CR ≤ 20. Significance. Studies conducted on datasets acquired by two different types of scanners confirmed capabilities of proposed framework that, for a wide range of CRs, yields de-speckled 3D OCT images suitable for clinical data archiving and remote consultation, for visual inspection-based diagnosis and for machine learning-based diagnosis by using segmented retina layers.

A Krylov-Schur-like method for computing the best rank-(r1,r2,r3) approximation of large and sparse tensors

The paper is concerned with methods for computing the best low multilinear rank approximation of large and sparse tensors. Krylov-type methods have been used for this problem; here block versions are introduced. For the computation of partial eigenvalue and singular value decompositions of matrices the Krylov-Schur (restarted Arnoldi) method is used. A generalization of this method to tensors is described, for computing the best low multilinear rank approximation of large and sparse tensors. In analogy to the matrix case, the large tensor is only accessed in multiplications between the tensor and blocks of vectors, thus avoiding excessive memory usage. It is proved that if the starting approximation is good enough, then the tensor Krylov-Schur method is convergent. Numerical examples are given for synthetic tensors and sparse tensors from applications, which demonstrate that for most large problems the Krylov-Schur method converges faster and more robustly than higher order orthogonal iteration.

From multilinear SVD to multilinear UTV decomposition

Across a range of applications, low multilinear rank approximation (LMLRA) is used to compress large tensors into a more compact form, while preserving most of their information. A specific instance of LMLRA is the multilinear singular value decomposition (MLSVD), which can be used for multilinear principal component analysis (MLPCA). MLSVDs are obtained by computing SVDs of all tensor unfoldings, but, in practical applications, it is often not necessary to compute full SVDs. In this article, we therefore propose a new decomposition, called the truncated multilinear UTV decomposition (TMLUTVD). This is a tensor decomposition that is also multilinear rank-revealing, yet less expensive to compute than a truncated MLSVD (TMLSVD); it can even be computed in a finite number of steps. We present its properties in an algorithm-independent manner. In particular, we derive bounds on the accuracy in function of the truncation level. Experiments illustrate the good performance in practice.

Updating the Multilinear UTV Decomposition

Low multilinear rank approximation (LMLRA) is a basic tool for compressing a tensor into a more compact form, while preserving most of its information. An LMLRA of an <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula>th order tensor consists of <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> factor matrices which are combined using a core tensor of smaller dimensions than the original tensor. When the tensor is non-static, for example when over time new tensor slices are being added along a certain mode, one can expect that the LMLRA of the tensor changes, but that an approximation of the new tensor can be derived from the previous one. By updating the factor matrices and the core tensor in an efficient way when a new tensor slice is added, this can indeed be achieved. In this paper, a new method is proposed to track the factor matrices in both the evolving and non-evolving modes, leading to efficient and accurate updates of the LMLRA. We track a truncated multilinear UTV decomposition (TMLUTVD), which is a tensor decomposition that is multilinear rank-revealing, yet less expensive to compute than the popular multilinear singular value decomposition. We derive accuracy bounds for the tracked TMLUTVD and give strategies to speed up its computation. Experiments show that the TMLUTVD tracking method works well in practice, even if the mode-<inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> principal subspaces change drastically between updates. As an illustration of the possibilities of the technique, we extend the use of LMLRA in a compression step, prior to the computation of a canonical polyadic decomposition (CPD), to the setting of updating and tracking.

Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

The low multilinear rank approximation, also known as the truncated Tucker decomposition, has been extensively utilized in many applications that involve higher-order tensors. Popular methods for low multilinear rank approximation usually rely directly on matrix SVD, therefore often suffer from the notorious intermediate data explosion issue and are not easy to parallelize, especially when the input tensor is large. In this paper, we propose a new class of truncated HOSVD algorithms based on alternating least squares (ALS) for efficiently computing the low multilinear rank approximation of tensors. The proposed ALS-based approaches are able to eliminate the redundant computations of the singular vectors of intermediate matrices and are therefore free of data explosion. Also, the new methods are more flexible with adjustable convergence tolerance and are intrinsically parallelizable on high-performance computers. Theoretical analysis reveals that the ALS iteration in the proposed algorithms is q-linear convergent with a relatively wide convergence region. Numerical experiments with large-scale tensors from both synthetic and real-world applications demonstrate that ALS-based methods can substantially reduce the total cost of the original ones and are highly scalable for parallel computing.

Randomized algorithms for the low multilinear rank approximations of tensors

In this paper, we focus on developing randomized algorithms for the computation of low multilinear rank approximations of tensors based on the random projection and the singular value decomposition. Following the theory of the singular values of sub-Gaussian matrices, we make a probabilistic analysis for the error bounds for the randomized algorithm. We demonstrate the effectiveness of proposed algorithms via several numerical examples.

The Computation of Low Multilinear Rank Approximations of Tensors via Power Scheme and Random Projection

This paper is devoted to the computation of low multilinear rank approximations of tensors. Combining the stretegy of power scheme, random projection, and singular value decomposition, we derive a ...

Synthesis Analysis Methods for Underwater Video Compression with Tensor Based Minimized Side Information

Synthesis analysis is a common approach used to compress videos with more amounts of dynamic textures. Underwater videos contain more moving species captured by moving camera. These kinds of videos have two types of motion registered by both the species and the camera. In this paper, tensor, an N-way representation of data is used to store the side information obtained from the synthesis analysis approach. The Low multilinear rank approximation (LMLRA) with error correction using residual tensor is applied on the side information to reduce the memory space for side information. The host encoder in synthesis analysis approach plays an important role in providing high compression rate with minimal loss and hence H.265 is used as the host encoder. The results show that the proposed method achieves highest compression ratio with minimal loss due to distortion and saved bit rate which is highly consumed by dynamic textures.

An Efficient Compressive Hyperspectral Imaging Algorithm Based on Sequential Computations of Alternating Least Squares

Hyperspectral imaging is widely used to many applications as it includes both spatial and spectral distributions of a target scene. However, a compression, or a low multilinear rank approximation of hyperspectral imaging data, is required owing to the difficult manipulation of the massive amount of data. In this paper, we propose an efficient algorithm for higher order singular value decomposition that enables the decomposition of a tensor into a compressed tensor multiplied by orthogonal factor matrices. Specifically, we sequentially compute low rank factor matrices from the Tucker-1 model optimization problems via an alternating least squares approach. Experiments with real world hyperspectral imaging revealed that the proposed algorithm could compute the compressed tensor with a higher computational speed, but with no significant difference in accuracy of compression compared to the other tensor decomposition-based compression algorithms.

Randomized algorithms for the approximations of Tucker and the tensor train decompositions

Randomized algorithms provide a powerful tool for scientific computing. Compared with standard deterministic algorithms, randomized algorithms are often faster and robust. The main purpose of this paper is to design adaptive randomized algorithms for computing the approximate tensor decompositions. We give an adaptive randomized algorithm for the computation of a low multilinear rank approximation of the tensors with unknown multilinear rank and analyze its probabilistic error bound under certain assumptions. Finally, we design an adaptive randomized algorithm for computing the tensor train approximations of the tensors. Based on the bounds about the singular values of sub-Gaussian matrices with independent columns or independent rows, we analyze these randomized algorithms. We illustrate our adaptive randomized algorithms via several numerical examples.

Compact video content representation for video coding using low multi-linear tensor rank approximation with dynamic core tensor order

Tensor is a multi-way representation of an entity that is widely used in many applications of mathematics and physics. The wide range of tensor algebra operations attract the researchers of graphical and vision field to identify unique methods for various applications such as segmentation, compression, indexing and so on. In this paper, low multi-linear rank approximation of a tensor is used for decomposition so as to perform compression on a video that is represented as a 4D tensor. Thus, the encoder has blocks for (1) multilinear rank approximation method so as to aid decomposition, (2) sparsity removal of residual data to reduce the required memory storage to a small level, (3) quantization on the sparse less residual data and (4) LZ77 dictionary coding for the video. The decoder uses a tensor reconstruction block along with residual error correction block that reduces loss/error in the reconstructed video. The size of core tensor is an important factor in rank approximation. It is identified at run time depending on the video contents. This size is determined adaptively using Tikhonov’s regularization method. The best value of core tensor size is identified with the corner of L-curve which preserves the video contents from heavy loss/error. Experimental results of the proposed work on comparison with H.264/AVC compression method indicate that the proposed method achieves up to 10% improvement in compression ratio with the least amount of loss and a maximum signal to noise ratio of 0.14. The proposed system gives better compression ratio than H.265 for videos with more number of frames without compromising quality.

On the convergence of higher-order orthogonal iteration

The higher-order orthogonal iteration (HOOI) has been popularly used for finding a best low-multilinear rank approximation of a tensor. However, its convergence is still an open question. In this paper, we first analyse a greedy HOOI, which updates each factor matrix by selecting from the best candidates one that is closest to the current iterate. Assuming the existence of a block-nondegenerate cluster point, we establish its global iterate sequence convergence through the so-called Kurdyka–ᴌojasiewicz property. In addition, we show that if the starting point is sufficiently close to any block-nondegenerate globally optimal solution, the greedy HOOI produces an iterate sequence convergent to a globally optimal solution. Relating the iterate sequence by the original HOOI to that by the greedy HOOI, we then show that the original HOOI has global convergence on the multilinear subspace sequence and thus positively address the open question.

Structure-Preserving Low Multilinear Rank Approximation of Antisymmetric Tensors

This paper is concerned with low multilinear rank approximations to antisymmetric tensors, that is, multivariate arrays for which the entries change sign when permuting pairs of indices. We show which ranks can be attained by an antisymmetric tensor and discuss the adaption of existing approximation algorithms to preserve antisymmetry, most notably a Jacobi algorithm. Particular attention is paid to the important special case when choosing the rank equal to the order of the tensor. It is shown that this case can be addressed with an unstructured rank-$1$ approximation. This allows for the straightforward application of the higher-order power method, for which we discuss effective initialization strategies.

Processing Hyperspectral Images using Non-Linear Least Square Algorithm as an Optimization Method for Tensor Decomposition Model

ABSTRACT Due to large size and huge availability of unwanted or missing information in hyperspectral image, development of data effective compression and denoising methods is of prior importance. Compression removes unmeaningful information and thereby reducing data which ultimately leads to noise free image. This study deals with execution of two lossless decomposition methods Low Multi-linear Rank Approximation, four types of Block Term Decomposition to the input image cube to make it noise free using non-linear least square method as an optimization method and their performance were assessed. BTD (L r , L r , 1) was selected as the best tensor algorithm based on residual error and frobenius norm value with a limitation that the image cube to be processed by the method should have good spatial resolution. General Terms Remote Sensing Keywords Hyperspectral imaging, Data Compression, Tensor decomposition models, Low Multi-linear Rank Approximation, Block Term Decomposition, Frobenius Norm.

Low Multilinear Rank Approximation of Tensors and Application in Missing Traffic Data

The problem of missing data in multiway arrays (i.e., tensors) is common in many fields such as bibliographic data analysis, image processing, and computer vision. We consider the problems of approximating a tensor by another tensor with low multilinear rank in the presence of missing data and possibly reconstructing it (i.e., tensor completion). In this paper, we propose a weighted Tucker model which models only the known elements for capturing the latent structure of the data and reconstructing the missing elements. To treat the nonuniqueness of the proposed weighted Tucker model, a novel gradient descent algorithm based on a Grassmann manifold, which is termed Tucker weighted optimization (Tucker-Wopt), is proposed for guaranteeing the global convergence to a local minimum of the problem. Based on extensive experiments, Tucker-Wopt is shown to successfully reconstruct tensors with noise and up to 95% missing data. Furthermore, the experiments on traffic flow volume data demonstrate the usefulness of our algorithm on real-world application.

A preconditioned low-rank CG method for parameter-dependent Lyapunov matrix equations

This paper is concerned with the numerical solution of symmetric large-scale Lyapunov equations with low-rank right-hand sides and coefficient matrices depending on a parameter. Specifically, we consider the situation when the parameter dependence is sufficiently smooth, and the aim is to compute solutions for many different parameter samples. On the basis of existing results for Lyapunov equations and parameter-dependent linear systems, we prove that the tensor containing all solution samples typically allows for an excellent low multilinear rank approximation. Stacking all sampled equations into one huge linear system, this fact can be exploited by combining the preconditioned CG method with low-rank truncation. Our approach is flexible enough to allow for a variety of preconditioners based, for example, on the sign function iteration or the alternating direction implicit method.

Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors

The problem discussed in this paper is the symmetric best low multilinear rank approximation of third-order symmetric tensors. We propose an algorithm based on Jacobi rotations, for which symmetry is preserved at each iteration. Two numerical examples are provided indicating the need for such algorithms. An important part of the paper consists of proving that our algorithm converges to stationary points of the objective function. This can be considered an advantage of the proposed algorithm over existing symmetry-preserving algorithms in the literature.

Krylov-type methods for tensor computations I

Several Krylov-type procedures are introduced that generalize matrix Krylov methods for tensor computations. They are denoted minimal Krylov recursion, maximal Krylov recursion, and contracted tensor product Krylov recursion. It is proved that, for a given tensor A with multilinear rank-(p,q,r), the minimal Krylov recursion extracts the correct subspaces associated to the tensor in p+q+r number of tensor–vector–vector multiplications. An optimized minimal Krylov procedure is described that, for a given multilinear rank of an approximation, produces a better approximation than the standard minimal recursion. We further generalize the matrix Krylov decomposition to a tensor Krylov decomposition. The tensor Krylov methods are intended for the computation of low multilinear rank approximations of large and sparse tensors, but they are also useful for certain dense and structured tensors for computing their higher order singular value decompositions or obtaining starting points for the best low-rank computations of tensors. A set of numerical experiments, using real-world and synthetic data sets, illustrate some of the properties of the tensor Krylov methods.

Tucker compression and local optima

This paper deals with a particular Tucker compression problem. Formally, given a higher-order tensor, we are interested in its best low multilinear rank approximation. We study the local minima of the cost function that measures the quality of putative low multilinear rank approximations. This discussion is specific to higher-order tensors since, in the matrix case, the low rank approximation problem has only one local, hence global, minimum. Furthermore, in the higher-order tensor case, convergence to the solution with lowest value of the cost function cannot be guaranteed with existing algorithms even if they are initialized with the truncated higher-order singular value decomposition. We observed that the values of the cost function at different local minima can be very close to each other. Thus, for memory savings based on Tucker compression, the choice between these local minima does not really matter. On the other hand, we found that the subspaces on which the original tensor is projected can be very different. If the subspaces are of importance, different local minima may yield very different results. We provide numerical examples and indicate a relation between the number of local minima that are found and the distribution of the higher-order singular values of the tensor.