Tensor Approximation Research Articles

Coupled Tensor Decomposition for Hyperspectral and Multispectral Image Fusion With Inter-Image Variability

Coupled tensor approximation has recently emerged as a promising approach for the fusion of hyperspectral and multispectral images, reconciling state of the art performance with strong theoretical guarantees. However, tensor-based approaches previously proposed assume that the different observed images are acquired under exactly the same conditions. A recent work proposed to accommodate inter-image spectral variability in the image fusion problem using a matrix factorization-based formulation, but did not account for spatially-localized variations. Moreover, it lacks theoretical guarantees and has a high associated computational complexity. In this paper, we consider the image fusion problem while accounting for both spatially and spectrally localized changes in an additive model. We first study how the general identifiability of the model is impacted by the presence of such changes. Then, assuming that the high-resolution image and the variation factors admit a Tucker decomposition, two new algorithms are proposed -- one purely algebraic, and another based on an optimization procedure. Theoretical guarantees for the exact recovery of the high-resolution image are provided for both algorithms. Experimental results show that the proposed method outperforms state-of-the-art methods in the presence of spectral and spatial variations between the images, at a smaller computational cost.

Efficient Simulation of Random Fields by Trigonometric Polynomial and Low‐rank Tensor

AbstractWe propose a fast and economic representation of stationary random fields in trigonometric polynomial, utilizing the prowess of fast Fourier transform (FFT) and low‐rank tensor approximation. With the method we are able to generate large random fields with discretization size up to 220 which are otherwise well beyond the capacity of PCs. We also illustrate the approach to the specified property of random field by increasing rank in the tensor approximation.

Cross Tensor Approximation Methods for Compression and Dimensionality Reduction

Cross Tensor Approximation (CTA) is a generalization of Cross/skeleton matrix and CUR Matrix Approximation (CMA) and is a suitable tool for fast low-rank tensor approximation. It facilitates interpreting the underlying data tensors and decomposing/compressing tensors so that their structures, such as nonnegativity, smoothness, or sparsity, can be potentially preserved. This paper reviews and extends state-of-the-art deterministic and randomized algorithms for CTA with intuitive graphical illustrations. We discuss several possible generalizations of the CMA to tensors, including CTAs: based on fiber selection, slice-tube selection, and lateral-horizontal slice selection. The main focus is on the CTA algorithms using Tucker and tubal SVD (t-SVD) models while we provide references to other decompositions such as Tensor Train (TT), Hierarchical Tucker (HT), and Canonical Polyadic (CP) decompositions. We evaluate the performance of the CTA algorithms by extensive computer simulations to compress color and medical images and compare their performance.

Adaptive Hierarchical Subtensor Partitioning for Tensor Compression

In this work a numerical method is proposed to compress a tensor by constructing a piece-wise tensor approximation. This is defined by partitioning a tensor into sub-tensors and by computing a low-rank tensor approximation (in a given format) in each sub-tensor. Neither the partition nor the ranks are fixed a priori, but, instead, are obtained in order to fulfill a prescribed accuracy and optimize, to some extent, the storage. The different steps of the method are detailed and some numerical experiments are proposed to assess its performances.

Time Integration of Tree Tensor Networks

Dynamical low-rank approximation by tree tensor networks is studied for the data-sparse approximation of large time-dependent data tensors and unknown solutions to tensor differential equations. A time integration method for tree tensor networks of prescribed tree rank is presented and analyzed. It extends the known projector-splitting integrators for dynamical low-rank approximation by matrices and rank-constrained Tucker tensors and is shown to inherit their favorable properties. The integrator is based on recursively applying the low-rank Tucker tensor integrator. In every time step, the integrator climbs up and down the tree: it uses a recursion that passes from the root to the leaves of the tree for the construction of initial value problems on subtree tensor networks using appropriate restrictions and prolongations, and another recursion that passes from the leaves to the root for the update of the factors in the tree tensor network. The integrator reproduces given time-dependent tree tensor networks of the specified tree rank exactly and is robust to the typical presence of small singular values in matricizations of the connection tensors, in contrast to standard integrators applied to the differential equations for the factors in the dynamical low-rank approximation by tree tensor networks.

A tensor rank theory and maximum full rank subtensors

A matrix always has a full rank submatrix such that the rank of this matrix is equal to the rank of that submatrix. This property is one of the corner stones of the matrix rank theory. We call this property the max-full-rank-submatrix property. Tensor ranks play a crucial role in low rank tensor approximation, tensor completion and tensor recovery. However, their theory is still not matured yet. Can we set an axiom system for tensor ranks? Can we extend the max-full-rank-submatrix property to tensors? We explore these in this paper. We first propose some axioms for tensor rank functions. Then we introduce proper tensor rank functions. The CP rank is a tensor rank function, but is not proper. There are two proper tensor rank functions, the max-Tucker rank and the submax-Tucker rank, which are associated with the Tucker decomposition. We define a partial order among tensor rank functions and show that there exists a unique smallest tensor rank function. We introduce the full rank tensor concept, and define the max-full-rank-subtensor property. We show the max-Tucker tensor rank function and the smallest tensor rank function have this property. We define the closure for an arbitrary proper tensor rank function, and show that it is still a proper tensor rank function and has the max-full-rank-subtensor property. An application of the submax-Tucker rank is also presented.

Generalized Nonconvex Low-Rank Tensor Approximation for Multi-View Subspace Clustering.

The low-rank tensor representation (LRTR) has become an emerging research direction to boost the multi-view clustering performance. This is because LRTR utilizes not only the pairwise relation between data points, but also the view relation of multiple views. However, there is one significant challenge: LRTR uses the tensor nuclear norm as the convex approximation but provides a biased estimation of the tensor rank function. To address this limitation, we propose the generalized nonconvex low-rank tensor approximation (GNLTA) for multi-view subspace clustering. Instead of the pairwise correlation, GNLTA adopts the low-rank tensor approximation to capture the high-order correlation among multiple views and proposes the generalized nonconvex low-rank tensor norm to well consider the physical meanings of different singular values. We develop a unified solver to solve the GNLTA model and prove that under mild conditions, any accumulation point is a stationary point of GNLTA. Extensive experiments on seven commonly used benchmark databases have demonstrated that the proposed GNLTA achieves better clustering performance over state-of-the-art methods.

Tensor Linear Regression: Degeneracy and Solution

Tensor regression is an important and useful tool for analyzing multidimensional array data. To deal with high dimensionality, CANDECOMP/PARAFAC (CP) low-rank constraints are often imposed on the coefficient tensor parameter in the (penalized) loss functions. However, besides the well-known non-identifiability issue of the CP parameters, we demonstrate that the corresponding optimization may not have any attainable solutions, and thus the estimation of the coefficient tensor is not well-defined when this happens. This is closely related to a phenomenon, called CP degeneracy, in low-rank tensor approximation problems. In this article, we show some useful results of CP degeneracy in the context of tensor regression problems. To overcome the theoretical and numerical issues associated with the degeneracy, we provide a general penalized strategy as a solution to the degeneracy. The related results also explain why some of the existing methods are more stable than the others. The asymptotic properties of the resulting estimation are also studied. Numerical experiments are conducted to illustrate our findings.

Prior-Based Tensor Approximation for Anomaly Detection in Hyperspectral Imagery.

The key to hyperspectral anomaly detection is to effectively distinguish anomalies from the background, especially in the case that background is complex and anomalies are weak. Hyperspectral imagery (HSI) as an image-spectrum merging cube data can be intrinsically represented as a third-order tensor that integrates spectral information and spatial information. In this article, a prior-based tensor approximation (PTA) is proposed for hyperspectral anomaly detection, in which HSI is decomposed into a background tensor and an anomaly tensor. In the background tensor, a low-rank prior is incorporated into spectral dimension by truncated nuclear norm regularization, and a piecewise-smooth prior on spatial dimension can be embedded by a linear total variation-norm regularization. For anomaly tensor, it is unfolded along spectral dimension coupled with spatial group sparse prior that can be represented by the l2,1 -norm regularization. In the designed method, all the priors are integrated into a unified convex framework, and the anomalies can be finally determined by the anomaly tensor. Experimental results validated on several real hyperspectral data sets demonstrate that the proposed algorithm outperforms some state-of-the-art anomaly detection methods.

Robust brain network identification from multi-subject asynchronous fMRI data

We describe a novel method for robust identification of common brain networks and their corresponding temporal dynamics across subjects from asynchronous functional MRI (fMRI) using tensor decomposition. We first temporally align asynchronous fMRI data using the orthogonal BrainSync transform, allowing us to study common brain networks across sessions and subjects. We then map the synchronized fMRI data into a 3D tensor (vertices × time × subject/session). Finally, we apply Nesterov-accelerated adaptive moment estimation (Nadam) within a scalable and robust sequential Canonical Polyadic (CP) decomposition framework to identify a low rank tensor approximation to the data. As a result of CP tensor decomposition, we successfully identified twelve known brain networks with their corresponding temporal dynamics from 40 subjects using the Human Connectome Project's language task fMRI data without any prior information regarding the specific task designs. Seven of these networks show distinct subjects’ responses to the language task with differing temporal dynamics; two show sub-components of the default mode network that exhibit deactivation during the tasks; the remaining three components reflect non-task-related activities. We compare results to those found using group independent component analysis (ICA) and canonical ICA. Bootstrap analysis demonstrates increased robustness of networks found using the CP tensor approach relative to ICA-based methods.

ScTenifoldNet: A Machine Learning Workflow for Constructing and Comparing Transcriptome-wide Gene Regulatory Networks from Single-Cell Data

SummaryWe present scTenifoldNet—a machine learning workflow built upon principal-component regression, low-rank tensor approximation, and manifold alignment—for constructing and comparing single-cell gene regulatory networks (scGRNs) using data from single-cell RNA sequencing. scTenifoldNet reveals regulatory changes in gene expression between samples by comparing the constructed scGRNs. With real data, scTenifoldNet identifies specific gene expression programs associated with different biological processes, providing critical insights into the underlying mechanism of regulatory networks governing cellular transcriptional activities.

The Effect of Translation on the Approximated First Order Polarization Tensor of Sphere and Cube

Throughout this paper, the translation effect on the first order polarization tensor approximation for different type of objects will be highlighted. Numerical integration involving quadratic element as well as linear element for polarization tensor approximation will be presented. Here, we used different positions of an object of fixed size and conductivity when computing the first order polarization tensor. From the numerical results of computed first order polarization tensor, the convergence for every translation is observed. Moreover, discretization of the geometric objects into triangular meshes was done by using meshing software called NETGEN mesh generator while for the numerical computation, MATLAB software was used. We found that the translation has no effect on the approximated first order PT for sphere and cube after we have computed the first order PT for both geometries with a few center of masses. The numerical results of approximated first order polarization tensor is plotted by comparing the numerical results with analytical solution provided.

Recovery of image and video based on compressive sensing via tensor approximation and Spatio-temporal correlation

In recent years, block-based compressive sensing (BCS) has been extensively studied because it can reduce computational complexity and data storage by dividing the image into smaller patches, but the performance of the reconstruction algorithm is not satisfactory. In this paper, a new reconstruction model for image and video is proposed. The model makes full use of spatio-temporal correlation and utilizes low-rank tensor approximation to improve the quality of the reconstructed image and video. For image recovery, the proposed model obtains a low-rank approximation of a tensor formed by non-local similar patches, and improves the reconstruction quality from a spatial perspective by combining non-local similarity and low-rank property. For video recovery, the reconstruction process is divided into two phases. In the first phase, each frame of the video sequence is regarded as an independent image to be reconstructed by taking advantage of spatial property. The second phase performs tensor approximation through searching similar patches within frames near the target frame, to achieve reconstruction by putting the spatio-temporal correlation into full play. The resulting model is solved by an efficient Alternating Direction Method of Multipliers (ADMM) algorithm. A series of experiments show that the quality of the proposed model is comparable to the current state-of-the-art recovery methods.

Tensor product method for fast solution of optimal control problems with fractional multidimensional Laplacian in constraints

We introduce the tensor numerical method for solution of the d-dimensional optimal control problems (d=2,3) with spectral fractional Laplacian type operators in constraints discretized on large n⊗d tensor-product Cartesian grids. The approach is based on the rank-structured approximation of the matrix valued functions of the corresponding fractional finite difference Laplacian. We solve the equation for the control function, where the system matrix includes the sum of the spectral fractional d-dimensional Laplacian and its inverse. The matrix valued functions of discrete Laplace operator on a tensor grid are diagonalized by using the fast Fourier transform (FFT). Then the low rank approximation of the d-dimensional tensors obtained by folding of the corresponding large diagonal matrices of eigenvalues are computed, which allows to solve the governing equation for the control function in a tensor-structured format. The existence of low rank canonical approximation to the class of matrix valued functions involved is justified by using the sinc quadrature approximation method applied to the Laplace transform of the generating function. The linear system of equations for the control function is solved by the PCG iterative method with the rank truncation at each iteration step, where the low Kronecker rank preconditioner is pre-computed. The right-hand side, the solution vector, and the governing system matrix are maintained in the rank-structured tensor format which beneficially reduces the numerical cost to O(nlog⁡n), outperforming the standard FFT based methods of complexity O(n3log⁡n) for 3D case. Numerical tests for the 2D and 3D control problems confirm the linear complexity scaling of the method in the univariate grid size n.

A two-stage surrogate model for Neo-Hookean problems based on adaptive proper orthogonal decomposition and hierarchical tensor approximation

The evaluation of robustness and reliability of realistic structures in the presence of uncertainty is numerically costly. This motivates model order reduction techniques like the proper orthogonal decomposition (POD), which gives an approximate model on the basis of a set of precomputations of a full model. This reduces the computational time. The reduction achieved by POD is usually not sufficient in the uncertainty quantification or optimization context where a large number of evaluations has to be carried out. In this context, it is also common that only a few quantities are of interest and a further reduction is possible. The second reduction may be represented by a mapping from a possibly high-dimensional parameter space onto each quantity of interest (QoI). In general, it is difficult to construct such a mapping from an unreduced model. Hence, in this paper a two-stage surrogate model that combines both reduction approaches is introduced. This surrogate model is tailored and applied to Neo-Hookean model problems with plasticity, hardening and damage. Here, the knowledge of the model allows an adaptive selection of the POD basis on the first stage. This idea is elaborated into a generalized framework called adaptive proper orthogonal decomposition (APOD). The second stage consists of the hierarchical tensor approximation (HTA) which is easily adjusted to the accuracy and computational cost of the first stage such that the two-stage surrogate model becomes an efficient option for uncertainty quantification and optimization for Neo-Hookean problems. Additionally, with both stages at hand, the HTA is utilized for a greedy snapshot search to improve the basis of APOD and POD. This is useful if no expert knowledge is available to guide the snapshot selection in the offline phase.

Hyperspectral image denoising via global spatial-spectral total variation regularized nonconvex local low-rank tensor approximation

Hyperspectral image (HSI) denoising aims to restore clean HSI from the noise-contaminated one which is usually caused during data acquisition and conversion. In this paper, we propose a novel spatial-spectral total variation (SSTV) regularized nonconvex local low-rank (LR) tensor approximation method to remove mixed noise in HSIs. From one aspect, the clean HSI data have its underlying local LR tensor property, even though the real HSI data is not globally low-rank due to the non-independent and non-identically distributed noise and out-liers. According to this fact, we propose a novel tensor Lγ-norm to formulate the local LR prior. From another aspect, HSIs are assumed to be piecewisely smooth in the global spatial and spectral domains. Instead of traditional bandwise total variation, we use the SSTV regularization to simultaneously consider global spatial and spectral smoothness. Results on simulated and real HSI datasets indicate that the use of local LR tensor penalty and global SSTV can boost the preserving of local details and overall structural information in HSIs.

A Modified Higher-Order Singular Value Decomposition Framework With Adaptive Multilinear Tensor Rank Approximation for Three-Dimensional Magnetic Resonance Rician Noise Removal.

The magnetic resonance (MR) images are acknowledged to be inevitably corrupted by Rician distributed noise, which adversely affected the image quality for diagnosis purpose. However, the traditional denoising methods may recover the images from corruptions with severe loss of detailed structure and edge information, which would affect the lesion detections and diagnostic decision making. In this study, we challenged improving the Rician noise removal from three-dimensional (3D) MR volumetric data through a modified higher-order singular value decomposition (MHOSVD) method. The proposed framework of MHOSVD involved a parameterized logarithmic nonconvex penalty function for low-rank tensor approximation (LRTA) algorithm optimization to suppress the image noise in MR dataset. Reference cubes were extracted from the noisy image volume, and block matching was performed according to nonlocal similarity for a fourth-order tensor construction. Then the LRTA problem was implemented by tensor factorization approaches, and the ranks of unfolding matrices along different modes of the tensor were estimated utilizing an adaptive nonconvex low-rank method. The denoised MR images were finally restored through aggregating all recovered cubes. We investigated the proposed algorithm MHOSVD on both the synthetic and real clinic 3D MR images for Rician noise removal, and relative results demonstrated that the MHOSVD can recover images with fine structures and detailed edge preservation with heavy noise even as high as 15% of the maximum intensity. The experimental results were also compared along with several classical denoising methods; the MHOSVD exhibited a sufficient improvement in noise-removal performance at various noise conditions in terms of different measurement indices such as peak signal-to-noise ratio and structural similarity index metrics. Based upon the comparison, the proposed MHOSVD has proved a relative state-of-the-art performance with excellent detailed structure reservation.

Innovative Approaches to the Numerical Approximation of PDEs

This workshop was about the numerical solution of PDEs for which classical approaches, such as the finite element method, are not well suited or need further (theoretical) underpinnings. A prominent example of PDEs for which classical methods are not well suited are PDEs posed in high space dimensions. New results on low rank tensor approximation for those problems were presented. Other presentations dealt with regularity of PDEs, the numerical solution of PDEs on surfaces, PDEs of fractional order, numerical solvers for PDEs that converge with exponential rates, and the application of deep neural networks for solving PDEs.

Noise robust image clustering based on reweighted low rank tensor approximation and $$l_{\frac{1}{2}}$$ regularization

In this paper, a noise robust tensor based image clustering approach is proposed which can also perform well in the presence of gross errors. Our major contribution is the improved submodule identification technique in noisy environment by incorporating three important improvements: better low rank representation using reweighted nuclear norm, $$l_\frac{1}{2}$$ regularization to accurately capture sparseness and an error term in the model for noise robustness. Reweighted nuclear norm is introduced in the clustering model to capture self-expressiveness property in a better manner. The $$l_\frac{1}{2}$$ norm regularization is applied in place of $$l_1$$ -norm to properly capture the correlation among data members. An error term is introduced into the model to separate noise and data, which brings a noise robust image clustering technique. Combined effect all the three factors results an accurate clustering method even under the presence of severe noise. The performance of the proposed method is tested on different datasets with varying amount of noise. It is found that the proposed method provides better classification accuracy in almost all conditions as compared with existing methods.

Hyperspectral Image Recovery Using Non-Convex Low-Rank Tensor Approximation

Low-rank tensors have received more attention in hyperspectral image (HSI) recovery. Minimizing the tensor nuclear norm, as a low-rank approximation method, often leads to modeling bias. To achieve an unbiased approximation and improve the robustness, this paper develops a non-convex relaxation approach for low-rank tensor approximation. Firstly, a non-convex approximation of tensor nuclear norm (NCTNN) is introduced to the low-rank tensor completion. Secondly, a non-convex tensor robust principal component analysis (NCTRPCA) method is proposed, which aims at exactly recovering a low-rank tensor corrupted by mixed-noise. The two proposed models are solved efficiently by the alternating direction method of multipliers (ADMM). Three HSI datasets are employed to exhibit the superiority of the proposed model over the low rank penalization method in terms of accuracy and robustness.