Low-rank Tensor Approximation Research Articles

Nonlocal Tensor-Ring Decomposition for Hyperspectral Image Denoising

Hyperspectral image (HSI) denoising is a fundamental problem in remote sensing and image processing. Recently, nonlocal low-rank tensor approximation-based denoising methods have attracted much attention due to their advantage of being capable of fully exploiting the nonlocal self-similarity and global spectral correlation. Existing nonlocal low-rank tensor approximation methods were mainly based on two common decomposition [Tucker or CANDECOMP/PARAFAC (CP)] methods and achieved the state-of-the-art results, but they are subject to certain issues and do not produce the best approximation for a tensor. For example, the number of parameters for Tucker decomposition increases exponentially according to its dimensions, and CP decomposition cannot better preserve the intrinsic correlation of the HSI. In this article, a novel nonlocal tensor-ring (TR) approximation is proposed for HSI denoising by using TR decomposition to explore the nonlocal self-similarity and global spectral correlation simultaneously. TR decomposition approximates a high-order tensor as a sequence of cyclically contracted third-order tensors, which has strong ability to explore these two intrinsic priors and to improve the HSI denoising results. Moreover, an efficient proximal alternating minimization algorithm is developed to optimize the proposed TR decomposition model efficiently. Extensive experiments on three simulated data sets under several noise levels and two real data sets verify that the proposed TR model provides better HSI denoising results than several state-of-the-art methods in terms of quantitative and visual performance evaluations.

Nonnegative Tensor Train Factorization with DMRG Technique

Tensor train is one of the modern decompositions used as low-rank tensor approximations of multidimensional arrays. If the original data is nonnegative we sometimes want the approximant to keep this property. In this work new methods for nonnegative tensor train factorization are proposed. Low-rank approximation approach helps to speed up the computations whereas DMRG technique allows to adapt nonnegative TT ranks for better accuracy. The performance analysis of the proposed algorithms as well as comparison with other nonnegative TT factorization method are presented.

Hyperspectral Image Denoising Using Global Weighted Tensor Norm Minimum and Nonlocal Low-Rank Approximation

A hyperspectral image (HSI) contains abundant spatial and spectral information, but it is always corrupted by various noises, especially Gaussian noise. Global correlation (GC) across spectral domain and nonlocal self-similarity (NSS) across spatial domain are two important characteristics for an HSI. To keep the integrity of the global structure and improve the details of the restored HSI, we propose a global and nonlocal weighted tensor norm minimum denoising method which jointly utilizes GC and NSS. The weighted multilinear rank is utilized to depict the GC information. To preserve structural information with NSS, a patch-group-based low-rank-tensor-approximation (LRTA) model is designed. The LRTA makes use of Tucker decompositions of 4D patches, which are composed of a similar 3D patch group of HSI. The alternating direction method of multipliers (ADMM) is adapted to solve the proposed models. Experimental results show that the proposed algorithm can preserve the structural information and outperforms several state-of-the-art denoising methods.

Low-Rank Tensor Models for Improved Multi-Dimensional MRI: Application to Dynamic Cardiac T 1 Mapping.

Multi-dimensional, multi-contrast magnetic resonance imaging (MRI) has become increasingly available for comprehensive and time-efficient evaluation of various pathologies, providing large amounts of data and offering new opportunities for improved image reconstructions. Recently, a cardiac phase-resolved myocardial T 1 mapping method has been introduced to provide dynamic information on tissue viability. Improved spatio-temporal resolution in clinically acceptable scan times is highly desirable but requires high acceleration factors. Tensors are well-suited to describe inter-dimensional hidden structures in such multi-dimensional datasets. In this study, we sought to utilize and compare different tensor decomposition methods, without the use of auxiliary navigator data. We explored multiple processing approaches in order to enable high-resolution cardiac phase-resolved myocardial T 1 mapping. Eight different low-rank tensor approximation and processing approaches were evaluated using quantitative analysis of accuracy and precision in T 1 maps acquired in six healthy volunteers. All methods provided comparable T 1 values. However, the precision was significantly improved using local processing, as well as a direct tensor rank approximation. Low-rank tensor approximation approaches are well-suited to enable dynamic T 1 mapping at high spatio-temporal resolutions.

Robust block tensor principal component analysis

Robust tensor principal component analysis based on tensor singular value decomposition (t-SVD) is a very effective tool to extract the low rank and sparse components in multi-way signals. In this paper, instead of the tensor nuclear norm (TNN) based on t-SVD for the whole tensor, we propose using the sum of TNN for its small blocks in the same size aiming to do the extraction in a more appropriate scale. The alternating direction method of multipliers can divide the optimization model into two sub-problems, i.e. low rank tensor approximation and sparse component approximation. The iterative block tensor singular value soft thresholding and iterative soft thresholding are used to solve these two sub-problems, respectively. In numerical experiments, the results demonstrate the performance improvement of the proposal method in face image denoising, color image denoising, and illumination normalization for face images.

Despeckling of 3D ultrasound image using tensor low rank approximation

In this paper, a new method for despeckling of 3D ultrasound images based on tensor low rank approximation technique is presented. The main motive of this paper is to establish a new denoising method which can incorporate the redundancy present in 3D ultrasound image in all directions unlike the existing techniques. Hence, a substantial improvement in the overall performance can be achieved by preserving the edges and other finer details of the image. In order to achieve this goal, the 3D ultrasound image is modeled in terms of low rank tensor approximation of grouped tensors which is solved by minimizing the Tensor Nuclear Norm (TNN) in tensor-Singular Value Decomposition (t-SVD) framework. The resulting optimization problem is solved by the method of inexact Augmented Lagrangian Multipliers (ALM). To the best of our understanding, this is the first attempt to model the 3D ultrasound image despeckling problem as tensor low rank recovery technique. The proposed algorithm is compared with popular state-of-art works in literature including a low rank based approach termed as DLRA [1] designed for 2D ultrasound images, and it is found that our scheme offers superior performance than its counterparts.

Low-rank Parameterization of Volumetric Domains for Isogeometric Analysis

Constructing a volumetric spline parameterization of a three dimensional domain with given boundary is a fundamental problem in isogeometric analysis. Recently it is shown that the rank of the parameterization has great impact in subsequent numerical simulation, and lower rank leads to higher computational efficiency. In this paper, we propose a method for low-rank spline parameterization of volumetric domains using low-rank tensor approximation technique. We model the problem as a non-linear optimization problem, the objective function of which consists of the MIPS (Most Isometric Parameterizations) term and the low-rank regularization term. An algorithm based on the alternating direction method of multipliers (ADMM) is presented to solve the optimization problem efficiently. To further obtain a low-rank spline representation of the parameterization, a modified CANDECOMP/PARAFAC (CP) decomposition algorithm is proposed. Experimental examples demonstrate that our approach can produce low-rank and bijective parameterizations with high quality.

Fast and Error-Bounded Space-Variant Bilateral Filtering

The traditional space-invariant isotropic kernel utilized by a bilateral filter (BF) frequently leads to blurry edges and gradient reversal artifacts due to the existence of a large amount of outliers in the local averaging window. However, the efficient and accurate estimation of space-variant kernels which adapt to image structures, and the fast realization of the corresponding space-variant bilateral filtering are challenging problems. To address these problems, we present a space-variant BF (SVBF), and its linear time and error-bounded acceleration method. First, we accurately estimate spacevariant anisotropic kernels that vary with image structures in linear time through structure tensor and minimum spanning tree. Second, we perform SVBF in linear time using two error-bounded approximation methods, namely, low-rank tensor approximation via higher-order singular value decomposition and exponential sum approximation. Therefore, the proposed SVBF can efficiently achieve good edge-preserving results. We validate the advantages of the proposed filter in applications including: image denoising, image enhancement, and image focus editing. Experimental results demonstrate that our fast and error-bounded SVBF is superior to state-of-the-art methods.

PLS-based adaptation for efficient PCE representation in high dimensions

Uncertainty quantification of engineering systems modeled by computationally intensive numerical models remains a challenging task, despite the increase in computer power. Efficient uncertainty propagation of such models can be performed by use of surrogate models, such as polynomial chaos expansions (PCE). A major drawback of standard PCE is that its predictive ability decreases with increase of the problem dimension for a fixed computational budget. This is related to the fact that the number of terms in the expansion increases fast with the input variable dimension. To address this issue, Tipireddy and Ghanem (2014) introduced a sparse PCE representation based on a transformation of the coordinate system in Gaussian input variable spaces. In this contribution, we propose to identify the projection operator underlying this transformation and approximate the coefficients of the resulting PCE through partial least squares (PLS) analysis. The proposed PCE-driven PLS algorithm identifies the directions with the largest predictive significance in the PCE representation based on a set of samples from the input random variables and corresponding response variable. This approach does not require gradient evaluations, which makes it efficient for high dimensional problems with black-box numerical models. We assess the proposed approach with three numerical examples in high-dimensional input spaces, comparing its performance with low-rank tensor approximations. These examples demonstrate that the PLS-based PCE method provides accurate representations even for strongly non-linear problems.

Low CP Rank and Tucker Rank Tensor Completion for Estimating Missing Components in Image Data

Tensor completion recovers missing components of multi-way data. The existing methods use either the Tucker rank or the CANDECOMP/PARAFAC (CP) rank in low-rank tensor optimization for data completion. In fact, these two kinds of tensor ranks represent different high-dimensional data structures. In this paper, we propose to exploit the two kinds of data structures simultaneously for image recovery through jointly minimizing the CP rank and Tucker rank in the low-rank tensor approximation. We use the alternating direction method of multipliers (ADMM) to reformulate the optimization model with two tensor ranks into its two sub-problems, and each has only one tensor rank optimization. For the two main sub-problems in the ADMM, we apply rank-one tensor updating and weighted sum of matrix nuclear norms minimization methods to solve them, respectively. The numerical experiments on some image and video completion applications demonstrate that the proposed method is superior to the state-of-the-art methods.

Computing the density of states for optical spectra of molecules by low-rank and QTT tensor approximation

In this paper, we introduce a new interpolation scheme to approximate the density of states (DOS) for a class of rank-structured matrices with application to the Tamm–Dancoff approximation (TDA) of the Bethe–Salpeter equation (BSE). The presented approach for approximating the DOS is based on two main techniques. First, we propose an economical method for calculating the traces of parametric matrix resolvents at interpolation points by taking advantage of the block-diagonal plus low-rank matrix structure described in [6,3] for the BSE/TDA problem. This allows us to overcome the computational difficulties of the traditional schemes since we avoid the construction of the matrix inverse and hence the need of stochastic sampling. Second, we show that a regularized or smoothed DOS discretized on a fine grid of size N can be accurately represented in a low rank quantized tensor train (QTT) format that can be determined through a least squares fitting procedure. The QTT tensor provides good approximation properties for strictly oscillating DOS functions with multiple gaps, in contrast to interpolation by problem independent functions like polynomials, trigonometric functions, etc. Moreover, the QTT approximant requires asymptotically much fewer (e.g., O(log⁡N)) functional calls compared with the full grid size N. Numerical tests indicate that the QTT approach yields accurate recovery of DOS associated with problems that contain relatively large spectral gaps. The QTT tensor rank only weakly depends on the size of a molecular system that paves the way for treatment of large-scale spectral problems.

Estimating Functional Connectivity Networks via Low-Rank Tensor Approximation With Applications to MCI Identification.

Functional connectivity network (FCN) has become an increasingly important approach to gain a better understanding of the brain, as well as discover informative biomarkers for diagnosis of neurodegenerative diseases. Due to its importance, many FCN estimation methods have been developed in the past decades, including methods based on the classical Pearson's correlation, (regularized) partial correlation, and some higher-order variants. However, most of the existing methods estimate one FCN at a time, thus ignoring the possibly shared structure among FCNs from different subjects. Recently, researchers introduce group constraints (or population priors) into FCN estimation by assuming that FCNs are topologically identical across subjects. Obviously, such a constraint/prior is too strong to be satisfied in practice, especially when both patients and healthy subjects are involved simultaneously in the group. To address this problem, we propose a novel FCN estimation approach based on an assumption that the involved FCNs have similar but not necessarily identical topology. More specifically, we implement this idea under a two-step learning framework. First, we independently estimate FCNs based on traditional methods, such as Pearson's correltion and sparse representation, making sure that each FCN captures the specific properties of the corresponding subject. Then, we stack the estimated FCNs (in fact, their adjacency matrices) into a tensor, and refine the stacked FCNs via low-rank tensor approximation. Finally, we apply the improved FCNs to identify subjects with mild cognitive impairment (MCI) from healthy controls, and achieve a higher classification accuracy.

Iterative Adaptive Nonconvex Low-Rank Tensor Approximation to Image Restoration Based on ADMM

In this paper, in order to recover more finer details of the image and to avoid the loss of image structure information for image restoration problem, we develop an iterative adaptive weighted core tensor thresholding (IAWCTT) approach based on the alternating direction method of multipliers (ADMM). By observing the decoupling property of the ADMM algorithm, we first propose that the key step to image restoration is to tackle the denoising subproblem efficiently using appropriate prior information. Secondly, by analyzing the properties of the core tensor, we propose that low-rank tensor approximation can be implemented by penalizing the core tensor itself, instead of penalizing the CP rank, Tucker rank or the multilinear rank and Tubal rank. The IAWCTT approach is proposed to solve the denoising subproblem in the ADMM framework, and we claim that such an adaptive weighted scheme is equivalent to a kind of nonconvex penalty for the core tensor; thus, it is unnecessary to use the nonconvex penalty term to induce strong sparse/low-rank solution in image restoration optimization problem, because the scheme that selecting appropriate weights to the convex penalty term can also lead to strong sparse/low-rank solution. Numerical experiments show that our proposed model and algorithm are comparable to other state-of-the-art models and methods.

Numerical Computation for Orthogonal Low-Rank Approximation of Tensors

In this paper we study the orthogonal low-rank approximation problem of tensors in the general setting in the sense that more than one matrix factor is required to be mutually orthonormal, which in...

A Survey on Tensor Techniques and Applications in Machine Learning

This survey gives a comprehensive overview of tensor techniques and applications in machine learning. Tensor represents higher order statistics. Nowadays, many applications based on machine learning algorithms require a large amount of structured high-dimensional input data. As the set of data increases, the complexity of these algorithms increases exponentially with the increase of vector size. Some scientists found that using tensors instead of the original input vectors can effectively solve these high-dimensional problems. This survey introduces the basic knowledge of tensor, including tensor operations, tensor decomposition, some tensor-based algorithms, and some applications of tensor in machine learning and deep learning for those who are interested in learning tensors. The tensor decomposition is highlighted because it can effectively extract structural features of data and many algorithms and applications are based on tensor decomposition. The organizational framework of this paper is as follows. In part one, we introduce some tensor basic operations, including tensor decomposition. In part two, applications of tensor in machine learning and deep learning, including regression, supervised classification, data preprocessing, and unsupervised classification based on low rank tensor approximation algorithms are introduced detailly. Finally, we briefly discuss urgent challenges, opportunities and prospects for tensor.

Matrix Rigidity and the Ill-Posedness of Robust PCA and Matrix Completion

Robust principal component analysis (RPCA) [J. Candès et al., J. ACM, 58 (2011), pp. 1--37] and low-rank matrix completion [B. Recht, M. Fazel, and P. A. Parrilo, SIAM Rev., 52 (2010), pp. 471--501] are extensions of PCA that allow for outliers and missing entries, respectively. It is well known that solving these problems requires a low coherence between the low-rank matrix and the canonical basis, since in the extreme cases---when the low-rank matrix we wish to recover is also sparse--- there is an inherent ambiguity. However, in both problems the well-posedness issue is even more fundamental; in some cases, both RPCA and matrix completion can fail to have any solutions due to the set of low-rank plus sparse matrices not being closed, which in turn is equivalent to the notion of the matrix rigidity function not being lower semicontinuous [Kumar et al., Comput. Complex., 23 (2014), pp. 531--563]. By constructing infinite families of matrices, we derive bounds on the rank and sparsity such that the set of low-rank plus sparse matrices is not closed. We also demonstrate numerically that a wide range of nonconvex algorithms for both RPCA and matrix completion have diverging components when applied to our constructed matrices. This is analogous to the case of sets of higher order tensors not being closed under canonical polyadic (CP) tensor rank, rendering the best low-rank tensor approximation unsolvable [V. de Silva and L.-H. Lim, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1084--1127] and hence encouraging the use of multilinear tensor rank [L. De Lathauwer, B. De Moor, and J. Vandewalle, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324--1342].

Weighted Tensor Rank-1 Decomposition for Nonlocal Image Denoising.

Natural images often contain patches with high similarity. In this paper, to effectively utilize the local and nonlocal self-similarity for low-rank models, we propose a novel weighted tensor rank-1 decomposition method (terms as WTR1) for nonlocal image denoising. Although the low-rank approximation problem has been well studied for matrices, it remains elusive of the theoretically extension to tensors due to the NPhard tensor decomposition. To tackle this problem, the proposed WTR1 method designs a new efficient CANDECOMP/PARAFAC (CP) decomposition algorithm and constructs a straightforward low-rank tensor approximation strategy. This is achieved by elegantly manipulating the CP-rank, called intrinsic low-rank tensor approximation. Specifically, the WTR1 method first groups similar patches into a 3-D stack and converts the stack into a finite sum of rank-1 products. Then, we deploy the intrinsic low-rank tensor approximation to produce the final denoised image. The proposed WTR1 method can jointly exploit the local and nonlocal self-similarity, thus improving the nonlocal image denoising quality. Experimental results have shown that the proposed WTR1 outperforms several state-of-the-art denoising methods.

Robust Tensor Approximation With Laplacian Scale Mixture Modeling for Multiframe Image and Video Denoising

Sparse and low-rank models have been widely studied in the literature of signal processing and computer vision. However, as the dimensionality of dataset increases (e.g., multispectral images, dynamic MRI images, and video sequences), the optimality of vector and matrix-based data representations and modeling tools becomes questionable. Inspired by recent advances in sparse and low-rank tensor analysis, we propose a novel robust tensor approximation (RTA) framework with the Laplacian Scale Mixture (LSM) modeling for three-dimensional (3-D) data and beyond. Our technical contributions are summarized as follows: first, conceptually similar to robust PCA, we consider its tensor extension here—i.e., low-rank tensor approximation in the presence of outliers modeled by sparse noise; second, built upon previous work on tensor sparsity, we propose to model tensor coefficients with an LSM prior and formulate a maximum a posterior estimation problem for noisy observations. Both unknown sparse coefficients and hidden LSM parameters can be efficiently estimated by the method of alternating optimization; and third, we have derived closed-form solutions for both subproblems and developed computationally efficient denoising techniques for multiframe images and video. Experimental results on three datasets have shown that the proposed algorithm can better preserve the sharpness of important image structures and outperform several existing state-of-the-art image/video denoising methods (e.g., BM4D/VBM4D and tensor dictionary learning).

Children exposure to femtocell in indoor environments estimated by sparse low-rank tensor approximations

The exposure of an 8-year-old child to a femtocell operating at 2600 MHz, both (child and source) freely located in random positions in an indoor environment, was assessed. In order to develop surrogate models of the exposure, stochastic dosimetry based on sparse low-rank tensor approximation method (sparse LRA) was used. The surrogate models were used for fastly estimating the specific absorption rate (SAR) in all the possible positions of femtocell and child. Results showed that, for all the possible positions in the room, the exposure values were significantly below the International Commission of Non-Ionizing Radiation Protection (ICNIRP) guidelines for general public and that the probability of reaching SAR values higher than 1% of the ICNIRP guidelines value was lower than 0.006. The variation of the distance between femtocell and child influenced greatly the exposure, resulting in quartile coefficient of dispersion values always higher than 48%.

Feature Extraction for Incomplete Data Via Low-Rank Tensor Decomposition With Feature Regularization.

Multidimensional data (i.e., tensors) with missing entries are common in practice. Extracting features from incomplete tensors is an important yet challenging problem in many fields such as machine learning, pattern recognition, and computer vision. Although the missing entries can be recovered by tensor completion techniques, these completion methods focus only on missing data estimation instead of effective feature extraction. To the best of our knowledge, the problem of feature extraction from incomplete tensors has yet to be well explored in the literature. In this paper, we therefore tackle this problem within the unsupervised learning environment. Specifically, we incorporate low-rank tensor decomposition with feature variance maximization (TDVM) in a unified framework. Based on orthogonal Tucker and CP decompositions, we design two TDVM methods, TDVM-Tucker and TDVM-CP, to learn low-dimensional features viewing the core tensors of the Tucker model as features and viewing the weight vectors of the CP model as features. TDVM explores the relationship among data samples via maximizing feature variance and simultaneously estimates the missing entries via low-rank Tucker/CP approximation, leading to informative features extracted directly from observed entries. Furthermore, we generalize the proposed methods by formulating a general model that incorporates feature regularization into low-rank tensor approximation. In addition, we develop a joint optimization scheme to solve the proposed methods by integrating the alternating direction method of multipliers with the block coordinate descent method. Finally, we evaluate our methods on six real-world image and video data sets under a newly designed multiblock missing setting. The extracted features are evaluated in face recognition, object/action classification, and face/gait clustering. Experimental results demonstrate the superior performance of the proposed methods compared with the state-of-the-art approaches.