Low-rank Tensor Approximation Research Articles

A Multilinear Nyström Algorithm for Low-Rank Approximation of Tensors in Tucker Format

A Multilinear Nyström Algorithm for Low-Rank Approximation of Tensors in Tucker Format

A sequential multilinear Nyström algorithm for streaming low-rank approximation of tensors in Tucker format

We present a sequential version of the multilinear Nyström algorithm which is suitable for low-rank Tucker approximation of tensors given in a streaming format. Accessing the tensor A exclusively through random sketches of the original data, the algorithm effectively leverages structures in A, such as low-rankness, and linear combinations. We present a deterministic analysis of the algorithm and demonstrate its superior speed and efficiency in numerical experiments including an application in video processing.

Saliency-Guided Sparse Low-Rank Tensor Approximation for Unsupervised Anomaly Detection of Hyperspectral Remote Sensing Images

Hyperspectral anomaly detection can separate sparse anomalies from the low-rank background component under an unsupervised behavior due to sufficient spectral information. Therefore, hyperspectral image anomaly detection technology has great application potential and value in public security and national defense. Currently, most existing models attempt to detect anomalous targets with a sparsity prior, without further considering the visual saliency of the targets themselves. To tackle this issue, this paper proposes a saliency-guided sparse low-rank tensor approximation model, called SSLR, to detect anomalous targets from hyperspectral remote sensing images in an unsupervised manner. Specifically, we first explore the saliency information of each pixel for regularizing the sparse anomaly matrix. We then suggest a three-directional tensor nuclear norm to obtain a low-rank background to characterize the background component. We solve the SSLR optimization problem by an efficient alternating direction method of multipliers framework. Experiments conducted on benchmark hyperspectral datasets demonstrate that the proposed SSLR outperforms some state-of-the-art anomaly detection methods.

Randomized two-sided subspace iteration for low-rank matrix and tensor decomposition

The low-rank approximation of big data matrices and tensors plays a pivotal role in many modern applications. Although, a truncated version of the singular value decomposition (SVD) furnishes the best approximation, its computation is challenging on modern, multicore architectures. Recently, the randomized subspace iteration has shown to be a powerful tool in approximating large-scale matrices. In this paper we present a two-sided variant of the randomized subspace iteration. Novel in our work is the exploitation of the unpivoted QR factorization, rather than the SVD, for factorizing the compressed matrix. Hence our algorithm is a randomized rank-revealing URV decomposition. We prove the rank-revealingness of our algorithm by establishing bounds for the singular values as well as the other blocks of the compressed matrix. We further provide bounds on the error of the low-rank approximations of the proposed algorithm, in both 2- and Frobenius norm. In addition, we employ the proposed algorithm to efficiently compute low rank tensor decompositions: we present two randomized algorithms, one for the truncated higher-order SVD, and the other for the tensor SVD. We conduct numerical tests on (i) various classes of matrices, and (ii) synthetic tensors and real datasets to demonstrate the efficacy of the proposed algorithms.

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

MvTuckER: Multi-view knowledge graphs representation learning based on tensor tucker model

Recently, more and more multi-view knowledge graphs with various important attributes are being constructed and applied, such as temporal information, geolocation, commonsense knowledge, multi-language and multi-modal. Knowledge graphs (KGs) have been widely applied in question answering systems, drug discovery, information retrieval, and so on. Knowledge graph representation learning (KGRL) is emerging as the most effective approach to tasks such as knowledge graph completion, entity alignment, knowledge reasoning and knowledge query. However, most previous work on KGs and KGRL are mainly designed for traditional KGs and only consider single-view, and the KGRL methods for special multi-view KGs faces the challenges of lacking a unified framework, insufficient interactive information and underutilizing important attributes. To bridge this gap, in this paper, we proposes a multi-view knowledge graphs (e.g., temporal KGs) representation learning method based on tensor tucker model (MvTuckER), which models the multi-view KGs from tuples to a nth-order binary tensor, and adopts the tensor n-mode product to capture complete interaction between and within different views. We also introduce low-rank and sparse approximations of core tensors to balance expressivity and complexity of MvTuckER. Moreover, we theoretically illustrate that our model is fully expressive for modeling multi-view KGs, discuss the working mechanism of MvTuckER from the perspective of logical operation, and explain how to migrate and extend our method to other types of views. We conducted extensive experiments on three multi-view knowledge graphs, and obtain 4.7%, 2.7%, and 5.3% effective improvement in Hit@1 respectively, which proves that the proposed MvTuckER can achieve state-of-the-art performance.

LRTA-SP: Low Rank Tensor Approximation with Saliency Prior for Small Target Detection in Infrared Videos

LRTA-SP: Low Rank Tensor Approximation with Saliency Prior for Small Target Detection in Infrared Videos

CP decomposition and low-rank approximation of antisymmetric tensors

CP decomposition and low-rank approximation of antisymmetric tensors

Nonconvex Robust High-Order Tensor Completion Using Randomized Low-Rank Approximation.

Within the tensor singular value decomposition (T-SVD) framework, existing robust low-rank tensor completion approaches have made great achievements in various areas of science and engineering. Nevertheless, these methods involve the T-SVD based low-rank approximation, which suffers from high computational costs when dealing with large-scale tensor data. Moreover, most of them are only applicable to third-order tensors. Against these issues, in this article, two efficient low-rank tensor approximation approaches fusing random projection techniques are first devised under the order-d ( d ≥ 3 ) T-SVD framework. Theoretical results on error bounds for the proposed randomized algorithms are provided. On this basis, we then further investigate the robust high-order tensor completion problem, in which a double nonconvex model along with its corresponding fast optimization algorithms with convergence guarantees are developed. Experimental results on large-scale synthetic and real tensor data illustrate that the proposed method outperforms other state-of-the-art approaches in terms of both computational efficiency and estimated precision.

Anomaly Detection in Satellite-Borne Push-Broom Hyperspectral Imagery Based on Joint Low-Rank Tensor Approximation

Anomaly Detection in Satellite-Borne Push-Broom Hyperspectral Imagery Based on Joint Low-Rank Tensor Approximation

High-Fidelity compressive spectral image reconstruction through a novel Non-Convex Non-Local Low-Rank tensor approximation model

Low-quality reconstruction is one main factor that greatly limits the development of snapshot compressive imaging (SCI) systems. The common algorithms usually unfold a 3D hyperspectral image (HSI) into a vector or matrix which inevitably corrupts the intrinsic structure of the HSI and insufficiently discovers its prior information. To fully exploit the underlying structures and internal correlations of the HSIs, this paper proposes a novel non-convex model based on Non-Local low-rank Tensor Approximation with Hyper-Laplacian priori (HL), named HL-NLTA, which effectively combines local and non-local similarity information from fused spectral and spatial perspectives. Specifically, a non-local low-rank tensor model based on the minimax concave plus (MCP) penalty and log sum (LS) penalty separately is developed which can simultaneously exploit two intrinsic properties of HSI, i.e., global correlation along the spectral domain (GCAS) and non-local self-similarity along the spatial domain (NSAS). Furthermore, the Hyper-Laplacian priori regularized through a non-convex ℓp (0 < p < 1) norm can preserve well the spectral and spatial structure. An optimization algorithm based on the alternating direction multiplier method (ADMM) framework is designed and accelerated by the Generalized shrinkage/thresholding (GST) algorithm and the fast Fourier transform (FFT) to solve the new nonconvex model. Extensive simulation results on public datasets and experimental results on our real CASSI system indicate that the proposed method enables high-fidelity HSI reconstructions in terms of simultaneous spatial structure detail recovery and spectral preservation. These results verify the effectiveness of the proposed model for snapshot compressive imaging (e.g., Coded aperture snapshot spectral imaging CASSI).

Uncertainty quantification in low-probability response estimation using sliced inverse regression and polynomial chaos expansion

For wave energy converters (WECs), wind turbines, etc., estimation of response extremes over a selected exposure time is important during design. Sources of uncertainty arising from background slowly-varying environmental conditions and from shorter time-scale fluctuations in ocean winds, turbulence, etc. must all be considered. Together, these different sources can comprise a high-dimensional vector of stochastic variables (often on the order of hundreds or thousands). To accurately propagate the influence of these uncertainty sources to model outputs, conventional surrogate model building approaches such as polynomial chaos expansion (PCE), stochastic collocation, low-rank tensor approximations, etc. must consider dimension reduction. We explore the use of sliced inverse regression (SIR) combined with polynomial chaos expansion. SIR first reduces the original high-dimensional problem to a low-dimensional one; then, an optimal polynomial PCE model is proposed and applied on “effective” components in the low-dimensional space. SIR-PCE can mitigate the curse of dimensionality. It is employed here in the prediction of the long-term extreme response of offshore structures; it is demonstrated using classical benchmark analytical functions as well as offshore applications including extreme waves and the response of a wave energy converter. Efficiency and accuracy gains over Monte Carlo simulation and other methods in literature are found.

Reducing the Control Overhead of Intelligent Reconfigurable Surfaces via a Tensor-Based Low-Rank Factorization Approach

Intelligent reconfigurable surfaces (IRS) are becoming an attractive component of cellular networks due to their ability to shape the propagation environment and thereby improve coverage. While IRS nodes incorporate a great number of phase-shifting elements and a controller entity, the phase shifts are typically determined by the cellular base station (BS) due to its computational capability. Since controlling a large number of phase shifts may become prohibitive in practice, it is important to reduce the control overhead between the BS and the IRS controller. To this end, in this paper, we propose a low-rank modeling approach for the IRS phase shifts. The key idea is to represent the IRS phase shift vector using a low-rank tensor approximation model, where each rank-one component is modeled as the Kronecker product of a predefined number of factors of smaller sizes, obtained via tensor decomposition algorithms. We show that the proposed low-rank models drastically reduce the required feedback requirements associated with the BS-IRS control links. Our simulation results indicate that the proposed method is especially attractive in scenarios with a strong line of sight component, in which case nearly the same spectral efficiency is reached as in the cases with near-optimal phase shifts, but with significantly lower feedback overhead.

Sensitivity and reliability assessment of buckling restrained braces using machine learning assisted-simulation

Buckling Restrained Braces (BRBs) play an essential role in seismic protection due to their remarkable energy dissipation capacity, ductility, and stiffness. This paper represents the first thorough evaluation of the reliability and uncertainty of BRBs, accounting for various sources of uncertainty and varied conditions, such as different gap sizes, friction coefficients between the encasing and the steel core, and other geometric and mechanical properties of the connecting parts. This paper introduces an innovative application of an integrated probabilistic approach that combined Finite Element (FE) modeling, Artificial Neural Networks (ANN), and Monte Carlo simulation to characterize the seismic behavior of BRBs. Experimental results from literature is used to validate the FE model constructed using ABAQUS. This validated model is then integrated into a Low-rank Tensor Approximation process to yield variance-based sensitivity measures for the considered variables. The most influential variables identified in this process along with the calibrated FE model is utilized to conduct a reliability assessment through an ANN-assisted Monte Carlo simulation. This analysis is conducted to quantify the reliability level of the investigated BRBs as a function of the gap size and establish the resistance factors necessary to maintain the reliability level above prescribed thresholds. It was found that the highest reliability can be achieved at a small gap size equal to half the radius of gyration of the main steel core, requiring a resistance factor of 0.9. For gap sizes higher than this value, a resistance factor ranging between 0.70 and 0.85 is needed.

Tensor based low rank representation of hyperspectral images for wheat seeds varieties identification

Hyperspectral image (HSI) based methods are widely used in identifying seeds varieties with high accuracy. However, the excessive spectral bands in HSI may contain redundant information and degrade the model performance. To address this challenge, we propose a novel feature extraction method called low rank tensor approximation (LRTA). Unlike traditional methods, LRTA extracts joint discriminative information of all wheat seeds from hyperspectral images in a 3-order tensor form, preserving intrinsic information. Our model has three key steps: extracting the region of interest from hyperspectral images and presenting average spectral information in tensor form, using LRTA to extract jointly discriminative information, and feeding this information into a classifier to identify seeds varieties. Experiments on our proposed dataset show that the proposed method has 4% improved to the conventional methods on average. Our method shows promise for improving the accuracy of seeds identification while reducing the dimensionality of HSI.

Superpixel-based robust tensor low-rank approximation for multimedia data recovery

Recently, the low-rank matrix/tensor-based methods have attracted increasing attention for multi-dimensional multimedia data (e.g., image and video) recovery, which assumes the holistic data is low-rank. For most multimedia data, the low-rank assumption is usually violated due to their spatially diverse local similarity. The cube-based method, which breaks the holistic image into small local cubes with a fixed size, can alleviate this issue to some extent. However, cubes with a fixed size are not flexible to cover the regions of local similarity at different scales. Inspired by the superpixel, we suggest the size-adaptive super-tensor as the generic unit instead of the cube with the fixed spatial size, which allows us to flexibly exploit the local similarity at different scales. The super-tensor enjoys significant low-rankness and avoids excessive overlapped pixels as compared with cubes of fixed sizes. Empowering with the super-tensor, we propose a low-rank super-tensor approximation (LRSTA) model for multi-dimensional data recovery, which can fully exploit the local similarity and low-rankness at different scales. Moreover, we develop an efficient alternating direction method of multipliers (ADMM) algorithm to solve the proposed model. Extensive experiments on multimedia data, including color images, multispectral images (MSIs), and videos, verify that the proposed method outperforms the other competing methods.

Robust Low-Rank Graph Multi-View Clustering via Cauchy Norm Minimization

Graph-based multi-view clustering methods aim to explore the partition patterns by utilizing a similarity graph. However, many existing methods construct a consensus similarity graph based on the original multi-view space, which may result in the lack of information on the underlying low-dimensional space. Additionally, these methods often fail to effectively handle the noise present in the graph. To address these issues, a novel graph-based multi-view clustering method which combines spectral embedding, non-convex low-rank approximation and noise processing into a unit framework is proposed. In detail, the proposed method constructs a tensor by stacking the inner product of normalized spectral embedding matrices obtained from each similarity matrix. Then, the obtained tensor is decomposed into a low-rank tensor and a noise tensor. The low-rank tensor is constrained via nonconvex low-rank tensor approximation and a novel Cauchy norm with an upper bound is proposed to handle the noise. Finally, we derive the consensus similarity graph from the denoised low-rank tensor. The experiments on five datasets demonstrate that the proposed method outperforms other state-of-the-art methods on five datasets.

Optical coherence tomography image despeckling based on tensor singular value decomposition and fractional edge detection

Optical coherence tomography (OCT) imaging is a technique that is frequently used to diagnose medical conditions. However, coherent noise, sometimes referred to as speckle noise, can dramatically reduce the quality of OCT images, which has an adverse effect on how OCT images are used. In order to enhance the quality of OCT images, a speckle noise reduction technique is developed, and this method is modelled as a low-rank tensor approximation issue. The grouped 3D tensors are first transformed into the transform domain using tensor singular value decomposition (t-SVD). Then, to cut down on speckle noise, transform coefficients are thresholded. Finally, the inverse transform can be used to produce images with speckle suppression. To further enhance the despeckling results, a feature-guided thresholding approach based on fractional edge detection and an adaptive backward projection technique are also presented. Experimental results indicate that the presented algorithm outperforms several comparison methods in relation to speckle suppression, objective metrics, and edge preservation.

On Best Low Rank Approximation of Positive Definite Tensors

Tensors, or multi-indexed arrays, play an important role in machine learning and signal processing. These higher-order generalizations of matrices allow for preservation of higher-order structure in data, and low rank decompositions of tensors allow for recovery of underlying information. In many cases, the underlying tensor of interest is positive (semi)definite, i.e., the homogeneous polynomial associated to the tensor has nonnegative evaluation on all inputs. An archetypal problem is that one has a noisy measurement of some low rank signal tensor of interest. This measurement itself does not have low rank, so one must compute a best low rank approximation of the measured tensor to recover component information. As it turns out, the set of tensors of rank less than or equal to is, in general, not closed when , and, as a consequence, best low rank approximations can fail to exist. In the case when a best low rank approximation does not exist, near optimal low rank approximations suffer numerical issues and cannot be used to reliably approximate underlying component information. As a consequence, existence guarantees for best low rank tensor approximations are of great practical and theoretical interest. This article develops deterministic guarantees for the existence of best low rank approximations of tensors which are positive semidefinite. In particular, we show that the set of low rank positive semidefinite tensors is relatively closed as a subset of the set of positive semidefinite tensors. We use this fact to give a deterministic bound which may be used to guarantee the existence of a best low rank approximation of a noisy low rank positive semidefinite tensor. Furthermore, we show that this condition guarantees uniqueness of the canonical polyadic decomposition for best low rank approximations. In addition, for order three tensors, we prove that our bound is sharp and that it can be computed using semidefinite programming. The main results of this article are illustrated through numerical experiments, which show that our bound is highly predictive of numerical issues when attempting to recover underlying component information from a noisy low rank positive semidefinite tensor.

Tracking online low-rank approximations of higher-order incomplete streaming tensors

In this paper, we propose two new provable algorithms for tracking online low-rank approximations of high-order streaming tensors with missing data. The first algorithm, dubbed adaptive Tucker decomposition (ATD), minimizes a weighted recursive least-squares cost function to obtain the tensor factors and the core tensor in an efficient way, thanks to an alternating minimization framework and a randomized sketching technique. Under the canonical polyadic (CP) model, the second algorithm, called ACP, is developed as a variant of ATD when the core tensor is imposed to be identity. Both algorithms are low-complexity tensor trackers that have fast convergence and low memory storage requirements. A unified convergence analysis is presented for ATD and ACP to justify their performance. Experiments indicate that the two proposed algorithms are capable of streaming tensor decomposition with competitive performance with respect to estimation accuracy and runtime on both synthetic and real data.