Rank Tensor Approximation Research Articles

Embedded Feature Selection on Graph-Based Multi-View Clustering

Recently, anchor graph-based multi-view clustering has been proven to be highly efficient for large-scale data processing. However, most existing anchor graph-based clustering methods necessitate post-processing to obtain clustering labels and are unable to effectively utilize the information within anchor graphs. To solve these problems, we propose an Embedded Feature Selection on Graph-Based Multi-View Clustering (EFSGMC) approach to improve the clustering performance. Our method decomposes anchor graphs, taking advantage of memory efficiency, to obtain clustering labels in a single step without the need for post-processing. Furthermore, we introduce the l2,p-norm for graph-based feature selection, which selects the most relevant data for efficient graph factorization. Lastly, we employ the tensor Schatten p-norm as a tensor rank approximation function to capture the complementary information between different views, ensuring similarity between cluster assignment matrices. Experimental results on five real-world datasets demonstrate that our proposed method outperforms state-of-the-art approaches.

Relaxation of the rank-1 tensor approximation using different norms

Relaxation of the rank-1 tensor approximation using different norms

LRTA-SP: Low Rank Tensor Approximation with Saliency Prior for Small Target Detection in Infrared Videos

LRTA-SP: Low Rank Tensor Approximation with Saliency Prior for Small Target Detection in Infrared Videos

Tensor based low rank representation of hyperspectral images for wheat seeds varieties identification

Hyperspectral image (HSI) based methods are widely used in identifying seeds varieties with high accuracy. However, the excessive spectral bands in HSI may contain redundant information and degrade the model performance. To address this challenge, we propose a novel feature extraction method called low rank tensor approximation (LRTA). Unlike traditional methods, LRTA extracts joint discriminative information of all wheat seeds from hyperspectral images in a 3-order tensor form, preserving intrinsic information. Our model has three key steps: extracting the region of interest from hyperspectral images and presenting average spectral information in tensor form, using LRTA to extract jointly discriminative information, and feeding this information into a classifier to identify seeds varieties. Experiments on our proposed dataset show that the proposed method has 4% improved to the conventional methods on average. Our method shows promise for improving the accuracy of seeds identification while reducing the dimensionality of HSI.

On Best Low Rank Approximation of Positive Definite Tensors

Tensors, or multi-indexed arrays, play an important role in machine learning and signal processing. These higher-order generalizations of matrices allow for preservation of higher-order structure in data, and low rank decompositions of tensors allow for recovery of underlying information. In many cases, the underlying tensor of interest is positive (semi)definite, i.e., the homogeneous polynomial associated to the tensor has nonnegative evaluation on all inputs. An archetypal problem is that one has a noisy measurement of some low rank signal tensor of interest. This measurement itself does not have low rank, so one must compute a best low rank approximation of the measured tensor to recover component information. As it turns out, the set of tensors of rank less than or equal to is, in general, not closed when , and, as a consequence, best low rank approximations can fail to exist. In the case when a best low rank approximation does not exist, near optimal low rank approximations suffer numerical issues and cannot be used to reliably approximate underlying component information. As a consequence, existence guarantees for best low rank tensor approximations are of great practical and theoretical interest. This article develops deterministic guarantees for the existence of best low rank approximations of tensors which are positive semidefinite. In particular, we show that the set of low rank positive semidefinite tensors is relatively closed as a subset of the set of positive semidefinite tensors. We use this fact to give a deterministic bound which may be used to guarantee the existence of a best low rank approximation of a noisy low rank positive semidefinite tensor. Furthermore, we show that this condition guarantees uniqueness of the canonical polyadic decomposition for best low rank approximations. In addition, for order three tensors, we prove that our bound is sharp and that it can be computed using semidefinite programming. The main results of this article are illustrated through numerical experiments, which show that our bound is highly predictive of numerical issues when attempting to recover underlying component information from a noisy low rank positive semidefinite tensor.

Hyper-Laplacian Regularized Multi-View Clustering with Exclusive L21 Regularization and Tensor Log-Determinant Minimization Approach

Multi-view clustering aims to capture the multiple views inherent information by identifying the data clustering that reflects distinct features of datasets. Since there is a consensus in literature that different views of a dataset share a common latent structure, most existing multi-view subspace learning methods rely on the nuclear norm to seek the low-rank representation of the underlying subspace. However, the nuclear norm often fails to distinguish the variance of features for each cluster due to its convex nature and data tends to fall in multiple non-linear subspaces for multi-dimensional datasets. To address these problems, we propose a new and novel multi-view clustering method (HL-L21-TLD-MSC) that unifies the Hyper-Laplacian (HL) and exclusive ℓ 2,1 (L21) regularization with the Tensor Log-Determinant Rank Minimization (TLD) setting. Specifically, the hyper-Laplacian regularization maintains the local geometrical structure that makes the estimation prune to nonlinearities, and the mixed ℓ 2,1 and ℓ 1,2 regularization provides the joint sparsity within-cluster as well as the exclusive sparsity between-cluster. Furthermore, a log-determinant function is used as a tighter tensor rank approximation to discriminate the dimension of features. An efficient alternating algorithm is then derived to optimize the proposed model, and the construction of a convergent sequence to the Karush-Kuhn-Tucker (KKT) critical point solution is mathematically validated in detail. Extensive experiments are conducted on ten well-known datasets to demonstrate that the proposed approach outperforms the existing state-of-the-art approaches with various scenarios, in which, six of them achieve perfect results under our framework developed in this article, demonstrating highl effectiveness for the proposed approach.

Low Rank Tensor Decompositions and Approximations

AbstractThere exist linear relations among tensor entries of low rank tensors. These linear relations can be expressed by multi-linear polynomials, which are called generating polynomials. We use generating polynomials to compute tensor rank decompositions and low rank tensor approximations. We prove that this gives a quasi-optimal low rank tensor approximation if the given tensor is sufficiently close to a low rank one.

Enhanced tensor low-rank representation learning for multi-view clustering

Multi-view subspace clustering (MSC), assuming the multi-view data are generated from a latent subspace, has attracted considerable attention in multi-view clustering. To recover the underlying subspace structure, a successful approach adopted recently is subspace clustering based on tensor nuclear norm (TNN). But there are some limitations to this approach that the existing TNN-based methods usually fail to exploit the intrinsic cluster structure and high-order correlations well, which leads to limited clustering performance. To address this problem, the main purpose of this paper is to propose a novel tensor low-rank representation (TLRR) learning method to perform multi-view clustering. First, we construct a 3rd-order tensor by organizing the features from all views, and then use the t-product in the tensor space to obtain the self-representation tensor of the tensorial data. Second, we use the ℓ1,2 norm to constrain the self-representation tensor to make it capture the class-specificity distribution, that is important for depicting the intrinsic cluster structure. And simultaneously, we rotate the self-representation tensor, and use the tensor singular value decomposition-based weighted TNN as a tighter tensor rank approximation to constrain the rotated tensor. For the challenged mathematical optimization problem, we present an effective optimization algorithm with a theoretical convergence guarantee and relatively low computation complexity. The constructed convergent sequence to the Karush–Kuhn–Tucker (KKT) critical point solution is mathematically validated in detail. We perform extensive experiments on four datasets and demonstrate that TLRR outperforms state-of-the-art multi-view subspace clustering methods.

Nonconvex multi-view subspace clustering via simultaneously learning the representation tensor and affinity matrix* *This research was supported by the National Natural Science Foundations of China (12071159, U1811464) and the NSF-DMS 1854638 of the United States.

Multi-view subspace clustering, which aims to partition a dataset into its relevant subspaces based on their multi-view features, has been widely applied to identify various characteristics of datasets. The typical model of multi-view subspace clustering in literature often makes use of the nuclear norm to seek the underlying low-rank representation. However, due to the sum property of the singular values defined by tensor nuclear norm, the existing multi-view subspace clustering does not well handle the noise and the illumination variations embedded in multi-view data. To address and improve the robustness and clustering performance, we propose a new nonconvex multi-view subspace clustering model via tensor minimax concave penalty (MCP) approximation associated with rank minimization (NMSC-MCP), which can simultaneously construct the low-rank representation tensor and affinity matrix in a unified framework. Specifically, the nonconvex MCP approximation rank function is adopted to as a tighter tensor rank approximation to discriminate the dimension of features so that better accuracy can be achieved. In addition, we also address the local structure by including both hyper-Laplacian regularization and auto-weighting scheme into the objective function to promote the clustering performance. A corresponding iterative algorithm is then developed to solve the proposed model and the constructed iterative sequence generated by the proposed algorithm is shown to converge to the desirable KKT critical point. Extensive experiments on benchmark datasets have demonstrate the highly desirable effectiveness of our proposed method.

TTV Regularized LRTA Technique for the Estimation of Haze Model Parameters in Video Dehazing

Nowadays, intelligent transport systems have a major role in providing a safe and secure traffic society for passengers, pedestrians, and vehicles. However, some bad weather conditions such as haze or fog may affect the visual clarity of video footage captured by the camera. This will cause a malfunction in further video processing algorithms performed by such automated systems. This article proposes an efficient technique for estimating the atmospheric light and the transmission map in the haze model entirely in tensor domain for video dehazing. In this work, the atmospheric light is appraised using the Mie scattering principle of visible light and the temporal coherency among the frames is achieved by means of tensor algebra. Furthermore, the transmission map is computed using Low Rank Tensor Approximation (LRTA) based on Weighted Tensor Nuclear Norm (WTNN) minimization and Tensor Total Variation (TTV) regularization. WTNN minimization is used to smooth the coarse transmission map, and TTV regularization is employed to maintain spatio-temporal continuity by preserving the details of salient structures and edges. The novelty of the proposed model is confined in the efficient formulation of a unified optimization model for the estimation of transmission map and atmospheric light in the tensor domain with fine-tuned regularization terms, which is not reported till now in the direction of video dehazing. Extensive experiments show that the proposed method outperforms state-of-the-art methods in video dehazing.

An orthogonal equivalence theorem for third order tensors

<p style='text-indent:20px;'>In 2011, Kilmer and Martin proposed tensor singular value decomposition (T-SVD) for third order tensors. Since then, T-SVD has applications in low rank tensor approximation, tensor recovery, multi-view clustering, multi-view feature extraction, tensor sketching, etc. By going through the Discrete Fourier Transform (DFT), matrix SVD and inverse DFT, a third order tensor is mapped to an f-diagonal third order tensor. We call this a Kilmer-Martin mapping. We show that the Kilmer-Martin mapping of a third order tensor is invariant if that third order tensor is taking T-product with some orthogonal tensors. We define singular values and T-rank of that third order tensor based upon its Kilmer-Martin mapping. Thus, tensor tubal rank, T-rank, singular values and T-singular values of a third order tensor are invariant when it is taking T-product with some orthogonal tensors. Some properties of singular values, T-rank and best T-rank one approximation are discussed.</p>

Infrared dim target detection via mode-k1k2 extension tensor tubal rank under complex ocean environment

Infrared dim target detection under complex ocean environment plays a key role in military and civilian fields. Many state-of-the-art methods have disadvantages such as low generalization ability, poor robustness to noise and stubborn edges, high time complexity, and the existence of background residuals or target defects in detection results. To further solve these shortcomings, based on the infrared patch tensor (IPT) model, a robust infrared dim target detection algorithm is proposed in this paper, which converts the target detection task into a convex optimization problem. Aiming at the current situation that the tensor rank approximation in the IPT model has not been well resolved, a new vector form of tensor rank named mode-k1k2 extension tensor tubal rank (METTR) is defined, the elements of which include the tubal ranks of all tensors expanded via mode-k1k2 extension. Through the mode-k1k2 extension of the tensor, the hidden information among the different modes of the tensor is better mined. To minimize the METTR efficiently, we propose its convex approximation norm METTR, and establish a tensor robust principal component analysis (TRPCA) model joint l1 norm. Then we use the alternating direction multiplier method (ADMM) and set the optimal parameters to solve the proposed model. A series of experimental results show that the proposed algorithm outperforms the baselines in terms of background suppression and target detection.

Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

The low multilinear rank approximation, also known as the truncated Tucker decomposition, has been extensively utilized in many applications that involve higher-order tensors. Popular methods for low multilinear rank approximation usually rely directly on matrix SVD, therefore often suffer from the notorious intermediate data explosion issue and are not easy to parallelize, especially when the input tensor is large. In this paper, we propose a new class of truncated HOSVD algorithms based on alternating least squares (ALS) for efficiently computing the low multilinear rank approximation of tensors. The proposed ALS-based approaches are able to eliminate the redundant computations of the singular vectors of intermediate matrices and are therefore free of data explosion. Also, the new methods are more flexible with adjustable convergence tolerance and are intrinsically parallelizable on high-performance computers. Theoretical analysis reveals that the ALS iteration in the proposed algorithms is q-linear convergent with a relatively wide convergence region. Numerical experiments with large-scale tensors from both synthetic and real-world applications demonstrate that ALS-based methods can substantially reduce the total cost of the original ones and are highly scalable for parallel computing.

Randomized algorithms for the low multilinear rank approximations of tensors

In this paper, we focus on developing randomized algorithms for the computation of low multilinear rank approximations of tensors based on the random projection and the singular value decomposition. Following the theory of the singular values of sub-Gaussian matrices, we make a probabilistic analysis for the error bounds for the randomized algorithm. We demonstrate the effectiveness of proposed algorithms via several numerical examples.

A tensor rank theory and maximum full rank subtensors

A matrix always has a full rank submatrix such that the rank of this matrix is equal to the rank of that submatrix. This property is one of the corner stones of the matrix rank theory. We call this property the max-full-rank-submatrix property. Tensor ranks play a crucial role in low rank tensor approximation, tensor completion and tensor recovery. However, their theory is still not matured yet. Can we set an axiom system for tensor ranks? Can we extend the max-full-rank-submatrix property to tensors? We explore these in this paper. We first propose some axioms for tensor rank functions. Then we introduce proper tensor rank functions. The CP rank is a tensor rank function, but is not proper. There are two proper tensor rank functions, the max-Tucker rank and the submax-Tucker rank, which are associated with the Tucker decomposition. We define a partial order among tensor rank functions and show that there exists a unique smallest tensor rank function. We introduce the full rank tensor concept, and define the max-full-rank-subtensor property. We show the max-Tucker tensor rank function and the smallest tensor rank function have this property. We define the closure for an arbitrary proper tensor rank function, and show that it is still a proper tensor rank function and has the max-full-rank-subtensor property. An application of the submax-Tucker rank is also presented.

Robust brain network identification from multi-subject asynchronous fMRI data

We describe a novel method for robust identification of common brain networks and their corresponding temporal dynamics across subjects from asynchronous functional MRI (fMRI) using tensor decomposition. We first temporally align asynchronous fMRI data using the orthogonal BrainSync transform, allowing us to study common brain networks across sessions and subjects. We then map the synchronized fMRI data into a 3D tensor (vertices × time × subject/session). Finally, we apply Nesterov-accelerated adaptive moment estimation (Nadam) within a scalable and robust sequential Canonical Polyadic (CP) decomposition framework to identify a low rank tensor approximation to the data. As a result of CP tensor decomposition, we successfully identified twelve known brain networks with their corresponding temporal dynamics from 40 subjects using the Human Connectome Project's language task fMRI data without any prior information regarding the specific task designs. Seven of these networks show distinct subjects’ responses to the language task with differing temporal dynamics; two show sub-components of the default mode network that exhibit deactivation during the tasks; the remaining three components reflect non-task-related activities. We compare results to those found using group independent component analysis (ICA) and canonical ICA. Bootstrap analysis demonstrates increased robustness of networks found using the CP tensor approach relative to ICA-based methods.

Innovative Approaches to the Numerical Approximation of PDEs

This workshop was about the numerical solution of PDEs for which classical approaches, such as the finite element method, are not well suited or need further (theoretical) underpinnings. A prominent example of PDEs for which classical methods are not well suited are PDEs posed in high space dimensions. New results on low rank tensor approximation for those problems were presented. Other presentations dealt with regularity of PDEs, the numerical solution of PDEs on surfaces, PDEs of fractional order, numerical solvers for PDEs that converge with exponential rates, and the application of deep neural networks for solving PDEs.

Noise robust image clustering based on reweighted low rank tensor approximation and $$l_{\frac{1}{2}}$$ regularization

In this paper, a noise robust tensor based image clustering approach is proposed which can also perform well in the presence of gross errors. Our major contribution is the improved submodule identification technique in noisy environment by incorporating three important improvements: better low rank representation using reweighted nuclear norm, $$l_\frac{1}{2}$$ regularization to accurately capture sparseness and an error term in the model for noise robustness. Reweighted nuclear norm is introduced in the clustering model to capture self-expressiveness property in a better manner. The $$l_\frac{1}{2}$$ norm regularization is applied in place of $$l_1$$ -norm to properly capture the correlation among data members. An error term is introduced into the model to separate noise and data, which brings a noise robust image clustering technique. Combined effect all the three factors results an accurate clustering method even under the presence of severe noise. The performance of the proposed method is tested on different datasets with varying amount of noise. It is found that the proposed method provides better classification accuracy in almost all conditions as compared with existing methods.

Nonlinear power-like iteration by polar decomposition and its application to tensor approximation

Low rank tensor approximation is an important subject with a wide range of applications. Most prevailing techniques for computing the low rank approximation in the Tucker format often first assemble relevant factors into matrices and then update by turns one factor matrix at a time. In order to improve two factor matrices simultaneously, a special system of nonlinear matrix equations over a certain product Stiefel manifold must be resolved at every update. The solution to the system consists of orbit varieties invariant under the orthogonal group action, which thus imposes challenges on its analysis. This paper proposes a scheme similar to the power method for subspace iterations except that the polar decomposition is used as the normalization process and that the iteration can be applied to both the orbits and the cross-sections. The notion of quotient manifold is employed to factor out the effect of orbital solutions. The dynamics of the iteration is completely characterized. An isometric isomorphism between the tangent spaces of two properly identified Riemannian manifolds is established to lend a hand to the proof of convergence.

Surrogate-assisted global sensitivity analysis: an overview

Surrogate models are popular tool to approximate the functional relationship of expensive simulation models in multiple scientific and engineering disciplines. Successful use of surrogate models can provide significant savings of computational cost. However, with a variety of surrogate model approaches available in literature, it is a difficult task to select an appropriate one at hand. In this paper, we present an overview of surrogate model approaches with an emphasis of their application for variance-based global sensitivity analysis, including polynomial regression model, high-dimensional model representation, state-dependent parameter, polynomial chaos expansion, Kriging/Gaussian Process, support vector regression, radial basis function, and low rank tensor approximation. The accuracy and efficiency of these approaches are compared with several benchmark examples. The strengths and weaknesses of these surrogate models are discussed, and the recommendations are provided for different types of applications. For ease of implementations, the packages, as well as toolboxes, of surrogate model techniques and their applications for global sensitivity analysis are collected.