Rank-one Tensors Research Articles

A Tensor Space for Multi-View and Multitask Learning Based on Einstein and Hadamard Products: A Case Study on Vehicle Traffic Surveillance Systems

Since multi-view learning leverages complementary information from multiple feature sets to improve model performance, a tensor-based data fusion layer for neural networks, called Multi-View Data Tensor Fusion (MV-DTF), is used. It fuses M feature spaces X1,⋯,XM, referred to as views, in a new latent tensor space, S, of order P and dimension J1×⋯×JP, defined in the space of affine mappings composed of a multilinear map T:X1×⋯×XM→S—represented as the Einstein product between a (P+M)-order tensor A anda rank-one tensor, X=x(1)⊗⋯⊗x(M), where x(m)∈Xm is the m-th view—and a translation. Unfortunately, as the number of views increases, the number of parameters that determine the MV-DTF layer grows exponentially, and consequently, so does its computational complexity. To address this issue, we enforce low-rank constraints on certain subtensors of tensor A using canonical polyadic decomposition, from which M other tensors U(1),⋯,U(M), called here Hadamard factor tensors, are obtained. We found that the Einstein product A⊛MX can be approximated using a sum of R Hadamard products of M Einstein products encoded as U(m)⊛1x(m), where R is related to the decomposition rank of subtensors of A. For this relationship, the lower the rank values, the more computationally efficient the approximation. To the best of our knowledge, this relationship has not previously been reported in the literature. As a case study, we present a multitask model of vehicle traffic surveillance for occlusion detection and vehicle-size classification tasks, with a low-rank MV-DTF layer, achieving up to 92.81% and 95.10% in the normalized weighted Matthews correlation coefficient metric in individual tasks, representing a significant 6% and 7% improvement compared to the single-task single-view models.

Parsimonious Tensor Dimension Reduction

Tensor data is emerging in many scientific applications, such as multi-tissue transcriptomics. In such cases, the covariates for each individual are no longer a vector. To apply traditional vector-based methods to this type of data, we need to either do the vectorization or analyze data marginally, which suffers a significant information loss. We propose a novel parsimonious tensor dimension reduction (pTDR) approach to directly link the response and tensor covariate through an unknown function g. In pTDR, the response variable, continuous or discrete, depends on K rank-one projections of the covariates, with the projections estimated via a sequential iterative dimension reduction algorithm. We further propose an asymptotic sequential statistical test to select the correct number of rank-one tensors. In contrast to the classic low-rank tensor regression, pTDR model is not restricted to the linear relationship between response and covariates. We apply pTDR to two modern genomic studies. We find that the gene expression of multiple tissues has a stronger association with aging and obesity than was apparent using previous approaches. Numerical results demonstrate the advantages of pTDR over competitors in terms of prediction accuracy and computing efficiency. Our software is publicly available on GitHub (https://github.com/BioAlgs/pTDR).

Rank-One Boolean Tensor Factorization and the Multilinear Polytope

We consider the NP-hard problem of finding the closest rank-one binary tensor to a given binary tensor, which we refer to as the rank-one Boolean tensor factorization (BTF) problem. This optimization problem can be used to recover a planted rank-one tensor from noisy observations. We formulate rank-one BTF as the problem of minimizing a linear function over a highly structured multilinear set. Leveraging on our prior results regarding the facial structure of multilinear polytopes, we propose novel linear programming relaxations for rank-one BTF. We then establish deterministic sufficient conditions under which our proposed linear programs recover a planted rank-one tensor. To analyze the effectiveness of these deterministic conditions, we consider a semirandom model for the noisy tensor and obtain high probability recovery guarantees for the linear programs. Our theoretical results as well as numerical simulations indicate that certain facets of the multilinear polytope significantly improve the recovery properties of linear programming relaxations for rank-one BTF. Funding: A. Del Pia is partially funded by the Air Force Office of Scientific Research [Grant FA9550-23-1-0433]. A. Khajavirad is partially funded by the Air Force Office of Scientific Research [Grant FA9550-23-1-0123].

TENSOR AND VECTOR APPROACHES TO OBJECTS RECOGNITION BY INVERSE FEATURE FILTERS

The investigation of the extraction of image objects features by filters based on tensor and vector data presentation is considered. The tensor data is obtained as a sum of rank-one tensors, given by the tensor product of the vector of lexicographic representation of image fragments pixels with itself. The accumulated tensor is approximated by one rank tensor obtained using singular values decomposition. It has been shown that the main vector of the decomposition can be considered as the object feature vector. The vector data is obtained by accumulating analogous vectors of image fragments pixels. The accumulated vector is also considered as an object feature. The filter banks of a set of objects are obtained by regularized inversion of the matrices compiled by object features vectors. Optimized regularization of the inversion is used to expand the regions of object features capture with minimal error. The object fragments and corresponding feature vectors are selected through a training iterative process. The tensor and vector approaches create two channels for recognition. High efficiency of object recognition can be achieved by choosing the filter capture band and creating filter branches according to the given bands. The filters create a convolutional network to recognize a set of objects. It has been shown that the obtained filters have an advantage over known correlation filters when recognizing objects with small fragments.

Parallel Factorization to Implement Group Analysis in Brain Networks Estimation.

When dealing with complex functional brain networks, group analysis still represents an open issue. In this paper, we investigated the potential of an innovative approach based on PARAllel FActorization (PARAFAC) for the extraction of the grand average connectivity matrices from both simulated and real datasets. The PARAFAC approach was solved using three different numbers of rank-one tensors (PAR-FACT). Synthetic data were parametrized according to different levels of three parameters: network dimension (NODES), number of observations (SAMPLE-SIZE), and noise (SWAP-CON) in order to investigate the way they affect the grand average estimation. PARAFAC was then tested on a real connectivity dataset, derived from EEG data of 17 healthy subjects performing wrist extension with left and right hand separately. Findings on both synthetic and real data revealed the potential of the PARAFAC algorithm as a useful tool for grand average extraction. As expected, the best performances in terms of FPR, FNR, and AUC were achieved for great values of sample size and low noise level. A crucial role has been revealed for the PAR-FACT parameter, revealing that an increase in the number of rank-one tensors solving the PARAFAC problem leads to an increase in FPR values and, thus, to a worse grand average estimation.

Reconfigurable Intelligent Surface-Aided 6G Massive Access: Coupled Tensor Modeling and Sparse Bayesian Learning

This paper investigates a reconfigurable intelligent surface (RIS)-aided unsourced random access (URA) scheme for the sixth-generation (6G) wireless networks with massive sporadic traffic devices. First of all, this paper proposes a novel joint active device separation (the message recovery of active device) and channel estimation architecture for the RIS-aided URA. Specifically, the RIS passive reflection is optimized before the successful device separation. Then, by associating the data sequences to multiple rank-one tensors and exploiting the angular sparsity of the RIS-BS channel, the detection problem is cast as a high-order coupled tensor decomposition problem without the need of exploiting pilot sequences. However, the inherent coupling among multiple sparse device-RIS channels, together with the unknown number of active devices make the detection problem at hand deviate from the widely-used coupled tensor decomposition format. To overcome this challenge, this paper judiciously devises a probabilistic model that captures both the element-wise sparsity from the angular channel model and the low-rank property due to the sporadic nature of URA. Then, based on such a probabilistic model, a iterative detection algorithm is developed under the framework of sparse variational inference, where each update iteration is obtained in a closed-form and the number of active devices can be automatically estimated for effectively avoiding the overfitting of noise. Extensive simulation results confirm the excellence of the proposed URA algorithm, especially for the case of a large number of reflecting elements for accommodating a significantly large number of devices.

Rank Properties and Computational Methods for Orthogonal Tensor Decompositions

The orthogonal decomposition factorizes a tensor into a sum of an orthogonal list of rank-one tensors. The corresponding rank is called orthogonal rank. We present several properties of orthogonal rank, which are different from those of tensor rank in many aspects. For instance, a subtensor may have a larger orthogonal rank than the whole tensor. To fit the orthogonal decomposition, we propose an algorithm based on the augmented Lagrangian method. The gradient of the objective function has a nice structure, inspiring us to use gradient-based optimization methods to solve it. We guarantee the orthogonality by a novel orthogonalization process. Numerical experiments show that the proposed method has a great advantage over the existing methods for strongly orthogonal decompositions in terms of the approximation error.

Separable Symmetric Tensors and Separable Anti-symmetric Tensors

In this paper, we first initialize the S-product of tensors to unify the outer product, contractive product, and the inner product of tensors. Then, we introduce the separable symmetry tensors and separable anti-symmetry tensors, which are defined, respectively, as the sum and the algebraic sum of rank-one tensors generated by the tensor product of some vectors. We offer a class of tensors to achieve the upper bound for \(\texttt {rank}({\mathcal {A}}) \leqslant 6\) for all tensors of size \(3\times 3\times 3\). We also show that each \(3\times 3\times 3\) anti-symmetric tensor is separable.

Maximum relative distance between real rank-two and rank-one tensors

It is shown that the relative distance in Frobenius norm of a real symmetric order-d tensor of rank-two to its best rank-one approximation is upper bounded by sqrt{1-(1-1/d)^{d-1}}. This is achieved by determining the minimal possible ratio between spectral and Frobenius norm for symmetric tensors of border rank two, which equals left( 1-{1}/{d}right) ^{(d-1)/{2}}. These bounds are also verified for arbitrary real rank-two tensors by reducing to the symmetric case.

Efficient and interpretable monitoring of high-dimensional categorical processes

High-Dimensional (HD) processes have become prevalent in many data-intensive scientific domains and engineering applications. The monitoring of HD categorical data, where each variable of interest is evaluated by attribute levels or nominal values, however, has seldom been studied. As the joint distribution of HD categorical variables can be fully characterized by a high-way contingency table or a high-order tensor, we propose a Probabilistic Tensor Decomposition (PTD) which factorizes a huge tensor into a few latent classes (rank-one tensors) to dramatically reduce the number of model parameters. Moreover, to enable high interpretability of this latent-class-type PTD model, a novel polarization regularization is devised, which makes each latent class focus on only a few vital combinations of attribute levels of categorical variables. An Expectation-Maximization algorithm is designed for parameter estimation from a historical normal dataset in Phase I, and an exponentially weighted moving average control chart is built in Phase II to monitor the proportions of latent classes that act as surrogates for each original categorical vector. Extensive simulations and a real case study validate the superior inference and monitoring performance of our proposed efficient and interpretable method.

Random Tensor Theory for Tensor Decomposition

We propose a new framework for tensor decomposition based on trace invariants, which are particular cases of tensor networks. In general, tensor networks are diagrams/graphs that specify a way to "multiply" a collection of tensors together to produce another tensor, matrix or scalar. The particularity of trace invariants is that the operation of multiplying copies of a certain input tensor that produces a scalar obeys specific symmetry constraints. In other words, the scalar resulting from this multiplication is invariant under some specific transformations of the involved tensor. We focus our study on the O(N)-invariant graphs, i.e. invariant under orthogonal transformations of the input tensor. The proposed approach is novel and versatile since it allows to address different theoretical and practical aspects of both CANDECOMP/PARAFAC (CP) and Tucker decomposition models. In particular we obtain several results: (i) we generalize the computational limit of Tensor PCA (a rank-one tensor decomposition) to the case of a tensor with axes of different dimensions (ii) we introduce new algorithms for both decomposition models (iii) we obtain theoretical guarantees for these algorithms and (iv) we show improvements with respect to state of the art on synthetic and real data which also highlights a promising potential for practical applications.

Local and Global Convergence of General Burer-Monteiro Tensor Optimizations

Tensor optimization is crucial to massive machine learning and signal processing tasks. In this paper, we consider tensor optimization with a convex and well-conditioned objective function and reformulate it into a nonconvex optimization using the Burer-Monteiro type parameterization. We analyze the local convergence of applying vanilla gradient descent to the factored formulation and establish a local regularity condition under mild assumptions. We also provide a linear convergence analysis of the gradient descent algorithm started in a neighborhood of the true tensor factors. Complementary to the local analysis, this work also characterizes the global geometry of the best rank-one tensor approximation problem and demonstrates that for orthogonally decomposable tensors the problem has no spurious local minima and all saddle points are strict except for the one at zero which is a third-order saddle point.

Minimality of tensors of fixed multilinear rank

We discover a geometric property of the space of tensors of fixed multilinear (Tucker) rank. Namely, it is shown that real tensors of the fixed multilinear rank form a minimal submanifold of the Euclidean space of tensors endowed with the Frobenius inner product. We also establish the absence of local extrema for linear functionals restricted to the submanifold of rank-one tensors, finding application in statistics.

Inference for low-rank tensors—no need to debias

In this paper, we consider the statistical inference for several low-rank tensor models. Specifically, in the Tucker low-rank tensor PCA or regression model, provided with any estimates achieving some attainable error rate, we develop the data-driven confidence regions for the singular subspace of the parameter tensor based on the asymptotic distribution of an updated estimate by two-iteration alternating minimization. The asymptotic distributions are established under some essential conditions on the signal-to-noise ratio (in PCA model) or sample size (in regression model). If the parameter tensor is further orthogonally decomposable, we develop the methods and nonasymptotic theory for inference on each individual singular vector. For the rank-one tensor PCA model, we establish the asymptotic distribution for general linear forms of principal components and confidence interval for each entry of the parameter tensor. Finally, numerical simulations are presented to corroborate our theoretical discoveries. In all of these models, we observe that different from many matrix/vector settings in existing work, debiasing is not required to establish the asymptotic distribution of estimates or to make statistical inference on low-rank tensors. In fact, due to the widely observed statistical-computational-gap for low-rank tensor estimation, one usually requires stronger conditions than the statistical (or information-theoretic) limit to ensure the computationally feasible estimation is achievable. Surprisingly, such conditions “incidentally” render a feasible low-rank tensor inference without debiasing.

Kronecker Product-Based Space-Time Block Codes

This letter presents a new approach to construct space-time block codes (STBCs) based on the Kronecker product. The proposed Kronecker-structured STBCs (K-STBCs) are specially designed for constant modulus constellations and include the classical linear dispersion (LD) codes as a special case. Its inherent Kronecker structure offers additional coding gains due to the mutual spreading of multiple space-time codewords. At the receiver, an efficient algorithm is derived by recasting the decoding of the K-STBCs as a rank-one tensor approximation problem. The proposed decoding algorithm provides a good bit error ratio (BER) performance especially, in the low signal-to-noise ratio (SNR) regime. In particular, our numerical results show that K-STBCs outperform the existing quasi-orthogonal STBCs due to the denoising gain offered by the proposed rank-one approximation based detector.

NeuLFT: A Novel Approach to Nonlinear Canonical Polyadic Decomposition on High-Dimensional Incomplete Tensors

A High-Dimensional and Incomplete (HDI) tensor is frequently encountered in a big data-related application concerning the complex dynamic interactions among numerous entities. Traditional tensor factorization-based models cannot handle an HDI tensor efficiently, while existing latent factorization of tensors models are all linear models unable to model an HDI tensor's nonlinearity. Motivated by this critical discovery, this paper proposes a Neural Latent Factorization of Tensors model, which provides a novel approach to nonlinear Canonical Polyadic decomposition on an HDI tensor. It is implemented with three-fold interesting ideas: a) adopting the density-oriented modeling principle to build rank-one tensor series with high computational efficiency and affordable storage cost; b) treating each rank-one tensor as a hidden neuron to achieve an efficient neural network structure; and c) developing an adaptive backward propagation (ABP) learning scheme for efficient model training. Experimental results on six HDI tensors from a real system demonstrate that compared with state-of-the-art models, the proposed model achieves significant performance gain in both convergence rate and accuracy. Hence, it is of great significance in performing challenging HDI tensor analysis.

One‐class tensor machine with randomized projection for large‐scale anomaly detection in high‐dimensional and noisy data

The modern industrial sector generates enormous amounts of high-dimensional heterogeneous data daily. However, mostly the vectored data (rank-one tensor) have been considered for anomaly detection, whereas the data in real-life is high dimensional. The expressive power of methods based on vector data is restrictive as they may destroy the structural information embedded in data and lead to the curse-of-dimensionality and overfitting. In this paper, we present a novel anomaly detection approach for large-scale tensor data. We first present novel one-class support tensor machines (OCSTM) with bounded loss function. We further extend it by leveraging the randomness to design a scalable approach that can also be used for large-scale anomaly detection. To solve the corresponding optimization of the objective function, we utilize half-quadratic optimization followed by solving it like a traditional OCSTM optimization at each iteration. We demonstrate the proposed randomized OCSTM with bounded hinge loss through experiments on 14 benchmark data sets. Experimental results demonstrate the effectiveness of the proposed approach against anomalies and a significant reduction in the computational complexity.

On the Realization of Hidden Markov Models and Tensor Decomposition

The minimum realization problem of hidden Markov models (HMM’s) is a fundamental question of stationary discrete-time processes with a finite alphabet. It was shown in the literature that tensor decomposition methods give the hidden Markov model with the minimum number of states generically. However, the tensor decomposition approach does not solve the minimum HMM realization problem when the observation is a deterministic function of the state, which is an important class of HMM’s not captured by a generic argument. In this paper, we show that the reduction of the number of rank-one tensors necessary to decompose the third-order tensor constructed from the probabilities of the process is possible when the reachable subspace is not the whole space or the null space is not the zero space. In fact, the rank of the tensor is not greater than the dimension of the effective subspace or the rank of the generalized Hankel matrix.

Polyhedral faces in Gram spectrahedra of binary forms

The positive semidefinite Gram matrices of a form f with real coefficients parametrize the sum-of-squares representations of f. The convex body formed by the entirety of these matrices is the so-called Gram spectrahedron of f. We analyze the facial structures of symmetric and Hermitian Gram spectrahedra in the case of binary forms. We give upper bounds for the dimensions of polyhedral faces in Hermitian Gram spectrahedra and show that, if the form f is sufficiently generic, they can be realized by faces that are simplices and whose extreme points are rank-one tensors. We use our construction to prove a similar statement for the real symmetric case.

CP decomposition and weighted clique problem

The determination of the optimal subdecomposition based on CP decomposition for high-order tensors is not straightforward because the rank-one tensors are not orthogonal. We show that this is equivalent to the weighted clique problem. Thus, it is NP-hard.