Low-rank Tensor Model Research Articles

A novel weighted-guided tensor completion missing data imputation method for health monitoring data of planar parallel mechanism

Abstract High-end mechanical equipment plays a crucial role in the manufacturing industry, making the monitoring of its operational status highly significant. Due to various factors such as environmental influences, the absence of monitoring signals in mechanical equipment status is a common issue, leading to a decline in data quality. To ensure data quality, this paper proposes an adaptive weighted low-rank tensor missing data imputation method. Firstly, based on the motion characteristics of a planar parallel mechanism (PPM), a new low-rank tensor model is established using periodicity, sliding windows, and time series. Secondly, the tensor truncation nuclear norm is defined, and a novel rate parameter is introduced to control the truncation degree of all tensor modes, thereby obtaining weights for each dimension. Finally, within the framework of the alternating direction method of multipliers (ADMM), adaptive weights for each dimension are obtained during each iteration, completing the filling of missing data. Two types of missing patterns are studied on a PPM experimental platform, and the results showed that as the missing rate increased, the mean absolute percentage error is less than 0.018, and root mean square error is less than 0.001, and the rate of change is less than 0.08, which was significantly better than other compared algorithms.

Accelerated 3D metabolite T1 mapping of the brain using variable-flip-angle SPICE.

To develop a practical method to enable 3D T1 mapping of brain metabolites. Due to the high dimensionality of the imaging problem underlying metabolite T1 mapping, measurement of metabolite T1 values has been currently limited to a single voxel or slice. This work achieved 3D metabolite T1 mapping by leveraging a recent ultrafast MRSI technique called SPICE (spectroscopic imaging by exploiting spatiospectral correlation). The Ernst-angle FID MRSI data acquisition used in SPICE was extended to variable flip angles, with variable-density sparse sampling for efficient encoding of metabolite T1 information. In data processing, a novel generalized series model was used to remove water and subcutaneous lipid signals; a low-rank tensor model with prelearned subspaces was used to reconstruct the variable-flip-angle metabolite signals jointly from the noisy data. The proposed method was evaluated using both phantom and healthy subject data. Phantom experimental results demonstrated that high-quality 3D metabolite T1 maps could be obtained and used for correction of T1 saturation effects. In vivo experimental results showed metabolite T1 maps with a large spatial coverage of 240 × 240 × 72 mm3 and good reproducibility coefficients (< 11%) in a 14.5-min scan. The metabolite T1 times obtained ranged from 0.99 to 1.44 s in gray matter and from 1.00 to 1.35 s in white matter. We successfully demonstrated the feasibility of 3D metabolite T1 mapping within a clinically acceptable scan time. The proposed method may prove useful for both T1 mapping of brain metabolites and correcting the T1-weighting effects in quantitative metabolic imaging.

Hyperspectral Image Mixed Noise Removal via Double Factor Total Variation Nonlocal Low-Rank Tensor Regularization

A hyperspectral image (HSI) is often corrupted by various types of noise during image acquisition, e.g., Gaussian noise, impulse noise, stripes, deadlines, and more. Thus, as a preprocessing step, HSI denoising plays a vital role in many subsequent tasks. Recently, a variety of mixed noise removal approaches have been developed for HSI, and the methods based on spatial–spectral double factor and total variation (DFTV) regularization have achieved comparable performance. Additionally, the nonlocal low-rank tensor model (NLR) is often employed to characterize spatial nonlocal self-similarity (NSS). Generally, fully exploring prior knowledge can improve the denoising performance, but it significantly increases the computational cost when the NSS prior is employed. To solve this problem, this article proposes a novel DFTV-based NLR regularization (DFTVNLR) model for HSI mixed noise removal. The proposed model employs low-rank tensor factorization (LRTF) to characterize the spectral global low-rankness (LR), introduces 2-D and 1-D TV constraints on double-factor to characterize the spatial and spectral local smoothness (LS), respectively. Meanwhile, the NLR is applied to the spatial factor to characterize the NSS. Then, we developed an algorithm based on proximal alternating minimization (PAM) to solve the proposed model effectively. Particularly, we effectively controlled the computational cost from two aspects, namely taking small-sized double factor as regularization object and putting the time-consuming NLR model before the main loop with fewer iterations to solve it independently. Finally, considerable experiments on simulated and real noisy HSI substantiate that the proposed method is superior to the related state-of-the-art methods in balancing the denoising effect and speed.

A Novel Tensor Ring Sparsity Measurement for Image Completion.

As a promising data analysis technique, sparse modeling has gained widespread traction in the field of image processing, particularly for image recovery. The matrix rank, served as a measure of data sparsity, quantifies the sparsity within the Kronecker basis representation of a given piece of data in the matrix format. Nevertheless, in practical scenarios, much of the data are intrinsically multi-dimensional, and thus, using a matrix format for data representation will inevitably yield sub-optimal outcomes. Tensor decomposition (TD), as a high-order generalization of matrix decomposition, has been widely used to analyze multi-dimensional data. In a direct generalization to the matrix rank, low-rank tensor modeling has been developed for multi-dimensional data analysis and achieved great success. Despite its efficacy, the connection between TD rank and the sparsity of the tensor data is not direct. In this work, we introduce a novel tensor ring sparsity measurement (TRSM) for measuring the sparsity of the tensor. This metric relies on the tensor ring (TR) Kronecker basis representation of the tensor, providing a unified interpretation akin to matrix sparsity measurements, wherein the Kronecker basis serves as the foundational representation component. Moreover, TRSM can be efficiently computed by the product of the ranks of the mode-2 unfolded TR-cores. To enhance the practical performance of TRSM, the folded-concave penalty of the minimax concave penalty is introduced as a nonconvex relaxation. Lastly, we extend the TRSM to the tensor completion problem and use the alternating direction method of the multipliers scheme to solve it. Experiments on image and video data completion demonstrate the effectiveness of the proposed method.

High-Fidelity compressive spectral image reconstruction through a novel Non-Convex Non-Local Low-Rank tensor approximation model

Low-quality reconstruction is one main factor that greatly limits the development of snapshot compressive imaging (SCI) systems. The common algorithms usually unfold a 3D hyperspectral image (HSI) into a vector or matrix which inevitably corrupts the intrinsic structure of the HSI and insufficiently discovers its prior information. To fully exploit the underlying structures and internal correlations of the HSIs, this paper proposes a novel non-convex model based on Non-Local low-rank Tensor Approximation with Hyper-Laplacian priori (HL), named HL-NLTA, which effectively combines local and non-local similarity information from fused spectral and spatial perspectives. Specifically, a non-local low-rank tensor model based on the minimax concave plus (MCP) penalty and log sum (LS) penalty separately is developed which can simultaneously exploit two intrinsic properties of HSI, i.e., global correlation along the spectral domain (GCAS) and non-local self-similarity along the spatial domain (NSAS). Furthermore, the Hyper-Laplacian priori regularized through a non-convex ℓp (0 < p < 1) norm can preserve well the spectral and spatial structure. An optimization algorithm based on the alternating direction multiplier method (ADMM) framework is designed and accelerated by the Generalized shrinkage/thresholding (GST) algorithm and the fast Fourier transform (FFT) to solve the new nonconvex model. Extensive simulation results on public datasets and experimental results on our real CASSI system indicate that the proposed method enables high-fidelity HSI reconstructions in terms of simultaneous spatial structure detail recovery and spectral preservation. These results verify the effectiveness of the proposed model for snapshot compressive imaging (e.g., Coded aperture snapshot spectral imaging CASSI).

Low-rank plus sparse joint smoothing model based on tensor singular value decomposition for dynamic MRI reconstruction

Dynamic magnetic resonance imaging (DMRI) is an important medical imaging modality, but the long imaging time limits its practical applications. This paper proposes a low-rank plus sparse joint smoothing model based on tensor singular value decomposition (T-SVD) to reconstruct DMR images from highly under-sampled k-t space data. The low-rank plus sparse tensor (ℒ+S) model decomposes the DMR data into a low-rank and sparse tensor, which naturally fits the dynamic MR images characteristics and exploits the spatiotemporal correlation of DMRI data to improve reconstruction effect. T-SVD is utilized in the ℒ+S model to maintain the intrinsic structure of the low-rank tensor and further enhance the low-rank property. In addition, considering the global multi-dimensional smoothness of the DMR images, the proposed method joint tensor total variation (TTV) constraints to utilize the smoothness of DMR images to obtain more reconstruction details while protecting the global structure. We conducted experiments on the dynamic cardiac datasets, and the experiment results show that the proposed method has superior performance to several state-of-the-art imaging methods.

Accelerating Magnetic Resonance T1ρ Mapping Using Simultaneously Spatial Patch-Based and Parametric Group-Based Low-Rank Tensors (SMART).

Quantitative magnetic resonance (MR) [Formula: see text] mapping is a promising approach for characterizing intrinsic tissue-dependent information. However, long scan time significantly hinders its widespread applications. Recently, low-rank tensor models have been employed and demonstrated exemplary performance in accelerating MR [Formula: see text] mapping. This study proposes a novel method that uses spatial patch-based and parametric group-based low-rank tensors simultaneously (SMART) to reconstruct images from highly undersampled k-space data. The spatial patch-based low-rank tensor exploits the high local and nonlocal redundancies and similarities between the contrast images in [Formula: see text] mapping. The parametric group-based low-rank tensor, which integrates similar exponential behavior of the image signals, is jointly used to enforce multidimensional low-rankness in the reconstruction process. In vivo brain datasets were used to demonstrate the validity of the proposed method. Experimental results demonstrated that the proposed method achieves 11.7-fold and 13.21-fold accelerations in two-dimensional and three-dimensional acquisitions, respectively, with more accurate reconstructed images and maps than several state-of-the-art methods. Prospective reconstruction results further demonstrate the capability of the SMART method in accelerating MR [Formula: see text] imaging.

Asymmetry total variation and framelet regularized nonconvex low-rank tensor completion

The low-rank tensor representation has shown enormous potential and advantages in diverse applications, such as image inpainting, image restoration. However, (1) due to the inherent limitations of the low-rank tensor model, the tensor nuclear norm is utilized yet as a biased estimate of the tensor rank, which limits the image recovery performance. (2) most current low-rank approximation-based methods focus on the preservation of global information, while ignoring the local details preservation such as the spatial piece-wise smoothness and sharp edges. To overcome the above obstacles, we propose a novel asymmetry three-dimensional total variation and framelet regularized nonconvex low-rank tensor completion (ATV-FNTC) model, which integrates the nonconvex penalty function and asymmetric three-dimensional total variation into one unified model. Specifically, ATV-FNTC introduces one nonconvex penalty function as a tighter regularizer to approximate the tensor rank. Different from existing methods, the asymmetric three-dimensional total variation (ATV) regularization is developed to achieve more accurate recovery of detailed information and flexibly control the smoothing strength of different dimensions of tensor data. Furthermore, we design an iterative algorithm to solve the nonconvex ATV-FNTC model by the alternating direction methods of multipliers. Experimental results reveal the superiority of the proposed method compared with several state-of-the-art methods when handling different completion tasks.

A structure noise-aware tensor dictionary learning method for high-dimensional data clustering

With the development of data acquisition technology, high-dimensional data clustering is an important yet challenging task in data mining. Despite advances achieved by current clustering methods, they can be further improved. First, many of them usually unfold the high-dimensional data into a large matrix, consequently resulting in destroying the intrinsic structural property. Second, some methods assume that the noise in the dataset conforms to a predefined distribution (e.g., the Gaussian or Laplacian distribution), which violates real-world applications and eventually decreases the clustering performance. To address these issues, in this paper, we propose a novel tensor dictionary learning method for clustering high-dimensional data with the coexistence of structure noise. We adopt tensors, the natural and powerful tools for the generalizations of vectors and matrices, to characterize high-dimensional data. Meanwhile, to depict the noise accurately, we decompose the observed data into clean data, structure noise, and Gaussian noise. Furthermore, we use low-rank tensor modeling to characterize the inherent correlations of clean data and adopt tensor dictionary learning to adaptively and accurately describe the structure noise instead of using the predefined distribution. We design the proximal alternating minimization algorithm to solve the proposed model with the theoretical convergence guarantee. Experimental results on both simulated and real datasets show that the proposed method outperforms the compared methods for high-dimensional data clustering.

Hyperspectral image denoising by low-rank models with hyper-Laplacian total variation prior

The total variation (TV) regularized low-rank models have emerged as a powerful tool for hyperspectral image (HSI) denoising. TV, defined by the ℓ1-norm of gradients, is assumed that gradients obey the Laplacian distribution from the statistics point of view. By investigating the histogram of HSI’s gradients, we find that gradients in real HSIs are actually distributed as the hyper-Laplacian distribution with the power parameter q=1/2. Taking this prior into account, a hyper-Laplacian spectral-spatial total variation (HTV), defined by the ℓ1/2-norm of gradients, is proposed for HSI denoising. Furthermore, by incorporating HTV as the regularizer, a low-rank matrix model and a low-rank tensor model are proposed. The two models can be solved by the augmented Lagrange multiplier algorithm. To validate the effectiveness of HTV, we formulate baseline models by replacing HTV with ℓ1-norm and ℓ0-norm based TV regularizations, and it is revealed that our proposed HTV outperforms them. Furthermore, compared with several popular HSI denoising algorithms, the experiments conducted on both the simulated and real data demonstrate the superiority of proposed models.

Active Sampling for Accelerated MRI with Low-Rank Tensors.

Magnetic resonance imaging (MRI) is a powerful imaging modality that revolutionizes medicine and biology. The imaging speed of high -dimensional MRI is often limited, which constrains its practical utility. Recently, low-rank tensor models have been exploited to enable fast MR imaging with sparse sampling. Most existing methods use some pre-defined sampling design, and active sensing has not been explored for low-rank tensor imaging. In this paper, we introduce an active low-rank tensor model for fast MR imaging. We propose an active sampling method based on a Query-by-Committee model, making use of the benefits of low-rank tensor structure. Numerical experiments on a 3-D MRI data set with Cartesian sampling designs demonstrate the effectiveness of the proposed method.

Inference for low-rank tensors—no need to debias

In this paper, we consider the statistical inference for several low-rank tensor models. Specifically, in the Tucker low-rank tensor PCA or regression model, provided with any estimates achieving some attainable error rate, we develop the data-driven confidence regions for the singular subspace of the parameter tensor based on the asymptotic distribution of an updated estimate by two-iteration alternating minimization. The asymptotic distributions are established under some essential conditions on the signal-to-noise ratio (in PCA model) or sample size (in regression model). If the parameter tensor is further orthogonally decomposable, we develop the methods and nonasymptotic theory for inference on each individual singular vector. For the rank-one tensor PCA model, we establish the asymptotic distribution for general linear forms of principal components and confidence interval for each entry of the parameter tensor. Finally, numerical simulations are presented to corroborate our theoretical discoveries. In all of these models, we observe that different from many matrix/vector settings in existing work, debiasing is not required to establish the asymptotic distribution of estimates or to make statistical inference on low-rank tensors. In fact, due to the widely observed statistical-computational-gap for low-rank tensor estimation, one usually requires stronger conditions than the statistical (or information-theoretic) limit to ensure the computationally feasible estimation is achievable. Surprisingly, such conditions “incidentally” render a feasible low-rank tensor inference without debiasing.

Low-Rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation

Learning generative probabilistic models is a core problem in machine learning, which presents significant challenges due to the curse of dimensionality. This paper proposes a joint dimensionality reduction and non-parametric density estimation framework, using a novel estimator that can explicitly capture the underlying distribution of appropriate reduced-dimension representations of the input data. The idea is to jointly design a nonlinear dimensionality reducing auto-encoder to model the training data in terms of a parsimonious set of latent random variables, and learn a canonical low-rank tensor model of the joint distribution of the latent variables in the Fourier domain. The proposed latent density model is non-parametric and universal, as opposed to the predefined prior that is assumed in variational auto-encoders. Joint optimization of the auto-encoder and the latent density estimator is pursued via a formulation which learns both by minimizing a combination of the negative log-likelihood in the latent domain and the auto-encoder reconstruction loss. We demonstrate that the proposed model achieves very promising results on toy, tabular, and image datasets on regression tasks, sampling, and anomaly detection.

Likelihood equations and scattering amplitudes

We relate scattering amplitudes in particle physics to maximum likelihood estimation for discrete models in algebraic statistics. The scattering potential plays the role of the log-likelihood function, and its critical points are solutions to rational function equations. We study the ML degree of low-rank tensor models in statistics, and we revisit physical theories proposed by Arkani-Hamed, Cachazo and their collaborators. Recent advances in numerical algebraic geometry are employed to compute and certify critical points. We also discuss positive models and how to compute their string amplitudes.

Information-Theoretic Feature Selection via Tensor Decomposition and Submodularity

Feature selection by maximizing high-order mutual information between the selected feature vector and a target variable is the gold standard in terms of selecting the best subset of relevant features that maximizes the performance of prediction models. However, such an approach typically requires knowledge of the multivariate probability distribution of all features and the target, and involves a challenging combinatorial optimization problem. Recent work has shown that any joint Probability Mass Function (PMF) can be represented as a naive Bayes model, via Canonical Polyadic (tensor rank) Decomposition. In this paper, we introduce a low-rank tensor model of the joint PMF of all variables and indirect targeting as a way of mitigating complexity and maximizing the classification performance for a given number of features. Through low-rank modeling of the joint PMF, it is possible to circumvent the curse of dimensionality by learning principal components of the joint distribution. By indirectly aiming to predict the latent variable of the naive Bayes model instead of the original target variable, it is possible to formulate the feature selection problem as maximization of a monotone submodular function subject to a cardinality constraint - which can be tackled using a greedy algorithm that comes with performance guarantees. Numerical experiments with several standard datasets suggest that the proposed approach compares favorably to the state-of-art for this important problem.

Hyperspectral and Multispectral Image Fusion via Nonlocal Low-Rank Tensor Decomposition and Spectral Unmixing

Hyperspectral (HS) imaging has shown its superiority in many real applications. However, it is usually difficult to obtain high-resolution (HR) HS images through existing imaging techniques due to the hardware limitations. To improve the spatial resolution of HS images, this article proposes an effective HS-multispectral (HS-MS) image fusion method by combining the ideas of nonlocal low-rank tensor modeling and spectral unmixing. To be more precise, instead of unfolding the HS image into a matrix as done in the literature, we directly represent it as a tensor, then a designed nonlocal Tucker decomposition is used to model its underlying spatial–spectral correlation and the spatial self-similarity. The MS image serves mainly as a data constraint to maintain spatial consistency. To further reduce the spectral distortions in spatial enhancement, endmembers, and abundances from the spectral are used for spectral regularization. An efficient algorithm based on the alternating direction method of multipliers (ADMM) is developed to solve the resulting model. Extensive experiments on four HS image data sets demonstrate the superiority of the proposed method over several state-of-the-art HS-MS image fusion methods.

Hyperspectral Image Restoration: Where Does the Low-Rank Property Exist

Hyperspectral image (HSI) restoration is to recover the clean image from degraded version, such as the noisy, blurred, or damaged. Recent low-rank tensor-based recovery methods have been widely explored in HSIs restoration. Most of previous methods, however, neglect an inconspicuous but important phenomenon that the physical meaning and dimension along the spatial, spectral, and nonlocal mode are markedly different. In this work, we discover the low-rank property discrepancy along spatial, spectral, and nonlocal self-similarity mode in the HSIs, and argue that the intrinsic low-rank correlations along each mode contribute different to the final restoration results. Consequently, we figure out that the combination of the spectral and nonlocal-induced low-rank is most beneficial for HSIs modeling, and propose an optimal low-rank tensor (OLRT) model for HSIs restoration. Furthermore, we not only explore the low-rank property in the image component, but also in the sparse error component (stripe noise in HSIs). Thus, we extend OLRT to the OLRT-robust principal component analysis (RPCA) with low-rank tensor priors for both the HSIs and sparse error. Besides, previous methods are usually designed for one specific HSI task, which is less robust to various tasks. We prove that the proposed optimal low-rank prior is very flexible for various HSI restoration problems including denoising, deblurring, inpainting, and destriping. The proposed methods have been extensively evaluated on several benchmarks and tasks, and greatly outperform state-of-the-art (STOA). We show the simple yet effective OLRT strategy is also beneficial to STOA.

Further Development of Subspace Imaging to Magnetic Resonance Fingerprinting: A Low-rank Tensor Approach.

Magnetic resonance fingerprinting is a recent quantitative MRI technique that simultaneously acquires multiple tissue parameter maps (e.g., T1, T2, and spin density) in a single imaging experiment. In our early work, we demonstrated that the low-rank/subspace reconstruction significantly improves the accuracy of tissue parameter maps over the conventional MR fingerprinting reconstruction that utilizes simple pattern matching. In this paper, we generalize the low-rank/subspace reconstruction by introducing a multilinear low-dimensional image model (i.e., a low-rank tensor model). With this model, we further estimate the subspace associated with magnetization evolutions to simplify the image reconstruction problem. The proposed formulation results in a nonconvex optimization problem which we solve by an alternating minimization algorithm. We evaluate the performance of the proposed method with numerical experiments, and demonstrate that the proposed method improves the conventional reconstruction method and the state-of-the-art low-rank reconstruction method.

Robust Low-Rank Tensor Minimization via a New Tensor Spectral k-Support Norm.

Recently, based on a new tensor algebraic framework for third-order tensors, the tensor singular value decomposition (t-SVD) and its associated tubal rank definition have shed new light on low-rank tensor modeling. Its applications to robust image/video recovery and background modeling show promising performance due to its superior capability in modeling cross-channel/frame information. Under the t-SVD framework, we propose a new tensor norm called tensor spectral k-support norm (TSP-k) by an alternative convex relaxation. As an interpolation between the existing tensor nuclear norm (TNN) and tensor Frobenius norm (TFN), it is able to simultaneously drive minor singular values to zero to induce low-rankness, and to capture more global information for better preserving intrinsic structure. We provide the proximal operator and the polar operator for the TSP-k norm as key optimization blocks, along with two showcase optimization algorithms for medium-and large-size tensors. Experiments on synthetic, image and video datasets in medium and large sizes, all verify the superiority of the TSP-k norm and the effectiveness of both optimization methods in comparison with the existing counterparts.

Dynamic cardiac MRI reconstruction using motion aligned locally low rank tensor (MALLRT)

Various sparse transform models have been explored for compressed sensing-based dynamic cardiac MRI reconstruction from vastly under-sampled k-space data. Recently emerged low rank tensor model using Tucker decomposition could be viewed as a special form of sparse model, where the core tensor, which is obtained using high-order singular value decomposition, is sparse in the sense that only a few elements have dominantly large magnitude. However, local details tend to be over-smoothed when the entire image is conventionally modeled as a global tensor. Moreover, low rankness is sensitive to motion as spatiotemporal correlation is corrupted by spatial misalignment between temporal frames. To overcome these limitations, this paper presents a novel motion aligned locally low rank tensor (MALLRT) model for dynamic MRI reconstruction. In MALLRT, low rank constraint is enforced on image patch-based local tensors, which correspond to overlapping blocks extracted from the reconstructed high-dimensional image after group-wise inter-frame motion registration. For solving the proposed model, this paper presents an efficient optimization algorithm by using variable splitting and alternating direction method of multipliers (ADMM). MALLRT demonstrated promising performance as validated on one cardiac perfusion MRI dataset and two cardiac cine MRI datasets using retrospective under-sampling with various acceleration factors, as well as one prospectively under-sampled cardiac perfusion MRI dataset. Compared to four state-of-the-art methods, MALLRT achieved substantially better image reconstruction quality in terms of both signal to error ratio (SER) and structural similarity index (SSIM) metrics, and visual perception in preserving spatial details and capturing temporal variations.