Low-rank Tensor Model Research Articles

Convex regularization for high-dimensional multiresponse tensor regression

In this paper, we present a general convex optimization approach for solving high-dimensional multiple response tensor regression problems under low-dimensional structural assumptions. We consider using convex and weakly decomposable regularizers assuming that the underlying tensor lies in an unknown low-dimensional subspace. Within our framework, we derive general risk bounds of the resulting estimate under fairly general dependence structure among covariates. Our framework leads to upper bounds in terms of two very simple quantities, the Gaussian width of a convex set in tensor space and the intrinsic dimension of the low-dimensional tensor subspace. To the best of our knowledge, this is the first general framework that applies to multiple response problems. These general bounds provide useful upper bounds on rates of convergence for a number of fundamental statistical models of interest including multiresponse regression, vector autoregressive models, low-rank tensor models and pairwise interaction models. Moreover, in many of these settings we prove that the resulting estimates are minimax optimal. We also provide a numerical study that both validates our theoretical guarantees and demonstrates the breadth of our framework.

Deep learning within a priori temporal feature spaces for large-scale dynamic MR image reconstruction: Application to 5-D cardiac MR Multitasking.

High spatiotemporal resolution dynamic magnetic resonance imaging (MRI) is a powerful clinical tool for imaging moving structures as well as to reveal and quantify other physical and physiological dynamics. The low speed of MRI necessitates acceleration methods such as deep learning reconstruction from under-sampled data. However, the massive size of many dynamic MRI problems prevents deep learning networks from directly exploiting global temporal relationships. In this work, we show that by applying deep neural networks inside a priori calculated temporal feature spaces, we enable deep learning reconstruction with global temporal modeling even for image sequences with >40,000 frames. One proposed variation of our approach using dilated multi-level Densely Connected Network (mDCN) speeds up feature space coordinate calculation by 3000x compared to conventional iterative methods, from 20 minutes to 0.39 seconds. Thus, the combination of low-rank tensor and deep learning models not only makes large-scale dynamic MRI feasible but also practical for routine clinical application.

Denoising of low-dose CT images via low-rank tensor modeling and total variation regularization.

Low-dose Computed Tomography (CT) imaging is a most commonly used medical imaging modality. Though the reduction in dosage reduces the risk due to radiation, it leads to an increase in noise level. Hence, it is a mandatory requirement to include a noise reduction technique as a pre- and/or post-processing step for better disease diagnosis. The nuclear norm minimization has attracted a great deal of research interest in contemporary years. This paper proposes a low-rank approximation based approach for denoising of CT images by effectively utilizing the global spatial correlation and local smoothness properties. The tensor nuclear norm is used to describe the global properties and the tensor total variation is used to characterize the local smoothness as well as to improve global smoothness. The resulting optimization problem is solved by the Alternative Direction Method of Multipliers (ADMM) technique. Experimental results on simulated and real CT data prove that the proposed methods outperform the state-of-art works.

Quantitative 3D dynamic contrast-enhanced (DCE) MR imaging of carotid vessel wall by fast T1 mapping using Multitasking.

To develop a dynamic contrast-enhanced (DCE) MRI method capable of high spatiotemporal resolution, 3D carotid coverage, and T1-based quantification of contrast agent concentration for the assessment of carotid atherosclerosis using a newly developed Multitasking technique. 5D imaging with 3 spatial dimensions, 1 T1 recovery dimension, and 1 DCE time dimension was performed using MR Multitasking based on low-rank tensor modeling, which allows direct T1 quantification with high spatiotemporal resolution (0.7 mm isotropic and 595 ms, respectively). Saturation recovery preparations followed by 3D segmented fast low angle shot readouts were implemented with Gaussian-density random 3D Cartesian sampling. A bulk motion removal scheme was developed to improve image quality. The proposed protocol was tested in phantom and human studies. In vivo scans were performed on 14 healthy subjects and 7 patients with carotid atherosclerosis. Kinetic parameters including area under the concentration versus time curve (AUC), vp , Ktrans , and ve were evaluated for each case. Phantom experiments showed that T1 measurements using the proposed protocol were in good agreement with reference value ( ). In vivo studies demonstrated that AUC, vp , and Ktrans in the patient group were significantly higher than in the control group (0.63 ± 0.13 versus 0.42 ± 0.12, P < 0.001; 0.14 ± 0.05 versus 0.11 ± 0.03, P = 0.034; and 0.13 ± 0.04 versus 0.08 ± 0.02, P < 0.001, respectively). Results from repeated subjects showed good interscan reproducibility (intraclass correlation coefficient: vp , 0.83; Ktrans , 0.87; ve , 0.92; AUC, 0.94). Multitasking DCE is a promising approach for quantitatively assessing the vascularity properties of the carotid vessel wall.

Online Robust Low-Rank Tensor Modeling for Streaming Data Analysis.

Tensor data (i.e., the data having multiple dimensions) are quickly growing in scale in many practical applications, which poses new challenges for data modeling and analysis approaches, such as high-order relations of large complexity, gross noise, and varying data scale. Existing low-rank data analysis methods, which are effective at analyzing matrix data, may fail in the regime of tensor data due to these challenges. A robust and scalable low-rank tensor modeling method is heavily desired. In this paper, we develop an online robust low-rank tensor modeling (ORLTM) method to address these challenges. The ORLTM method leverages the high-order correlations among all tensor modes to model an intrinsic low-rank structure of streaming tensor data online and can effectively analyze data residing in a mixture of multiple subspaces by virtue of dictionary learning. ORLTM consumes a very limited memory space that remains constant regardless of the increase of tensor data size, which facilitates processing tensor data at a large scale. More concretely, it models each mode unfolding of streaming tensor data using the bilinear formulation of tensor nuclear norms. With this reformulation, ORLTM employs a stochastic optimization algorithm to learn the tensor low-rank structure alternatively for online updating. To capture the final tensors, ORLTM uses an average pooling operation on folded tensors in all modes. We also provide the analysis regarding computational complexity, memory cost, and convergence. Moreover, we extend ORLTM to the image alignment scenario by incorporating the geometrical transformations and linearizing the constraints. Extensive empirical studies on synthetic database and three practical vision tasks, including video background subtraction, image alignment, and visual tracking, have demonstrated the superiority of the proposed method.

A Low-Rank Tensor Model for Imputation of Missing Vehicular Traffic Volume

This paper presents a low-rank tensor model for vehicular traffic volume data. Contrarily to previous works, we capitalize on a definition of rank, called the tensor train, that is as effective as possible; so that it exploits all the correlation between local structures that are present in the multiple modes, but practical enough that efficient optimization algorithms still hold. From our model, a formulation to find balanced (higher order) tensors is derived. The resulting optimally-balanced tensor improves the imputation accuracy of the tensor train rank. Then, we design specific experiments, which are numerically evaluated using real-world traffic data from Tampere city, Finland. The experimental results are promising, our proposed approach outperforms existing algorithms in both imputation accuracy and, in some instances, computation time.

Tensor-Based Large-Scale Blind System Identification Using Segmentation

Many real-life signals can be described in terms of much fewer parameters than the actual number of samples. Such compressible signals can often be represented very compactly with low-rank matrix and tensor models. The authors have adopted this strategy to enable large-scale instantaneous blind source separation. In this paper, we generalize the approach to the blind identification of large-scale convolutive systems. In particular, we apply the same idea to the system coefficients of finite impulse response systems. This allows us to reformulate blind system identification as a structured tensor decomposition. The tensor is obtained by applying a deterministic tensorization technique called segmentation on the observed output data. Exploiting the low-rank structure of the system coefficients enables a unique identification of the system and estimation of the inputs. We obtain a new type of deterministic uniqueness conditions. Moreover, the compactness of the low-rank models allows one to solve large-scale problems. We illustrate our method for direction-of-arrival estimation in large-scale antenna arrays and neural spike sorting in high-density microelectrode arrays.

Convergence rate of Bayesian supervised tensor modeling with multiway shrinkage priors

This article studies the convergence rate of the posterior for Bayesian low rank supervised tensor modeling with multiway shrinkage priors. Multiway shrinkage priors constitute a new class of shrinkage prior distributions for tensor parameters in Bayesian low rank supervised tensor modeling to regress a scalar response on a tensor predictor with the primary aim to identify cells in the tensor predictor which are predictive of the scalar response. This novel and computationally efficient framework stems from pressing needs in many applications, including functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). This article shows that the convergence rate is nearly optimal in terms of in-sample predictive accuracy of the Bayesian supervised low rank tensor model with a multiway shrinkage prior distribution when the number of observations grows. The conditions under which this nearly optimal convergence rate is achieved are seen to be very mild. More importantly, the rate is achieved for an easily computable method, even when the true CP/PARAFAC rank of the tensor coefficient corresponding to the tensor predictor is unknown.

High-resolution dynamic 31 P-MRSI using a low-rank tensor model.

To develop a rapid 31 P-MRSI method with high spatiospectral resolution using low-rank tensor-based data acquisition and image reconstruction. The multidimensional image function of 31 P-MRSI is represented by a low-rank tensor to capture the spatial-spectral-temporal correlations of data. A hybrid data acquisition scheme is used for sparse sampling, which consists of a set of "training" data with limited k-space coverage to capture the subspace structure of the image function, and a set of sparsely sampled "imaging" data for high-resolution image reconstruction. An explicit subspace pursuit approach is used for image reconstruction, which estimates the bases of the subspace from the "training" data and then reconstructs a high-resolution image function from the "imaging" data. We have validated the feasibility of the proposed method using phantom and in vivo studies on a 3T whole-body scanner and a 9.4T preclinical scanner. The proposed method produced high-resolution static 31 P-MRSI images (i.e., 6.9 × 6.9 × 10 mm3 nominal resolution in a 15-min acquisition at 3T) and high-resolution, high-frame-rate dynamic 31 P-MRSI images (i.e., 1.5 × 1.5 × 1.6 mm3 nominal resolution, 30 s/frame at 9.4T). Dynamic spatiospectral variations of 31 P-MRSI signals can be efficiently represented by a low-rank tensor. Exploiting this mathematical structure for data acquisition and image reconstruction can lead to fast 31 P-MRSI with high resolution, frame-rate, and SNR. Magn Reson Med 78:419-428, 2017. © 2017 International Society for Magnetic Resonance in Medicine.

Tensor-based dynamic reconstruction method for electrical capacitance tomography

Electrical capacitance tomography (ECT) is an attractive visualization measurement method, in which the acquisition of high-quality images is beneficial for the understanding of the underlying physical or chemical mechanisms of the dynamic behaviors of the measurement objects. In real-world measurement environments, imaging objects are often in a dynamic process, and the exploitation of the spatial-temporal correlations related to the dynamic nature will contribute to improving the imaging quality. Different from existing imaging methods that are often used in ECT measurements, in this paper a dynamic image sequence is stacked into a third-order tensor that consists of a low rank tensor and a sparse tensor within the framework of the multiple measurement vectors model and the multi-way data analysis method. The low rank tensor models the similar spatial distribution information among frames, which is slowly changing over time, and the sparse tensor captures the perturbations or differences introduced in each frame, which is rapidly changing over time. With the assistance of the Tikhonov regularization theory and the tensor-based multi-way data analysis method, a new cost function, with the considerations of the multi-frames measurement data, the dynamic evolution information of a time-varying imaging object and the characteristics of the low rank tensor and the sparse tensor, is proposed to convert the imaging task in the ECT measurement into a reconstruction problem of a third-order image tensor. An effective algorithm is developed to search for the optimal solution of the proposed cost function, and the images are reconstructed via a batching pattern. The feasibility and effectiveness of the developed reconstruction method are numerically validated.

The Twist Tensor Nuclear Norm for Video Completion.

In this paper, we propose a new low-rank tensor model based on the circulant algebra, namely, twist tensor nuclear norm (t-TNN). The twist tensor denotes a three-way tensor representation to laterally store 2-D data slices in order. On one hand, t-TNN convexly relaxes the tensor multirank of the twist tensor in the Fourier domain, which allows an efficient computation using fast Fourier transform. On the other, t-TNN is equal to the nuclear norm of block circulant matricization of the twist tensor in the original domain, which extends the traditional matrix nuclear norm in a block circulant way. We test the t-TNN model on a video completion application that aims to fill missing values and the experiment results validate its effectiveness, especially when dealing with video recorded by a nonstationary panning camera. The block circulant matricization of the twist tensor can be transformed into a circulant block representation with nuclear norm invariance. This representation, after transformation, exploits the horizontal translation relationship between the frames in a video, and endows the t-TNN model with a more powerful ability to reconstruct panning videos than the existing state-of-the-art low-rank models.

Sampling and recovery of MRI data using low rank tensor models.

In this paper we investigate the utility of several low-rank models for recovery of Magnetic Resonance Imaging (MRI) data from limited sampling in the k - t space for dynamic imaging. In particular, for 3D temporal (2D space + time) MRI data we employ several tensor factorization techniques and assess the degree of dimensionality reduction, or compressibility, that can be obtained. This algebraic approach is more data adaptive, in contrast to existing compressed sensing (CS) based methods that exploit sparsity in a transform domain, such as wavelets or total variation. Further, we compare these tensor factorization approaches in recovering temporal MRI data under limited sampling. Respecting traditional MRI data acquisition methods, the sampling process is restricted to be uniformly random along only one k space direction. Experimental results on synthetically sub-sampled MRI data show promise in using tensor factorization for sampling and recovery of MRI data.

Fast dynamic electron paramagnetic resonance (EPR) oxygen imaging using low-rank tensors

Hypoxic tumors are resistant to radiotherapy, motivating the development of tools to image local oxygen concentrations. It is generally believed that stable or chronic hypoxia is the source of resistance, but more recent work suggests a role for transient hypoxia. Conventional EPR imaging (EPRI) is capable of imaging tissue pO2in vivo, with high pO2 resolution and 1mm spatial resolution but low imaging speed (10min temporal resolution for T1-based pO2 mapping), which makes it difficult to investigate the oxygen changes, e.g., transient hypoxia. Here we describe a new imaging method which accelerates dynamic EPR oxygen imaging, allowing 3D imaging at 2 frames per minute, fast enough to image transient hypoxia at the “speed limit” of observed pO2 change. The method centers on a low-rank tensor model that decouples the tradeoff between imaging speed, spatial coverage/resolution, and number of inversion times (pO2 accuracy). We present a specialized sparse sampling strategy and image reconstruction algorithm for use with this model. The quality and utility of the method is demonstrated in simulations and in vivo experiments in tumor bearing mice.