Fast Sampling and Reconstruction for Linear Inverse Problems: From Vectors to Tensors

Abstract

Signals typical in the real world have different modes, expressed as vectors, matrices, or higher-order tensors. In practice, a target signal is commonly assumed to be linear in the residing factor mode(s) with low-dimensional parameters, and thus can be recovered from partial samples by solving a linear inverse problem (LIP). Sampling for LIPs is to partially query signals for better recovery. There exist many fast sampling methods for vector signals, but they are not applicable for higher-order tensors due to high computation and storage costs. In this paper, we propose a fast sampling algorithm for vector signals first and then extend it to sample tensors with low complexity. Specifically, a tensor signal with <inline-formula><tex-math notation="LaTeX">$R$</tex-math></inline-formula> modes is modelled as multilinear on <inline-formula><tex-math notation="LaTeX">$R$</tex-math></inline-formula> factor matrices <inline-formula><tex-math notation="LaTeX">${\mathbf \{U_{i}\}}^{R}_{i=1}$</tex-math></inline-formula>, whose vectorized version is linear on a matrix <inline-formula><tex-math notation="LaTeX">$\pmb {\Phi }$</tex-math></inline-formula>&#x2014;the Kronecker product of <inline-formula><tex-math notation="LaTeX">$\mathbf {U}_{i}$</tex-math></inline-formula>&#x0027;s. Thus, it can be estimated from partial noisy samples via the least-squares (LS) method, where the mean square error (MSE) depends on chosen samples and matrix <inline-formula><tex-math notation="LaTeX">$\pmb {\Phi }$</tex-math></inline-formula>. Instead of MSE, we minimize a modified MSE problem and prove it has the same optimal and greedy solutions to a problem with a sample-dependent sub-matrix of <inline-formula><tex-math notation="LaTeX">$\pmb {\Phi }\pmb {\Phi }^\top$</tex-math></inline-formula>. For one-order vector signals, where <inline-formula><tex-math notation="LaTeX">$\pmb {\Phi }=\mathbf {U}_{1}$</tex-math></inline-formula> is given, we propose a fast algorithm to solve the sampling problem via simple vector-vector multiplications based on a matrix inversion formula and greedy solution reuse. For higher-order tensors, we extend our sampling algorithm to select entries by addressing matrices <inline-formula><tex-math notation="LaTeX">$\mathbf {U}_{i}$</tex-math></inline-formula>&#x0027;s directly, thus removing the burden of obtaining <inline-formula><tex-math notation="LaTeX">$\pmb {\Phi }$</tex-math></inline-formula> explicitly. Accompanying our sampling strategy, we propose an iterative reconstruction scheme to estimate the LS solution. Experiments on synthetic and real-world signals from vectors to tensors validated the performance and efficacy of our strategy compared to previous methods.