Low-rank Structure Research Articles

A Majorization-Minimization Gauss-Newton Method for 1-Bit Matrix Completion

In 1-bit matrix completion, the aim is to estimate an underlying low-rank matrix from a partial set of binary observations. We propose a novel method for 1-bit matrix completion called Majorization-Minimization Gauss-Newton (MMGN). Our method is based on the majorization-minimization principle, which converts the original optimization problem into a sequence of standard low-rank matrix completion problems. We solve each of these sub-problems by a factorization approach that explicitly enforces the assumed low-rank structure and then apply a Gauss-Newton method. Using simulations and a real data example, we illustrate that in comparison to existing 1-bit matrix completion methods, MMGN outputs comparable if not more accurate estimates. In addition, it is often significantly faster, and less sensitive to the spikiness of the underlying matrix. In comparison with three standard generic optimization approaches that directly minimize the original objective, MMGN also exhibits a clear computational advantage, especially when the fraction of observed entries is small.

Large-Scale Low-Rank Gaussian Process Prediction with Support Points

Low-rank approximation is a popular strategy to tackle the “big n problem” associated with large-scale Gaussian process regressions. Basis functions for developing low-rank structures are crucial and should be carefully specified. Predictive processes simplify the problem by inducing basis functions with a covariance function and a set of knots. The existing literature suggests certain practical implementations of knot selection and covariance estimation; however, theoretical foundations explaining the influence of these two factors on predictive processes are lacking. In this article, the asymptotic prediction performance of the predictive process and Gaussian process predictions are derived and the impacts of the selected knots and estimated covariance are studied. The use of support points as knots, which best represent data locations, is advocated. Extensive simulation studies demonstrate the superiority of support points and verify our theoretical results. Real data of precipitation and ozone are used as examples, and the efficiency of our method over other widely used low-rank approximation methods is verified. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Collocation methods for nonlinear differential equations on low-rank manifolds

We introduce new methods for integrating nonlinear differential equations on low-rank manifolds. These methods rely on interpolatory projections onto the tangent space, enabling low-rank time integration of vector fields that can be evaluated entry-wise. A key advantage of our approach is that it does not require the vector field to exhibit low-rank structure, thereby overcoming significant limitations of traditional dynamical low-rank methods based on orthogonal projection. To construct the interpolatory projectors, we develop a sparse tensor sampling algorithm based on the discrete empirical interpolation method (DEIM) that parameterizes tensor train manifolds and their tangent spaces with cross interpolation. Using these projectors, we propose two time integration schemes on low-rank tensor train manifolds. The first scheme integrates the solution at selected interpolation indices and constructs the solution with cross interpolation. The second scheme generalizes the well-known orthogonal projector-splitting integrator to interpolatory projectors. We demonstrate the proposed methods with applications to several tensor differential equations arising from the discretization of partial differential equations.

Microseismic Signal Reconstruction From Strong Complex Noise Using Low-Rank Structure Extraction and Dual Convolutional Neural Networks.

Microseismic signal reconstruction from complex nonrandom noise is challenging, especially when the signal is disrupted or completely covered by strong field noise. Various methods often assume that signals are laterally coherent or the noise is predictable. In this article, we propose a dual convolutional neural network preceded by a low-rank structure extraction module to reconstruct signals hidden by strong complex field noise. Preconditioning by low-rank structure extraction is the first step in removing high-energy regular noise. The module is followed by two convolutional neural networks with different complexity to achieve better signal reconstruction and noise removal. In addition to the combination of synthetic and field microseismic data, natural images are also used in the training due to their correlation, complexity, and completeness, which contributes to increasing the generalization of the networks. The results from synthetic and real datasets demonstrate superior signal recovery, which cannot be achieved by using solely deep learning, low-rank structure extraction, or curvelet thresholding. Algorithmic generalization is demonstrated using independently acquired array data excluded during training.

Learnable Transform-Assisted Tensor Decomposition for Spatio-Irregular Multidimensional Data Recovery

Tensor decompositions have been successfully applied to multidimensional data recovery. However, classical tensor decompositions are not suitable for emerging spatio-irregular multidimensional data (i.e., spatio-irregular tensor), whose spatial domain is non-rectangular, e.g., spatial transcriptomics data from bioinformatics and semantic units from computer vision. By using preprocessing (e.g., zero-padding or element-wise 0-1 weighting), the spatio-irregular tensor can be converted to a spatio-regular tensor and then classical tensor decompositions can be applied, but this strategy inevitably introduces bias information, leading to artifacts. How to design a tensor-based method suitable for emerging spatio-irregular tensors is an imperative challenge. To address this challenge, we propose a learnable transform-assisted tensor singular value decomposition (LTA-TSVD) for spatio-irregular tensor recovery, which allows us to leverage the intrinsic structure behind the spatio-irregular tensor. Specifically, we design a learnable transform to project the original spatio-irregular tensor into its latent spatio-regular tensor, and then the latent low-rank structure is captured by classical TSVD on the resulting regular tensor. Empowered by LTA-TSVD, we develop spatio-irregular low-rank tensor completion (SIR-LRTC) and spatio-irregular tensor robust principal component analysis (SIR-TRPCA) models for the spatio-irregular tensor imputation and denoising respectively, and we design corresponding solving algorithms with theoretical convergence. Extensive experiments including the spatial transcriptomics data imputation and hyperspectral image denoising show SIR-LRTC and SIR-TRPCA are superior performance to competing approaches and benefit downstream applications.

Simultaneous seismic interpolation and statics estimation of land data

Imaging and inversion of land seismic data affected by complex weathering layers near the surface are challenging. When the data are additionally subsampled for economical reasons such as monitoring of sequestrated carbon dioxide and hydrogen, the problem is further exacerbated due to the combined influence of subsampling and weathering layers. On the one hand, interpolation performs poorly since the weathering layers reduce the data's coherency. On the other hand, near-surface corrections require knowledge of the subsurface model, separation between primaries and multiples as well as subsurface velocity estimation, which are difficult to perform from subsampled data. To overcome these hurdles, we combine seismic interpolation and statics estimation into a joint single rank-reduction-based algorithm. To our knowledge, this is the first time that this has been done. The proposed method simultaneously accounts for the weathering and subsampling effects, which both contribute to the low-rank structure destruction typically associated with statics-free densely-sampled data, to provide accurate reconstruction. Since a low-rank approximation is used for statics estimation, we also use it in rank-minimization interpolation as a cost-free initial solution to the optimization problem. As both statics estimation and interpolation operate in the midpoint-offset domain, we avoid the cost of transformations back and forth from the source-receiver to midpoint-offset transform domain. Consequently, the proposed reconstruction, which shows its potential on synthetic and field data is also computationally efficient.

CP decomposition-based algorithms for completion problem of motion capture data

Motion capture (MoCap) technology is an essential tool for recording and analyzing movements of objects or humans. However, MoCap systems frequently encounter the challenge of missing data, stemming from mismatched markers, occlusion, or equipment limitations. Recovery of these missing data is imperative to maintain the reliability and integrity of MoCap recordings. This paper introduces a novel application of the tensor framework for MoCap data completion. We propose three completion algorithms based on the canonical polyadic (CP) decomposition of tensors. The first algorithm utilizes CP decomposition to capture the low-rank structure of the tensor. However, relying only on low-rank assumptions may be insufficient to deal with complex motion data. Thus, we propose two modified CP decompositions that incorporate additional information, SmoothCP and SparseCP decompositions. SmoothCP integrates piecewise smoothness prior, while SparseCP incorporates sparsity prior, each aiming to improve the accuracy and robustness of MoCap data recovery. To compare and evaluate the merit of the proposed algorithms over other tensor completion methods in terms of several evaluation metrics, we conduct numerical experiments with different MoCap sequences from the CMU motion capture dataset.

Robust Low-Tubal-Rank Tensor Completion Based on Tensor Factorization and Maximum Correntopy Criterion.

The goal of tensor completion is to recover a tensor from a subset of its entries, often by exploiting its low-rank property. Among several useful definitions of tensor rank, the low tubal rank was shown to give a valuable characterization of the inherent low-rank structure of a tensor. While some low-tubal-rank tensor completion algorithms with favorable performance have been recently proposed, these algorithms utilize second-order statistics to measure the error residual, which may not work well when the observed entries contain large outliers. In this article, we propose a new objective function for low-tubal-rank tensor completion, which uses correntropy as the error measure to mitigate the effect of the outliers. To efficiently optimize the proposed objective, we leverage a half-quadratic minimization technique whereby the optimization is transformed to a weighted low-tubal-rank tensor factorization problem. Subsequently, we propose two simple and efficient algorithms to obtain the solution and provide their convergence and complexity analysis. Numerical results using both synthetic and real data demonstrate the robust and superior performance of the proposed algorithms.

PARAFAC under non-negativity constraint is adapted to recover the underlying Beer-Lambert law of the excitation-emission fluorescence matrix measurements acquired from analyte-triggered semiconductor QDs photoluminescence modulation. When and why?

BackgroundAnalyte-triggered semiconductor quantum dots (QDs) modulation in the presence of non-consistently responsive fluorescent species represents a challenging analytical issue in concrete multi-way data handling. QDs with heterogeneous sizes and/or uneven distribution of functional moieties on their surfaces exhibit significant fluctuations in the fluorescent response components, known as chemical rank, across different excitation/emission modes. This phenomenon may lead to a substantial deviation from the proportionality prescribed by Beer-Lambert law. Nonetheless, even in the presence of such deviation, a multi-way model may be successfully selected after determining a proper chemical rank in a QDs system. ResultsWe show that in a valid PARAllel FACtor (PARAFAC) model under properly determined chemical rank, meaningfully resolved pure spectral profiles can be reached for each fluorescent responsive constituent in the original excitation-emission fluorescence matrix (EEFM) measurements. This was thoroughly illustrated by applying PARAFAC trilinear decomposition of a three-way data array of two distinct datasets acquired from semiconductor QDs sensing systems with low-rank trilinear assumption. The first dataset, presented here for the first time, comprises EEFM measurements of the ligand-driven quenching of thiomalic acid (TMA)-capped AgInS2 (AIS) QDs by vomitoxin. The second dataset, employed for illustrative purposes, comprises EEFM measurements of the quenching, via cation bridging, of glutathione (GSH)-capped CdTe QDs by Pb(II). The results of this study enabled the determination of vomitoxin at a ppb level in real samples of fish feeds, showcasing the efficacy of the PARAFAC model in resolving spectral signatures (loadings) and pure concentration profiles (scores). SignificancePARAFAC under a properly examined chemical rank can be easily adapted for retrieval the underlying Beer-Lambert law of the original EEFM measurements with a low-rank trilinear structure through the chemically meaningful information either when (i) no deviation of Beer-Lambert law was observed as deeply discussed in connection with the dataset acquired from vomitoxin-driven molecular sensing through TMA-capped AIS QDs, or when (ii) substantial deviations of the Beer-Lambert law are evident, as discussed in connection with the dataset collected from sensing ionic species through Pb(II) bridging of GSH-capped CdTe QDs.

Quaternion Nuclear Norm Minus Frobenius Norm Minimization for color image reconstruction

Color image restoration methods typically represent images as vectors in Euclidean space or combinations of three monochrome channels. However, they often overlook the correlation between these channels, leading to color distortion and artifacts in the reconstructed image. To address this, we present Quaternion Nuclear Norm Minus Frobenius Norm Minimization (QNMF), a novel approach for color image reconstruction. QNMF utilizes quaternion algebra to capture the relationships among RGB channels comprehensively. By employing a regularization technique that involves nuclear norm minus Frobenius norm, QNMF approximates the underlying low-rank structure of quaternion-encoded color images. Theoretical proofs are provided to ensure the method’s mathematical integrity. Demonstrating versatility and efficacy, the QNMF regularizer excels in various color low-level vision tasks, including denoising, deblurring, inpainting, and random impulse noise removal, achieving state-of-the-art results.

Accelerated SVD-based initialization for nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is a popular dimensionality reduction technique. NMF is typically cast as a non-convex optimization problem solved via standard iterative schemes, such as coordinate descent methods. Hence the choice of the initialization for the variables is crucial as it will influence the factorization quality and the convergence speed. Different strategies have been proposed in the literature, the most popular ones rely on singular value decomposition (SVD). In particular, Atif et al. (Pattern Recognit Lett 122:53–59, 2019) have introduced a very efficient SVD-based initialization, namely NNSVD-LRC, that overcomes the drawbacks of previous methods, namely, it guarantees that (i) the error decreases as the factorization rank increases, (ii) the initial factors are sparse, and (iii) the computational cost is low. In this paper, we improve upon NNSVD-LRC by using the low-rank structure of the residual matrix; this allows us to obtain NMF initializations with similar quality to NNSVD-LRC (in terms of error and sparsity) while reducing the computational load. We evaluate our proposed solution over other NMF initializations on several real dense and sparse datasets.

Multiview Subspace Clustering via Low-Rank Symmetric Affinity Graph.

Multiview subspace clustering (MVSC) has been used to explore the internal structure of multiview datasets by revealing unique information from different views. Most existing methods ignore the consistent information and angular information of different views. In this article, we propose a novel MVSC via low-rank symmetric affinity graph (LSGMC) to tackle these problems. Specifically, considering the consistent information, we pursue a consistent low-rank structure across views by decomposing the coefficient matrix into three factors. Then, the symmetry constraint is utilized to guarantee weight consistency for each pair of data samples. In addition, considering the angular information, we utilize the fusion mechanism to capture the inherent structure of data. Furthermore, to alleviate the effect brought by the noise and the high redundant data, the Schatten p-norm is employed to obtain a low-rank coefficient matrix. Finally, an adaptive information reduction strategy is designed to generate a high-quality similarity matrix for spectral clustering. Experimental results on 11 datasets demonstrate the superiority of LSGMC in clustering performance compared with ten state-of-the-art multiview clustering methods.

Reduced order model enhanced source iteration with synthetic acceleration for parametric radiative transfer equation

Applications such as uncertainty quantification, shape optimization, and optical tomography, require solving the radiative transfer equation (RTE) many times for various parameters. Efficient solvers for RTE are highly desired.Source Iteration with Synthetic Acceleration (SISA) is a popular and successful iterative solver for RTE. Synthetic Acceleration (SA) acts as a preconditioning step to accelerate the convergence of Source Iteration (SI). After each source iteration, classical SA strategies introduce a correction to the macroscopic particle density by solving a low order approximation to a kinetic correction equation. For example, Diffusion Synthetic Acceleration (DSA) uses the diffusion limit. However, these strategies may become less effective when the underlying low order approximations are not accurate enough. Furthermore, they do not exploit low rank structures concerning the parameters of parametric problems.To address these issues, we propose enhancing SISA with data-driven ROMs for the parametric problem and the corresponding kinetic correction equation. First, the ROM for the parametric problem can be utilized to obtain an improved initial guess. Second, the ROM for the kinetic correction equation can be utilized to design a low rank approximation to it. Unlike the diffusion limit, this ROM-based approximation builds on the kinetic description of the correction equation and leverages low rank structures concerning the parameters. We further introduce a novel SA strategy called ROMSAD. ROMSAD initially adopts our ROM-based approximation to exploit its greater efficiency in the early stage of SISA, and then automatically switches to DSA to leverage its robustness in the later stage. Additionally, we propose an approach to construct the ROM for the kinetic correction equation without directly solving it.Through a series of numerical tests, we compare the proposed methods with SI-DSA and DSA preconditioned Krylov solver. Particularly, for a multiscale parametric pin-cell problem, ROMSAD achieves approximately 10 times the acceleration compared to SI-DSA and 4 times acceleration compared to DSA preconditioned GMRES.

Cyclic tensor singular value decomposition with applications in low-rank high-order tensor recovery

The rapid advancements in emerging technologies have increased the demand for recovery tasks involving high-dimensional data with complex structures. Effectively utilizing tensor decomposition techniques to capture the low-rank structure of such data is crucial. Recently, the high-order t-SVD has demonstrated strong adaptability. However, this decomposition approach can only capture the low-rank correlation of two modes along other modes individually, while disregarding the structural correlation between different modes. In this paper, we propose a novel cyclic tensor singular value decomposition (CTSVD) method that effectively characterizes the low-rank structures of high-order tensors along all modes. Specifically, our method decomposes an order-N tensor into N factor tensors and one core tensor, connecting them using a defined mode-k tensor-tensor product (t-product). Building upon this, we establish the corresponding tensor rank and its convex relaxation. To address the issue of dimensional imbalance between adjacent modes in high-dimensional data, we propose and integrate a square reshaping strategy into the recovery models for tensor completion (TC) and tensor principal component analysis (TRPCA) tasks. Effective alternating direction method of multipliers (ADMM)-based algorithms are designed to these tasks. Extensive experiments on both synthetic and real data demonstrate that our methods outperform state-of-the-art approaches.

Reconstruction and denoising of high-dimensional seismic data via Frobenius-nuclear mixed norm constraints

Abstract The seismic data acquisition design with ‘two-wide and one-high’ geometry effectively improves the imaging quality of seismic records. However, when data are acquired in the field, complex near surface conditions and environmental factors can introduce a variety of noises and gaps in seismic data, impacting the accuracy of seismic imaging. Currently, the method of low-rank matrix/tensor completion is commonly employed for data reconstruction after normal moveout (NMO). In a complex subsurface medium, common midpoint data processed with NMO may not satisfy the linear or quasi-linear assumptions within local data windows. Therefore, this paper exploits the inherent low-rank structure of high-dimensional data to propose a high-dimensional tensor completion method under the Frobenius-nuclear mixed norm constraint (FN-TC). This method unfolds the 4D data tensor into the frequency-space domain along its modes (m, n) and subsequently imposes a non-convex Frobenius-nuclear mixed norm constraint on the unfolded approximate matrices. This approach closely approximates the rank function of the factor matrices, thereby enhancing the accuracy of data modeling. Theoretical and practical studies demonstrate that the novel FN-TC approach can effectively reconstruct high-dimensional seismic data and suppress noise, thereby providing data support for subsequent high-precision seismic imaging.

A nonlinear high-order transformations-based method for high-order tensor completion

The high-order tensor nuclear norm model (HTNN) has recently shown promising results in tensor completion problems. The HTNN-based approaches rely on the low-rank structure of tensor slices during reversible transformations. However, under reversible transformations, the low-rank configuration within the slice-wise modality of the tensor’s structure is not markedly pronounced. In order to more effectively describe the low-rank characteristics, we propose a model based on the nonlinear high-order transform-based tensor nuclear norm (NHTNN). Specifically, our framework consists of a linearly semi-orthogonal transformation along the high-dimensional modality and an element-wise nonlinear transformation. We introduce a model for tensor completion, grounded in the suggested measure of tensor low-rank, i.e., NHTNN. Utilizing this non-convex nonlinear model, we formulate a proximal alternating minimization (PAM) algorithm, establishing its convergence through a rigorous proof. In experiments on datasets such as hyperspectral videos (HSVs) and color videos (CVs), our approach demonstrates superior quantitative numerical results and qualitative visual effects compared to cutting-edge tensor completion techniques.

Nonnegative low multi‐rank third‐order tensor approximation via transformation

AbstractThe main aim of this paper is to develop a new algorithm for computing a nonnegative low multi‐rank tensor approximation for a nonnegative tensor. In the literature, there are several nonnegative tensor factorizations or decompositions, and their approaches are to enforce the nonnegativity constraints in the factors of tensor factorizations or decompositions. In this paper, we study nonnegativity constraints in tensor entries directly, and a low rank approximation for the transformed tensor by using discrete Fourier transformation matrix, discrete cosine transformation matrix, or unitary transformation matrix. This strategy is particularly useful in imaging science as nonnegative pixels appear in tensor entries and a low rank structure can be obtained in the transformation domain. We propose an alternating projections algorithm for computing such a nonnegative low multi‐rank tensor approximation. The convergence of the proposed projection method is established. Numerical examples for multidimensional images are presented to demonstrate that the performance of the proposed method is better than that of nonnegative low Tucker rank tensor approximation and the other nonnegative tensor factorizations and decompositions.

Efficient enhancement of low-rank tensor completion via thin QR decomposition.

Low-rank tensor completion (LRTC), which aims to complete missing entries from tensors with partially observed terms by utilizing the low-rank structure of tensors, has been widely used in various real-world issues. The core tensor nuclear norm minimization (CTNM) method based on Tucker decomposition is one of common LRTC methods. However, the CTNM methods based on Tucker decomposition often have a large computing cost due to the fact that the general factor matrix solving technique involves multiple singular value decompositions (SVDs) in each loop. To address this problem, this article enhances the method and proposes an effective CTNM method based on thin QR decomposition (CTNM-QR) with lower computing complexity. The proposed method extends the CTNM by introducing tensor versions of the auxiliary variables instead of matrices, while using the thin QR decomposition to solve the factor matrix rather than the SVD, which can save the computational complexity and improve the tensor completion accuracy. In addition, the CTNM-QR method's convergence and complexity are analyzed further. Numerous experiments in synthetic data, real color images, and brain MRI data at different missing rates demonstrate that the proposed method not only outperforms in terms of completion accuracy and visualization, but also conducts more efficiently than most state-of-the-art LRTC methods.

CoNoT: Coupled Nonlinear Transform-Based Low-Rank Tensor Representation for Multidimensional Image Completion.

Recently, the transform-based tensor nuclear norm (TNN) methods have shown promising performance and drawn increasing attention in tensor completion (TC) problems. The main idea of these methods is to exploit the low-rank structure of frontal slices of the tensor under the transform. However, the transforms in TNN methods usually treat all modes equally and do not consider the different traits of different modes (i.e., spatial and spectral/temporal modes). To address this problem, we suggest a new low-rank tensor representation based on the coupled nonlinear transform (called CoNoT) for a better low-rank approximation. Concretely, spatial and spectral/temporal transforms in the CoNoT, respectively, exploit the different traits of different modes and are coupled together to boost the implicit low-rank structure. Here, we use the convolutional neural network (CNN) as the CoNoT, which can be learned solely from an observed multidimensional image in an unsupervised manner. Based on this low-rank tensor representation, we build a new multidimensional image completion model. Moreover, we also propose an enhanced version (called Ms-CoNoT) to further exploit the spatial multiscale nature of real-world data. Extensive experiments on real-world data substantiate the superiority of the proposed models against many state-of-the-art methods both qualitatively and quantitatively.

Tensor Ring Decomposition Guided Dictionary Learning for OCT Image Denoising.

Optical coherence tomography (OCT) is a non-invasive and effective tool for the imaging of retinal tissue. However, the heavy speckle noise, resulting from multiple scattering of the light waves, obscures important morphological structures and impairs the clinical diagnosis of ocular diseases. In this paper, we propose a novel and powerful model known as tensor ring decomposition-guided dictionary learning (TRGDL) for OCT image denoising, which can simultaneously utilize two useful complementary priors, i.e., three-dimensional low-rank and sparsity priors, under a unified framework. Specifically, to effectively use the strong correlation between nearby OCT frames, we construct the OCT group tensors by extracting cubic patches from OCT images and clustering similar patches. Then, since each created OCT group tensor has a low-rank structure, to exploit spatial, non-local, and its temporal correlations in a balanced way, we enforce the TR decomposition model on each OCT group tensor. Next, to use the beneficial three-dimensional inter-group sparsity, we learn shared dictionaries in both spatial and temporal dimensions from all of the stacked OCT group tensors. Furthermore, we develop an effective algorithm to solve the resulting optimization problem by using two efficient optimization approaches, including proximal alternating minimization and the alternative direction method of multipliers. Finally, extensive experiments on OCT datasets from various imaging devices are conducted to prove the generality and usefulness of the proposed TRGDL model. Experimental simulation results show that the suggested TRGDL model outperforms state-of-the-art approaches for OCT image denoising both qualitatively and quantitatively.