Low-rank Approximation Method Research Articles

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

Windowed variation kernel Wiener filter model for image denoising with edge preservation

Due to the inherent characteristics of redundancy in natural images, sparsity and low-rank approximation can be applied to image processing tasks. However, most existing denoising algorithms based on low-rank approximation are pron to arise over-smoothing problem, resulting in a loss of details and structures. In this paper, a windowed variation kernel Wiener filter (WV-KWF) image denoising algorithm based on low-rank approximation is proposed. We first estimate the reference image of kernel Wiener filtering by using a low-rank approximation method. Then a windowed inherent variation is used to describe image local gradient information, and a shape-aware kernel function is introduced to describe image local complex structures. Finally the optimal kernel Wiener filter approach can be obtained for image denoising while preserving edges. The experimental results show that compared with the existing state-of-the-art methods, our proposed method is more competitive in maintaining image structures and removing noises both subjectively and objectively.

A guidable nonlocal low-rank approximation model for hyperspectral image denoising

Hyperspectral image (HSI) denoising is an essential preprocessing step for improving HSI applications. Recently, subspace-based nonlocal low-rank approximation (SNLR) methods have shown their superiority. However, most of these methods ignore such a potentially important phenomenon, that is, real HSIs contain many high signal-to-noise ratio bands (HSNRBs), and thus result in the underutilization of information. In this paper, we propose a new method called Subspace-based Guided Nonlocal Low-Rank Approximation (SGNLR) for HSI denoising. Our method takes advantage of the abundant spatial information in the representation coefficients of HSNRBs to improve denoising performances. Specifically, we employ low-rank subspace representation to exploit the global spectral correlation of HSI and transform the HSI denoising task as the estimation of spectral basis and spatial representation coefficients (SRCs). Motivated by the consistency of coefficient features between the whole HSI and HSNRBs, we employ the SRCs of HSNRBs to guide the restoration of target coefficients. To restore the SRCs accurately, we design a powerful nonlocal low-rank approximation that takes into account the nonlocal self-similarity (NSS) of SRCs. An efficient algorithm based on alternating minimization is developed to optimize the proposed model. Extensive experiments on both simulated and real-world data demonstrate the outperformance of our method.

Link prediction by adversarial Nonnegative Matrix Factorization

Networks are now more crucial than ever for modeling complex systems with interconnected components. Many methods have been developed to infer unobserved links or predict latent links based on observed network topology. However, the effectiveness of current methods is often limited due to the nature of sparseness in large-scale networks. Nonnegative Matrix Factorization (NMF) is one of the most popular low-rank approximation methods which has been successfully applied to large-scale data. However, maintaining generalization from limited observations is still an open challenge. To overcome these challenges, we propose a novel link prediction method based on adversarial NMF, which reconstructs a sparse network by an efficient adversarial training algorithm. Unlike the conventional NMF methods, our model considers potential test adversaries beyond the pre-defined bounds and provides a robust reconstruction with good generalization power. Besides, to preserve the local structure of a network, we use the common neighbor algorithm to extract the node similarity and apply it to low-dimensional latent representation. Simultaneously, we use Frobenius-norm regularization to prevent the factorization from overfitting. We provide an effective Majorization-Minimization method to learn the model parameters. Our method outperforms the state-of-the-art methods, as shown by extensive experiment results on twelve real-world datasets.

Pass-efficient randomized LU algorithms for computing low-rank matrix approximation

Abstract Low-rank matrix approximation is extremely useful in the analysis of data that arises in scientific computing, engineering applications, and data science. However, as data sizes grow, traditional low-rank matrix approximation methods, such as singular value decomposition (SVD) and column pivoting QR decomposition (CPQR), are either prohibitively expensive or cannot provide sufficiently accurate results. A solution is to use randomized low-rank matrix approximation methods such as randomized SVD, and randomized LU decomposition on extremely large data sets. In this paper, we focus on the randomized LU decomposition method. Then we propose a novel randomized LU algorithm, called SubspaceLU, for the fixed low-rank approximation problem. SubspaceLU is based on the sketch of the co-range of input matrices and allows for an arbitrary number of passes of the input matrix, v ≥ 2 {v\geq 2} . Numerical experiments on CPU show that our proposed SubspaceLU is generally faster than the existing randomized LU decomposition, while remaining accurate. Experiments on GPU shows that our proposed SubspaceLU can gain more speedup than the existing randomized LU decomposition. We also propose a version of SubspaceLU, called SubspaceLU_FP, for the fixed precision low-rank matrix approximation problem. SubspaceLU_FP is a post-processing step based on an efficient blocked adaptive rank determination Algorithm 5 proposed in this paper. We present numerical experiments that show that SubspaceLU_FP can achieve close results to SVD but faster in speed. We finally propose a single-pass algorithm based on LU factorization. Tests show that the accuracy of our single-pass algorithm is comparable with the existing single-pass algorithms.

Imaging point diffractors using a low-rank approximation method

Diffraction separation plays a critical role in obtaining high-resolution images of subsurface diffractors by eliminating the diffractions’ contribution from reflections. It is usually based on the difference in spatial coherency between seismic reflection and diffraction energy in the time domain. Methods based on local slope differences, e.g., plane-wave destruction (PWD) method, tend to separate point and line diffractors simultaneously. This paper applies a rank-based diffraction separation method, known as the localized rank-reduction (LRR) method, to distinguish point diffractors in poststack data sets from line diffractors, thereby obtaining a higher resolution when characterizing point discontinuities such as karst caves. To make the LRR method more adaptive, a novel rank determination method based on a newly defined accumulated singular value ratio sequence is developed. The effectiveness of the proposed method is proved via a variety of 2D/3D synthetic and real data examples and a detailed discussion of its practical potential. Despite a larger computational cost compared to the PWD method, the proposed method helps interpretation by enabling us to distinguish between line and point diffractors.

A new nonconvex low-rank tensor approximation method with applications to hyperspectral images denoising

Hyperspectral images (HSIs) are frequently corrupted by mixing noise during their acquisition and transmission. Such complicated noise may reduce the quality of the obtained HSIs and limit the accuracy of the subsequent processing. By using the low-rank prior of the tensor formed by spatial and spectral information and further exploring the intrinsic structure of the underlying HSI from noisy observations, in this paper, we propose a new nonconvex low-rank tensor approximation method including optimization model and efficient iterative algorithm to eliminate multiple types of noise. The proposed mathematical model consists of a nonconvex low-rank regularization term using the γ nuclear norm, which is nonconvex surrogate to Tucker rank, and two data fidelity terms representing sparse and Gaussian noise components, which are regularized by the -norm and the Frobenius norm, respectively. To solve this model, we propose an efficient augmented Lagrange multiplier algorithm. We also study the convergence and parameter setting of the algorithm. Extensive experimental results show that the proposed method has better denoising performance than the state-of-the-art competing methods for low-rank tensor approximation and noise modeling.

A robust collision source method for rank adaptive dynamical low-rank approximation in radiation therapy

Deterministic models for radiation transport describe the density of radiation particles moving through a background material. In radiation therapy applications, the phase space of this density is composed of energy, spatial position and direction of flight. The resulting six-dimensional phase space prohibits fine numerical discretizations, which are essential for the construction of accurate and reliable treatment plans. In this work, we tackle the high dimensional phase space through a dynamical low-rank approximation of the particle density. Dynamical low-rank approximation (DLRA) evolves the solution on a low-rank manifold in time. Interpreting the energy variable as a pseudo-time lets us employ the DLRA framework to represent the solution of the radiation transport equation on a low-rank manifold for every energy. Stiff scattering terms are treated through an efficient implicit energy discretization and a rank adaptive integrator is chosen to dynamically adapt the rank in energy. To facilitate the use of boundary conditions and reduce the overall rank, the radiation transport equation is split into collided and uncollided particles through a collision source method. Uncollided particles are described by a directed quadrature set guaranteeing low computational costs, whereas collided particles are represented by a low-rank solution. It can be shown that the presented method is L2-stable under a time step restriction which does not depend on stiff scattering terms. Moreover, the implicit treatment of scattering does not require numerical inversions of matrices. Numerical results for radiation therapy configurations as well as the line source benchmark underline the efficiency of the proposed method.

Regularized Low-Rank Approximation Method for Diffraction Enhancement

Regularized Low-Rank Approximation Method for Diffraction Enhancement

A two-step singular spectrum analysis method for robust low-rank approximation of seismic data

The singular spectrum analysis (SSA) method can detect the low-rank structure of data and therefore has become a powerful tool in seismic data processing and analysis. In particular, the SSA method can effectively suppress seismic random noise according to the different behaviors of coherent signal and random noise in the singular spectrum. However, there has been much research and experimentation indicating that the basic SSA method performs poorly when the noise becomes erratic. One of the reasons is that the quadratic misfit adopted in SSA is sensitive to non-Gaussian disturbances. One solution to this problem is to iteratively reweight the low-rank approximation of using the SSA method and finally achieve a particular kind of robust misfit. The low-rank approximation is robustified in this study by a more direct strategy, which consists of two main steps, prediction and elimination, and is called the two-step SSA method. The whole algorithm only runs the SSA filtering twice and hence is more computationally efficient compared to the iteratively reweighted SSA technique. The two-step SSA method adopts two criteria to calculate a weighting matrix and predict the erratic disturbance. This strategy also can be generalized into an iterative reweighting-based technique (e.g., the iteratively reweighted SSA technique) for robust denoising. The performance of the proposed two-step SSA method is tested using simulated and real seismic data. The results demonstrate its feasibility.

The legitimacy of decoupled dynamic flow stress equations and their representation based on discrete experimental data

In this study, the legitimacy of a decoupled empirical dynamic flow stress equations, typically the type of Johnson-Cook (J-C) (flow stress) equations, is mathematically verified and a criterion of J-C type equation is provided. The mechanical experimental data are assembled as 2D matrix (two variables of strain and strain-rate or temperature) or 3D array (three variables of strain, strain-rate and temperature). With low-rank approximation method, i.e. singular value decomposition (SVD) for 2D matrix and CANDECOMP/PARAFAC (CP) for 3D array, the experimental data matrix/array (N) can be decomposed into a heavily-weighted matrix/array (N1) and several matrices/arrays with reduced weights (N2, N3, …). The criterion of J-C type equation is that N1 can approximately represent N with acceptable error. Otherwise, the use of J-C type equation is invalid. A method to describe the dynamic flow stress based on discrete experimental dataset is further proposed based on the SVD/CP method. The advantages of coupled and decoupled flow stress equations and the problems associated with the original J-C equation are discussed.

High-order tensor low-rank approximation with application in color video recovery

Video denoising is an elementary but critical task in computer vision and has been widely studied in recent years. However, the existing denoising methods have inevitable drawbacks: some need to predefine rank, some ignore the local information, and most cannot deal with higher-order data. To overcome these shortcomings, we consider two high-order tensor low-rank approximation methods, aiming to achieve color video denoising in a mixed noise environment. First, we establish a high-order tensor framework. Based on this framework, high-order tensor robust principal component analysis (HRPCA) is proposed. Although HRPCA is capable of processing high-order data, there is still a loss of recovery details. Then, we develop another method called high-order tensor low-rank approximation with total variation regularization (HTV). In particular, the TV consists of frontal total variation (FTV) and global total variation (GTV), thus extending the HTV into HFTV and HGTV, respectively. Extensive experimental results of color videos show that the HRPCA and HTV are more efficient in dealing with denoising problems than other state-of-the-art methods.

Multi-Fidelity Low-Rank Approximations for Uncertainty Quantification of a Supersonic Aircraft Design

Uncertainty quantification has proven to be an indispensable study for enhancing reliability and robustness of engineering systems in the early design phase. Single and multi-fidelity surrogate modelling methods have been used to replace the expensive high fidelity analyses which must be repeated many times for uncertainty quantification. However, since the number of analyses required to build an accurate surrogate model increases exponentially with the number of random input variables, most surrogate modelling methods suffer from the curse of dimensionality. As an alternative approach, the Low-Rank Approximation method can be applied to high-dimensional uncertainty quantification studies with a low computational cost, where the number of coefficients for building the surrogate model increases only linearly with the number of random input variables. In this study, the Low-Rank Approximation method is implemented for multi-fidelity applications with additive and multiplicative correction approaches to make the high-dimensional uncertainty quantification analysis more efficient and accurate. The developed uncertainty quantification methodology is tested on supersonic aircraft design problems and its predictions are compared with the results of single- and multi-fidelity Polynomial Chaos Expansion and Monte Carlo methods. For the same computational cost, the Low-Rank Approximation method outperformed both in surrogate modeling and uncertainty quantification cases for all the benchmarks and real-world engineering problems addressed in the present study.

Accelerating a spatially varying aberration correction of holographic displays with low-rank approximation.

Correction of spatially varying aberrations in holographic displays often requires intractable computational loads. In this Letter, we introduce a low-rank approximation method that decomposes sub-holograms into a small number of modes, thereby reformulating the computer-generated hologram calculation into a summation of a few convolutions. The low-rank approximation is carried out with two different algorithms: the Karhunen-Loeve transform as the optimum solution with respect to the mean-squared error criterion and a novel, to the best of our knowledge, optimization method to provide uniform image quality over the entire field of view. The proposed method is two orders of magnitude faster than the conventional point-wise integration method in our experimental setup, with comparable image quality.

A Block Bidiagonalization Method for Fixed-Accuracy Low-Rank Matrix Approximation

A Block Bidiagonalization Method for Fixed-Accuracy Low-Rank Matrix Approximation

Multi-scale low-rank approximation method for image denoising

Multi-scale low-rank approximation method for image denoising

Nonconvex Tensor Low-Rank Approximation for Infrared Small Target Detection

Infrared small target detection is an important fundamental task in the infrared system. Therefore, many infrared small target detection methods have been proposed, in which the low-rank model has been used as a powerful tool. However, most low-rank-based methods assign the same weights for different singular values, which will lead to inaccurate background estimation. Considering that different singular values have different importance and should be treated discriminatively, in this paper, we propose a non-convex tensor low-rank approximation (NTLA) method for infrared small target detection. In our method, NTLA regularization adaptively assigns different weights to different singular values for accurate background estimation. Based on the proposed NTLA, we propose asymmetric spatial-temporal total variation (ASTTV) regularization to achieve more accurate background estimation in complex scenes. Compared with the traditional total variation approach, ASTTV exploits different smoothness intensities for spatial and temporal regularization. We design an efficient algorithm to find the optimal solution of our method. Compared with some state-of-the-art methods, the proposed method achieves an improvement in terms of various evaluation metrics. Extensive experimental results in various complex scenes demonstrate that our method has strong robustness and low false-alarm rate. Code is available at https://github.com/LiuTing20a/ASTTV-NTLA.

Hyperspectral Image Denoising Using Factor Group Sparsity-Regularized Nonconvex Low-Rank Approximation

Hyperspectral image (HSI) mixed noise removal is a fundamental problem and an important preprocessing step in remote sensing fields. The low-rank approximation-based methods have been verified effective to encode the global spectral correlation for HSI denoising. However, due to the large scale and complexity of real HSI, previous low-rank HSI denoising techniques encounter several problems, including coarse rank approximation (such as nuclear norm), the high computational cost of singular value decomposition (SVD) (such as Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm), and adaptive rank selection (such as low-rank factorization). In this article, two novel factor group sparsity-regularized nonconvex low-rank approximation (FGSLR) methods are introduced for HSI denoising, which can simultaneously overcome the mentioned issues of previous works. The FGSLR methods capture the spectral correlation via low-rank factorization, meanwhile utilizing factor group sparsity regularization to further enhance the low-rank property. It is SVD-free and robust to rank selection. Moreover, FGSLR is equivalent to Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm approximation ( <xref ref-type="theorem" rid="theorem1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Theorem 1</xref> ), and thus FGSLR is tighter than the nuclear norm in terms of rank approximation. To preserve the spatial information of HSI in the denoising process, the total variation regularization is also incorporated into the proposed FGSLR models. Specifically, the proximal alternating minimization is designed to solve the proposed FGSLR models. Experimental results have demonstrated that the proposed FGSLR methods significantly outperform existing low-rank approximation-based HSI denoising methods.

Exploring Nonlocal Group Sparsity Under Transform Learning for Hyperspectral Image Denoising

Hyperspectral image (HSI) denoising has been regarded as an effective and economical preprocessing step in data subsequent applications. Recent nonlocal low-rank approximation on each full band patch group has demonstrated their superiority for HSI denoising. These methods, however, directly design the low-rank regularization to the grouped patch image itself (i.e., original domain), which ignores the spatial information of the grouped patch image and cannot explores the potential structure. To address these issues, this paper proposes a nonlocal group sparsifying transform learning method (dubbed TLNLGS) for HSI denoising. Motivated by the global spectral correlation in the HSI, we firstly impose a certain low-dimensional subspace hypothesis over the HSI to prevent the heavy computation burden with the spectral band increases, and then explore a discriminatively intrinsic nonlocal group sparse prior of the reduced image by transform model. The learned group sparse prior can not only excavate the nonlocal self-similarity as recent nonlocal low-rank approximation methods but also preserve the local spatial smooth structure of the image. Moreover, compared with the fixed transform domain (e.g., gradient and discrete cosine transformation domains), the transform learning scheme can improve the sparse representation ability. An efficient block coordinate descent (BCD) algorithm is developed to solve the proposed model. Extensive experiments, including simulated and real HSI datasets, indicate the superiority of the proposed TLNLGS method over the state-of-the-art HSI denoising approaches.

Image denoising based on global image similar patches searching and HOSVD to patches tensor

Nonlocal self-similarity of natural image is an essential property for image processing. But how to measure the similarity between different patches and how to better exploit the similarity of patches are two crucial problems for image denoising. In this article, we establish a novel image denoising method based on a global image similar patches searching method and the similar patches tensor high-order singular value decomposition theory. Particularly, in order to find the reasonable similar patches to a reference, a variant of Gaussian mixture model (GMM) global similar patches searching method is proposed, and the graph process unit-based GMM model training method is performed to speed up the training process. Furthermore, the k-means and the local inter-class searching are used to improve on the similarity measure between patches. To better exploit the similarity of patches to image denoising, we rearrange similar patches to a tensor for each reference, and an iterative adaptive weighted tensor low-rank approximation method is established to perform image denoising. Experimental results clearly show that the proposed method is comparable to many recent denoising algorithms from the PNSR, SSIM and NCC viewpoint. For higher noise level (e.g., sigma=50), the performance of our proposed algorithm evinces a 0.06-unit, 0.05-unit and 0.12-unit improvement in the PNSR, SSIM and NCC score, respectively, when compared to state-of-the-art.