Low-rank Matrix Approximation Research Articles

The spectral spread of Hermitian matrices

Let A be an n×n complex Hermitian matrix and let λ(A)=(λ1,…,λn)∈Rn denote the eigenvalues of A, counting multiplicities and arranged in non-increasing order. Motivated by problems arising in the theory of low rank matrix approximation, we study the spectral spread of A, denoted Spr+(A), given by Spr+(A)=(λ1−λn,λ2−λn−1,…,λk−λn−k+1)∈Rk, where k=[n/2] (integer part). The spectral spread is a vector-valued measure of dispersion of the spectrum of A, that allows one to obtain several submajorization inequalities. In the present work we obtain inequalities that are related to Tao's inequality for anti-diagonal blocks of positive semidefinite matrices, Zhan's inequalities for the singular values of differences of positive semidefinite matrices, extremal properties of direct rotations between subspaces, generalized commutators and distances between matrices in the unitary orbit of a Hermitian matrix.

Signed Network Representation by Preserving Multi-order Signed Proximity

Signed network representation is a key problem for signed network data. Previous studies have shown that by preserving multi-order signed proximity (SP), expressive node representations can be learned. However, multi-order SP cannot be perfectly encoded using limited samples extracted from random walks, which reduces effectiveness. To perfectly encode multi-order SP, we have innovatively integrated the informativeness of infinite samples to construct high-level summaries of multi-order SP without explicit sampling. Based on these summaries, we propose a method called SPMF, in which node representations are obtained using low-rank matrix approximation. Furthermore, we theoretically investigate the rationality of SPMF by examining its relationship with a powerful representation learning architecture. In sign inference and link prediction tasks with several real-world datasets, SPMF is empirically competitive compared with state-of-the-art methods. Additionally, two tricks are designed for improving the scalability of SPMF. One trick aims to filter out less informative summaries, and another one is inspired by kernel techniques. Both tricks empirically improve scalability while preserving effective performance. The code for our methods is publicly available.

Low Rank Pure Quaternion Approximation for Pure Quaternion Matrices

Quaternion matrices are employed successfully in many color image processing applications. In particular, a pure quaternion matrix can be used to represent red, green, and blue channels of color images. A low-rank approximation for a pure quaternion matrix can be obtained by using the quaternion singular value decomposition. However, this approximation is not optimal in the sense that the resulting low-rank approximation matrix may not be pure quaternion, i.e., the low-rank matrix contains a real component which is not useful for the representation of a color image. The main contribution of this paper is to find an optimal rank-$r$ pure quaternion matrix approximation for a pure quaternion matrix (a color image). Our idea is to use a projection on a low-rank quaternion matrix manifold and a projection on a quaternion matrix with zero real component, and develop an alternating projections algorithm to find such optimal low-rank pure quaternion matrix approximation. The convergence of the projection algorithm can be established by showing that the low-rank quaternion matrix manifold and the zero real component quaternion matrix manifold has a nontrivial intersection point. Numerical examples on synthetic pure quaternion matrices and color images are presented to illustrate the projection algorithm can find optimal low-rank pure quaternion approximation for pure quaternion matrices or color images.

On Low-Rank Hankel Matrix Denoising

The low-complexity assumption in linear systems can often be expressed as rank deficiency in data matrices with generalized Hankel structure. This makes it possible to denoise the data by estimating the underlying structured low-rank matrix. However, standard low-rank approximation approaches are not guaranteed to perform well in estimating the noise-free matrix. In this paper, recent results in matrix denoising by singular value shrinkage are reviewed. A novel approach is proposed to solve the low-rank Hankel matrix denoising problem by using an iterative algorithm in structured low-rank approximation modified with data-driven singular value shrinkage. It is shown numerically in both the input-output trajectory denoising and the impulse response denoising problems, that the proposed method performs the best in terms of estimating the noise-free matrix among existing algorithms of low-rank matrix approximation and denoising.

Measuring Heterogeneous Thermal Patterns in Infrared-Based Diagnostic Systems Using Sparse Low-Rank Matrix Approximation: Comparative Study

Active and passive thermography are two efficient techniques extensively used to measure heterogeneous thermal patterns leading to subsurface defects for diagnostic evaluations. This study conducts a comparative analysis on low-rank matrix approximation methods in thermography with applications of semi-, convex-, and sparse- non-negative matrix factorization (NMF) methods for detecting subsurface thermal patterns. These methods inherit the advantages of principal component thermography (PCT) and sparse PCT, whereas tackle negative bases in sparse PCT with non-negative constraints, and exhibit clustering property in processing data. The practicality and efficiency of these methods are demonstrated by the experimental results for subsurface defect detection in three specimens (for different depth and size defects) and preserving thermal heterogeneity for distinguishing breast abnormality in breast cancer screening dataset (accuracy of 74.1%, 75.8%, and 77.8%).

Accelerating patch-based low-rank image restoration using kd-forest and Lanczos approximation

Patch-based low-rank approximation (PLRA) via truncated singular value decomposition is a powerful and effective tool for recovering the underlying low-rank structure in images. Generally, it first performs an approximate nearest neighbors (ANN) search algorithm to group similar patches into a collection of matrices with reshaping them as vectors. The inherent correlation among similar patches makes these matrices have a low-rank structure. Then the singular value decomposition (SVD) is used to derive a low-rank approximation of each matrix by truncating small singular values. However, the conventional implementation of patch-based low-rank image restoration suffers from high computational cost of the ANN search and full SVD. To address this limitation, we propose a fast approximation method that accelerates the computation of PLRA using multiple kd-trees and Lanczos approximation. The basic idea of this method is to exploit an index kd-tree built from patch samples of the observed image and several small kd-trees built from overlapping regions of the image to accelerate the search for similar patches, and apply the Lanczos bidiagonalization procedure to obtain a fast low-rank approximation of patch matrix without computing the full SVD. Experimental results on image denoising and inpainting tasks demonstrate the efficiency and accuracy of our method.

Detecting Vasodilation as Potential Diagnostic Biomarker in Breast Cancer Using Deep Learning-Driven Thermomics.

Breast cancer is the most common cancer in women. Early diagnosis improves outcome and survival, which is the cornerstone of breast cancer treatment. Thermography has been utilized as a complementary diagnostic technique in breast cancer detection. Artificial intelligence (AI) has the capacity to capture and analyze the entire concealed information in thermography. In this study, we propose a method to potentially detect the immunohistochemical response to breast cancer by finding thermal heterogeneous patterns in the targeted area. In this study for breast cancer screening 208 subjects participated and normal and abnormal (diagnosed by mammography or clinical diagnosis) conditions were analyzed. High-dimensional deep thermomic features were extracted from the ResNet-50 pre-trained model from low-rank thermal matrix approximation using sparse principal component analysis. Then, a sparse deep autoencoder designed and trained for such data decreases the dimensionality to 16 latent space thermomic features. A random forest model was used to classify the participants. The proposed method preserves thermal heterogeneity, which leads to successful classification between normal and abnormal subjects with an accuracy of 78.16% (73.3–81.07%). By non-invasively capturing a thermal map of the entire tumor, the proposed method can assist in screening and diagnosing this malignancy. These thermal signatures may preoperatively stratify the patients for personalized treatment planning and potentially monitor the patients during treatment.

Low Rank Matrix Approximation for 3D Geometry Filtering.

We propose a robust normal estimation method for both point clouds and meshes using a low rank matrix approximation algorithm. First, we compute a local isotropic structure for each point and find its similar, non-local structures that we organize into a matrix. We then show that a low rank matrix approximation algorithm can robustly estimate normals for both point clouds and meshes. Furthermore, we provide a new filtering method for point cloud data to smooth the position data to fit the estimated normals. We show the applications of our method to point cloud filtering, point set upsampling, surface reconstruction, mesh denoising, and geometric texture removal. Our experiments show that our method generally achieves better results than existing methods.

Enhanced low-rank constraint for temporal subspace clustering and its acceleration scheme

Inspired by the temporal subspace clustering (TSC) method and low-rank matrix approximation constraint, a new model is proposed termed as temporal plus low-rank subspace clustering (TLRSC) by utilizing both the local and global structural information. On one hand, to solve the drawback that the nuclear norm-based constraint usually results in a suboptimal solution, we incorporate certain nonconvex surrogates into our model, which approximates the low-rank constraint closely and holds the potential for the convexity of the whole cost function. On the other hand, to ensure fast convergence, we propose an efficient iteratively reweighted singular value minimization (IRSVD) algorithm under the majorization-minimization framework. Moreover, we show that for the weighted low-rank constraint, a cutoff can be derived to automatically threshold the singular values computed from the proximal operator. This guarantees the thresholding operation can be reduced to that of two smaller matrices. Accordingly, an efficient singular value thresholding scheme is proposed for acceleration. Comprehensive experiments are conducted on several public available datasets for quantitative evaluation. Results demonstrate the efficacy and efficiency of TLRSC compared with several state-of-the-art methods.

Continuous-time Laguerre-based subspace identification utilising nuclear norm minimisation

This paper presents a continuous-time Laguerre-based subspace identification method utilising nuclear norm minimisation. The input–output matrix equation of the systems is deduced by a bank of Laguerre filters in all-pass domain, which can deal with the un-equidistant data. Nuclear norm minimisation is adopted, instead of the truncation of dominant singular values, to obtain low-rank matrix approximations which can easily obtain the system order. Furthermore, the optimisation problem is solved by the alternating direction method of multipliers. Simulation results are provided to show the effectiveness of the proposed method.

Skin effect and proximity effect analysis of stranded conductor based on mixed order MoM with adaptive cross approximation algorithm

In this paper, an efficient adaptive cross approximation (ACA) method is employed for the skin effect and proximity effect analysis. Multi-layer stranded cables with multiple strands and different materials are used for analysis to approximate the effect. The skin and proximity matrices are computed by the ACA low-rank matrix approximation technique. The ACA method can provide better solutions through different iterations. The ACA algorithm is very stable and useful for calculating a low rank matrix approximation, because the adaptive mesh refinement for generating optimal computation. The ACA approach provides the less computational memory and less CPU time. Compare the calculated results with standard work results (such as reference) to verify the effectiveness of the adaptive cross-approximation algorithm.

Boolean decomposition of binary matrices using a post-nonlinear mixture approach

We introduce a novel binary matrix factorization (BMF) approach based on a post-nonlinear mixture model. Unlike the existing BMF methods, which are based on the classical matrix product, the proposed mixture model is equivalent to the Boolean matrix factorization model when the entries of the factor matrices are exactly binary. Consequently, our approach yields interpretable results in the case of overlapping sources and more accurate low-rank binary matrix approximations compared to the state-of-the-art. We propose a simple yet efficient algorithm for solving the proposed BMF problem based on multiplicative update rules. In addition, we provide for the first time in the binary data literature, a necessary and sufficient condition for the uniqueness of the Boolean matrix factorization, as well as several other uniqueness results. The interest of this new approach is illustrated in numerical simulation and on real datasets.

Low-rank matrix approximations over canonical subspaces

In this paper we derive closed form expressions for the nearest rank-\(k\) matrix on canonical subspaces. 
 
 We start by studying three kinds of subspaces. Let \(X\) and \(Y\) be a pair of given matrices. The first subspace contains all the \(m\times n\) matrices \(A\) that satisfy \(AX=O\). The second subspace contains all the \(m \times n\) matrices \(A\) that satisfy \(Y^TA = O\), while the matrices in the third subspace satisfy both \(AX =O\) and \(Y^TA = 0\).
 
 The second part of the paper considers a subspace that contains all the symmetric matrices \(S\) that satisfy \(SX =O\). In this case, in addition to the nearest rank-\(k\) matrix we also provide the nearest rank-\(k\) positive approximant on that subspace. 
 
 A further insight is gained by showing that the related cones of positive semidefinite matrices, and negative semidefinite matrices, constitute a polar decomposition of this subspace.
 The paper ends with two examples of applications. The first one regards the problem of computing the nearest rank-\(k\) centered matrix, and adds new insight into the PCA of a matrix.
 The second application comes from the field of Euclidean distance matrices. The new results on low-rank positive approximants are used to derive an explicit expression for the nearest source matrix. This opens a direct way for computing the related positions matrix.

Weighted Nuclear Norm Minimization-Based Regularization Method for Image Restoration

Regularization methods have been substantially applied in image restoration due to the ill-posedness of the image restoration problem. Different assumptions or priors on images are applied in the construction of image regularization methods. In recent years, matrix low-rank approximation has been successfully introduced in the image denoising problem and significant denoising effects have been achieved. Low-rank matrix minimization is an NP-hard problem and it is often replaced with the matrix’s weighted nuclear norm minimization (WNNM). The assumption that an image contains an extensive amount of self-similarity is the basis for the construction of the matrix low-rank approximation-based image denoising method. In this paper, we develop a model for image restoration using the sum of block matching matrices’ weighted nuclear norm to be the regularization term in the cost function. An alternating iterative algorithm is designed to solve the proposed model and the convergence analyses of the algorithm are also presented. Numerical experiments show that the proposed method can recover the images much better than the existing regularization methods in terms of both recovered quantities and visual qualities.

A Priori-Based Subarray Selection Algorithm for DOA Estimation.

A finer direction-of-arrival (DOA) estimation result needs a large and dense array; it may, however, encounter the mutual coupling effect, which degrades the performance of DOA estimation. There is a new approach to mitigating this effect by using a nonuniform array to achieve DOA estimation. In this paper, we consider a priori DOA estimation, which is easily obtained from tracking results. The a priori DOA requires us to pay close attention to the high possibility of where the DOA will appear; then, a weight according to the prior probability distribution of DOA is added to each direction, which leads the sensing matrix of DOA estimation to be near low-rank. Thus, according to the low-rank matrix approximation theory, an optimal low-rank approximate matrix is obtained and an algorithm is proposed to select the elements of the original array according to right singular vectors of the approximate matrix. After that, the impacts of different weights are analyzed, and a mixed weight is presented which has flexibility for common use. Finally, a number of numerical simulations are carried out, and the results verify the effectiveness of the proposed methods.

A Novel Adaptive Mode Decomposition Method Based on Reassignment Vector and Its Application to Fault Diagnosis of Rolling Bearing

To solve the problem that the random distribution of noise in the time-frequency (TF) plane largely affects the readability of TF representations, a novel signal adaptive decomposition algorithm processed in TF domain, which provides adequate information about the time-varying instantaneous frequency, is presented in this paper. The theoretical basis of this algorithm is short-time Fourier transform (STFT). The research into the algorithm comprises two steps: the TF plane denoising takes sparse low-rank matrix estimation as a priority and then achieves signal decomposition based on reassignment vector (RV). A low-rank matrix approximation scheme, which exploits the sparse properties of the TF transformation coefficient and uses non-convex penalty, is put forward to obtain clean STFT. Then, a new approach called RV, which is different from the traditional mode decomposition methods such as Empirical Mode Decomposition (EMD), is used to estimate the characteristic curve corresponding to the TF ridges of the interested modes. Based on the classical reassignment method, RV has a solid theory foundation. Moreover, it can identify different signal components such as stationary signal, modulating signal and impulse characteristic. Combining the advantages of low-rank matrix approximation approach and those of RV defined in TF plane, a novel signal adaptive decomposition method is proposed in this paper to identify fault characteristics. To illustrate the effectiveness of the method, fault signals of rolling bearing under stationary condition and time-varying speed are respectively analyzed.

Randomized matrix approximation to enhance regularized projection schemes in inverse problems

The authors consider a randomized solution to ill-posed operator equations in Hilbert spaces. In contrast to statistical inverse problems, where randomness appears in the noise, here randomness arises in the low-rank matrix approximation of the forward operator, which results in using a Monte Carlo method to solve the inverse problems. In particular, this approach follows the paradigm of the study N. Halko et al 2011 SIAM Rev. 53 217–288, and hence regularization is performed based on the low-rank matrix approximation. Error bounds for the mean error are obtained which take into account solution smoothness and the inherent noise level. Based on the structure of the error decomposition the authors propose a novel algorithm which guarantees (on the mean) a prescribed error tolerance. Numerical simulations confirm the theoretical findings.

Interpolative Separable Density Fitting Decomposition for Accelerating Hartree-Fock Exchange Calculations within Numerical Atomic Orbitals.

The high cost associated with the evaluation of Hartree-Fock exchange (HFX) makes hybrid functionals computationally challenging for large systems. In this work, we present an efficient way to accelerate HFX calculations with numerical atomic basis sets. Our approach is based on the recently proposed interpolative separable density fitting (ISDF) decomposition to construct a low-rank approximation of the HFX matrix, which avoids explicit calculations of the electron repulsion integrals (ERIs) and significantly reduces the computational cost. We implement the ISDF method for hybrid functional (PBE0) calculations in the HONPAS package. We take benzene and polycyclic aromatic hydrocarbon molecules as examples and demonstrate that hybrid functionals with ISDF yield quite promising results at a significantly reduced computational cost. Especially, the ISDF approach reduces the total cost of the evaluating HFX matrix by nearly 2 orders of magnitude compared to conventional approaches of direct evaluation of ERIs.

Quick and Accurate False Data Detection in Mobile Crowd Sensing

The attacks, faults, and severe communication/system conditions in Mobile Crowd Sensing (MCS) make false data detection a critical problem. Observing the intrinsic low dimensionality of general monitoring data and the sparsity of false data, false data detection can be performed based on the separation of normal data and anomalies. Although the existing separation algorithm based on Direct Robust Matrix Factorization (DRMF) is proven to be effective, requiring iteratively performing Singular Value Decomposition (SVD) for low-rank matrix approximation would result in a prohibitively high accumulated computation cost when the data matrix is large. In this work, we observe the quick false data location feature from our empirical study of DRMF, based on which we propose an intelligent Light weight Low Rank and False Matrix Separation algorithm (LightLRFMS) that can reuse the previous result of the matrix decomposition to deduce the one for the current iteration step. Depending on the type of data corruption, random or successive/mass, we design two versions of LightLRFMS. From a theoretical perspective, we validate that LightLRFMS only requires one round of SVD computation and thus has very low computation cost. We have done extensive experiments using a PM 2.5 air condition trace and a road traffic trace. Our results demonstrate that LightLRFMS can achieve very good false data detection performance with the same highest detection accuracy as DRMF but with up to 20 times faster speed thanks to its lower computation cost.

Multi-band weighted [formula omitted] norm minimization for image denoising

Low rank matrix approximation (LRMA) has a wide range of applications in computer vision and has drawn much attention in recent years. A typical nuclear norm minimization (NNM) is often used to solve a LRMA, but this is likely to overshrink the rank components due to having the same threshold. To address this problem, we propose a flexible and precise model named multi-band weighted lp norm minimization (MBWPNM). We have reformulated it into nonconvex lp norm subproblems under certain weight conditions, and we solve these subproblems via a generalized soft-thresholding algorithm. We then adopt MBWPNM for image (grayscale, color and multispectral) denoising. The proposed MBWPNM not only guarantees a more accurate approximation with a Schatten p-norm in case of a change of noise levels, but it also considers prior knowledge, for which different rank components have varying importance. Extensive experiments on additive white Gaussian noise removal and realistic noise removal demonstrate that the proposed MBWPNM achieves better performance than several state-of-the-art algorithms.