Rank Approximation Research Articles

Pseudo-skeleton approximations with better accuracy estimates

We propose a priori accuracy estimates for low-rank matrix approximations that use just a small number of the rows and columns. This number is greater than the approximation rank, unlike the existing methods of pseudo-skeleton approximation. But the estimates are more accurate than previously known ones. This paper generalizes the results of [12,13].

Multiple structure recovery via robust preference analysis

This paper address the extraction of multiple models from outlier-contaminated data by exploiting preference analysis and low rank approximation. First points are represented in the preference space, then Robust PCA (Principal Component Analysis) and Symmetric NMF (Non negative Matrix Factorization) are used to break the multi-model fitting problem into many single-model problems, which in turn are tackled with an approach inspired to MSAC (M-estimator SAmple Consensus) coupled with a model-specific scale estimate. Experimental validation on public, real data-sets demonstrates that our method compares favorably with the state of the art.

Spatial Random Sampling: A Structure-Preserving Data Sketching Tool

Random column sampling is not guaranteed to yield data sketches that preserve the underlying structures of the data and may not sample sufficiently from less-populated data clusters. Also, adaptive sampling can often provide accurate low rank approximations, yet may fall short of producing descriptive data sketches, especially when the cluster centers are linearly dependent. Motivated by that, this paper introduces a novel randomized column sampling tool dubbed Spatial Random Sampling (SRS), in which data points are sampled based on their proximity to randomly sampled points on the unit sphere. The most compelling feature of SRS is that the corresponding probability of sampling from a given data cluster is proportional to the surface area the cluster occupies on the unit sphere, independently from the size of the cluster population. Although it is fully randomized, SRS is shown to provide descriptive and balanced data representations. The proposed idea addresses a pressing need in data science and holds potential to inspire many novel approaches for analysis of big data.

Adjustable Robust Singular Value Decomposition: Design, Analysis and Application to Finance

The Singular Value Decomposition (SVD) is a fundamental algorithm used to understand the structure of data by providing insight into the relationship between the row and column factors. SVD aims to approximate a rectangular data matrix, given some rank restriction, especially lower rank approximation. In practical data analysis, however, outliers and missing values maybe exist that restrict the performance of SVD, because SVD is a least squares method that is sensitive to errors in the data matrix. This paper proposes a robust SVD algorithm by applying an adjustable robust estimator. Through adjusting the tuning parameter in the algorithm, the method can be both robust and efficient. Moreover, a sequential robust SVD algorithm is proposed in order to decrease the computation volume in sequential and streaming data. The advantages of the proposed algorithms are proved with a financial application.

Hyperspectral Image Classification Using Graph Clustering Methods

Hyperspectral imagery is a challenging modality due to the dimension of the pixels which can range from hundreds to over a thousand frequencies depending on the sensor. Most methods in the literature reduce the dimension of the data using a method such as principal component analysis, however this procedure can lose information. More recently methods have been developed to address classification of large datasets in high dimensions. This paper presents two classes of graph-based classification methods for hyperspectral imagery. Using the full dimensionality of the data, we consider a similarity graph based on pairwise comparisons of pixels. The graph is segmented using a pseudospectral algorithm for graph clustering that requires information about the eigenfunctions of the graph Laplacian but does not require computation of the full graph. We develop a parallel version of the Nyström extension method to randomly sample the graph to construct a low rank approximation of the graph Laplacian. With at most a few hundred eigenfunctions, we can implement the clustering method designed to solve a variational problem for a graph-cut-based semi-supervised or unsupervised classification problem. We implement OpenMP directive-based parallelism in our algorithms and show performance improvement and strong, almost ideal, scaling behavior. The method can handle very large datasets including a video sequence with over a million pixels, and the problem of segmenting a data set into a pre-determined number of classes.

Low rank approximation methods for MR fingerprinting with large scale dictionaries

This work proposes new low rank approximation approaches with significant memory savings for large scale MR fingerprinting (MRF) problems. We introduce a compressed MRF with randomized singular value decomposition method to significantly reduce the memory requirement for calculating a low rank approximation of large sized MRF dictionaries. We further relax this requirement by exploiting the structures of MRF dictionaries in the randomized singular value decomposition space and fitting them to low-degree polynomials to generate high resolution MRF parameter maps. In vivo 1.5T and 3T brain scan data are used to validate the approaches. T1 , T2 , and off-resonance maps are in good agreement with that of the standard MRF approach. Moreover, the memory savings is up to 1000 times for the MRF-fast imaging with steady-state precession sequence and more than 15 times for the MRF-balanced, steady-state free precession sequence. The proposed compressed MRF with randomized singular value decomposition and dictionary fitting methods are memory efficient low rank approximation methods, which can benefit the usage of MRF in clinical settings. They also have great potentials in large scale MRF problems, such as problems considering multi-component MRF parameters or high resolution in the parameter space. Magn Reson Med 79:2392-2400, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

Nested equivalence source approximation with adaptive group size for multiscale simulations

A nested equivalence source approximation (NESA) of the electric field integral equation with adaptive octree is explored for multiscale problems in this paper. The NESA low rank approximation formulation previously for far coupling groups with uniform size is derived for coupling groups with adaptive size, while preserves the kernel free and multiscale property. With the proposed adaptive group decomposition, reasonable separation of near and far region can be obtained. Numerical tests of the conformal and non-conformal multiscale electromagnetic simulation to show the validity of the adaptive NESA.

Novel alternating update method for low rank approximation of structured matrices

This work is devoted to designing a unified alternating update method for solving a class of structured low rank approximations under the convex and unitarily invariant norm. By the aid of the variational inequality and monotone operator, the proposed method is proved to converge to the solution point of an equivalent variational inequality with a worst-case O(1/t) convergence rate in a nonergodic sense. We also analyze that the involved subproblems under the Frobenius norm are respectively equivalent to the structured least-squares problem and low rank least-squares problem, where the explicit solutions to some special cases are derived. In order to investigate the efficiency of the proposed method, several examples in system identification are tested to validate that the proposed method can outperform some state-of-the-art methods.

Trimmed autocalibrating k-space estimation based on structured matrix completion

PurposeParallel imaging allows the reconstruction of undersampled data from multiple coils. This provides a means to reject and regenerate corrupt data (e.g. from motion artefact). The purpose of this work is to approach this problem using the SAKE parallel imaging method. Theory and methodsParallel imaging methods typically require calibration by fully sampling the center of k-space. This is a challenge in the presence of corrupted data, since the calibration data may be corrupted which leads to an errors-in-variables problem that cannot be solved by least squares or even iteratively reweighted least squares. The SAKE method, based on matrix completion and structured low rank approximation, was modified to detect and trim these errors from the data. ResultsSimulated and actual corrupted datasets were reconstructed with SAKE, the proposed approach and a more standard reconstruction method (based on solving a linear equation) with a data rejection criterion. The proposed approach was found to reduce artefacts considerably in comparison to the other two methods. ConclusionSAKE with data trimming improves on previous methods for reconstructing images from grossly corrupted data.

Shot boundary detection via adaptive low rank and svd-updating

Usually considered as the first step in content-based video retrieval, shot boundary detection (SBD) is crucial to subsequent high-level applications like video summarization. The paper proposes an accurate video shot boundary detection method based on singular value decomposition (SVD), through the use of different mathematical theorems and interpretations. In our contribution, adaptive feature extraction is performed using the Frobenius norm of low rank approximation matrices. Each frame will be mapped into k˜-dimensional vector in the singular space according to each segment. The classification of continuity values is then achieved via double thresholding technique for detecting the hard cuts. For gradual transitions detection, the folding-in technique, known as SVD-updating, is used for the first time in video shot boundary detection. This allows for accurate detection in reduced computational time, as it is no longer necessary to recalculate the whole SVD decomposition for segment correction. Experimental results on four different datasets, including the TRECVid 2001, 2002 and 2005 SBD tasks, show the effectiveness of our detector, which outperforms recent related methods in detecting both hard and gradual transitions.

TECHNIQUE TO RECONSTRUCT BAND 6 REFLECTANCE INFORMATION OF AQUA MODIS

Remote sensing data could experience damage due to sensor failure or atmospheric condition. Reconstruction technique to retrieve the missing information had been widely developed in the past few years. This writing aimed to provide a technique to recover reflectance information of Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) Band 6. Since Band 6 Aqua MODIS experienced sensor failure, lots of information would be missing. There were three kinds of methods used in repairing such damage. Two of which were categorized as spatial-based methods, i.e. NaN interpolation method and tensor completion method. Whereas, another method was a spectral-based one. NaN was an interpolation method to reconstruct missing value; while tensor completion method utilized low rank approximation, and spectral method used correlation between Band 6 and Band 7 which had near wavelength. Implementation of these methods was resulted in reconstruction of Aqua Modis Band 6 data which was damaged due to detector disfunction on Aqua Satellite. Peak Signal to Noise Ratio (PSNR) value of this method was 41 dB, meaning that reconstruction technique provided positive impacts for data improvement.

Patch-Based Image Inpainting via Two-Stage Low Rank Approximation.

To recover the corrupted pixels, traditional inpainting methods based on low-rank priors generally need to solve a convex optimization problem by an iterative singular value shrinkage algorithm. In this paper, we propose a simple method for image inpainting using low rank approximation, which avoids the time-consuming iterative shrinkage. Specifically, if similar patches of a corrupted image are identified and reshaped as vectors, then a patch matrix can be constructed by collecting these similar patch-vectors. Due to its columns being highly linearly correlated, this patch matrix is low-rank. Instead of using an iterative singular value shrinkage scheme, the proposed method utilizes low rank approximation with truncated singular values to derive a closed-form estimate for each patch matrix. Depending upon an observation that there exists a distinct gap in the singular spectrum of patch matrix, the rank of each patch matrix is empirically determined by a heuristic procedure. Inspired by the inpainting algorithms with component decomposition, a two-stage low rank approximation (TSLRA) scheme is designed to recover image structures and refine texture details of corrupted images. Experimental results on various inpainting tasks demonstrate that the proposed method is comparable and even superior to some state-of-the-art inpainting algorithms.

Convex envelopes for fixed rank approximation

A convex envelope for the problem of finding the best approximation to a given matrix with a prescribed rank is constructed. This convex envelope allows the usage of traditional optimization techniques when additional constraints are added to the finite rank approximation problem. Expression for the dependence of the convex envelope on the singular values of the given matrix is derived and global minimization properties are derived. The corresponding proximity operator is also studied.

Tensor Methods for Solving Symmetric $${\mathcal {M}}$$ M -tensor Systems

Tensor systems involving tensor-vector products (or polynomial systems) are considered. We solve these tensor systems, especially focusing on symmetric $${\mathcal {M}}$$ -tensor systems, by some tensor methods. A new tensor method is proposed based on the rank-1 approximation of the coefficient tensor. Numerical examples show that the tensor methods could be more efficient than the Newton method for some $${\mathcal {M}}$$ -tensor systems.

Complexity-Aware Gabor Filter Bank Architecture Using Principal Component Analysis

A Gabor filter quantifies signal characteristics with the optimal resolution in both time and frequency domains, and it has been widely used in image processing and computer vision applications since the extracted features which mimic human vision to provide good capability for differentiation. We usually require many Gabor filters to extract multi-scale and multi-orientation features; and hence, it introduces heavy computational load. Based on Algorithm/Architecture Co-exploration (AAC), this paper proposes a computationally efficient method of Gabor filter bank through Principal Component Analysis (PCA) analysis and low rank approximation. PCA projects filter coefficients onto a more symmetric vector space, then, the computation load is reduced by sharing coefficients; on the other hand, while trading off between algorithmic accuracy and computational efficiency, low rank approximation approximates Gabor filter bank to reduce computational load via removing less important components measured by PCA. Furthermore, we also propose an efficient approximation evaluation method to measure potential loss in trading off algorithmic performance and computational load; hence, designers can adaptively select various approximation levels based on requirements. In a case study, we used Gabor filter bank to evaluate the proposed method, and we used AAC design paradigm for entire design flow. Experimental results showed that proposed method, which reduces ∼78.5% multiplications and ∼75.1% additions as compared to conventional approach, has the lowest complexity and the fastest computational speed in comparison with related works; furthermore, while deploying low rank approximation, the proposed method reduced ∼26% multiplications and ∼18% additions in comparison with the proposed method without approximation and it achieved comparative algorithmic performance.

A non-convex tensor rank approximation for tensor completion

• A non-convex approximation for tensors rank is proposed for low-rank tensor completion problem. • Develop an efficient ADMM algorithm for solving the proposed method. • The method can efficiently estimate the missing information, especially for the low sampling rates. Low-rankness has been widely exploited for the tensor completion problem. Recent advances have suggested that the tensor nuclear norm often leads to a promising approximation for the tensor rank. It treats the singular values equally to pursue the convexity of the objective function, while the singular values for the practical images have clear physical meanings with different importance and should be treated differently. In this paper, we propose a non-convex logDet function as a smooth approximation for tensor rank instead of the convex tensor nuclear norm and introduce it into the low-rank tensor completion problem. An alternating direction method of multiplier (ADMM)-based method is developed to solve the problem. Experimental results have shown that the proposed method can significantly outperform existing state-of-the-art nuclear norm-based methods for tensor completion.

Augmented Lagrangian alternating direction method for low-rank minimization via non-convex approximation

This paper concerns the low-rank minimization problems which consist of finding a matrix of minimum rank subject to linear constraints. Many existing approaches, which used the nuclear norm as a convex surrogate of the rank function, usually result in a suboptimal solution. To seek a tighter rank approximation, we develop a non-convex surrogate to approximate the rank function based on the Laplace function. An iterative algorithm based on the augmented Lagrangian multipliers method is developed. Empirical studies for practical applications including robust principal component analysis and low-rank representation demonstrate that our proposed algorithm outperforms many other state-of-the-art convex and non-convex methods developed recently in the literature.

Fixed-point algorithms for frequency estimation and structured low rank approximation

We develop fixed-point algorithms for the approximation of structured matrices with rank penalties. In particular we use these fixed-point algorithms for making approximations by sums of exponentials, i.e., frequency estimation. For the basic formulation of the fixed-point algorithm we show that it converges to the solution of a related minimization problem, namely the one obtained by replacing the original objective function with its convex envelope and keeping the structured matrix constraint unchanged. It often happens that this solution agrees with the solution to the original minimization problem, and we provide a simple criterion for when this is true. We also provide more general fixed-point algorithms that can be used to treat the problems of making weighted approximations by sums of exponentials given equally or unequally spaced sampling. We apply the method to the case of missing data, although the above mentioned convergence results do not hold in this case. However, it turns out that the method often gives perfect reconstruction (up to machine precision) in such cases. We also discuss multidimensional extensions, and illustrate how the proposed algorithms can be used to recover sums of exponentials in several variables, but when samples are available only along a curve.

Fast Estimation of Approximate Matrix Ranks Using Spectral Densities.

Many machine learning and data-related applications require the knowledge of approximate ranks of large data matrices at hand. This letter presents two computationally inexpensive techniques to estimate the approximate ranks of such matrices. These techniques exploit approximate spectral densities, popular in physics, which are probability density distributions that measure the likelihood of finding eigenvalues of the matrix at a given point on the real line. Integrating the spectral density over an interval gives the eigenvalue count of the matrix in that interval. Therefore, the rank can be approximated by integrating the spectral density over a carefully selected interval. Two different approaches are discussed to estimate the approximate rank, one based on Chebyshev polynomials and the other based on the Lanczos algorithm. In order to obtain the appropriate interval, it is necessary to locate a gap between the eigenvalues that correspond to noise and the relevant eigenvalues that contribute to the matrix rank. A method for locating this gap and selecting the interval of integration is proposed based on the plot of the spectral density. Numerical experiments illustrate the performance of these techniques on matrices from typical applications.

Gradient-based low rank method and its application in image inpainting

Conventional inpainting methods generally apply textures that are most similar to the areas around the missing region or use large image database. Recently, low rank property of data shows that the non-convex optimization decreases measurements. In this paper, we propose a new image prior, which implies the low rank prior knowledge of image gradients. The proposed detail-preserving image inpainting algorithm adopts the low rank regularization to gradient similarity minimization, termed gradient-based low rank approximation (Grad-LR), namely that we employ the low rank constraints in the horizontal and vertical gradients of the image and then reconstruct the desired image using the adaptive iterative singular-value thresholding of both derivatives. In the method, by incorporating the spatially adaptive iterative singular-value thresholding (SAIST) to optimize our gradient scheme, the deterministic annealing iterates the procedure efficiently. As a result, the strength of the algorithm is obvious when filling large missing region. Experimental results consistently demonstrate that the proposed algorithm works well for both structural and texture images and outperforms other techniques, in terms of both objective and subjective performance measures.