Low Numerical Rank Research Articles

Effectiveness of Low-Numerical Rank Approximation to Image Compression in Wavelet Domain

In this paper, we report a modeling of approximation for images by finding numerical rank in the wavelet domain through singular value decomposition of approximation coefficients. Firstly, the digital image is transformed into the frequency domain. Then high-frequency sub-bands are quantized to zero. This is quite obvious in wavelet-based image compression. Simultaneously, the low-frequency sub-bands are compressing by using truncated singular value decomposition (TSVD) through a numerical rank. Finally, reconstruct the approximation matrix via inverse discrete wavelet transform with low computational intricacy. This mathematical model is more adequate for solving engineering problems arises in digital image processing such as the transmission of image (reducing the bandwidth size of a communication channel) and storage capacity (space saving). The simulation results on gray and color images show that there is a gain in: (i) the compression ratio with acceptable visual quality as per human vision system; (ii) balancing of performance measures over conventional SVD methods.

Highly Efficient Spatial-Temporal Correlation Basis for 5G IoT Networks.

One of the major concerns in 5G IoT networks is that most of the sensor nodes are powered through limited lifetime, which seriously affects the performance of the networks. In this article, Compressive sensing (CS) technique is used to decrease transmission cost in 5G IoT networks. Sparse basis is one of the important steps in the CS. However, most of the existing sparse basis-based method such as DCT (Discrete cosine transform) and DFT (Discrete Fourier Transform) basis do not capture data structure characteristics in the networks. They also do not take into consideration multi-resolution representations. In addition, some of sparse basis-driven methods exploit either spatial or temporal features, resulting in performance degradation of CS-based strategies. To address these challenging problems, we propose a novel spatial–temporal correlation basis algorithm (SCBA). Subsequently, an optimal basis algorithm (OBA) is provided considering greedy scoring criteria. To evaluate the efficiency of OBA, orthogonal wavelet basis algorithm (OWBA) by employing NS (Numerical Sparsity) and GI (Gini Index) sparse metrics is also presented. In addition, we discuss the complexity of the above three algorithms, and prove that OBA has low numerical rank. After experimental evaluation, we found that OBA is capable of the sparsest representing original signal compared to spatial, DCT, haar-1, haar-2, and rbio5.5. Furthermore, OBA has the low recovery error and the highest efficiency.

Projection-Based QLP Algorithm for Efficiently Computing Low-Rank Approximation of Matrices

Matrices with low numerical rank are omnipresent in many signal processing and data analysis applications. The pivoted QLP (p-QLP) algorithm constructs a highly accurate approximation to an input low-rank matrix. However, it is computationally prohibitive for large matrices. In this paper, we introduce a new algorithm termed Projection-based Partial QLP (PbP-QLP) that efficiently approximates the p-QLP with high accuracy. Fundamental in our work is the exploitation of randomization and in contrast to the p-QLP, PbP-QLP does not use the pivoting strategy. As such, PbP-QLP can harness modern computer architectures, even better than competing randomized algorithms. The efficiency and effectiveness of our proposed PbP-QLP algorithm are investigated through various classes of synthetic and real-world data matrices.

Randomized Recompression of $$\mathcal {H}$$ H -Matrices for BEM

The technique of $$\mathcal {H}$$ -matrices was introduced to deal with (large-scale) dense matrices in an elegant way. It provides a data-sparse format and allows an approximate matrix algebra of nearly optimal complexity. The primary step in the construction of an $$\mathcal {H}$$ -matrix is how to approximate the subblocks of a given dense matrix by matrices that have low numerical rank. Randomized algorithms have recently been introduced as a highly efficient tool for computing approximate factorization of low-rank matrices. In this paper, we consider various randomized algorithms to construct an $$\mathcal {H}$$ -matrix and perform the corresponding $$\mathcal {H}$$ -matrix algebra. In particular, by taking advantages of randomization, we present a simple but fast algorithm of complexity $$\mathcal {O}(4k^2n+k^3 )$$ for truncating an $$n\times n$$ low-rank matrix of (fixed) rank k. The corresponding efficient deterministic algorithm has the complexity $$\mathcal {O}(12k^2n+23k^3 )$$ . We provide numerical examples applying to a BEM model associated with Laplace equation in one and two dimensions to show the efficiency of the proposed randomized methods versus the previously known efficient deterministic methods. The gain of efficiency about 10–25% is achievable when a moderate oversampling parameter $$p=5$$ is used in the computations. In this case, the experimental results show that the proposed algorithm not only has a lower cost but also is more accurate than the conventional methods.

A loop-counting method for covariate-corrected low-rank biclustering of gene-expression and genome-wide association study data.

A common goal in data-analysis is to sift through a large data-matrix and detect any significant submatrices (i.e., biclusters) that have a low numerical rank. We present a simple algorithm for tackling this biclustering problem. Our algorithm accumulates information about 2-by-2 submatrices (i.e., ‘loops’) within the data-matrix, and focuses on rows and columns of the data-matrix that participate in an abundance of low-rank loops. We demonstrate, through analysis and numerical-experiments, that this loop-counting method performs well in a variety of scenarios, outperforming simple spectral methods in many situations of interest. Another important feature of our method is that it can easily be modified to account for aspects of experimental design which commonly arise in practice. For example, our algorithm can be modified to correct for controls, categorical- and continuous-covariates, as well as sparsity within the data. We demonstrate these practical features with two examples; the first drawn from gene-expression analysis and the second drawn from a much larger genome-wide-association-study (GWAS).

Low-Rank Eigenvector Compression of Posterior Covariance Matrices for Linear Gaussian Inverse Problems

We consider the problem of estimating the uncertainty in statistical inverse problems using Bayesian inference. When the probability density of the noise and the prior are Gaussian, the solution of such a statistical inverse problem is also Gaussian. Therefore, the underlying solution is characterized by the mean and covariance matrix of the posterior probability density. However, the covariance matrix of the posterior probability density is full and large. Hence, the computation of such a matrix is impossible for large dimensional parameter spaces. It is shown that for many ill-posed problems, the Hessian matrix of the data misfit part has low numerical rank and it is therefore possible to perform a low-rank approach to approximate the posterior covariance matrix. For such a low-rank approximation, one needs to solve a forward partial differential equation (PDE) and the adjoint PDE in both space and time. This in turn gives $\mathcal{O}(n_x n_t)$ complexity for both, computation and storage, where $n_x$ is the dimension of the spatial domain and $n_t$ is the dimension of the time domain. Such computations and storage demand are infeasible for large problems. To overcome this obstacle, we develop a new approach that utilizes a recently developed low-rank in time algorithm together with the low-rank Hessian method. We reduce both the computational complexity and storage requirement from $\mathcal{O}(n_x n_t)$ to $\mathcal{O}(n_x + n_t)$. We use numerical experiments to illustrate the advantages of our approach.

Data-sparse approximation on the computation of a weakly singular Fredholm equation: A stellar radiative transfer application

Data-sparse representation techniques are emerging on computing approximate solutions for large scale problems involving matrices with low numerical rank. This representation provides both low memory requirements and cheap computational costs.In this work we consider the numerical solution of a large dimensional problem resulting from a finite rank discretization of an integral radiative transfer equation in stellar atmospheres. The integral operator, defined through the first exponential-integral function, is of convolution type and weakly singular.Hierarchically semiseparable representation of the matrix operator with low-rank blocks is built and data-sparse matrix computations can be performed with almost linear complexity. This representation of the original fully populated matrix is an algebraic multilevel structure built from a specific hierarchy of partitions of the matrix indices.Numerical tests illustrate the benefits of this matrix technique compared to standard storage schemes, dense and sparse, in terms of computational cost as well as memory requirements.This approach is particularly useful when a fine discretization of the integral equation is required and the resulting linear system of equations is of large dimension and numerically difficult to solve.

A Distributed-Memory Package for Dense Hierarchically Semi-Separable Matrix Computations Using Randomization

We present a distributed-memory library for computations with dense structured matrices. A matrix is considered structured if its off-diagonal blocks can be approximated by a rank-deficient matrix with low numerical rank. Here, we use Hierarchically Semi-Separable (HSS) representations. Such matrices appear in many applications, for example, finite-element methods, boundary element methods, and so on. Exploiting this structure allows for fast solution of linear systems and/or fast computation of matrix-vector products, which are the two main building blocks of matrix computations. The compression algorithm that we use, that computes the HSS form of an input dense matrix, relies on randomized sampling with a novel adaptive sampling mechanism. We discuss the parallelization of this algorithm and also present the parallelization of structured matrix-vector product, structured factorization, and solution routines. The efficiency of the approach is demonstrated on large problems from different academic and industrial applications, on up to 8,000 cores. This work is part of a more global effort, the STRUctured Matrices PACKage (STRUMPACK) software package for computations with sparse and dense structured matrices. Hence, although useful on their own right, the routines also represent a step in the direction of a distributed-memory sparse solver.

Matrix-free Krylov iteration for implicit convolution of numerically low-rank data

Evaluating the response of a linear shift-invariant system is a problem that occurs frequently in a wide variety of science and engineering problems. Calculating the system response via a convolution may be done efficiently with Fourier transforms. When one must compute the response of one system to m input signals, or the response of m systems to one signal, it may be the case that one may approximate all system responses without having to compute all m Fourier transforms. This can lead to substantial computational savings. Rather than process each point individually, one may only process basis vectors that span the output data space. However, to get a low-error approximation, it is necessary that the output vectors have low numerical rank if they were assembled into a matrix. We develop theory that shows how the singular value decay of a matrix ΦA that is a product of a convolution operator Φ and an arbitrary matrix A depends in a linear fashion on the singular value decays of Φ and A. We propose gap-rank, a measure of the relative numerical rank of a matrix. We show that convolution cannot destroy the numerical low-rank-ness of ΦA data with only modest assumptions. We then develop a new method that exploits low-rank problems with block Golub–Kahan iteration in a Krylov subspace to approximate the low-rank problem. Our method can exploit parallelism in both the individual convolutions and the linear algebra operations in the block Golub–Kahan algorithm. We present numerical examples from signal and image processing that show the low error and scalability of our method.

Compressing Rank-Structured Matrices via Randomized Sampling

Randomized sampling has recently been proven a highly efficient technique for computing approximate factorizations of matrices that have low numerical rank. This paper describes an extension of such techniques to a wider class of matrices that are not themselves rank-deficient but have off-diagonal blocks that are---specifically, the classes of so-called hierarchically off-diagonal low rank (HODLR) matrices and hierarchically block separable (HBS) matrices (a.k.a. hierarchically semiseparable (HSS) matrices). Such matrices arise frequently in numerical analysis and signal processing, in particular in the construction of fast methods for solving differential and integral equations numerically. These structures admit algebraic operations (matrix-vector multiplications, matrix factorizations, matrix inversion, etc.) to be performed very rapidly, but only once a data-sparse representation of the matrix has been constructed. This paper demonstrates that if an $N\times N$ matrix, and its transpose, can be appli...

On the solution of large-scale algebraic Riccati equations by using low-dimensional invariant subspaces

This article discusses an approach to solving large-scale algebraic Riccati equations (AREs) by computing a low-dimensional stable invariant subspace of the associated Hamiltonian matrix. We give conditions on AREs to admit solutions of low numerical rank and show that these can be approximated via Hamiltonian eigenspaces. We discuss strategies on choosing the proper eigenspace that yields a good approximation, and different formulas for building the approximation itself. Similarities of our approach with several other methods for solving AREs are shown: closely related are the projection-type methods that use various Krylov subspaces and the qADI algorithm. The aim of this paper is merely to analyze the possibilities of computing approximate Riccati solutions from low-dimensional subspaces related to the corresponding Hamiltonian matrix and to explain commonalities among existing methods rather than providing a new algorithm.

An algebraic multifrontal preconditioner that exploits the low‐rank property

SummaryWe present an algebraic structured preconditioner for the iterative solution of large sparse linear systems. The preconditioner is based on a multifrontal variant of sparse LU factorization used with nested dissection ordering. Multifrontal factorization amounts to a partial factorization of a sequence of logically dense frontal matrices, and the preconditioner is obtained if structured factorization is used instead. This latter exploits the presence of low numerical rank in some off‐diagonal blocks of the frontal matrices. An algebraic procedure is presented that allows to identify the hierarchy of the off‐diagonal blocks with low numerical rank based on the sparsity of the system matrix. This procedure is motivated by a model problem analysis, yet numerical experiments show that it is successful beyond the model problem scope. Further aspects relevant for the algebraic structured preconditioner are discussed and illustrated with numerical experiments. The preconditioner is also compared with other solvers, including the corresponding direct solver. Copyright © 2015 John Wiley & Sons, Ltd.

Cover-based bounds on the numerical rank of Gaussian kernels

A popular approach for analyzing high-dimensional datasets is to perform dimensionality reduction by applying non-parametric affinity kernels. Usually, it is assumed that the represented affinities are related to an underlying low-dimensional manifold from which the data is sampled. This approach works under the assumption that, due to the low-dimensionality of the underlying manifold, the kernel has a low numerical rank. Essentially, this means that the kernel can be represented by a small set of numerically-significant eigenvalues and their corresponding eigenvectors.We present an upper bound for the numerical rank of Gaussian convolution operators, which are commonly used as kernels by spectral manifold-learning methods. The achieved bound is based on the underlying geometry that is provided by the manifold from which the dataset is assumed to be sampled. The bound can be used to determine the number of significant eigenvalues/eigenvectors that are needed for spectral analysis purposes. Furthermore, the results in this paper provide a relation between the underlying geometry of the manifold (or dataset) and the numerical rank of its Gaussian affinities.The term cover-based bound is used because the computations of this bound are done by using a finite set of small constant-volume boxes that cover the underlying manifold (or the dataset). We present bounds for finite Gaussian kernel matrices as well as for the continuous Gaussian convolution operator. We explore and demonstrate the relations between the bounds that are achieved for finite and continuous cases. The cover-oriented methodology is also used to provide a relation between the geodesic length of a curve and the numerical rank of Gaussian kernel of datasets that are sampled from it.

A Fast Randomized Algorithm for Computing a Hierarchically Semiseparable Representation of a Matrix

Randomized sampling has recently been proven a highly efficient technique for computing approximate factorizations of matrices that have low numerical rank. This paper describes an extension of such techniques to a wider class of matrices that are not themselves rank-deficient but have off-diagonal blocks that are; specifically, the class of so-called hierarchically semiseparable (HSS) matrices. HSS matrices arise frequently in numerical analysis and signal processing, particularly in the construction of fast methods for solving differential and integral equations numerically. The HSS structure admits algebraic operations (matrix-vector multiplications, matrix factorizations, matrix inversion, etc.) to be performed very rapidly, but only once the HSS representation of the matrix has been constructed. How to rapidly compute this representation in the first place is much less well understood. The present paper demonstrates that if an $N\times N$ matrix can be applied to a vector in $O(N)$ time, and if individual entries of the matrix can be computed rapidly, then provided that an HSS representation of the matrix exists, it can be constructed in $O(N\,k^{2})$ operations, where $k$ is an upper bound for the numerical rank of the off-diagonal blocks. The point is that when legacy codes (based on, e.g., the fast multipole method) can be used for the fast matrix-vector multiply, the proposed algorithm can be used to obtain the HSS representation of the matrix, and then well-established techniques for HSS matrices can be used to invert or factor the matrix.

Detecting low-rank clusters via random sampling

We present an algorithm for detecting a low-rank cluster of vectors from within a much larger group of vectors. This algorithm relies on a basic geometric property of high-dimensional space: Most of the volume of a typical eccentric ellipsoid is confined to relatively few orthants within the ambient space. This simple fact can be used to quickly detect a collection of vectors with low numerical rank from amongst a larger group of vectors with higher numerical rank.

A Fast $ULV$ Decomposition Solver for Hierarchically Semiseparable Representations

We consider an algebraic representation that is useful for matrices with off‐diagonal blocks of low numerical rank. A fast and stable solver for linear systems of equations in which the coefficient matrix has this representation is presented. We also present a fast algorithm to construct the hierarchically semiseparable representation in the general case.

An Eigenspace Update Algorithm for Image Analysis

During the past few years several interesting applications of eigenspace representation of images have been proposed. These include face recognition, video coding, and pose estimation. However, the vision research community has largely overlooked parallel developments in signal processing and numerical linear algebra concerning efficient eigenspace updating algorithms. These new developments are significant for two reasons: Adopting them will make some of the current vision algorithms more robust and efficient. More important is the fact that incremental updating of eigenspace representations will open up new and interesting research applications in vision such as active recognition and learning. The main objective of this paper is to put these in perspective and discuss a new updating scheme for low numerical rank matrices that can be shown to be numerically stable and fast. A comparison with a nonadaptive SVD scheme shows that our algorithm achieves similar accuracy levels for image reconstruction and recognition at a significantly lower computational cost. We also illustrate applications to adaptive view selection for 3D object representation from projections.