Symmetric Diagonally Dominant Research Articles

Engineering a Combinatorial Laplacian Solver: Lessons Learned

Linear system solving is a main workhorse in applied mathematics. Recently, theoretical computer scientists contributed sophisticated algorithms for solving linear systems with symmetric diagonally-dominant (SDD) matrices in provably nearly-linear time. These algorithms are very interesting from a theoretical perspective, but their practical performance was unclear. Here, we address this gap. We provide the first implementation of the combinatorial solver by Kelner et al. (STOC 2013), which is appealing for implementation due to its conceptual simplicity. The algorithm exploits that a Laplacian matrix (which is SDD) corresponds to a graph; solving symmetric Laplacian linear systems amounts to finding an electrical flow in this graph with the help of cycles induced by a spanning tree with the low-stretch property. The results of our experiments are ambivalent. While they confirm the predicted nearly-linear running time, the constant factors make the solver much slower for reasonable inputs than basic methods with higher asymptotic complexity. We were also not able to use the solver effectively as a smoother or preconditioner. Moreover, while spanning trees with lower stretch indeed reduce the solver’s running time, we experience again a discrepancy in practice: in our experiments, simple spanning tree algorithms perform better than those with a guaranteed low stretch. We expect that our results provide insights for future improvements of combinatorial linear solvers.

A fast algorithm for manifold learning by posing it as a symmetric diagonally dominant linear system

We present a fast manifold learning algorithm by formulating a new linear constraint that we use to replace the weighted orthonormality constraints within Laplacian Eigenmaps; a popular manifold learning algorithm. We thereby convert a quadratically constrained quadratic optimization problem into a simpler formulation that is a linearly constrained quadratic optimization problem. We show that solving this problem is equivalent to solving a symmetric diagonally dominant (SDD) linear system which can be solved very fast using a combinatorial multigrid (CMG) solver. In addition to this we also suggest another method that can exploit any sparsity within the graph Laplacian matrix via a fast sparse Cholesky decomposition to produce an alternative solution in addition to the SDD based method. We compare the improvements in run-times using both our SDD system based method and our fast sparse Cholesky decomposition based method against the well known Nystrom method based fast manifold learning and present competitive results.

A scalable approach to spectral clustering with SDD solvers

The promise of spectral clustering is that it can help detect complex shapes and intrinsic manifold structure in large and high dimensional spaces. The price for this promise is the expensive computational cost for computing the eigen-decomposition of the graph Laplacian matrix--so far a necessary subroutine for spectral clustering. In this paper we bypass the eigen-decomposition of the original Laplacian matrix by leveraging the recently introduced near-linear time solver for symmetric diagonally dominant (SDD) linear systems and random projection. Experiments on several synthetic and real datasets show that the proposed approach has better clustering quality and is faster than the state-of-the-art approximate spectral clustering methods.

Combinatorial preconditioners for scalar elliptic finite‐element problems

AbstractWe present a new preconditioner for linear systems arising from finite‐elements discretizations of scalar elliptic partial differential equations. The solver is based on building a symmetric diagonally dominant (SDD) approximation of the stiffness matrix K. The approximation is built by approximating each element inside the collection {Ke } of element matrices by an SDD matrix Le. The SDD approximation L is built by assembling the collection {Le }. We then sparsify L using a graph algorithm, and use the sparsified matrix as a preconditioner. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Combinatorial characterization of the null spaces of symmetric H-matrices

We characterize the structure of null spaces of symmetric diagonally dominant (SDD) matrices and symmetric H-matrices with non-negative diagonal entries. We show that the structure of the null space of a symmetric SDD matrix or H-matrix A depends on the structure of the connected components of its underlying graph. Each connected component contributes at most one vector to the null space. This paper provides a combinatorial characterization of the rank of each connected component, and a combinatorial characterization of a null vector if one exists. For SDD matrices, we also present an efficient combinatorial algorithm for constructing an orthonormal basis for the null space. The paper also shows a close connection between gain graphs and H-matrices, which extends known results regarding the connection between undirected graphs and Laplacian matrices, and between signed graphs and SDD matrices. We show how to exploit these combinatorial algorithms to reliably solve certain singular linear systems in finite-precision arithmetic.