Spectral Sparsification Research Articles

Small-Space Spectral Sparsification via Bounded-Independence Sampling

We give a deterministic, nearly logarithmic-space algorithm for mild spectral sparsification of undirected graphs. Given a weighted, undirected graph G on n vertices described by a binary string of length N , an integer k ≤ log n , and an error parameter ɛ > 0, our algorithm runs in space \(\widetilde{O}(k\log (N\cdot w_{\mathrm{max}}/w_{\mathrm{min}})),\) where w max and w min are the maximum and minimum edge weights in G , and produces a weighted graph H with \(\widetilde{O}(n^{1+2/k}/\varepsilon ^2)\) edges that spectrally approximates G , in the sense of Spielman and Teng, up to an error of ɛ. Our algorithm is based on a new bounded-independence analysis of Spielman and Srivastava’s effective resistance-based edge sampling algorithm and uses results from recent work on space-bounded Laplacian solvers. In particular, we demonstrate an inherent trade-off (via upper and lower bounds) between the amount of (bounded) independence used in the edge sampling algorithm, denoted by k above, and the resulting sparsity that can be achieved.

DiGRASS: Di rected Gra ph S pectral S parsification via Spectrum-Preserving Symmetrization

Recent spectral graph sparsification research aims to construct ultra-sparse subgraphs for preserving the original graph spectral (structural) properties, such as the first few Laplacian eigenvalues and eigenvectors, which has led to the development of a variety of nearly-linear time numerical and graph algorithms. However, there is very limited progress for spectral sparsification of directed graphs. In this work, we prove the existence of nearly-linear-sized spectral sparsifiers for directed graphs under certain conditions. Furthermore, we introduce a practically-efficient spectral algorithm (diGRASS) for sparsifying real-world, large-scale directed graphs leveraging spectral matrix perturbation analysis. The proposed method has been evaluated using a variety of directed graphs obtained from real-world applications, showing promising results for solving directed graph Laplacians, spectral partitioning of directed graphs, and approximately computing (personalized) PageRank vectors.

Unleashing the Power of Graph Spectral Sparsification for Power Grid Analysis via Incomplete Cholesky Factorization

Graph spectral sparsification based preconditioning technique has shown promising results for power grid analysis. However, the conventional methods converge slowly for high accuracy requirement. In this work, we propose an efficient approach to address this issue. Instead of using Cholesky factorization, we employ incomplete Cholesky factorization to factorize the spectral sparsifier. We also propose a concept of graph spectral pattern, which can further reduce the preconditioned conjugate gradient (PCG) iterations using less number of nonzeros. Experiments show that under 10-6 relative tolerance, our proposed preconditioning technique achieves 1.17× speedup compared to AMGPCG in average; compared to the conventional spectral sparsification based preconditioning techniques, our proposed approach achieves up to 8.53× speedup of the factorization, 8.74× speedup of the PCG iteration, and 5.6× speedup of the total time. Moreover, the speedup of the total time continues to enlarge for higher accuracy requirement, e.g., 10-12. Last, but not the least, our method is compatible with existing graph spectral sparsification algorithms for power grid analysis.

Graph Sparsification by Universal Greedy Algorithms

Graph sparsification is to approximate an arbitrary graph by a sparse graph and is useful in many applications, such as simplification of social networks, least squares problems, numerical solution of symmetric positive definite linear systems and etc. In this paper, inspired by the well-known sparse signal recovery algorithm called orthogonal matching pursuit (OMP), we introduce a deterministic, greedy edge selection algorithm called universal greedy approach (UGA) for graph sparsification. For a general spectral sparsification problem, e.g., positive subset selection problem from a set of $m$ vectors from $\mathbb{R}^n$, we propose a nonnegative UGA algorithm which needs $O(mn^2+ n^3/\epsilon^2)$ time to find a $\frac{1+\epsilon/\beta}{1-\epsilon/\beta}$-spectral sparsifier with positive coefficients with sparsity $\le\lceil\frac{n}{\epsilon^2}\rceil$, where $\beta$ is the ratio between the smallest length and largest length of the vectors. The convergence of the nonnegative UGA algorithm will be established. For the graph sparsification problem, another UGA algorithm will be proposed which can output a $\frac{1+O(\epsilon)}{1-O(\epsilon)}$-spectral sparsifier with $\lceil\frac{n}{\epsilon^2}\rceil$ edges in $O(m+n^2/\epsilon^2)$ time from a graph with $m$ edges and $n$ vertices under some mild assumptions. This is a linear time algorithm in terms of the number of edges that the community of graph sparsification is looking for. The best result in the literature to the knowledge of the authors is the existence of a deterministic algorithm which is almost linear, i.e. $O(m^{1+o(1)})$ for some $o(1)=O(\frac{(\log\log(m))^{2/3}}{\log^{1/3}(m)})$. Finally, extensive experimental results, including applications to graph clustering and least squares regression, show the effectiveness of proposed approaches.

Spectral sparsification of speech signals and its interaction with top-down mechanisms in adult cochlear implant users

The ability to understand speech varies significantly among cochlear implant (CI) listeners as it depends on a variety of individual and environmental factors. Especially in adverse listening situations, good cognitive and linguistic (“top-down”) skills and the use of signal pre-processing algorithms are considered beneficial. However, not much is known about the interactions between these top-down and perceptive bottom-up processes in CI listeners. To shed light on the relation between the spectral representation of speech sounds and the ability of CI listeners to decode these signals, two methods for the simplification of speech signals through spectral sparsification were developed and evaluated in listening tests with postlingually deaf adult CI listeners. Speech signals were separated into transient and harmonic parts. After sparsification of the harmonic spectrum by principal component analysis (PCA) or by an individualized spectral peak picking approach, the transient and the sparsened harmonic parts were remixed. Furthermore, cognitive parameters of the subjects were assessed via a neurocognitive test battery (ALAcog) and their correlation with recognition scores evaluated. The PCA-based sparsification method showed a significant benefit in speech recognition relative to the unprocessed signal. Furthermore, subjects with better performance in working memory and mental flexibility showed larger improvements.

A Spectral Approach to Network Design

We present a spectral approach to design approximation algorithms for network design problems. We observe that the underlying mathematical questions are the spectral rounding problems, which were studied in spectral sparsification and in discrepancy theory. We extend these results to incorporate additional non-negative linear constraints, and show that they can be used to significantly extend the scope of network design problems that can be solved. Our algorithm for spectral rounding is an iterative randomized rounding algorithm based on the regret minimization framework. In some settings, this provides an alternative spectral algorithm to achieve constant factor approximation for the classical survivable network design problem, and partially answers a question of Bansal about survivable network design with concentration property. We also show many other applications of the spectral rounding results, including weighted experimental design and additive spectral sparsification.

SPROUT: spectral sparsification helps restore the spatial structure at single-cell resolution.

Single-cell RNA sequencing thoroughly quantifies the individual cell transcriptomes but renounces the spatial structure. Conversely, recently emerged spatial transcriptomics technologies capture the cellular spatial structure but skimp cell or gene resolutions. Ligand-receptor interactions reveal the potential of cell proximity since they are spatially constrained. Cell–cell affinity values estimated by ligand–receptor interaction can partially represent the structure of cells but falsely include the pseudo affinities between distant or indirectly interacting cells. Here, we develop a software package, SPROUT, to reconstruct the single-cell resolution spatial structure from the transcriptomics data through diminished pseudo ligand–receptor affinities. For spatial data, SPROUT first curates the representative single-cell profiles for each spatial spot from a candidate library, then reduces the pseudo affinities in the intercellular affinity matrix by partial correlation, spectral graph sparsification, and spatial coordinates refinement. SPROUT embeds the estimated interactions into a low-dimensional space with the cross-entropy objective to restore the intercellular structures, which facilitates the discovery of dominant ligand–receptor pairs between neighboring cells at single-cell resolution. SPROUT reconstructed structures achieved shape Pearson correlations ranging from 0.91 to 0.97 on the mouse hippocampus and human organ tumor microenvironment datasets. Furthermore, SPROUT can solely de novo reconstruct the structures at single-cell resolution, i.e., reaching the cell-type proximity correlations of 0.68 and 0.89 between reconstructed and immunohistochemistry-informed spatial structures on a human developing heart dataset and a tumor microenvironment dataset, respectively.

Scaling industrial applications for the Big Data era

Industrial applications tend to rely increasingly on large datasets for regular operations. In order to facilitate that need, we unite the increasingly available hardware resources with fundamental problems found in classical algorithms. We show solutions to the following problems: power flow and island detection in power networks, and the more general graph sparsification. At their core lie respectively algorithms for solving systems of linear equations, graph connectivity and matrix multiplication, and spectral sparsification of graphs, which are applicable on their own to a far greater spectrum of problems. The novelty of our approach lies in developing the first open source and distributed solutions, capable of handling large datasets. Such solutions constitute a toolkit, which, aside from the initial purpose, can be used for the development of unrelated applications and for educational purposes in the study of distributed algorithms.

BC tree-based spectral sampling for big complex network visualization

Graph sampling methods have been used to reduce the size and complexity of big complex networks for graph mining and visualization. However, existing graph sampling methods often fail to preserve the connectivity and important structures of the original graph. This paper introduces a new divide and conquer approach to spectral graph sampling based on graph connectivity, called the BC Tree (i.e., decomposition of a connected graph into biconnected components) and spectral sparsification. Specifically, we present two methods, spectral vertex sampling BC_SV and spectral edge sampling BC_SS by computing effective resistance values of vertices and edges for each connected component. Furthermore, we present DBC_SS and DBC_GD, graph connectivity-based distributed algorithms for spectral sparsification and graph drawing respectively, aiming to further improve the runtime efficiency of spectral sparsification and graph drawing by integrating connectivity-based graph decomposition and distributed computing. Experimental results demonstrate that BC_SV and BC_SS are significantly faster than previous spectral graph sampling methods while preserving the same sampling quality. DBC_SS and DBC_GD obtain further significant runtime improvement over sequential approaches, and DBC_GD further achieves significant improvements in quality metrics over sequential graph drawing layouts.

Geometric graphs from data to aid classification tasks with Graph Convolutional Networks

SummaryTraditional classification tasks learn to assign samples to given classes based solely on sample features. This paradigm is evolving to include other sources of information, such as known relations between samples. Here, we show that, even if additional relational information is not available in the dataset, one can improve classification by constructing geometric graphs from the features themselves, and using them within a Graph Convolutional Network. The improvement in classification accuracy is maximized by graphs that capture sample similarity with relatively low edge density. We show that such feature-derived graphs increase the alignment of the data to the ground truth while improving class separation. We also demonstrate that the graphs can be made more efficient using spectral sparsification, which reduces the number of edges while still improving classification performance. We illustrate our findings using synthetic and real-world datasets from various scientific domains.

Bounds on the spectral sparsification of symmetric and off-diagonal nonnegative real matrices

We say that a square real matrix [Formula: see text] is off-diagonal nonnegative if and only if all entries outside its diagonal are nonnegative real numbers. In this paper, we show that for any off-diagonal nonnegative symmetric matrix [Formula: see text], there exists a nonnegative symmetric matrix [Formula: see text] which is sparse and close in spectrum to [Formula: see text].

FeGRASS: Fast and Effective Graph Spectral Sparsification for Scalable Power Grid Analysis

Graph spectral sparsification aims to find a ultrasparse subgraph which can preserve the spectral properties of the original graph. The subgraph can be leveraged to construct a preconditioner to speed up the solution of the original graph’s Laplacian matrix. In this work, we propose feGRASS, a fast and effective graph spectral sparsification approach for the problem of large-scale power grid analysis and other problems with similar graphs. The proposed approach is based on two novel concepts: 1) effective edge weight and 2) spectral edge similarity. The former takes advantage of node degrees and breadth-first-search (BFS) distances, which leads to a scalable algorithm for generating low-stretch spanning trees (LSSTs). Then, the latter concept is leveraged during the recovery of spectrally critical off-tree edges to produce spectrally similar subgraphs. Compared with the most recent competitor <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> , the proposed approach is much faster for producing high-quality spectral sparsifiers. Extensive experimental results have been demonstrated to illustrate the superior efficiency of a preconditioned conjugate gradient (PCG) algorithm based on the proposed approach, for solving large power grid problems and many other real-world graph Laplacians. For instance, a power grid matrix with 60 million unknowns and 260 million nonzeros can be solved (at a 1E-3 accuracy level) within 196 s and 12 PCG iterations, on a single CPU core.

Deep sparse graph functional connectivity analysis in AD patients using fMRI data

Functional magnetic resonance imaging (fMRI) is a non-invasive method that helps to analyze brain function based on BOLD signal fluctuations. Functional Connectivity (FC) catches the transient relationship between various brain regions usually measured by correlation analysis. The elements of the correlation matrix are between -1 to 1. Some of them are very small values usually related to weak and spurious correlations due to noises and artifacts. They can not be concluded as real strong correlations between brain regions and their existence could make a misconception and leads to fake results. It is crucial to make a conclusion based on reliable and informative correlations. In order to eliminate weak correlations, thresholding is a common method. In this routine, by adjusting a threshold the values below the threshold turn to zero and the rest remains. In this paper, in addition to thresholding, two other methods including spectral sparsification based on Effective Resistance (ER) and autoencoders are investigated for sparsing the correlation matrices. Autoencoders are based on deep learning neural networks and ER considers the network as a resistive circuit. The fMRI data of the study correspond to Alzheimer's patients and control subjects. Graph global measures are calculated and a non-parametric permutation test is reported. Results show that the autoencoder and spectral sparsification achieved more distinctive brain graphs between healthy and AD subjects. Also, more graph global features were significantly different from these two methods due to better elimination of weak correlations and preserve more informative ones. Regardless of the sparsification method features including average strength, clustering, local efficiency, modularity, and transitivity are significantly different (P-value=0.05). On the other hand, the measures radius, diameter, and eccentricity showed no significant differences in none of the methods. In addition, according to three different methods, the brain regions show fragile and solid FCs are determined.

Multiclass classification of patients during different stages of Alzheimer’s disease using fMRI time-series

Alzheimer’s Disease (AD) begins several years before the symptoms develop. It starts with Mild Cognitive Impairment (MCI) which can be separated into Early MCI and Late MCI (EMCI and LMCI). Functional connectivity analysis and classification are done among the different stages of illness with Functional Magnetic Resonance Imaging (fMRI). In this study, in addition to the four stages including healthy, EMCI, LMCI, and AD, the patients have been tracked for a year. Indeed, the classification has been done among 7 groups to analyze the functional connectivity changes in one year in different stages. After generating the functional connectivity graphs for eliminating the weak links, three different sparsification methods were used. In addition to simple thresholding, spectral sparsification based on effective resistance and sparse autoencoder were performed in order to analyze the effect of sparsification routine on classification results. Also, instead of extracting common features, the correlation matrices were reshaped to a correlation vector and used as a feature vector to enter the classifier. Since the correlation matrix is symmetric, in another analysis half of the feature vector was used, moreover, the Genetic Algorithm (GA) also utilized for feature vector dimension reduction. The non-linear SVM classifier with a polynomial kernel applied. The results showed that the autoencoder sparsification method had the greatest discrimination power with the accuracy of 98.35% for classification when the feature vector was the full correlation matrix.

Spectral sparsification of simplicial complexes for clustering and label propagation

As a generalization of the use of graphs to describe pairwise interactions, simplicial complexes can be used to model higher-order interactions between three or more objects in complex systems. There has been a recent surge in activity for the development of data analysis methods applicable to simplicial complexes, including techniques based on computational topology, higher-order random processes, generalized Cheeger inequalities, isoperimetric inequalities, and spectral methods. In particular, spectral learning methods (e.g. label propagation and clustering) that directly operate on simplicial complexes represent a new direction emerging from the confluence of computational topology and machine learning. To apply spectral methods in learning to massive datasets modeled as simplicial complexes, we sparsify simplicial complexes, while preserving the spectrum of the associated Laplacian operators. We show that the theory of Spielman and Srivastava for the sparsification of graphs extends to simplicial complexes via the up Laplacian. In particular, we introduce a generalized effective resistance for simplexes, provide an algorithm for sparsifying simplicial complexes at a fixed dimension, and give a specific version of the generalized Cheeger inequality for weighted simplicial complexes. Finally, we introduce higher-order generalizations of spectral clustering and label propagation for simplicial complexes and demonstrate via experiments the utility of the proposed spectral sparsification method for these applications.

Sublinear Time Force Computation for Big Complex Network Visualization

AbstractIn this paper, we present a new framework for sublinear time force computation for visualization of big complex graphs. Our algorithm is based on the sampling of vertices for computing repulsion forces and edge sparsification for attraction force computation. More specifically, for vertex sampling, we present three types of sampling algorithms, including random sampling, geometric sampling, and combinatorial sampling, to reduce the repulsion force computation to sublinear in the number of vertices. We utilize a spectral sparsification approach to reduce the number of attraction force computations to sublinear in the number of edges for dense graphs. We also present a smart initialization method based on radial tree drawing of the BFS spanning tree rooted at the center.Experiments show that our new sublinear time force computation algorithms run quite fast, while producing good visualization of large and complex networks, with significant improvements in quality metrics such as shape‐based and edge crossing metrics.

Geometric Graphs from Data to Aid Classification Tasks with Graph Convolutional Networks

Traditional classification tasks learn to assign samples to given classes based solely on sample features. This paradigm is evolving to include other sources of information, such as known relations between samples. Here we show that, even if additional relational information is not available in the data set, one can improve classification by constructing geometric graphs from the features themselves, and using them within a Graph Convolutional Network. The improvement in classification accuracy is maximized by graphs that capture sample similarity with relatively low edge density. We show that such feature-derived graphs increase the alignment of the data to the ground truth while improving class separation. We also demonstrate that the graphs can be made more efficient using spectral sparsification, which reduces the number of edges while still improving classification performance. We illustrate our findings using synthetic and real-world data sets from various scientific domains.

Spectrum-preserving sparsification for visualization of big graphs

We present a novel spectrum-preserving sparsification algorithm for visualizing big graph data. Although spectral methods have many advantages, the high memory and computation costs due to the involved Laplacian eigenvalue problems could immediately hinder their applications in big graph analytics. In this paper, we introduce a practically efficient, nearly-linear time spectral sparsification algorithm for tackling real-world big graph data. Besides spectral sparsification, we further propose a node reduction scheme based on intrinsic spectral graph properties to allow more aggressive, level-of-detail simplification. To enable effective visual exploration of the resulting spectrally sparsified graphs, we implement spectral clustering and edge bundling. Our framework does not depend on a particular graph layout and can be integrated into different graph drawing algorithms. We experiment with publicly available graph data of different sizes and characteristics to demonstrate the efficiency and effectiveness of our approach. To further verify our solution, we quantitatively compare our method against different graph simplification solutions using a proxy quality metric and statistical properties of the graphs.

GRASS: Graph Spectral Sparsification Leveraging Scalable Spectral Perturbation Analysis

Spectral graph sparsification aims to find ultra-sparse subgraphs whose Laplacian matrix can well approximate the original Laplacian eigenvalues and eigenvectors. In recent years, spectral sparsification techniques have been extensively studied for accelerating various numerical and graph-related applications. Prior nearly-linear-time spectral sparsification methods first extract low-stretch spanning tree from the original graph to form the backbone of the sparsifier, and then recover small portions of spectrally-critical off-tree edges to the spanning tree to significantly improve the approximation quality. However, it is not clear how many off-tree edges should be recovered for achieving a desired spectral similarity level within the sparsifier. Motivated by recent graph signal processing techniques, this work proposes a similarity-aware spectral graph sparsification framework that leverages efficient spectral off-tree edge embedding and filtering schemes to construct spectral sparsifiers with guaranteed spectral similarity (relative condition number) level. An iterative graph densification scheme is also introduced to facilitate efficient and effective filtering of off-tree edges for highly ill-conditioned problems. The proposed method has been validated using various kinds of graphs obtained from public domain sparse matrix collections relevant to very large-scale integration computer-aided design, finite element analysis, as well as social and data networks frequently studied in many machine learning and data mining applications. For instance, a sparse SDD matrix with 40 million unknowns and 180 million nonzeros can be solved (1E-3 accuracy level) within 2 min using a single CPU core and about 6-GB memory.

Determinant-Preserving Sparsification of SDDM Matrices

We show that variants of spectral sparsification routines can preserve the total spanning tree counts of graphs. By Kirchhoff's matrix-tree theorem, this is equivalent to preserving the determinant...