Eigenvectors Of Graph Laplacian Research Articles

Minimax optimal regression over Sobolev spaces via Laplacian Eigenmaps on neighbourhood graphs

Abstract In this paper, we study the statistical properties of Principal Components Regression with Laplacian Eigenmaps (PCR-LE), a method for non-parametric regression based on Laplacian Eigenmaps (LE). PCR-LE works by projecting a vector of observed responses ${\textbf Y} = (Y_1,\ldots ,Y_n)$ onto a subspace spanned by certain eigenvectors of a neighbourhood graph Laplacian. We show that PCR-LE achieves minimax rates of convergence for random design regression over Sobolev spaces. Under sufficient smoothness conditions on the design density $p$, PCR-LE achieves the optimal rates for both estimation (where the optimal rate in squared $L^2$ norm is known to be $n^{-2s/(2s + d)}$) and goodness-of-fit testing ($n^{-4s/(4s + d)}$). We also consider the situation where the design is supported on a manifold of small intrinsic dimension $m$, and give upper bounds establishing that PCR-LE achieves the faster minimax estimation ($n^{-2s/(2s + m)}$) and testing ($n^{-4s/(4s + m)}$) rates of convergence. Interestingly, these rates are almost always much faster than the known rates of convergence of graph Laplacian eigenvectors to their population-level limits; in other words, for this problem regression with estimated features appears to be much easier, statistically speaking, than estimating the features itself. We support these theoretical results with empirical evidence.

Extending bootstrap AMG for clustering of attributed graphs

In this paper we propose a new approach to detect clusters in undirected graphs with attributed vertices. We incorporate structural and attribute similarities between the vertices in an augmented graph by creating additional vertices and edges as proposed in [1, 2]. The augmented graph is then embedded in a Euclidean space associated to its Laplacian and we cluster vertices via a modified K-means algorithm, using a new vector-valued distance in the embedding space. Main novelty of our method, which can be classified as an early fusion method, i.e., a method in which additional information on vertices are fused to the structure information before applying clustering, is the interpretation of attributes as new realizations of graph vertices, which can be dealt with as coordinate vectors in a related Euclidean space. This allows us to extend a scalable generalized spectral clustering procedure which substitutes graph Laplacian eigenvectors with some vectors, named algebraically smooth vectors, obtained by a linear-time complexity Algebraic MultiGrid (AMG) method. We discuss the performance of our proposed clustering method by comparison with recent literature approaches and public available results. Extensive experiments on different types of synthetic datasets and real-world attributed graphs show that our new algorithm, embedding attributes information in the clustering, outperforms structure-only-based methods, when the attributed network has an ambiguous structure. Furthermore, our new method largely outperforms the method which originally proposed the graph augmentation, showing that our embedding strategy and vector-valued distance are very effective in taking advantages from the augmented-graph representation.

Lipschitz Regularity of Graph Laplacians on Random Data Clouds

In this paper we study Lipschitz regularity of elliptic PDEs on geometric graphs, constructed from random data points. The data points are sampled from a distribution supported on a smooth manifold. The family of equations that we study arises in data analysis in the context of graph-based learning and contains, as important examples, the equations satisfied by graph Laplacian eigenvectors. In particular, we prove high probability interior and global Lipschitz estimates for solutions of graph Poisson equations. Our results can be used to show that graph Laplacian eigenvectors are, with high probability, essentially Lipschitz regular with constants depending explicitly on their corresponding eigenvalues. Our analysis relies on a probabilistic coupling argument of suitable random walks at the continuum level, and an interpolation method for extending functions on random point clouds to the continuum manifold. As a byproduct of our general regularity results, we obtain high probability $L^\infty$ and approximate $\mathcal{C}^{0,1}$ convergence rates for the convergence of graph Laplacian eigenvectors towards eigenfunctions of the corresponding weighted Laplace-Beltrami operators. The convergence rates we obtain scale like the $L^2$-convergence rates established by two of the authors in previous work.

Geary’s c and Spectral Graph Theory

Spatial autocorrelation, of which Geary’s c has traditionally been a popular measure, is fundamental to spatial science. This paper provides a new perspective on Geary’s c. We discuss this using concepts from spectral graph theory/linear algebraic graph theory. More precisely, we provide three types of representations for it: (a) graph Laplacian representation, (b) graph Fourier transform representation, and (c) Pearson’s correlation coefficient representation. Subsequently, we illustrate that the spatial autocorrelation measured by Geary’s c is positive (resp. negative) if spatially smoother (resp. less smooth) graph Laplacian eigenvectors are dominant. Finally, based on our analysis, we provide a recommendation for applied studies.

Multiscale representations of community structures in attractor neural networks.

Our cognition relies on the ability of the brain to segment hierarchically structured events on multiple scales. Recent evidence suggests that the brain performs this event segmentation based on the structure of state-transition graphs behind sequential experiences. However, the underlying circuit mechanisms are poorly understood. In this paper we propose an extended attractor network model for graph-based hierarchical computation which we call the Laplacian associative memory. This model generates multiscale representations for communities (clusters) of associative links between memory items, and the scale is regulated by the heterogenous modulation of inhibitory circuits. We analytically and numerically show that these representations correspond to graph Laplacian eigenvectors, a popular method for graph segmentation and dimensionality reduction. Finally, we demonstrate that our model exhibits chunked sequential activity patterns resembling hippocampal theta sequences. Our model connects graph theory and attractor dynamics to provide a biologically plausible mechanism for abstraction in the brain.

Natural Graph Wavelet Packet Dictionaries

We introduce a set of novel multiscale basis transforms for signals on graphs that utilize their “dual” domains by incorporating the “natural” distances between graph Laplacian eigenvectors, rather than simply using the eigenvalue ordering. These basis dictionaries can be seen as generalizations of the classical Shannon wavelet packet dictionary to arbitrary graphs, and do not rely on the frequency interpretation of Laplacian eigenvalues. We describe the algorithms (involving either vector rotations or orthogonalizations) to construct these basis dictionaries, use them to efficiently approximate graph signals through the best basis search, and demonstrate the strengths of these basis dictionaries for graph signals measured on sunflower graphs and street networks.

On graph Laplacian eigenvectors with components in [formula omitted

We characterize all graphs for which there are eigenvectors of the graph Laplacian having all their components in {−1,+1} or {−1,0,+1}. Graphs having eigenvectors with components in {−1,+1} are called bivalent and are shown to be the regular bipartite graphs and their extensions obtained by adding edges between vertices with the same value for the given eigenvector. Graphs with eigenvectors with components in {−1,0,+1} are called trivalent and are shown to be soft-regular graphs – graphs such that vertices associated with non-zero components have the same degree – and their extensions via certain transformations.

Sparse Laplacian Component Analysis for Internet Traffic Anomalies Detection

We consider the problem of anomaly detection in network traffic. It is a challenging problem because of high-dimensional and noisy nature of network traffic. A popularly used technique is subspace analysis . In particular, subspace analysis aims to separate the high-dimensional space of traffic signals into disjoint subspaces corresponding to normal and anomalous network conditions. Principal component analysis (PCA) and its improvements have been applied for this analysis. In this work, we take a different approach to determine the subspaces, and propose to capture the essence of the data using the eigenvectors of graph Laplacian, which we refer as Laplacian components (LCs). Our main contribution is to propose a regression framework to compute LCs followed by its application in anomaly detection. This framework provides much flexibility in incorporating different properties into the LCs, notably LCs with sparse loadings, which we exploit in detail. In other words, our contribution is a new framework to compute the graph Fourier transform (GFT). The proposed framework enables sparse loadings and potentially other properties to be incorporated into the analysis components of GFT to suit different tasks. Furthermore, different from previous work that uses a sample graph to preserve local structure, we advocate modeling with a dual-input feature graph that encodes the correlation of the time series data and prior information. Therefore, the proposed model can readily incorporate the “physics” of some applications as prior information to improve the analysis. We perform experiments on volume anomaly detection using three real data sets. We demonstrate that the proposed model can correctly uncover the essential low-dimensional principal subspace containing the normal Internet traffic and achieve outstanding detection performance.

Moments of the inverse participation ratio for the Laplacian on finite regular graphs

We investigate the first and second moments of the inverse participation ratio (IPR) for all eigenvectors of the Laplacian on finite random regular graphs with n vertices and degree z. By exactly diagonalizing a large set of z-regular graphs, we find that as n becomes large, the mean of the inverse participation ratio on each graph, when averaged over a large ensemble of graphs, approaches the numerical value 3. This universal number is understood as the large-n limit of the average of the quartic polynomial corresponding to the IPR over an appropriate -dimensional hypersphere of . For a large, but not exhaustive ensemble of graphs, the mean variance of the inverse participation ratio for all graph Laplacian eigenvectors deviates from its continuous hypersphere average due to large graph-to-graph fluctuations that arise from the existence of highly localized modes.

Image denoising via adaptive eigenvectors of graph Laplacian

An image denoising method via adaptive eigenvectors of graph Laplacian (EGL) is proposed. Unlike the trivial parameter setting of the used eigenvectors in the traditional EGL method, in our method, the eigenvectors are adaptively selected in the whole denoising procedure. In detail, a rough image is first built with the eigenvectors from the noisy image, where the eigenvectors are selected by using the deviation estimation of the clean image. Subsequently, a guided image is effectively restored with a weighted average of the noisy and rough images. In this operation, the average coefficient is adaptively obtained to set the deviation of the guided image to approximately that of the clean image. Finally, the denoised image is achieved by a group-sparse model with the pattern from the guided image, where the eigenvectors are chosen in the error control of the noise deviation. Moreover, a modified group orthogonal matching pursuit algorithm is developed to efficiently solve the above group sparse model. The experiments show that our method not only improves the practicality of the EGL methods with the dependence reduction of the parameter setting, but also can outperform some well-developed denoising methods, especially for noise with large deviations.

Spectral-based mesh segmentation

In design and manufacturing, mesh segmentation is required for FACE construction in boundary representation (B-Rep), which in turn is central for feature-based design, machining, parametric CAD and reverse engineering, among others. Although mesh segmentation is dictated by geometry and topology, this article focuses on the topological aspect (graph spectrum), as we consider that this tool has not been fully exploited. We pre-process the mesh to obtain a edge-length homogeneous triangle set and its Graph Laplacian is calculated. We then produce a monotonically increasing permutation of the Fiedler vector (2nd eigenvector of Graph Laplacian) for encoding the connectivity among part feature sub-meshes. Within the mutated vector, discontinuities larger than a threshold (interactively set by a human) determine the partition of the original mesh. We present tests of our method on large complex meshes, which show results which mostly adjust to B-Rep FACE partition. The achieved segmentations properly locate most manufacturing features, although it requires human interaction to avoid over segmentation. Future work includes an iterative application of this algorithm to progressively sever features of the mesh left from previous sub-mesh removals.

Matched signal detection on graphs: Theory and application to brain imaging data classification

Motivated by recent progress in signal processing on graphs, we have developed a matched signal detection (MSD) theory for signals with intrinsic structures described by weighted graphs. First, we regard graph Laplacian eigenvalues as frequencies of graph-signals and assume that the signal is in a subspace spanned by the first few graph Laplacian eigenvectors associated with lower eigenvalues. The conventional matched subspace detector can be applied to this case. Furthermore, we study signals that may not merely live in a subspace. Concretely, we consider signals with bounded variation on graphs and more general signals that are randomly drawn from a prior distribution. For bounded variation signals, the test is a weighted energy detector. For the random signals, the test statistic is the difference of signal variations on associated graphs, if a degenerate Gaussian distribution specified by the graph Laplacian is adopted. We evaluate the effectiveness of the MSD on graphs both with simulated and real data sets. Specifically, we apply MSD to the brain imaging data classification problem of Alzheimer's disease (AD) based on two independent data sets: 1) positron emission tomography data with Pittsburgh compound-B tracer of 30 AD and 40 normal control (NC) subjects, and 2) resting-state functional magnetic resonance imaging (R-fMRI) data of 30 early mild cognitive impairment and 20 NC subjects. Our results demonstrate that the MSD approach is able to outperform the traditional methods and help detect AD at an early stage, probably due to the success of exploiting the manifold structure of the data.

Distributed finite-time consensus of nonlinear systems under switching topologies

In this paper, the finite time consensus problem of distributed nonlinear systems is studied under the general setting of directed and switching topologies. Specifically, a contraction mapping argument is used to investigate performance of networked control systems, two classes of varying topologies are considered, and distributive control designs are presented to guarantee finite time consensus. The proposed control scheme employs a distributed observer to estimate the first left eigenvector of graph Laplacian and, by exploiting this knowledge of network connectivity, it can handle switching topologies. The proposed methodology ensures finite time convergence to consensus under varying topologies of either having a globally reachable node or being jointly strongly connected, and the topological requirements are less restrictive than those in the existing results. Numerical examples are provided to illustrate the effectiveness of the proposed scheme.

Hierarchical graph Laplacian eigen transforms

Hierarchical graph Laplacian eigen transforms