Spectral Graph Clustering Research Articles

Extending Undirected Graph Techniques to Directed Graphs via Category Theory

We use Category Theory to construct a ‘bridge’ relating directed graphs with undirected graphs, such that the notion of direction is preserved. Specifically, we provide an isomorphism between the category of simple directed graphs and a category we call ‘prime graphs category’; this has as objects labeled undirected bipartite graphs (which we call prime graphs), and as morphisms undirected graph morphisms that preserve the labeling (which we call prime graph morphisms). This theoretical bridge allows us to extend undirected graph techniques to directed graphs by converting the directed graphs into prime graphs. To give a proof of concept, we show that our construction preserves topological features when applied to the problems of network alignment and spectral graph clustering.

Equivalence between Graph Spectral Clustering and Column Subset Selection (Student Abstract)

The common criteria for evaluating spectral clustering are NCut and RatioCut. The seemingly unrelated column subset selection (CSS) problem aims to compute a column subset that linearly approximates the entire matrix. A common criterion is the approximation error in the Frobenius norm (ApproxErr). We show that any algorithm for CSS can be viewed as a clustering algorithm that minimizes NCut by applying it to a matrix formed from graph edges. Conversely, any clustering algorithm can be seen as identifying a column subset from that matrix. In both cases, ApproxErr and NCut have the same value. Analogous results hold for RatioCut with a slightly different matrix. Therefore, established results for CSS can be mapped to spectral clustering. We use this to obtain new clustering algorithms, including an optimal one that is similar to A*. This is the first nontrivial clustering algorithm with such an optimality guarantee. A variant of the weighted A* runs much faster and provides bounds on the accuracy. Finally, we use the results from spectral clustering to prove the NP-hardness of CSS from sparse matrices.

Pass-Efficient Algorithms for Graph Spectral Clustering (Student Abstract)

Graph spectral clustering is a fundamental technique in data analysis, which utilizes eigenpairs of the Laplacian matrix to partition graph vertices into clusters. However, classical spectral clustering algorithms require eigendecomposition of the Laplacian matrix, which has cubic time complexity. In this work, we describe pass-efficient spectral clustering algorithms that leverage recent advances in randomized eigendecomposition and the structure of the graph vertex-edge matrix. Furthermore, we derive formulas for their efficient implementation. The resulting algorithms have a linear time complexity with respect to the number of vertices and edges and pass over the graph constant times, making them suitable for processing large graphs stored on slow memory. Experiments validate the accuracy and efficiency of the algorithms.

Self-Supervised Framework Based on Subject-Wise Clustering for Human Subject Time Series Data

With the widespread adoption of IoT, wearable devices, and sensors, time series data from human subjects are significantly increasing in the healthcare domain. Due to the laborious nature of manual annotation in time series data and the requirement for human experts, self-supervised learning methods are attempted to alleviate the limited label situations. While existing self-supervised methods have been successful to achieve comparable performance to the fully supervised methods, there are still some limitations that need to be addressed, considering the nature of time series data from human subjects: In real-world clinical settings, data labels (e.g., sleep stages) are usually annotated by subject-level, and there is a substantial variation in patterns between subjects. Thus, a model should be designed to deal with not only the label scarcity but also subject-wise nature of data to ensure high performance in real-world scenarios. To mitigate these issues, we propose a novel self-supervised learning framework for human subject time series data: Subject-Aware Time Series Clustering (SA-TSC). In the unsupervised representation learning phase, SA-TSC adopts a subject-wise learning strategy rather than instance-wise learning which randomly samples data instances from different subjects within the batch during training. Specifically, we generate subject-graphs with our graph construction method based on Gumbel-Softmax and perform graph spectral clustering on each subject-graph. In addition, we utilize graph neural networks to capture dependencies between channels and design our own graph learning module motivated from self-supervised loss. Experimental results show the outstanding performance of our SA-TSC with the limited & subject-wise label setting, leading to its high applicability to the healthcare industry. The code is available at: https://github.com/DILAB-HYU/SA-TSC

Towards explainability of hashtags in the light of Graph Spectral Clustering methods

Towards explainability of hashtags in the light of Graph Spectral Clustering methods

Graph-based pattern recognition on spectral reduced graphs

Graph-based pattern recognition – in particular in conjunction with large graphs – is often computationally expensive. This hampers, or makes it at least challenging, to employ graph-based representations for real-world data. To address this issue, we propose a method for reducing the size of the underlying graphs to their most important substructures using spectral graph clustering. The proposed method partitions the nodes of the graphs into clusters and then merges each cluster into supernodes. The motivation of this procedure is to reduce the computational cost of any graph comparison algorithm while maintaining the accuracy of the final classification. To assess the benefits and limitations of our method, we conduct thorough experiments on nine real-world datasets with different levels of graph reductions. The classification is obtained by four different graph classifiers (viz. a KNN based on graph edit distance, two SVMs based on a shortest path graph and a Weisfeiler–Lehman graph kernel, as well as a graph neural network). The results indicate that we can reduce computation time by up to two orders of magnitude without substantially degrading the classification accuracy.

Spectral Graph Clustering for Intentional Islanding Operations in Resilient Hybrid Energy Systems

Establishing cleaner energy generation, therefore, improving the sustainability of the power system is a crucial task in this century, and one of the key strategies being pursued is to shift the dependence on fossil fuel to renewable technologies, such as wind, solar, and nuclear. However, with the increasing number of heterogeneous components included, the complexity of the hybrid energy system becomes more significant, and the complex system imposes a more stringent requirement of the contingency plan to enhance the overall system resilience. Among different strategies to ensure a reliable system, intentional islanding is commonly applied in practical applications for power systems and attracts abundant interest in the literature. In this study, we propose a hierarchical spectral clustering-based intentional islanding strategy at the transmission level with renewable generations. To incorporate the renewable generation that relies on the inverter technology, the frequency measurements are considered to represent the transient response, and it has been further used as embedded information in the clustering algorithm along with other important electrical information from the system to enrich the modeling capability of the proposed framework. To demonstrate the effectiveness of the islanding strategy, the modified IEEE 9-bus system coupled with wind farms is considered as the test case.

Circuit analysis of the Drosophila brain using connectivity-based neuronal classification reveals organization of key communication pathways.

We present a functionally relevant, quantitative characterization of the neural circuitry of Drosophila melanogaster at the mesoscopic level of neuron types as classified exclusively based on potential network connectivity. Starting from a large neuron-to-neuron brain-wide connectome of the fruit fly, we use stochastic block modeling and spectral graph clustering to group neurons together into a common "cell class" if they connect to neurons of other classes according to the same probability distributions. We then characterize the connectivity-based cell classes with standard neuronal biomarkers, including neurotransmitters, developmental birthtimes, morphological features, spatial embedding, and functional anatomy. Mutual information indicates that connectivity-based classification reveals aspects of neurons that are not adequately captured by traditional classification schemes. Next, using graph theoretic and random walk analyses to identify neuron classes as hubs, sources, or destinations, we detect pathways and patterns of directional connectivity that potentially underpin specific functional interactions in the Drosophila brain. We uncover a core of highly interconnected dopaminergic cell classes functioning as the backbone communication pathway for multisensory integration. Additional predicted pathways pertain to the facilitation of circadian rhythmic activity, spatial orientation, fight-or-flight response, and olfactory learning. Our analysis provides experimentally testable hypotheses critically deconstructing complex brain function from organized connectomic architecture.

GraphPlanner: Floorplanning with Graph Neural Network

Chip floorplanning has long been a critical task with high computation complexity in the physical implementation of VLSI chips. Its key objective is to determine the initial locations of large chip modules with minimized wirelength while adhering to the density constraint, which in essence is a process of constructing an optimized mapping from circuit connectivity to physical locations. Proven to be an NP-hard problem, chip floorplanning is difficult to be solved efficiently using algorithmic approaches. This article presents GraphPlanner, a variational graph-convolutional-network-based deep learning technique for chip floorplanning. GraphPlanner is able to learn an optimized and generalized mapping between circuit connectivity and physical wirelength and produce a chip floorplan using efficient model inference. GraphPlanner is further equipped with an efficient clustering method, a unification of hyperedge coarsening with graph spectral clustering, to partition a large-scale netlist into high-quality clusters with minimized inter-cluster weighted connectivity. GraphPlanner has been integrated with two state-of-the-art mixed-size placers. Experimental studies using both academic benchmarks and industrial designs demonstrate that compared to state-of-the-art mixed-size placers alone, GraphPlanner improves placement runtime by 25% with 4% wirelength reduction on average.

An improved Nyström spectral graph clustering using k-core decomposition as a sampling strategy for large networks

Clustering on graphs (networks) is becoming intractable due to increasing sizes. Nyström spectral graph clustering (NSC) is a popular method to circumvent the problem. However, NSC currently faces two issues: (1) how to efficiently obtain representative samples for large networks; (2) the NSC is irrational for the mapping of nodes. To address the issues, in this paper, we propose an improved Nyström spectral graph clustering based on k-core decomposition sampling for large networks. In the proposed method, we first employ k-core decomposition as a sampling method, then use the samples to conduct the NSC to acquire the clusters of the sample nodes and the nodes connected with the samples, and finally utilize the proposed label propagation algorithm to group the remaining nodes into previously found clusters. To evaluate the performance, we use spectral clustering and NSC as baselines and compare our algorithm with the baselines on 9 small networks. Moreover, the proposed method is applied to 5 large networks. The proposed method can achieve a Modularity of 0.355 and cost 1234.95 s for a large network with approximately 120 million edges, which demonstrates that the proposed method has higher accuracy than NSC and higher efficiency than spectral clustering.

Latent structure blockmodels for Bayesian spectral graph clustering

Spectral embedding of network adjacency matrices often produces node representations living approximately around low-dimensional submanifold structures. In particular, hidden substructure is expected to arise when the graph is generated from a latent position model. Furthermore, the presence of communities within the network might generate community-specific submanifold structures in the embedding, but this is not explicitly accounted for in most statistical models for networks. In this article, a class of models called latent structure block models (LSBM) is proposed to address such scenarios, allowing for graph clustering when community-specific one-dimensional manifold structure is present. LSBMs focus on a specific class of latent space model, the random dot product graph (RDPG), and assign a latent submanifold to the latent positions of each community. A Bayesian model for the embeddings arising from LSBMs is discussed, and shown to have a good performance on simulated and real-world network data. The model is able to correctly recover the underlying communities living in a one-dimensional manifold, even when the parametric form of the underlying curves is unknown, achieving remarkable results on a variety of real data.

Spectral graph clustering via the expectation-solution algorithm

The stochastic blockmodel (SBM) models the connectivity within and between disjoint subsets of nodes in networks. Prior work demonstrated that the rows of an SBM’s adjacency spectral embedding (ASE) and Laplacian spectral embedding (LSE) both converge in law to Gaussian mixtures where the components are curved exponential families. Maximum likelihood estimation via the Expectation-Maximization (EM) algorithm for a full Gaussian mixture model (GMM) can then perform the task of clustering graph nodes, albeit without appealing to the components’ curvature. Noting that EM is a special case of the Expectation-Solution (ES) algorithm, we propose two ES algorithms that allow us to take full advantage of these curved structures. After presenting the ES algorithm for the general curved-Gaussian mixture, we develop those corresponding to the ASE and LSE limiting distributions. Simulating from artificial SBMs and a brain connectome SBM reveals that clustering graph nodes via our ES algorithms can improve upon that of EM for a full GMM for a wide range of settings.

XDeathDB: a visualization platform for cell death molecular interactions

Lots of cell death initiator and effector molecules, signalling pathways and subcellular sites have been identified as key mediators in both cell death processes in cancer. The XDeathDB visualization platform provides a comprehensive cell death and their crosstalk resource for deciphering the signaling network organization of interactions among different cell death modes associated with 1461 cancer types and COVID-19, with an aim to understand the molecular mechanisms of physiological cell death in disease and facilitate systems-oriented novel drug discovery in inducing cell deaths properly. Apoptosis, autosis, efferocytosis, ferroptosis, immunogenic cell death, intrinsic apoptosis, lysosomal cell death, mitotic cell death, mitochondrial permeability transition, necroptosis, parthanatos, and pyroptosis related to 12 cell deaths and their crosstalk can be observed systematically by the platform. Big data for cell death gene-disease associations, gene-cell death pathway associations, pathway-cell death mode associations, and cell death-cell death associations is collected by literature review articles and public database from iRefIndex, STRING, BioGRID, Reactom, Pathway’s commons, DisGeNET, DrugBank, and Therapeutic Target Database (TTD). An interactive webtool, XDeathDB, is built by web applications with R-Shiny, JavaScript (JS) and Shiny Server Iso. With this platform, users can search specific interactions from vast interdependent networks that occur in the realm of cell death. A multilayer spectral graph clustering method that performs convex layer aggregation to identify crosstalk function among cell death modes for a specific cancer. 147 hallmark genes of cell death could be observed in detail in these networks. These potential druggable targets are displayed systematically and tailoring networks to visualize specified relations is available to fulfil user-specific needs. Users can access XDeathDB for free at https://pcm2019.shinyapps.io/XDeathDB/.

Simultaneous Dimensionality and Complexity Model Selection for Spectral Graph Clustering

Our problem of interest is to cluster vertices of a graph by identifying underlying community structure. Among various vertex clustering approaches, spectral clustering is one of the most popular methods because it is easy to implement while often outperforming more traditional clustering algorithms. However, there are two inherent model selection problems in spectral clustering, namely estimating both the embedding dimension and number of clusters. This article attempts to address the issue by establishing a novel model selection framework specifically for vertex clustering on graphs under a stochastic block model. The first contribution is a probabilistic model which approximates the distribution of the extended spectral embedding of a graph. The model is constructed based on a theoretical result of asymptotic normality for the informative part of the embedding, and on simulation results providing a conjecture for the limiting behavior of the redundant part of the embedding. The second contribution is a simultaneous model selection framework. In contrast with traditional approaches, our model selection procedure estimates embedding dimension and number of clusters simultaneously. Based on our conjectured distributional model, a theorem on the consistency of the estimates of model parameters is presented, providing support for the utility of our method. Algorithms for our simultaneous model selection (SMS) for vertex clustering are proposed, demonstrating superior performance in simulation experiments. We illustrate our method via application to a collection of brain graphs.

CLEAR: A Consistent Lifting, Embedding, and Alignment Rectification Algorithm for Multiview Data Association

Many robotics applications require alignment and fusion of observations obtained at multiple views to form a global model of the environment. Multiway data association methods provide a mechanism to improve alignment accuracy of pairwise associations and ensure their consistency. However, existing methods that solve this computationally challenging problem are often too slow for real-time applications. Furthermore, some of the existing techniques can violate the cycle consistency principle, thus drastically reducing the fusion accuracy. This article presents the consistent lifting, embedding, and alignment rectification (CLEAR) algorithm to address these issues. By leveraging insights from the multiway matching and spectral graph clustering literature, CLEAR provides cycle-consistent and accurate solutions in a computationally efficient manner. Numerical experiments on both synthetic and real datasets are carried out to demonstrate the scalability and superior performance of our algorithm in real-world problems. This algorithmic framework can provide significant improvement in the accuracy and efficiency of existing discrete assignment problems, which traditionally use pairwise (but potentially inconsistent) correspondences. An implementation of CLEAR is made publicly available online.

Hybridized Gradient Descent Spectral Graph and Local global Louvain Based Clustering of Temporal Relational Data

Temporal data clustering examines the time series data to determine the basic structure and other characteristics of the data. Many methodologies simply process the temporal dimension of data but it still faces the many challenges for extracting useful patterns due to complex data types. In order to analyze the complex temporal data, Hybridized Gradient Descent Spectral Graph and Local-Global Louvain Clustering (HGDSG-LGLC) technique are designed. The number of temporal data is gathered from input dataset. Then the HGDSG-LGLC technique performs graph-based clustering to partitions the vertices i.e. data into different clusters depending on similarity matrix spectrum. The distance similarity is measured between the data and cluster mean. The Gradient Descent function find minimum distance between data and cluster mean. Followed by, the Local-Global Louvain method performs the merging and filtering of temporal data to connect the local and global edges of the graph with similar data. Then for each data, the change in modularity is calculated for filtering the unwanted data from its own cluster and merging it into the neighboring cluster. As a result, optimal ‘k’ numbers of clusters are obtained with higher accuracy with minimum error rate. Experimental analysis is performed with various parameters like clustering accuracy ( ), error rate ( ), computation time ( ) and space complexity ( ) with respect to number of temporal data. The proposed HGDSG-LGLC technique achieves higher and lesser , minimum as well as than conventional methods.

Laplacian Matrix Based Spectral Graph Clustering

Recent attention in the research field of clustering is focused on grouping of clusters based on structure of a graph. At present, there are plentiful literature work has been proposed towards the clustering techniques but it is still an open challenge to find the best technique for clustering. This paper present a comprehensive review of our insights towards emerging clustering methods on graph based spectral clustering. Graph Laplacians have become a core technology for the spectral clustering which works based on the properties of the Laplacian matrix. In our study, we discuss correlation between similarity and Laplacian matrices within a graph and spectral graph theory concepts. Current studies on graph-based clustering methods requires a well defined good quality graph to achieve high clustering accuracy. This paper describes how spectral graph theory has been used in the literature of clustering concepts and how it helps to predict relationships that have not yet been identified in the existing literature. Some application areas on the graph clustering algorithms are discussed. This survey outlines the problems addressed by the existing research works on spectral clustering with its problems, methodologies, data sets and advantages. This paper identifies fundamental issues of graph clustering which provides a better direction for more applications in social network analysis, image segmentation, computer vision and other domains.

Residential Power Forecasting Using Load Identification and Graph Spectral Clustering

Forecasting energy or power usage is an important part of providing a stable supply of power to all customers on a power grid. We present a novel method that aims to forecast the power consumption of a single house, or a set of houses, based on non-intrusive load monitoring (NILM) and graph spectral clustering. In the proposed method, the aggregate power signal is decomposed into individual appliance signals and each appliance’s power is forecasted separately. Then the total power forecast is formed by aggregating forecasted power levels of individual appliances. We use four publicly available datasets (reference energy disaggregation dataset, rainforest automation energy, almanac of minutely power dataset version 2, tracebase ) to test our forecasting method and report its accuracy. The results show that our method is more accurate compared to popular existing approaches, such as autoregressive integrated moving average, similar profile load forecast, artificial neural network, and recent NILM-based forecasting.

On a two-truths phenomenon in spectral graph clustering

Clustering is concerned with coherently grouping observations without any explicit concept of true groupings. Spectral graph clustering-clustering the vertices of a graph based on their spectral embedding-is commonly approached via K-means (or, more generally, Gaussian mixture model) clustering composed with either Laplacian spectral embedding (LSE) or adjacency spectral embedding (ASE). Recent theoretical results provide deeper understanding of the problem and solutions and lead us to a "two-truths" LSE vs. ASE spectral graph clustering phenomenon convincingly illustrated here via a diffusion MRI connectome dataset: The different embedding methods yield different clustering results, with LSE capturing left hemisphere/right hemisphere affinity structure and ASE capturing gray matter/white matter core-periphery structure.

On-Orbit Event-Based Conformance Checking Using GPU

In the past few years, China's space science and application has entered the stage of the space station with large-scale space science experiments. The number of flight missions related to space science grows rapidly, meanwhile, the corresponding payloads from various fields of science become more complicated. This paper breaks through the traditional management mode in which the payloads of space science are usually monitored by space-ground telemetry links and proposes an approach to utilize workflow net models to monitor the experimental processes of on-orbit payloads in real time. This approach can effectively offload the pressure of space-ground communication as well as the pressure of ground monitor stations. In this approach, spectral graph clustering is used in decomposing a workflow net model when the model is too huge to be processed in parallel on one single GPU (graphic processing units) device, and hybrid parallel computing algorithms are designed to carry out conformance checking operations based on observed events from on-orbit payloads in real time. The algorithms are implemented with a gather-apply-scatter model in CUDA 9.0 (compute unified device architecture) on Jetson TX2i module and are benchmarked. The performance of the algorithms is acceptable for practical use on orbit.