Graph Filter Research Articles

NCGNN: Node-Level Capsule Graph Neural Network for Semisupervised Classification.

Message passing has evolved as an effective tool for designing graph neural networks (GNNs). However, most existing methods for message passing simply sum or average all the neighboring features to update node representations. They are restricted by two problems: 1) lack of interpretability to identify node features significant to the prediction of GNNs and 2) feature overmixing that leads to the oversmoothing issue in capturing long-range dependencies and inability to handle graphs under heterophily or low homophily. In this article, we propose a node-level capsule graph neural network (NCGNN) to address these problems with an improved message passing scheme. Specifically, NCGNN represents nodes as groups of node-level capsules, in which each capsule extracts distinctive features of its corresponding node. For each node-level capsule, a novel dynamic routing procedure is developed to adaptively select appropriate capsules for aggregation from a subgraph identified by the designed graph filter. NCGNN aggregates only the advantageous capsules and restrains irrelevant messages to avoid overmixing features of interacting nodes. Therefore, it can relieve the oversmoothing issue and learn effective node representations over graphs with homophily or heterophily. Furthermore, our proposed message passing scheme is inherently interpretable and exempt from complex post hoc explanations, as the graph filter and the dynamic routing procedure identify a subset of node features that are most significant to the model prediction from the extracted subgraph. Extensive experiments on synthetic as well as real-world graphs demonstrate that NCGNN can well address the oversmoothing issue and produce better node representations for semisupervised node classification. It outperforms the state of the arts under both homophily and heterophily.

CDC: A Simple Framework for Complex Data Clustering.

In today's digital era driven by data, the amount and complexity of the collected data, such as multiview, non-Euclidean, and multirelational, are growing exponentially or even faster. Clustering, which unsupervisedly extracts valid knowledge from data, is extremely useful in practice. However, existing methods are independently developed to handle one particular challenge at the expense of the others. In this work, we propose a simple but effective framework for complex data clustering (CDC) that can efficiently process different types of data with linear complexity. We first use graph filtering (GF) to fuse geometric structure and attribute information. We then reduce complexity with high-quality anchors that are adaptively learned via a novel similarity-preserving (SP) regularizer. We illustrate the cluster-ability of our proposed method theoretically and experimentally. In particular, we deploy CDC to graph data of size 111 M.

Joint Projection Learning and Tensor Decomposition-Based Incomplete Multiview Clustering.

Incomplete multiview clustering (IMVC) has received increasing attention since it is often that some views of samples are incomplete in reality. Most existing methods learn similarity subgraphs from original incomplete multiview data and seek complete graphs by exploring the incomplete subgraphs of each view for spectral clustering. However, the graphs constructed on the original high-dimensional data may be suboptimal due to feature redundancy and noise. Besides, previous methods generally ignored the graph noise caused by the interclass and intraclass structure variation during the transformation of incomplete graphs and complete graphs. To address these problems, we propose a novel joint projection learning and tensor decomposition (JPLTD)-based method for IMVC. Specifically, to alleviate the influence of redundant features and noise in high-dimensional data, JPLTD introduces an orthogonal projection matrix to project the high-dimensional features into a lower-dimensional space for compact feature learning. Meanwhile, based on the lower-dimensional space, the similarity graphs corresponding to instances of different views are learned, and JPLTD stacks these graphs into a third-order low-rank tensor to explore the high-order correlations across different views. We further consider the graph noise of projected data caused by missing samples and use a tensor-decomposition-based graph filter for robust clustering. JPLTD decomposes the original tensor into an intrinsic tensor and a sparse tensor. The intrinsic tensor models the true data similarities. An effective optimization algorithm is adopted to solve the JPLTD model. Comprehensive experiments on several benchmark datasets demonstrate that JPLTD outperforms the state-of-the-art methods. The code of JPLTD is available at https://github.com/weilvNJU/JPLTD.

Graph Filters for Signal Processing and Machine Learning on Graphs

Graph Filters for Signal Processing and Machine Learning on Graphs

Geometric Graph Filters and Neural Networks: Limit Properties and Discriminability Trade-Offs

Geometric Graph Filters and Neural Networks: Limit Properties and Discriminability Trade-Offs

Spatio-temporal data prediction of multiple air pollutants in multi-cities based on 4D digraph convolutional neural network.

In response to the problem that current multi-city multi-pollutant prediction methods based on one-dimensional undirected graph neural network models cannot accurately reflect the two-dimensional spatial correlations and directedness, this study proposes a four-dimensional directed graph model that can capture the two-dimensional spatial directed information and node correlation information related to multiple factors, as well as extract temporal correlation information at different times. Firstly, A four-dimensional directed GCN model with directed information graph in two-dimensional space was established based on the geographical location of the city. Secondly, Spectral decomposition and tensor operations were then applied to the two-dimensional directed information graph to obtain the graph Fourier coefficients and graph Fourier basis. Thirdly, the graph filter of the four-dimensional directed GCN model was further improved and optimized. Finally, an LSTM network architecture was introduced to construct the four-dimensional directed GCN-LSTM model for synchronous extraction of spatio-temporal information and prediction of atmospheric pollutant concentrations. The study uses the 2020 atmospheric six-parameter data of the Taihu Lake city cluster and applies canonical correlation analysis to confirm the data's temporal, spatial, and multi-factor correlations. Through experimentation, it is verified that the proposed 4D-DGCN-LSTM model achieves a MAE reduction of 1.12%, 4.91%, 5.62%, and 11.67% compared with the 4D-DGCN, GCN-LSTM, GCN, and LSTM models, respectively, indicating the good performance of the 4D-DGCN-LSTM model in predicting multiple types of atmospheric pollutants in various cities.

Brain-inspired GCN: Modularity-based Siamese simple graph convolutional networks

In graph representation learning, Graph Convolutional Networks (GCNs) and their variants have received much attention. However, GCNs encounter oversmoothing as the models get deeper, limiting their ability to aggregate node representations within high-order neighborhood. Inspired by the modular structure of the brain network, we propose Modularity-based Siamese Simple Graph Convolution (MS-SGC), a Siamese network architecture that incorporates the characteristics of brain modular structure into graph convolutional networks. Spectral clustering is leveraged to detect the modular structure in the graph, and then the weight of cross edges between modules is lowered. Siamese network is adopted to combine the modularity-preserved graph representation with the original graph representation, improving classification performance and reducing oversmoothing. Furthermore, a graph convolution method that functions as a linear low-pass graph filter is elaborated through spectral analysis to preserve the similarity of nodes within the same module and further alleviate the oversmoothing problem. We validate the effectiveness of MS-SGC on citation networks and extend our experimental analysis to various downstream tasks. Extensive experiments demonstrate that MS-SGC outperforms state-of-the-arts while reducing the time and computation complexity for node classification. Moreover, MS-SGC achieves competitive performance compared to other state-of-the-arts in node clustering, community prediction and text classification tasks.

AGCL: Adaptive Graph Contrastive Learning for graph representation learning

Unsupervised graph representation learning has attracted great attention as it can learn low-dimensional and compact node embeddings from high-dimensional and sparse graph data without labels. However, two major drawbacks exist in most previous methods, i.e., limited applications of the global graph structure and the problem of the false-negative samples.To address the above problems, we propose a novel Adaptive Graph Contrastive Learning (AGCL) method that utilizes multiple graph filters to capture both the local and the global view information and propose a novel adaptive graph contrastive learning framework to alleviate the false-negative sample problem. AGCL first utilizes a Graph Convolution Network (GCN) filter and our designed Diffusion-based filters to smooth the initial node features. Then, AGCL measures the node pair similarity and then iteratively selects the similar and the dissimilar node pairs as the positive and the negative samples to do the graph contrastive learning. Finally, AGCL leverages various aggregators to obtain node embeddings from multiple views. In addition, we further propose AGCL-Light which can reduce the complexity of updating the training samples, which pre-selects a similar node subset for each node for the subsequent updates. Extensive experiments on nine benchmark datasets demonstrate that our models outperform previous state-of-the-art methods.

Recognition of building shape in maps using deep graph filter neural network

Shape is one of the core features of the buildings which are the main elements of the map. The building shape recognition is widely used in many spatial applications. Due to the irregularity of the building contour, it is still challenging for building shape recognition. Inspired by graph signal processing theory, we propose a deep graph filter neural network (DGFN) for the shape recognition of buildings in maps. First, we regard shape recognition as a combination of subjective and objective graph signal filtering process. Second, we construct a shape features extraction framework from the perspective of shape details, shape structure and shape local information. Third, DGFN model can fulfil the tasks of shape classification and shape embedding of building at the same time. Finally, multi angle experiments verify our viewpoint of shape recognition mechanism, and the comparison with similar algorithms proves the high accuracy and availability of DGFN model.

Beyond low-pass filtering on large-scale graphs via Adaptive Filtering Graph Neural Networks

Graph Neural Networks (GNNs) have emerged as a crucial deep learning framework for graph-structured data. However, existing GNNs suffer from the scalability limitation, which hinders their practical implementation in industrial settings. Many scalable GNNs have been proposed to address this limitation. However, they have been proven to act as low-pass graph filters, which discard the valuable middle- and high-frequency information. This paper proposes a novel graph neural network named Adaptive Filtering Graph Neural Networks (AFGNN), which can capture all frequency information on large-scale graphs. AFGNN consists of two stages. The first stage utilizes low-, middle-, and high-pass graph filters to extract comprehensive frequency information without introducing additional parameters. This computation is a one-time task and is pre-computed before training, ensuring its scalability. The second stage incorporates a node-level attention-based feature combination, enabling the generation of customized graph filters for each node, contrary to existing spectral GNNs that employ uniform graph filters for the entire graph. AFGNN is suitable for mini-batch training, and can enhance scalability and efficiently capture all frequency information from large-scale graphs. We evaluate AFGNN by comparing its ability to capture all frequency information with spectral GNNs, and its scalability with scalable GNNs. Experimental results illustrate that AFGNN surpasses both scalable GNNs and spectral GNNs, highlighting its superiority.

Adaptive Consensus Clustering for Multiple K-Means Via Base Results Refining

Consensus clustering, which learns a consensus clustering result from multiple weak base results, has been widely studied. However, conventional consensus clustering methods only focus on the ensemble process while ignoring the quality improvement of the base results, and thus they just use the fixed base results for consensus learning. In this paper, we provide an alternative idea to improve the final consensus clustering performance by considering the base results refining. In our framework, we adaptively refine the base results in the process of the ensemble. In more detail, on one hand, we ensemble multiple K-means results to learn the consensus one by considering the consensus and diversity; on the other hand, we apply the consensus result to design a graph filter to learn a more cluster-friendly embedding for refining the base K-means results. In our framework, the consensus learning and base results refining are integrated into one unified objective function so that these two tasks can be boosted by each other. Then we design an effective iterative algorithm to optimize the carefully designed objective function. The extensive experiments on benchmark data sets demonstrate that the proposed method can outperform both the single clustering and the state-of-the-art consensus clustering methods. The codes of this paper are released in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://Doctor-Nobody.github.io/codes/ACMK.zip</uri> .

A multilayered graph-based framework to explore behavioural phenomena in social media conversations

ObjectiveSocial media is part of current health communications. This research aims to delve into the effects of social contagion, biased assimilation, and homophily in building and changing health opinions on social media. Materials and methodsConversations about COVID-19 vaccination on English and Spanish Twitter are the case studies. A new multilayered graph-based framework supports the integrated analysis of content similarity within and across posts, users, and conversations to interpret contrasting and confluent user stances. Deep learning models are applied to infer stance. Graph centrality and homophily scores support the interpretation of information reproduction. ResultsThe results show that semantically related English posts tend to present a similar stance about COVID-19 vaccination (rstance = 0.51) whereas Spanish posts are more heterophilic (rstance = 0.38). Neither case showed evidence of homophily regarding user influence or vaccine hashtags. Graph filters for Pfizer and Astrazeneca with a similarity threshold of 0.85 show stance homophily in English scenarios (i.e. rstance = 0.45 and rstance = 0.58, respectively) and small homophily in Spanish scenarios (i.e. r = 0.12 and r = 0.3, respectively). Highly connected users are a minority and are not socially influential. Spanish conversations showed stance homophily, i.e. most of the connected conversations promote vaccination (rstance = 0.42), whereas English conversations are more likely to offer contrasting stances. ConclusionThe methodology proposed for quantifying the impact of natural and intentional social behaviours in health information reproduction can be applied to any of the main social platforms and any given topic of conversation. Its effectiveness was demonstrated by two case studies describing English and Spanish demographic and sociocultural scenarios.

Multikernel adaptive filtering over graphs based on normalized LMS algorithm

To address the difficulty and inflexibility associated with choosing kernel parameters for single kernel adaptive filters (KAFs), this article proposes a multikernel adaptive filter for graph signals based on the least mean square (LMS) algorithm. First, normalized by its largest eigenvalue, the combinatorial graph Laplacian is adopted as the graph shift operator (GSO) to preprocess graph input signals. Then, the graph multikernel normalized least mean square (GMKNLMS) algorithm is developed to estimate nonlinear graph filter coefficients. To limit the growth in dictionary size, a coherence-check (CC) based sparsification method is introduced to form the new GMKNLMS-CC algorithm. In addition, numerical simulation examples are presented to demonstrate the improved performance of the proposed algorithms compared with the linear graph least mean square (GLMS) and graph kernel normalized least mean square (GKNLMS) algorithms. Finally, the real sensor measurement data taken from the Intel Lab is used as time-varying graph signals to demonstrate the tracking performance of the GMKNLMS-CC algorithm.

MLGAL: Multi-level Label Graph Adaptive Learning for node clustering in the attributed graph

Node clustering aims to divide nodes into disjoint groups. Recently, a considerable amount of research leverages Graph Neural Networks (GNNs) to learn compact node embeddings, which are then used as input of the traditional clustering methods to get better clustering results. While in most of these methods the node representation learning and the clustering are performed separately, a few works have further coupled them in a self-supervised learning manner. The coupling, however, should be carefully designed to avoid potential noises in the pseudo labels generated automatically during the training process.To address the above problems, in this article, we propose Multi-level Label Graph Adaptive Learning (MLGAL), a novel unsupervised learning algorithm for the node clustering problem. We first design a graph filter to smooth the node features. Then, we iteratively choose the similar and the dissimilar node pairs to perform the adaptive learning with the multi-level label, i.e., the node-level label and the cluster-level label generated automatically by our model. We conduct extensive experiments on four benchmark datasets to evaluate the performance of our model. The results demonstrate that our model outperforms the state-of-the-art baselines on four benchmark datasets.

High-order multi-view clustering for generic data

Graph-based multi-view clustering has achieved better performance than most non-graph approaches. However, in many real-world scenarios, the graph structure of data is not given or the quality of initial graph is poor. Additionally, existing methods largely neglect the high-order neighborhood information that characterizes complex intrinsic interactions. To tackle these problems, we introduce an approach called high-order multi-view clustering (HMvC) to explore the long-distance structural information of generic data. Firstly, graph filtering is applied to encode structure information, which unifies the processing of attributed graph data and non-graph data in a single framework. Secondly, up to infinity-order intrinsic relationships are exploited to enrich the learned graph. Thirdly, to explore the consistent and complementary information of various views, an adaptive graph fusion mechanism is proposed to achieve a consensus graph. Comprehensive experimental results on both non-graph and attributed graph data show the superior performance of our method with respect to various state-of-the-art techniques, including some deep learning methods.

Graph neural network-based remote target classification in hyperspectral imaging

ABSTRACT Hyperspectral Imagery (HSI) onboard target classification is a frequently used technique in remote sensing. Due to the small volume of labelled data, the classification of HSI has been a challenging subject in recent study. The graph neural network (GNN) has become a popular method for semi-supervised classification, and its use with hyperspectral images has gained a lot of interest. A graph filter or single graph neural network has been typically employed in the earlier GNN-based approaches, and it is used to obtain HSI characteristics. It has not fully utilized the benefits of different graph filters or graph neural networks. The issue of oversmoothing also persists with the classical GNNs. We propose a spectral filter and an autoregressive moving average filter for the multi-graph neural network; (SAF-GNN) to address such drawbacks as the one mentioned above. The first is excellent at extracting the nodes’ spectral characteristics, and the second is effective at suppressing graph distortion. With the help of performance measures like overall accuracy (OA), individual class accuracy (ICA), and Kappa coefficient (KC), the performance of the proposed method was compared to that of other state-of-the-art methods. The result shows that the SAF-GNN provides an improvement in OA, ICA and KC of 2.21%, 7.93% and 0.025 for the cuprite dataset and 4.84%, 11.78% and 0.025 for the Pavia University dataset, respectively. Comparison results show that the proposed method is suitable for remote target classification problem.

A diffusion strategy for robust distributed estimation based on streaming graph signals

The problem of robust distributed estimation over dynamic and streaming graph signals is investigated in this paper. Existing works related to distributed estimation over dynamic and streaming graph signals are mainly derived from the Mean-Square-Error criterion, and they are vulnerable to non-Gaussian noise. Therefore, a new kind of diffusion Mixture correntropy (d-MC) algorithm is developed to deal with the disadvantage in this paper. Incorporating the diffusion strategy and a novel cost function, the proposed algorithm could accurately estimate the graph filter parameter with the dynamic and streaming graph signals, and achieve desirable performance under both Gaussian and impulsive noise environment. Besides, the theoretical analysis results of mean and mean-square stability are derived. Simulations on various case studies indicate the desirable performance of proposed d-MC algorithm by comparing it to other benchmarks.

Minimax design of two-channel critically sampled graph QMF banks

We address here the issue of designing high spectral selectivity graph filter banks, which require high-degree polynomial approximation. To achieve sharp transition band roll required for high-selectivity, the minimax criterion is used in the design formulation. A constraint optimization method is used for the design process, and a spectral transformation is employed to circumvent the numerical problems that arise with high-degree polynomials. Using the transformation, approximation problem can be recast as an approximation problem with trigonometric polynomials. The transformed constraint optimization problem can be solved using the readily available sequential quadratic programming (SQP) method. High order filters, with low maximum approximation error (MAE) and low maximum reconstruction error (MRE), can be designed with this method. An algorithm is also developed that allows the user to choose design parameters that will give the lowest possible MRE. The method can also be adapted to trade-off between reconstruction quality and spectral selectivity of the resulting filter bank. The suite of techniques presented allow the user to easily shape the spectral response of the filters, to suit the application at hand. Numerous examples, with comparisons, will be presented to demonstrate the versatility of the proposed techniques. Applications to nonlinear approximation and denoising of graph signal will also be considered.

Fourier-based and Rational Graph Filters for Spectral Processing.

Data are represented as graphs in a wide range of applications, such as Computer Vision (e.g., images) and Graphics (e.g., 3D meshes), network analysis (e.g., social networks), and bio-informatics (e.g., molecules). In this context, our overall goal is the definition of novel Fourier-based and graph filters induced by rational polynomials for graph processing, which generalise polynomial filters and the Fourier transform to non-Euclidean domains. For the efficient evaluation of discrete spectral Fourier-based and wavelet operators, we introduce a spectrum-free approach, which requires the solution of a small set of sparse, symmetric, and well-conditioned linear systems and is oblivious of the evaluation of the Laplacian or kernel spectrum. Approximating arbitrary graph filters with rational polynomials provides a more accurate and numerically stable alternative with respect to polynomials. To achieve these goals, we also study the link between spectral operators, wavelets, and filtered convolution with integral operators induced by spectral kernels.

Graph Federated Learning for CIoT Devices in Smart Home Applications

This article deals with the problem of statistical and system heterogeneity in a cross-silo federated learning (FL) framework where there exist a limited number of Consumer Internet of Things (CIoT) devices in a smart building. We propose a novel graph signal processing (GSP)-inspired aggregation rule based on graph filtering dubbed “G-Fedfilt.” The proposed aggregator enables a structured flow of information based on the graph’s topology. This behavior allows capturing the interconnection of CIoT devices and training domain-specific models. The embedded graph filter is equipped with a tunable parameter which enables a continuous tradeoff between domain-agnostic and domain-specific FL. In the case of domain-agnostic, it forces G-Fedfilt to act similar to the conventional federated averaging (FedAvg) aggregation rule. The proposed G-Fedfilt also enables an intrinsic smooth clustering based on the graph connectivity without explicitly specified which further boosts the personalization of the models in the framework. In addition, the proposed scheme enjoys a communication-efficient time scheduling to alleviate the system heterogeneity. This is accomplished by adaptively adjusting the amount of training data samples and sparsity of the models’ gradients to reduce communication desynchronization and latency. Simulation results show that the proposed G-Fedfilt achieves up to 3.99% better classification accuracy than the conventional FedAvg when concerning model personalization on the statistically heterogeneous local data sets, while it is capable of yielding up to 2.41% higher accuracy than FedAvg in the case of testing the generalization of the models. Furthermore, the proposed communication optimization scheme can boost the framework’s efficiency by reducing the computation, communication desynchronization, and latency up to 70.21%, 99.65%, and 44.61%, respectively, at the cost of 0.36% accuracy and under the system heterogeneity.