Graph Perturbation Research Articles

Generating graph perturbations to enhance the generalization of GNNs

Graph neural networks (GNNs) have become the standard approach for performing machine learning on graphs. Such models need large amounts of training data, however, in several graph classification and regression tasks, only limited training data is available. Unfortunately, due to the complex nature of graphs, common augmentation strategies employed in other settings, such as computer vision, do not apply to graphs. This work aims to improve the generalization ability of GNNs by increasing the size of the training set of a given problem. The new samples are generated using an iterative contrastive learning procedure that augments the dataset during the training, in a task-relevant approach, by manipulating the graph topology. The proposed approach is general, assumes no knowledge about the underlying architecture, and can thus be applied to any GNN. We provided a theoretical analysis regarding the equivalence of the proposed approach to a regularization technique. We demonstrate instances of our framework on popular GNNs, and evaluate them on several real-world benchmark graph classification datasets. The experimental results show that the proposed approach, in several cases, enhances the generalization of the underlying prediction models reaching in some datasets state-of-the-art performance.

Distribution-aware hybrid noise augmentation in graph contrastive learning for recommendation

The recommender systems are one of the most effective big data tools for solving the information overload problem, but data sparsity greatly affects its performance. However, most of the existing graph-based contrastive learning methods perturb the original graph structure for data augmentation, which may lead to semantic loss and subsequently degrade the recommendation performance. Meanwhile, studies have shown that improving the uniformity of data distribution is more effective than data augmentation using graph perturbation. In this paper, to enhance the uniformity of data distribution, we propose Distribution-Aware Hybrid Noise augmentation in graph contrastive learning for Recommendation (DAHNRec). Specifically, we design distribution-aware hybrid noise augmentation (DAHN) method to generate noise suitable for the users and items, instead of simply utilizing random noise following uniform distributions. By applying DAHN, the model can learn the appropriate noise distribution during the training process to enhance the uniformity of the embedding space. Additionally, we propose Balanced Bayesian Personalized Ranking (BBPR) as the loss function for recommendation tasks to improve difference characterizing between positive and negative samples. This improves the ranking tasks’ performance. Furthermore, we conduct extensive experiments based on public benchmark datasets. The results indicate that DAHNRec outperforms several existing methods. Furthermore, DAHNRec is a general recommendation model based on user–item interactions, which can be used to improve the recommendation performance in various real-world scenarios, such as e-commerce platforms and social media.

Isospectral graphs via spectral bracketing

In this article, we develop a perturbative technique to construct families of non-isomorphic discrete graphs which are isospectral for the standard (also called normalised) Laplacian and its signless version. We use vertex contractions as a graph perturbation and spectral bracketing with auxiliary graphs which have certain eigenvalues with high multiplicity. There is no need to know explicitly the eigenfunctions of the corresponding graphs. In principle, one only needs to know the multiplicity of the eigenvalues of the auxiliary graphs, and that these eigenvalues are all different. We illustrate the method by presenting several families of examples of isospectral graphs including fuzzy balls, complete bipartite graphs and subdivision graphs obtained from the previous examples. All the examples constructed turn out to be also isospectral for the standard (Kirchhoff) Laplacian on the associated equilateral metric graph.

A graph transformer defence against graph perturbation by a flexible-pass filter

Graph perturbation hinders graph models in real applications, and thus defense methods against graph perturbation have been attracting increasing attention. However, current defense methods limit expressiveness and demand expert knowledge. To overcome these issues, in this paper, we propose a flexible-frequency graph transformer, building on the powerful expressive ability of self-attention. Specifically, we design a frequency-extraction self-attention with three heads to extract multi-frequency representations, i.e., low-frequency representation, hybrid-frequency representation, and high-frequency representation. An adaptive fusion method is then designed to combine diverse representations to output a flexible-frequency representation. This improves the model’s expressive ability and also enhances defense against graph perturbation by utilizing comprehensive information. In addition, we adaptively capture robust graph filters within self-attention, eliminating the need for expert knowledge. To boost self-attention’s effectiveness, we integrate graph learning to capture graph information before conducting node representation learning within self-attention. Furthermore, we theoretically analyze the feasibility of our proposed method. Extensive experiments demonstrate that our proposed method offers a dynamic and effective defense against graph perturbation compared to existing state-of-the-art methods.

Incorporating self-attentions into robust spatial-temporal graph representation learning against dynamic graph perturbations

Incorporating self-attentions into robust spatial-temporal graph representation learning against dynamic graph perturbations

Quantifying the Structural Stability of Simplicial Homology

Simplicial complexes are generalizations of classical graphs. Their homology groups are widely used to characterize the structure and the topology of data in e.g. chemistry, neuroscience, and transportation networks. In this work we assume we are given a simplicial complex and that we can act on its underlying graph, formed by the set of 1-simplices, and we investigate the stability of its homology with respect to perturbations of the edges in such graph. Precisely, exploiting the isomorphism between the homology groups and the higher-order Laplacian operators, we propose a numerical method to compute the smallest graph perturbation sufficient to change the dimension of the simplex’s Hodge homology. Our approach is based on a matrix nearness problem formulated as a matrix differential equation, which requires an appropriate weighting and normalizing procedure for the boundary operators acting on the Hodge algebra’s homology groups. We develop a bilevel optimization procedure suitable for the formulated matrix nearness problem and illustrate the method’s performance on a variety of synthetic quasi-triangulation datasets and real-world transportation networks.

Toward Certified Robustness of Graph Neural Networks in Adversarial AIoT Environments

Graph neural networks (GNNs) have transformed network analysis, leading to state-of-the-art performance across a variety of tasks. Especially, GNNs are increasingly been employed as detection tools in the AIoT environment in various security applications. However, GNNs have also been shown vulnerable to adversarial graph perturbation. We present the first approach for certifying robustness of general GNNs against attacks that add or remove graph edges either at training or prediction time. Extensive experiments demonstrate that our approach significantly outperforms prior art in certified robust predictions. In addition, we show that a non-certified adaptation of our method exhibits significantly better robust accuracy against state-of-the-art attacks that past approaches. Thus, we achieve both the best certified bounds and best practical robustness of GNNs to structural attacks to date.

Exploring Faithful Rationale for Multi-Hop Fact Verification via Salience-Aware Graph Learning

The opaqueness of the multi-hop fact verification model imposes imperative requirements for explainability. One feasible way is to extract rationales, a subset of inputs, where the performance of prediction drops dramatically when being removed. Though being explainable, most rationale extraction methods for multi-hop fact verification explore the semantic information within each piece of evidence individually, while ignoring the topological information interaction among different pieces of evidence. Intuitively, a faithful rationale bears complementary information being able to extract other rationales through the multi-hop reasoning process. To tackle such disadvantages, we cast explainable multi-hop fact verification as subgraph extraction, which can be solved based on graph convolutional network (GCN) with salience-aware graph learning. In specific, GCN is utilized to incorporate the topological interaction information among multiple pieces of evidence for learning evidence representation. Meanwhile, to alleviate the influence of noisy evidence, the salience-aware graph perturbation is induced into the message passing of GCN. Moreover, the multi-task model with three diagnostic properties of rationale is elaborately designed to improve the quality of an explanation without any explicit annotations. Experimental results on the FEVEROUS benchmark show significant gains over previous state-of-the-art methods for both rationale extraction and fact verification.

Unified robust network embedding framework for community detection via extreme adversarial attacks

Graph data are widely available in complex network systems. Numerous community detection algorithms have been investigated to study graph problems wherein the network topology offers abundant behavioral and functional information. In real scenarios, networks are extremely susceptible to external perturbations, primarily because of sparse topological information. Consequently, maintaining robust community detection performance when confronted with intricate network attacks or perturbations is challenging. Previous anti-graph perturbation methods have typically been adjusted to specific perturbation types, leading to the degradation or failure of these defences when confronted with other types of attack. Accordingly, a unified robust framework that leverages extreme adversarial attacks is proposed in this paper. Specifically, a novel graph perturbation module is introduced into the model to generate extreme undirected or directed perturbations through dynamic updating; this ensures that the model can fit the attacked network well. The proposed framework derives a unified perturbation formulation capable of simultaneously attacking symmetric and asymmetric networks. Furthermore, this framework can be applied to many existing non-negative matrix factorization-based community detection methods. Extensive experiments on artificial and real-world networks demonstrate that the proposed framework significantly improves the robustness of detection tasks, particularly in noisy networks.

SimDCL: dropout-based simple graph contrastive learning for recommendation

Representation learning of users and items is the core of recommendation, and benefited from the development of graph neural network (GNN), graph collaborative filtering (GCF) for capturing higher order connectivity has been successful in the recommendation domain. Nevertheless, the matrix sparsity problem in collaborative filtering and the tendency of higher order embeddings to smooth in GNN limit further performance improvements. Contrastive learning (CL) was introduced into GCF and alleviated these problems to some extent. However, existing methods usually require graph perturbation to construct augmented views or design complex CL tasks, which limits the further development of CL-based methods in the recommendation. We propose a simple CL framework that does not require graph augmentation, but is based on dropout techniques to generate contrastive views to address the aforementioned problem. Specifically, we first added dropout operation to the GNN computation, and then fed the same batch of samples twice into the network for computation. Using the randomness of dropout, a pair of views with random noise was obtained, and maximizing the similarity of the view pairs is set as an auxiliary task to complement the recommendation. In addition, we made a simple modification to the computation of the GNN to alleviate the information loss due to embedding smoothing by means of cross-layer connected graph convolution computation. We named our proposed method as Simple Contrastive Learning Graph Neural Network based on dropout (SimDCL). Extensive experiments on five public datasets demonstrate the effectiveness of the proposed SimDCL, especially on the Amazon Books and Ta-Feng datasets, where our approach achieves 44% and 43% performance gains compared to baseline.

Defense against membership inference attack in graph neural networks through graph perturbation.

Graph neural networks have demonstrated remarkable performance in learning node or graph representations for various graph-related tasks. However, learning with graph data or its embedded representations may induce privacy issues when the node representations contain sensitive or private user information. Although many machine learning models or techniques have been proposed for privacy preservation of traditional non-graph structured data, there is limited work to address graph privacy concerns. In this paper, we investigate the privacy problem of embedding representations of nodes, in which an adversary can infer the user's privacy by designing an inference attack algorithm. To address this problem, we develop a defense algorithm against white-box membership inference attacks, based on perturbation injection on the graph. In particular, we employ a graph reconstruction model and inject a certain size of noise into the intermediate output of the model, i.e., the latent representations of the nodes. The experimental results obtained on real-world datasets, along with reasonable usability and privacy metrics, demonstrate that our proposed approach can effectively resist membership inference attacks. Meanwhile, based on our method, the trade-off between usability and privacy brought by defense measures can be observed intuitively, which provides a reference for subsequent research in the field of graph privacy protection.

Perturbations of graphs for Newton maps I: bounded hyperbolic components

AbstractWe consider graphs consisting of finitely many internal rays for degenerating Newton maps and state a convergence result. As an application, we prove that a hyperbolic component in the moduli space of quartic Newton maps is bounded if and only if every element has degree $2$ on the immediate basin of each root. This provides the first complete description of bounded hyperbolic components in a complex two-dimensional moduli space.

Implications of topological imbalance for representation learning on biomedical knowledge graphs.

Adoption of recently developed methods from machine learning has given rise to creation of drug-discovery knowledge graphs (KGs) that utilize the interconnected nature of the domain. Graph-based modelling of the data, combined with KG embedding (KGE) methods, are promising as they provide a more intuitive representation and are suitable for inference tasks such as predicting missing links. One common application is to produce ranked lists of genes for a given disease, where the rank is based on the perceived likelihood of association between the gene and the disease. It is thus critical that these predictions are not only pertinent but also biologically meaningful. However, KGs can be biased either directly due to the underlying data sources that are integrated or due to modelling choices in the construction of the graph, one consequence of which is that certain entities can get topologically overrepresented. We demonstrate the effect of these inherent structural imbalances, resulting in densely connected entities being highly ranked no matter the context. We provide support for this observation across different datasets, models as well as predictive tasks. Further, we present various graph perturbation experiments which yield more support to the observation that KGE models can be more influenced by the frequency of entities rather than any biological information encoded within the relations. Our results highlight the importance of data modelling choices, and emphasizes the need for practitioners to be mindful of these issues when interpreting model outputs and during KG composition.

Defending Graph Convolutional Networks against Dynamic Graph Perturbations via Bayesian Self-Supervision

In recent years, plentiful evidence illustrates that Graph Convolutional Networks (GCNs) achieve extraordinary accomplishments on the node classification task. However, GCNs may be vulnerable to adversarial attacks on label-scarce dynamic graphs. Many existing works aim to strengthen the robustness of GCNs; for instance, adversarial training is used to shield GCNs against malicious perturbations. However, these works fail on dynamic graphs for which label scarcity is a pressing issue. To overcome label scarcity, self-training attempts to iteratively assign pseudo-labels to highly confident unlabeled nodes but such attempts may suffer serious degradation under dynamic graph perturbations. In this paper, we generalize noisy supervision as a kind of self-supervised learning method and then propose a novel Bayesian self-supervision model, namely GraphSS, to address the issue. Extensive experiments demonstrate that GraphSS can not only affirmatively alert the perturbations on dynamic graphs but also effectively recover the prediction of a node classifier when the graph is under such perturbations. These two advantages prove to be generalized over three classic GCNs across five public graph datasets.

On some bounds on the perturbation of invariant subspaces of normal matrices with application to a graph connection problem

We provide upper bounds on the perturbation of invariant subspaces of normal matrices measured using a metric on the space of vector subspaces of mathbb{C}^{n} in terms of the spectrum of both unperturbed and perturbed matrices as well as the spectrum of the unperturbed matrix only. The results presented give tighter bounds than the Davis–Kahan sinΘ theorem. We apply the result to a graph perturbation problem.

Structural brain splitting is a hallmark of Granulin-related frontotemporal dementia

Frontotemporal dementia associated with granulin (GRN) mutations presents asymmetric brain atrophy. We applied a Minimum Spanning Tree plus an Efficiency Cost Optimization approach to cortical thickness data in order to test whether graph theory measures could identify global or local impairment of connectivity in the presymptomatic phase of pathology, where other techniques failed in demonstrating changes.We included 52 symptomatic GRN mutation carriers (SC), 161 presymptomatic GRN mutation carriers (PSC) and 341 non-carriers relatives from the Genetic Frontotemporal dementia research Initiative cohort. Group differences of global, nodal and edge connectivity in (Minimum Spanning Tree plus an Efficiency Cost Optimization) graph were tested via Structural Equation Models.Global graph perturbation was selectively impaired in SC compared to non-carriers, with no changes in PSC. At the local level, only SC exhibited perturbation of frontotemporal nodes, but edge connectivity revealed a characteristic pattern of interhemispheric disconnection, involving homologous parietal regions, in PSC.Our results suggest that GRN-related frontotemporal dementia resembles a disconnection syndrome, with interhemispheric disconnection between parietal regions in presymptomatic phases that progresses to frontotemporal areas as symptoms emerge.

Investigation and Application of Differential Privacy in Bitcoin

Bitcoin is one of the best-known cryptocurrencies, which captivated researchers with its innovative blockchain structure. Examinations of this public blockchain resulted in many proposals for improvement in terms of anonymity and privacy. Generally used methods for improvement include mixing protocols, ring signatures, zero-knowledge proofs, homomorphic commitments, and off-chain storage systems. To the best of our knowledge, in the literature, there is no study examining Bitcoin in terms of differential privacy, which is a privacy notion coming up with some mechanisms that enable running useful statistical queries without identifying any personal information. In this paper, we provide a theoretical examination of differential privacy in Bitcoin. Our motivation arises from the idea that the Bitcoin public blockchain structure can benefit from differential privacy mechanisms for improved privacy, both making anonymization and privacy breaches by direct queries impossible, and preserving the checkability of the integrity of the blockchain. We first examine the current Bitcoin implementation for four query functions using the differential privacy formulation. Then, we present the feasibility of the utilization of two differential privacy mechanisms in Bitcoin; the noise addition to the transaction amounts and the user graph perturbation. We show that these mechanisms decrease the fraction of the cases violating differential privacy, therefore they can be used for improving anonymity and privacy in Bitcoin. Moreover, we showcase the noise addition to transaction amounts by using IBM Differential Privacy Library. We compare four differential privacy mechanisms for varying privacy parameter values and determine the feasible mechanisms and the parameters.

Stability of graph convolutional neural networks to stochastic perturbations

Graph convolutional neural networks (GCNNs) are nonlinear processing tools to learn representations from network data. A key property of GCNNs is their stability to graph perturbations. Current analysis considers deterministic perturbations but fails to provide relevant insights when topological changes are random. This paper investigates the stability of GCNNs to stochastic graph perturbations induced by link losses. In particular, it proves the expected output difference between the GCNN over random perturbed graphs and the GCNN over the nominal graph is upper bounded by a factor that is linear in the link loss probability. We perform the stability analysis in the graph spectral domain such that the result holds uniformly for any graph. This result also shows the role of the nonlinearity and the architecture width and depth, and allows identifying handle to improve the GCNN robustness. Numerical simulations on source localization and robot swarm control corroborate our theoretical findings.

Stochastic Graph Neural Networks

Graph neural networks (GNNs) model nonlinear representations in graph data with applications in distributed agent coordination, control, and planning among others. Current GNN architectures assume ideal scenarios and ignore link fluctuations that occur due to environment, human factors, or external attacks. In these situations, the GNN fails to address its distributed task if the topological randomness is not considered accordingly. To overcome this issue, we put forth the stochastic graph neural network (SGNN) model: a GNN where the distributed graph convolution module accounts for the random network changes. Since stochasticity brings in a new learning paradigm, we conduct a statistical analysis on the SGNN output variance to identify conditions the learned filters should satisfy for achieving robust transference to perturbed scenarios, ultimately revealing the explicit impact of random link losses. We further develop a stochastic gradient descent (SGD) based learning process for the SGNN and derive conditions on the learning rate under which this learning process converges to a stationary point. Numerical results corroborate our theoretical findings and compare the benefits of SGNN robust transference with a conventional GNN that ignores graph perturbations during learning.

Ordering trees by their ABC spectral radii

AbstractLet G = (V, E) be a connected graph, where V = {v1, v2, …, vn}. Let di denote the degree of vertex vi. The ABC matrix of G is defined as M(G) = (mij)n × n, where if vivj ∈ E, and 0 otherwise. The ABC spectral radius of G is the largest eigenvalue of M(G). In the present paper, two graph perturbations with respect to ABC spectral radius are established. By applying these perturbations, the trees with the third, fourth, and fifth largest ABC spectral radii are determined.