Graph Generation Models Research Articles

Topic scene graphs for image captioning

When describing an image, people can rapidly extract the topic from the image and find the main object, generating sentences that match the main idea of the image. However, most of the scene graph generation methods do not emphasise the importance of the topic of the image. Consequently, the captions generated by the scene graph-based image captioning models cannot reflect the topic in the image then expressing the central idea of the image. In this paper, we propose a method for image captioning based on topic scene graphs (TSG). Firstly, we propose the structure of topic scene graphs that express images' topics and the relationships between objects. Then, combined with the topic scene graph, we utilise the salient object detection to generate the topic scene graph highlighting the salient objects of the image. Note that our framework is agnostic to any scene graph-based image captioning model and thus can be widely applied in the community which seeks salient object predictions. We compare the performance of our topic scene graph with the state-of-the-art scene graph generation models and mainstream image captioning models on MSCOCO and Visual Genome datasets, both achieving better performance.

Representation Learning for Dynamic Functional Connectivities via Variational Dynamic Graph Latent Variable Models.

Latent variable models (LVMs) for neural population spikes have revealed informative low-dimensional dynamics about the neural data and have become powerful tools for analyzing and interpreting neural activity. However, these approaches are unable to determine the neurophysiological meaning of the inferred latent dynamics. On the other hand, emerging evidence suggests that dynamic functional connectivities (DFC) may be responsible for neural activity patterns underlying cognition or behavior. We are interested in studying how DFC are associated with the low-dimensional structure of neural activities. Most existing LVMs are based on a point process and fail to model evolving relationships. In this work, we introduce a dynamic graph as the latent variable and develop a Variational Dynamic Graph Latent Variable Model (VDGLVM), a representation learning model based on the variational information bottleneck framework. VDGLVM utilizes a graph generative model and a graph neural network to capture dynamic communication between nodes that one has no access to from the observed data. The proposed computational model provides guaranteed behavior-decoding performance and improves LVMs by associating the inferred latent dynamics with probable DFC.

Non-Graph Data Clustering via O(n) Bipartite Graph Convolution.

Since the representative capacity of graph-based clustering methods is usually limited by the graph constructed on the original features, it is attractive to find whether graph neural networks (GNNs), a strong extension of neural networks to graphs, can be applied to augment the capacity of graph-based clustering methods. The core problems mainly come from two aspects. On the one hand, the graph is unavailable in the most general clustering scenes so that how to construct graph on the non-graph data and the quality of graph is usually the most important part. On the other hand, given n samples, the graph-based clustering methods usually consume at least O(n2) time to build graphs and the graph convolution requires nearly O(n2) for a dense graph and O(|E|) for a sparse one with |E| edges. Accordingly, both graph-based clustering and GNNs suffer from the severe inefficiency problem. To tackle these problems, we propose a novel clustering method, AnchorGAE, with the self-supervised estimation of graph and efficient graph convolution. We first show how to convert a non-graph dataset into a graph dataset, by introducing the generative graph model and anchors. A bipartite graph is built via generating anchors and estimating the connectivity distributions of original points and anchors. We then show that the constructed bipartite graph can reduce the computational complexity of graph convolution from O(n2) and O(|E|) to O(n). The succeeding steps for clustering can be easily designed as O(n) operations. Interestingly, the anchors naturally lead to siamese architecture with the help of the Markov process. Furthermore, the estimated bipartite graph is updated dynamically according to the features extracted by GNN modules, to promote the quality of the graph by exploiting the high-level information by GNNs. However, we theoretically prove that the self-supervised paradigm frequently results in a collapse that often occurs after 2-3 update iterations in experiments, especially when the model is well-trained. A specific strategy is accordingly designed to prevent the collapse. The experiments support the theoretical analysis and show the superiority of AnchorGAE.

로봇 물체 조작 작업을 위한 동적 3차원 장면 그래프 생성

In this study, we proposed a novel dataset and a deep learning model that can generate three-dimensional (3D) dynamic scene graphs for robotic manipulation tasks. First, we defined a new 3D scene graph to effectively represent the dynamics of a robotic manipulation task environment. Subsequently, we collected a series of input sensory data by conducting multiple manipulation tasks in a simulated environment. Based on the collected sensory data and the corresponding 3D scene graphs, we constructed a dataset, namely, D3DSG, for training and validating a scene graph generation model. In addition, we proposed a ST-GCN based context reasoning module that can utilize both rich spatial and temporal contexts, after which an effective 3D scene graph generation model, namely, SG4RMT, which consisted of a 6DoF pose estimation module and a spatio-temporal context reasoning module, was presented. The superiority and high performance of the proposed SG4RMT model were demonstrated by performing multiple experiments using the D3DSG dataset.

Hierarchical recurrent neural networks for graph generation

Graph generation is widely used in various fields, such as social science, chemistry, and physics. Although the deep graph generative models have achieved considerable success in recent years, some problems still need to be addressed. First, some models learn only the structural information and cannot capture the semantic information. Second, many existing models cannot be applied to large-scale datasets owing to a complex training process and a large number of parameters to be learned. Third, most of the existing models require domain-specific rules to be set up, resulting in poor generalization. To address the aforementioned problems, we propose a domain-agnostic model with hierarchical recurrent neural networks, named GHRNN, which learns the distribution of graph data for generating new graphs. First, we utilize the minimum depth-first-search code to convert a graph into a unique edge sequence that naturally generalizes to various graphs. Then, we adopt the edge-level recurrent neural network (RNN) module to learn the global structure of a graph and the element-level RNN module to obtain semantic information, such as node labels, edge labels, and the timestamp of nodes. The element-level RNN module with a fixed and small number of time steps in GHRNN can efficiently scale to large datasets. Finally, we generate a large number of graphs and evaluate the effectiveness of our model by using 11 different metrics on diverse datasets ranging from molecules to citation networks. Extensive experimental results indicate that GHRNN can capture complex dependencies and generate the most realistic and diverse graphs, in contrast to several state-of-the-art baselines.

Stochastic blockmodeling for learning the structure of optimization problems

AbstractDecomposition‐based solution algorithms for optimization problems depend on the underlying latent block structure of the problem. Methods for detecting this structure are currently lacking. In this article, we propose stochastic blockmodeling (SBM) as a systematic framework for learning the underlying block structure in generic optimization problems. SBM is a generative graph model in which nodes belong to some blocks and the interconnections among the nodes are stochastically dependent on their block affiliations. Hence, through parametric statistical inference, the interconnection patterns underlying optimization problems can be estimated. For benchmark optimization problems, we show that SBM can reveal the underlying block structure and that the estimated blocks can be used as the basis for decomposition‐based solution algorithms which can reach an optimum or bound estimates in reduced computational time. Finally, we present a general software platform for automated block structure detection and decomposition‐based solution following distributed and hierarchical optimization approaches.

MolGrow: A Graph Normalizing Flow for Hierarchical Molecular Generation

We propose a hierarchical normalizing flow model for generating molecular graphs. The model produces new molecular structures from a single-node graph by recursively splitting every node into two. All operations are invertible and can be used as plug-and-play modules. The hierarchical nature of the latent codes allows for precise changes in the resulting graph: perturbations in the first layer cause global structural changes, while perturbations in the consequent layers change the resulting molecule only marginally. Proposed model outperforms existing generative graph models on the distribution learning task. We also show successful experiments on global and constrained optimization of chemical properties using latent codes of the model.

Characterization of robustness and resilience in graphs: a mini-review

Abstract We briefly survey recent proposals that seek to capture in numerical terms the resilience and the robustness of a graph. After a brief introduction and the establishment of notation and terminology, we catalogue characterizations proposed in journal articles published within the last two decades. We then describe some of the numerous application areas for such characterizations. We experiment with implementations of numerous characteristics on several graph-generation models, after which we conclude with a discussion of open problems and future directions.

An empirical characterization of community structures in complex networks using a bivariate map of quality metrics

Community detection emerges as an important task in the discovery of network mesoscopic structures. However, the concept of a “good” community is very context-dependent, and it is relatively complicated to deduce community characteristics using available community detection techniques. In reality, the existence of a gap between structural goodness quality metrics and expected topological patterns creates a confusion in evaluating community structures. We thus introduce an empirical multivariate analysis of different structural goodness properties in order to characterize several detectable community topologies. Specifically, we show that a combination of two representative structural dimensions including community transitivity and hub dominance allows to distinguish different topologies such as star-based, clique-based, string-based and grid-based structures. Additionally, these classes of topology disclose structural proximities with those of graphs created by Erdős–Renyi, Watts–Strogatz and Barabasi–Albert generative models. We illustrate popular community topologies identified by different detection methods on a large dataset composing many network categories and associate their structures with the most related graph generative model. Interestingly, this conjunctive representation sheds light on fundamental differences between mesoscopic structures in various network categories including communication, information, biological, technological, social, ecological, synthetic networks and more.

Graph Variational Autoencoder for Detector Reconstruction and Fast Simulation in High-Energy Physics

Accurate and fast simulation of particle physics processes is crucial for the high-energy physics community. Simulating particle interactions with the detector is both time consuming and computationally expensive. With its proton-proton collision energy of 13 TeV, the Large Hadron Collider is uniquely positioned to detect and measure the rare phenomena that can shape our knowledge of new interactions. The High-Luminosity Large Hadron Collider (HLLHC) upgrade will put a significant strain on the computing infrastructure and budget due to increased event rate and levels of pile-up. Simulation of highenergy physics collisions needs to be significantly faster without sacrificing the physics accuracy. Machine learning approaches can offer faster solutions, while maintaining a high level of fidelity. We introduce a graph generative model that provides effiective reconstruction of LHC events on the level of calorimeter deposits and tracks, paving the way for full detector level fast simulation.

AORM: Fast Incremental Arbitrary-Order Reachability Matrix Computation for Massive Graphs

Processing a reachability query in large-scale networks using existing methods remains one of the most challenging problems in graph mining. In this paper, we propose a novel incremental algorithmic framework for arbitrary-order reachability computation in massive graphs. The proposed method is intuitive and significantly outperforms the currently known methods in terms of computation time. We focus on the arbitrary-order reachability matrix framework called AORM, which can handle directed and disconnected networks such as citation networks. The AORM can handle diverse types of real-world datasets. We conduct extensive experimental studies with twenty synthetic networks generated from five random graph generation models and twenty massive real-world networks. The experimental results show the advantages of the method in terms of both computational efficiency and approximation controllability. In particular, the proposed method outperforms up to 10 times compared to NetworkX for incremental all-pairs shortest paths computation. Moreover, the computational results of the method rapidly converge to the ground truths. Thus, we can get the correct solution in the early stage of the incremental approximation. We can employ the method as a versatile feature extraction framework for network embedding. Overall, the experimental results present a remarkable improvement in speed-up for reachability computation.

Practical notes on building molecular graph generative models

AbstractHere are presented technical notes and tips on developing graph generative models for molecular design. Although this work stems from the development of GraphINVENT, a Python platform for iterative molecular generation using graph neural networks, this work is relevant to researchers studying other architectures for graph‐based molecular design. In this work, technical details that could be of interest to researchers developing their own molecular generative models are discussed, including an overview of previous work in graph‐based molecular design and strategies for designing new models. Advice on development and debugging tools which are helpful during code development is also provided. Finally, methods that were tested but which ultimately did not lead to promising results in the development of GraphINVENT are described here in the hope that this will help other researchers avoid pitfalls in development and instead focus their efforts on more promising strategies for graph‐based molecular generation.

Visual Experience-Based Question Answering with Complex Multimodal Environments

This paper proposes a novel visual experience-based question answering problem (VEQA) and the corresponding dataset for embodied intelligence research that requires an agent to do actions, understand 3D scenes from successive partial input images, and answer natural language questions about its visual experiences in real time. Unlike the conventional visual question answering (VQA), the VEQA problem assumes both partial observability and dynamics of a complex multimodal environment. To address this VEQA problem, we propose a hybrid visual question answering system, VQAS, integrating a deep neural network-based scene graph generation model and a rule-based knowledge reasoning system. The proposed system can generate more accurate scene graphs for dynamic environments with some uncertainty. Moreover, it can answer complex questions through knowledge reasoning with rich background knowledge. Results of experiments using a photo-realistic 3D simulated environment, AI2-THOR, and the VEQA benchmark dataset prove the high performance of the proposed system.

Graph Representation Learning

Graph-structured data is ubiquitous throughout the natural and social sciences, from telecommunication networks to quantum chemistry. Building relational inductive biases into deep learning architectures is crucial for creating systems that can learn, reason, and generalize from this kind of data. Recent years have seen a surge in research on graph representation learning, including techniques for deep graph embeddings, generalizations of convolutional neural networks to graph-structured data, and neural message-passing approaches inspired by belief propagation. These advances in graph representation learning have led to new state-of-the-art results in numerous domains, including chemical synthesis, 3D vision, recommender systems, question answering, and social network analysis. This book provides a synthesis and overview of graph representation learning. It begins with a discussion of the goals of graph representation learning as well as key methodological foundations in graph theory and network analysis. Following this, the book introduces and reviews methods for learning node embeddings, including random-walk-based methods and applications to knowledge graphs. It then provides a technical synthesis and introduction to the highly successful graph neural network (GNN) formalism, which has become a dominant and fast-growing paradigm for deep learning with graph data. The book concludes with a synthesis of recent advancements in deep generative models for graphs—a nascent but quickly growing subset of graph representation learning.

Beyond ranking nodes: Predicting epidemic outbreak sizes by network centralities.

Identifying important nodes for disease spreading is a central topic in network epidemiology. We investigate how well the position of a node, characterized by standard network measures, can predict its epidemiological importance in any graph of a given number of nodes. This is in contrast to other studies that deal with the easier prediction problem of ranking nodes by their epidemic importance in given graphs. As a benchmark for epidemic importance, we calculate the exact expected outbreak size given a node as the source. We study exhaustively all graphs of a given size, so do not restrict ourselves to certain generative models for graphs, nor to graph data sets. Due to the large number of possible nonisomorphic graphs of a fixed size, we are limited to ten-node graphs. We find that combinations of two or more centralities are predictive (R2 scores of 0.91 or higher) even for the most difficult parameter values of the epidemic simulation. Typically, these successful combinations include one normalized spectral centrality (such as PageRank or Katz centrality) and one measure that is sensitive to the number of edges in the graph.

Explaining the semantics capturing capability of scene graph generation models

Deep neural network is a effective way for scene graph generation tasks. However, it also makes the scene graph generation models difficult to explain. For instance, the current standard metric cannot explain how capable neural network models are of capturing the semantics of relations. In this paper, we try to understand the semantics capturing capability of scene graph generation models based on three types of metrics: conformance recall, violation recall, and non-violation recall, which measure semantic properties of relations that are reflected by triples in scene graph generated by models. Evaluation of these metrics on three representative state-of-the-art scene graph generation models based on deep neural network in Visual Genome dataset shows that the proposed metrics can effectively explain the capability of models to capture different semantic properties and identify design problems in models. By extending the Visual Genome dataset with different sets of additional annotations, these metrics can also explaining whether the semantics capturing capability of deep neural network models can be improved by data enhancement.

Scaffold-based molecular design with a graph generative model.

Searching for new molecules in areas like drug discovery often starts from the core structures of known molecules. Such a method has called for a strategy of designing derivative compounds retaining a particular scaffold as a substructure. On this account, our present work proposes a graph generative model that targets its use in scaffold-based molecular design. Our model accepts a molecular scaffold as input and extends it by sequentially adding atoms and bonds. The generated molecules are then guaranteed to contain the scaffold with certainty, and their properties can be controlled by conditioning the generation process on desired properties. The learned rule of extending molecules can well generalize to arbitrary kinds of scaffolds, including those unseen during learning. In the conditional generation of molecules, our model can simultaneously control multiple chemical properties despite the search space constrained by fixing the substructure. As a demonstration, we applied our model to designing inhibitors of the epidermal growth factor receptor and show that our model can employ a simple semi-supervised extension to broaden its applicability to situations where only a small amount of data is available.

Static malware detection and attribution in android byte-code through an end-to-end deep system

Android reflects a revolution in handhelds and mobile devices. It is a virtual machine based, an open source mobile platform that powers millions of smartphone and devices and even a larger no. of applications in its ecosystem. Surprisingly in a short lifespan, Android has also seen a colossal expansion in application malware with 99% of the total malware for smartphones being found in the Android ecosystem. Subsequently, quite a few techniques have been proposed in the literature for the analysis and detection of these malicious applications for the Android platform. The increasing and diversified nature of Android malware has immensely attenuated the usefulness of prevailing malware detectors, which leaves Android users susceptible to novel malware. Here in this paper, as a remedy to this problem, we propose an anti-malware system that uses customized learning models, which are sufficiently deep, and are ’End to End deep learning architectures which detect and attribute the Android malware via opcodes extracted from application bytecode’. Our results show that Bidirectional long short-term memory (BiLSTMs) neural networks can be used to detect static behavior of Android malware beating the state-of-the-art models without using handcrafted features. For our experiments in our system, we also choose to work with distinct and independent deep learning models leveraging sequence specialists like recurrent neural networks, Long Short Term Memory networks and its Bidirectional variation as well as those are more usual neural architectures like a network of all connected layers(fully connected), deep convnets, Diabolo network (autoencoders) and generative graphical models like deep belief networks for static malware analysis on Android. To test our system, we have also augmented a bytecode dataset from three open and independently maintained state-of-the-art datasets. Our bytecode dataset, which is on an order of magnitude large, essentially suffice for our experiments. Our results suggests that our proposed system can lead to better design of malware detectors as we report an accuracy of 0.999 and an F1-score of 0.996 on a large dataset of more than 1.8 million Android applications.

DP-FT: A Differential Privacy Graph Generation With Field Theory for Social Network Data Release

Many data analysis applications rely on social networks that contain abundant information about individuals. Nevertheless, these applications can leak private information about individuals in social networks. To protect the privacy of individuals in social networks, several approaches involving graph generation models or differential privacy were proposed for publishing a social network in place of the original graph for data analysis applications. However, these techniques can cause a serious loss of data utility, especially regarding the real social links. In this paper, we propose an approach of degree-differential privacy graph generation with field theory. The approach includes two steps for publishing a social network. The degrees of the nodes are first perturbed with the differential privacy by adding noise that follows a Laplacian distribution. Then, the edges of the social network are synthesized with field theory. We propose a field theory model for social networks by simulating the law of gravity in physics and establish the correspondence of the gravitational field in physics to the field theory model. When an edge is formed, the starting node is preferentially chosen with high probability from the nodes with high degrees, and then the ending node is selected with high probability when the interaction force between the starting node and the ending node is large. Extensive experiments over four datasets show that our approach can preserve more real social ties compared with previous approaches and will not incur a loss of structure features over the datasets, such as the degree distribution and clustering coefficients.

A generative graph model for electrical infrastructure networks

We propose a generative graph model for electrical infrastructure networks that accounts for heterogeneity in both node and edge type. To inform the model design, we analyze the properties of power grid graphs derived from the U.S. Eastern Interconnection, Texas Interconnection, and Poland transmission system power grids. Across these datasets, we find subgraphs induced by nodes of the same voltage level exhibit shared structural properties atypical to small-world networks, including low local clustering, large diameter, and large average distance. On the other hand, we find subgraphs induced by transformer edges linking nodes of different voltage types contain a more limited structure, consisting mainly of small, disjoint star graphs. The goal of our proposed model is to match both these inter and intra-network properties by proceeding in two phases: the first phase adapts the Chung-Lu random graph model, taking desired vertex degrees and desired diameter as inputs, while the second phase of the model is based on a simpler random star graph generation process. We test the model's performance by comparing its output across many runs to the aforementioned real data. In nearly all categories tested, we find our model is more accurate in reproducing the unusual mixture of properties apparent in the data than the Chung-Lu model. We also include graph visualization comparisons, a brief analysis of edge-deletion resiliency, and guidelines for artificially generating the model inputs in the absence of real data.