Graph Convolutional Layers Research Articles

Recommendation System Based on Generative Adversarial Network with Graph Convolutional Layers

A graph convolutional generative adversarial network (GCGAN) is proposed to provide recommendations for new users or items. To maintain scalability, the discriminator was improved to capture the latent features of users and items, using graph convolution from a minibatch-sized bipartite graph. In the experiment using MovieLens, it was confirmed that the proposed GCGAN had better performance than the conventional CFGAN, when MovieLens 1M was employed with sufficient data. The proposed method is characterized in such a manner that it can learn domain information of both, users and items, and it does not require to relearn a model for a new node. Further, it can be developed for any service having such conditions, in the information recommendation field.

Variational graph autoencoders for multiview canonical correlation analysis

We present a novel approach for multiview canonical correlation analysis based on a variational graph neural network model. We propose a nonlinear model which takes into account the available graph-based geometric constraints while being scalable to large-scale datasets with multiple views. This model combines the probabilistic interpretation of CCA with an autoencoder architecture based on graph convolutional neural network layers. Experiments with the proposed method are conducted on classification, clustering, and recommendation tasks on real datasets. The algorithm is competitive with state-of-the-art multiview representation learning techniques, in addition to being scalable and robust to instances with missing views.

Brain graph super-resolution using adversarial graph neural network with application to functional brain connectivity.

Brain image analysis has advanced substantially in recent years with the proliferation of neuroimaging datasets acquired at different resolutions. While research on brain image super-resolution has undergone a rapid development in the recent years, brain graph super-resolution is still poorly investigated because of the complex nature of non-Euclidean graph data. In this paper, we propose the first-ever deep graph super-resolution (GSR) framework that attempts to automatically generate high-resolution (HR) brain graphs with N' nodes (i.e., anatomical regions of interest (ROIs)) from low-resolution (LR) graphs with N nodes where N<N'. First, we formalize our GSR problem as a node feature embedding learning task. Once the HR nodes' embeddings are learned, the pairwise connectivity strength between brain ROIs can be derived through an aggregation rule based on a novel Graph U-Net architecture. While typically the Graph U-Net is a node-focused architecture where graph embedding depends mainly on node attributes, we propose a graph-focused architecture where the node feature embedding is based on the graph topology. Second, inspired by graph spectral theory, we break the symmetry of the U-Net architecture by super-resolving the low-resolution brain graph structure and node content with a GSR layer and two graph convolutional network layers to further learn the node embeddings in the HR graph. Third, to handle the domain shift between the ground-truth and the predicted HR brain graphs, we incorporate adversarial regularization to align their respective distributions. Our proposed AGSR-Net framework outperformed its variants for predicting high-resolution functional brain graphs from low-resolution ones. Our AGSR-Net code is available on GitHub at https://github.com/basiralab/AGSR-Net.

Vision-Language-Knowledge Co-Embedding for Visual Commonsense Reasoning.

Visual commonsense reasoning is an intelligent task performed to decide the most appropriate answer to a question while providing the rationale or reason for the answer when an image, a natural language question, and candidate responses are given. For effective visual commonsense reasoning, both the knowledge acquisition problem and the multimodal alignment problem need to be solved. Therefore, we propose a novel Vision–Language–Knowledge Co-embedding (ViLaKC) model that extracts knowledge graphs relevant to the question from an external knowledge base, ConceptNet, and uses them together with the input image to answer the question. The proposed model uses a pretrained vision–language–knowledge embedding module, which co-embeds multimodal data including images, natural language texts, and knowledge graphs into a single feature vector. To reflect the structural information of the knowledge graph, the proposed model uses the graph convolutional neural network layer to embed the knowledge graph first and then uses multi-head self-attention layers to co-embed it with the image and natural language question. The effectiveness and performance of the proposed model are experimentally validated using the VCR v1.0 benchmark dataset.

Ground-Based Remote Sensing Cloud Classification via Context Graph Attention Network

Most ground-based remote sensing cloud classification methods focus on learning representation features for cloud images while ignoring the correlations among cloud images. Recently, graph convolutional network (GCN) is applied to provide the correlations for ground-based remote sensing cloud classification, in which the graph convolutional layer aggregates information from the connected nodes of graph in a weighted way. However, the weights assigned by GCN cannot reflect the importance of connected nodes precisely, which declines the discrimination of the aggregated features (AFs). To overcome the limitation, in this article, we propose the context graph attention network (CGAT) for ground-based remote sensing cloud classification. Specifically, the context graph attention layer (CGA layer) of CGAT is proposed to learn the context attention coefficients (CACs) and obtain the AFs of nodes based on the CACs. We compute the CACs not only considering the two connected nodes but also their neighborhood nodes in order to stabilize the aggregation process. In addition, we propose to utilize two different transformation matrices to transform the node and its connected nodes into new feature spaces, which could enhance the discrimination of AFs. We concatenate the AFs with the deep features (DFs) as final representations for cloud classification. Since existing ground-based cloud data sets (GCDs) have limited cloud images, we release a new data set named GCD that is the largest one for ground-based cloud classification. We conduct a series of experiments on GCD, and the experimental results verify the effectiveness of CGAT.

Multicomponent Spatial-Temporal Graph Attention Convolution Networks for Traffic Prediction with Spatially Sparse Data.

Predicting traffic data on traffic networks is essential to transportation management. It is a challenging task due to the complicated spatial-temporal dependency. The latest studies mainly focus on capturing temporal and spatial dependencies with spatially dense traffic data. However, when traffic data become spatially sparse, existing methods cannot capture sufficient spatial correlation information and thus fail to learn the temporal periodicity sufficiently. To address these issues, we propose a novel deep learning framework, Multi-component Spatial-Temporal Graph Attention Convolutional Networks (MSTGACN), for traffic prediction, and we successfully apply it to predicting traffic flow and speed with spatially sparse data. MSTGACN mainly consists of three independent components to model three types of periodic information. Each component in MSTGACN combines dilated causal convolution, graph convolution layer, and the weight-shared graph attention layer. Experimental results on three real-world traffic datasets, METR-LA, PeMS-BAY, and PeMSD7-sparse, demonstrate the superior performance of our method in the case of spatially sparse data.

Embedding Disentanglement in Graph Convolutional Networks for Recommendation

To improve recommendation performance, many efforts have been made to study how to equip the conventional methods with auxiliary information such as item relations. Meanwhile, a growing body of work has focused on applying graph convolutional networks to recommendation tasks. Thus, it is promising to use graph convolution to model multi-order relations for improving recommendation performance. However, the existing graph convolution-based recommendation methods may suffer from structural design problems: for methods with embedding transformations in graph convolutional layers, the MLP makes the updated embedding dimensions coupled, hurting the embedding expressivity. While for methods based on simplified graph convolution, removing the parameter matrices makes the model attach a same weight to embeddings in different layers, limiting the model expressivity. In this paper, we propose a novel graph convolution-based recommendation method, namely Channel-Independent Graph Convolutional Network (CIGCN). To learn disentangled embeddings, CIGCN uses diagonal parameter matrices as filters in graph convolution, keeping the updated embedding dimensions independent. In addition, with layer-aggregation strategies, the parameters in the diagonal matrices act as trainable weights that attach different importance to the embeddings in each layer and each dimension, enhancing the model expressivity. Results of extensive experiments show the superior performance of our proposed method.

Graph Convolutional Embeddings for Recommender Systems

Modern recommender systems (RS) work by processing a number of signals that can be inferred from large sets of user-item interaction data. The main signal to analyze stems from the raw matrix that represents interactions. However, we can increase the performance of RS by considering other kinds of signals like the context of interactions, which could be, for example, the time or date of the interaction, the user location, or sequential data corresponding to the historical interactions of the user with the system. These complex, context-based interaction signals are characterized by a rich relational structure that can be represented by a multi-partite graph. Graph Convolutional Networks (GCNs) have been used successfully in collaborative filtering with simple user-item interaction data. In this work, we generalize the use of GCNs for N-partite graphs by considering N multiple context dimensions and propose a simple way for their seamless integration in modern deep learning RS architectures. More specifically, we define a graph convolutional embedding layer for N-partite graphs that processes user-item-context interactions and constructs node embeddings by leveraging their relational structure. Experiments on several datasets show the benefits of the introduced GCN embedding layer by measuring the performance of different context-enriched tasks.

Adaptive Spatiotemporal Graph Convolutional Networks for Motor Imagery Classification

Classification of electroencephalogram-based motor imagery (MI-EEG) tasks is crucial in brain computer interfaces (BCI). In view of the characteristics of non-stationarity, time-variability and individual diversity of EEG signals, a novel framework based on graph neural network is proposed for MI-EEG classification. First, an adaptive graph convolutional layer (AGCL) is constructed, by which the electrode channel information are integrated dynamically. We further propose an adaptive spatiotemporal graph convolutional network (ASTGCN), which fully exploits the characteristics of EEG signals in time domain and the channel correlations in spatial domain simultaneously. We execute the experiments using EEG signals recorded at motor imagery scenarios, where twenty-five healthy subjects performed MI movements of the right hand and feet to generate motor commands. Experimental results reveal that the proposed method outperforms state-of-the-art methods in terms of both classification quality and robustness. The advantages of ASTGCN include high accuracy, high efficiency, and robustness to cross-trial and cross-subject variations, making it an ideal candidate for long-term MI-EEG applications.

DCNet: diffusion convolutional networks for semantic image segmentation

Semantic image segmentation makes a pixel-level classification play an essential role in scene understanding. Recently, most approaches exploit deep learning neural networks, especially convolutional neural networks (CNNs), to tackle the image segmentation challenge. Common issues of these CNN-based methods are the loss of spatial features during learning representations and the limited capacity for capturing contextual information in a large receptive field. This paper proposes a diffusion convolutional network (DCNet) to combine the CNN and graph convolutional neural network (GCNN) for semantic image segmentation. In the proposed model, diffusion convolution is formulated as a graph convolutional layer to aggregate structural and contextual information without losing spatial features. The final segmentation results on the PASCAL VOC 2012 and Cityscapes datasets show better performance than baseline approaches and can be competitive with state-of-the-art methods.

Graph Neural Networks With Convolutional ARMA Filters.

Popular graph neural networks implement convolution operations on graphs based on polynomial spectral filters. In this paper, we propose a novel graph convolutional layer inspired by the auto-regressive moving average (ARMA) filter that, compared to polynomial ones, provides a more flexible frequency response, is more robust to noise, and better captures the global graph structure. We propose a graph neural network implementation of the ARMA filter with a recursive and distributed formulation, obtaining a convolutional layer that is efficient to train, localized in the node space, and can be transferred to new graphs at test time. We perform a spectral analysis to study the filtering effect of the proposed ARMA layer and report experiments on four downstream tasks: semi-supervised node classification, graph signal classification, graph classification, and graph regression. Results show that the proposed ARMA layer brings significant improvements over graph neural networks based on polynomial filters.

Emotion Knowledge Driven Video Highlight Detection

This paper addresses video highlight detection which aims to select a small subset of frames according to user's major or special interest. The performances of conventional methods highly depend on large-scale manually labeled training data which are time-consuming and labor-intensive to collect. To deal with this problem, we trace back to the original problem definition and find that whether a user is interested in a specific video segment heavily depends on human's subjective emotions. Leveraging this insight, we introduce an emotion knowledge driven video detection framework for modeling human's general emotion and inferencing highlight strength. Firstly, we obtain the concept-level representation of the video clip with a front-end network. The concepts are used as nodes to build an emotion-related knowledge graph, and their relationships in the graph are modeled via external public knowledge graphs. Then we adopt Siamese GCNs to model the dependencies between nodes in the graph and propagate messages along the edges. Finally, we compute the emotion-aware representation of the video clip based on the GCN layers and further use it to predict the highlight score. Our framework, including the front-end network, graph convolution layers and the highlight mapping network, can be trained in an end-to-end manner with the constraint of a ranking loss. Experiments on two benchmark datasets show that our proposed method performs favorably against the state-of-the-art methods.

Multi-scale spatiotemporal graph convolution network for air quality prediction

Air pollution is a serious environmental problem that has attracted much attention. Air quality prediction can provide useful information for urban environmental governance decision-making and residents’ daily health control. However, existing research methods have suffered from a weak ability to capture the spatial correlations and fail to model the long-term temporal dependencies of air quality. To overcome these limitations, we propose a multi-scale spatiotemporal graph convolution network (MST-GCN), which consists of a multi-scale block, several spatial-temporal blocks and a fusion block. We first divide the extracted features into several groups based on their domain categories, and represent the spatial correlations across stations as two graphs. Then we combine the grouped features and the constructed graphs in pairs to form a multi-scale block that feeds into spatial-temporal blocks. Each spatial-temporal block contains a graph convolution layer and a temporal convolution layer, which can model the spatial correlations and long-term temporal dependencies. To capture the group interactions, we use a fusion block to fuse multiple groups. Extensive experiments on a real-world dataset demonstrate that our model achieves the highest performance compared with state-of-the-art and baseline models for air quality prediction.

Predicting drug-disease associations through layer attention graph convolutional network.

Determining drug-disease associations is an integral part in the process of drug development. However, the identification of drug-disease associations through wet experiments is costly and inefficient. Hence, the development of efficient and high-accuracy computational methods for predicting drug-disease associations is of great significance. In this paper, we propose a novel computational method named as layer attention graph convolutional network (LAGCN) for the drug-disease association prediction. Specifically, LAGCN first integrates the known drug-disease associations, drug-drug similarities and disease-disease similarities into a heterogeneous network, and applies the graph convolution operation to the network to learn the embeddings of drugs and diseases. Second, LAGCN combines the embeddings from multiple graph convolution layers using an attention mechanism. Third, the unobserved drug-disease associations are scored based on the integrated embeddings. Evaluated by 5-fold cross-validations, LAGCN achieves an area under the precision-recall curve of 0.3168 and an area under the receiver-operating characteristic curve of 0.8750, which are better than the results of existing state-of-the-art prediction methods and baseline methods. The case study shows that LAGCN can discover novel associations that are not curated in our dataset. LAGCN is a useful tool for predicting drug-disease associations. This study reveals that embeddings from different convolution layers can reflect the proximities of different orders, and combining the embeddings by the attention mechanism can improve the prediction performances.

Enhanced Spatial and Extended Temporal Graph Convolutional Network for Skeleton-Based Action Recognition.

In the skeleton-based human action recognition domain, the spatial-temporal graph convolution networks (ST-GCNs) have made great progress recently. However, they use only one fixed temporal convolution kernel, which is not enough to extract the temporal cues comprehensively. Moreover, simply connecting the spatial graph convolution layer (GCL) and the temporal GCL in series is not the optimal solution. To this end, we propose a novel enhanced spatial and extended temporal graph convolutional network (EE-GCN) in this paper. Three convolution kernels with different sizes are chosen to extract the discriminative temporal features from shorter to longer terms. The corresponding GCLs are then concatenated by a powerful yet efficient one-shot aggregation (OSA) + effective squeeze-excitation (eSE) structure. The OSA module aggregates the features from each layer once to the output, and the eSE module explores the interdependency between the channels of the output. Besides, we propose a new connection paradigm to enhance the spatial features, which expand the serial connection to a combination of serial and parallel connections by adding a spatial GCL in parallel with the temporal GCLs. The proposed method is evaluated on three large scale datasets, and the experimental results show that the performance of our method exceeds previous state-of-the-art methods.

GSSA: Pay attention to graph feature importance for GCN via statistical self-attention

Graph convolutional network (GCN) has been proved to be an effective framework for graph-based semi-supervised learning applications. The core operation block of GCN is the convolutional layer, which enables the network to construct node embeddings by fusing both attributes of nodes and relationships between nodes. Different features or feature interactions inherently have various influences on the convolutional layers. However, there are very limited studies about the impact of feature importance in GCN-related communities. In this work, we attempt to augment convolutional layers in GCNs with statistical attention-based feature importance by modeling the latent interactions of features, which is complementary to the standard GCNs and only needs simple calculations with statistics rather than heavy trainings. To this end, we treat the feature input of each convolutional layer as a separate multi-layer heterogeneous graph, and propose Graph Statistical Self-Attention (GSSA) method to automatically learn the hierarchical structure of feature importance. More specifically, we propose two modules in GSSA, Channel-wise Self-Attention (CSA) to capture the dependencies between feature channels, and Mean-based Self-Attention (MSA) to reweight similarities among features. Aiming at each graph convolutional layer, GSSA can be applied in a ”plug and play” way for a wide range of GCN variants. To the best of our knowledge, this is the first implementation that optimizes GCNs from the feature importance perspective. Extensive experiments demonstrate that GSSA can promote existing popular baselines remarkably in semi-supervised node classification tasks. We further employ multiple qualitative evaluations to get deep insights into our method.

An end-to-end graph convolutional kernel support vector machine

A novel kernel-based support vector machine (SVM) for graph classification is proposed. The SVM feature space mapping consists of a sequence of graph convolutional layers, which generates a vector space representation for each vertex, followed by a pooling layer which generates a reproducing kernel Hilbert space (RKHS) representation for the graph. The use of a RKHS offers the ability to implicitly operate in this space using a kernel function without the computational complexity of explicitly mapping into it. The proposed model is trained in a supervised end-to-end manner whereby the convolutional layers, the kernel function and SVM parameters are jointly optimized with respect to a regularized classification loss. This approach is distinct from existing kernel-based graph classification models which instead either use feature engineering or unsupervised learning to define the kernel function. Experimental results demonstrate that the proposed model outperforms existing deep learning baseline models on a number of datasets.

Joint extraction of entities and relations using graph convolution over pruned dependency trees

We present a novel end-to-end deep neural network model based on graph convolutional networks for simultaneous joint extraction of entities and relations among them. Our model captures context and syntactic information from sentence by stacking a graph convolutional layer over bidirectional sequential LSTM layers. We sequentially concatenate the subject, object and sentence representations for obtaining the directionality of relations. Besides, in order to address long entity-distances problem, we further apply a path-centric pruning procedure to input trees in order to preserve useful information while maximally removing irrelevant words. Experiments are conducted on NYT dataset, and the proposed model achieves the state-of-the-art results on entity and relation extraction task. Our source code is available on Github: https://github.com/michael-hon/LSTM-GCN-ER.

Unsupervised generative and graph representation learning for modelling cell differentiation

Using machine learning techniques to build representations from biomedical data can help us understand the latent biological mechanism of action and lead to important discoveries. Recent developments in single-cell RNA-sequencing protocols have allowed measuring gene expression for individual cells in a population, thus opening up the possibility of finding answers to biomedical questions about cell differentiation. In this paper, we explore unsupervised generative neural methods, based on the variational autoencoder, that can model cell differentiation by building meaningful representations from the high dimensional and complex gene expression data. We use disentanglement methods based on information theory to improve the data representation and achieve better separation of the biological factors of variation in the gene expression data. In addition, we use a graph autoencoder consisting of graph convolutional layers to predict relationships between single-cells. Based on these models, we develop a computational framework that consists of methods for identifying the cell types in the dataset, finding driver genes for the differentiation process and obtaining a better understanding of relationships between cells. We illustrate our methods on datasets from multiple species and also from different sequencing technologies.

Hyperspectral Image Classification With Context-Aware Dynamic Graph Convolutional Network

In hyperspectral image (HSI) classification, spatial context has demonstrated its significance in achieving promising performance. However, conventional spatial context-based methods simply assume that spatially neighboring pixels should correspond to the same land-cover class, so they often fail to correctly discover the contextual relations among pixels in complex situations, and thus leading to imperfect classification results on some irregular or inhomogeneous regions such as class boundaries. To address this deficiency, we develop a new HSI classification method based on the recently proposed Graph Convolutional Network (GCN), as it can flexibly encode the relations among arbitrarily structured non-Euclidean data. Different from traditional GCN, there are two novel strategies adopted by our method to further exploit the contextual relations for accurate HSI classification. First, since the receptive field of traditional GCN is often limited to fairly small neighborhood, we proposed to capture long range contextual relations in HSI by performing successive graph convolutions on a learned region-induced graph which is transformed from the original 2D image grids. Second, we refine the graph edge weight and the connective relationships among image regions by learning the improved adjacency matrix and the 'edge filter', so that the graph can be gradually refined to adapt to the representations generated by each graph convolutional layer. Such updated graph will in turn result in accurate region representations, and vice versa. The experiments carried out on three real-world benchmark datasets demonstrate that the proposed method yields significant improvement in the classification performance when compared with some state-of-the-art approaches.