Variational Graph Autoencoder Research Articles

Graph Embedding-Based Sensitive Link Protection in IoT Systems

In the Internet of Things (IoT), massive interconnected intelligent terminal devices constitute diverse networks. Link prediction can serve as a powerful inference attack to speculate the sensitive links in the networks, posing a security threat to entity privacy in IoT. Most antilink prediction methods reduce the prediction ability of link prediction models through link disturbance to hide sensitive links but fail to consider the impact of node attributes on link prediction. This paper proposes a sensitive link protection method based on graph embedding (SLPGE) to combat link prediction attacks. This method is aimed at compressing network topology data into an embedding matrix and lessening private information by combining Variational Graph Autoencoder (VGAE) and Adversarially Regularized Variational Graph Autoencoder (ARVGA). Based on our experiment on two datasets, SLPGE reduces the prediction accuracy of two attack models for sensitive links by up to 30.05% and 15.03% compared to the original data, and the corresponding utility sees a drop of 7.54% and 7.79% at most, which verifies the feasibility of SLPGE—achieving the tradeoff between privacy protection and data utility effectively.

Weakly supervised setting for learning concept prerequisite relations using multi-head attention variational graph auto-encoders

An increasing number of learners can benefit from educational resources in Massive Open Online Courses (MOOCs) through self-regulated learning. However, it is difficult for learners to organize a suitable learning path from educational resources in MOOCs if they don’t know the prerequisite relation between concepts. Manually labeling the prerequisite relation between concepts is time-consuming and requires significant domain knowledge. In addition, it isn’t notably easy for large concept datasets to label. How should learners start learning when they face massive knowledge concepts in MOOCs? To address these problems, we propose an end-to-end graph network-based model called Multi-Head Attention Variational Graph Auto-Encoders (MHAVGAE) to automatically label the prerequisite relation between concepts via a resource-concept graph. Firstly, we model a resource-concept graph according to learning resources, concept, and their relations, then introduce the multi-head attention mechanism to operate and compute the hidden representations of each vertex over the resource-concept graph. The purpose is to reduce the cognitive difference of manually labeling and consider the mutual influence between vertices in the resource concept graph. Secondly, we design a gated fusion mechanism to fuse the feature of the resource and concept graphs to enrich concept features. Thirdly, we propose a metric named Resource Prerequisite Reference Distance (RPRD), which generates inaccurate concept prerequisite relations to help us reduce manual labeling and then extend MHAVGAE with the weakly supervised setting for learning concept prerequisite relations. Finally, we conduct numerous experiments to demonstrate the effectiveness of MHAVGAE across multiple widely used metrics compared with the state-of-the-art methods. The experimental results show that the performance of MHAVGAE almost outperforms all the baseline methods and the weakly supervised setting of MHAVGAE is beneficial.

Network Embedding Algorithm Taking in Variational Graph AutoEncoder

Complex networks with node attribute information are employed to represent complex relationships between objects. Research of attributed network embedding fuses the topology and the node attribute information of the attributed network in the common latent representation space, to encode the high-dimensional sparse network information to the low-dimensional dense vector representation, effectively improving the performance of the network analysis tasks. The current research on attributed network embedding is presently facing problems of high-dimensional sparsity of attribute eigenmatrix and underutilization of attribute information. In this paper, we propose a network embedding algorithm taking in a variational graph autoencoder (NEAT-VGA). This algorithm first pre-processes the attribute features, i.e., the attribute feature learning of the network nodes. Then, the feature learning matrix and the adjacency matrix of the network are fed into the variational graph autoencoder algorithm to obtain the Gaussian distribution of the potential vectors, which more easily generate high-quality node embedding representation vectors. Then, the embedding of the nodes obtained by sampling this Gaussian distribution is reconstructed with structural and attribute losses. The loss function is minimized by iterative training until the low-dimension vector representation, containing network structure information and attribute information of nodes, can be better obtained, and the performance of the algorithm is evaluated by link prediction experimental results.

Spatiotemporal air quality inference of low-cost sensor data: Evidence from multiple sensor testbeds

Recent advances in sensor and IoT technologies allow for denser and mobile air quality measurements. These measurements are still spatiotemporally sparse at city-level, but can be interpolated using data-driven techniques. This work presents validation results of two machine-learning models to infer air quality sensor data in both space and time. Temporal validation exercises are performed at available regulatory monitoring stations following the FAIRMODE protocol. Both models show scalable to different mobile datasets with comparable prediction performance for PM2.5 (R2 = 0.68–0.75, MAE = 2.99–2.82 μg m−3) and NO2 (R2 = 0.8–0.82, MAE = 8.81–9.83 μg m−3) in Utrecht and Antwerp. In Oakland (Atlanta), we observed a lower performance for NO2 (R2 = 0.46–0.41, MAE = 4.06–5.07) and BC (R2 = 0.31–0.28, MAE = 0.48–0.27), likely caused by the less representative monitoring coverage. Although comparable in terms of prediction performance, the Geographical Random Forest (GRF) model seems to achieve slightly better accuracies, while the correlations are typically higher for the Air Variational Graph Autoencoder (AVGAE) model. This work demonstrates the potential of data-driven techniques for spatiotemporal air quality inference of complementary sensor data. The observed performance metrics approach current state-of-the-art chemical transport models in terms of performance while needing much lower resources, computational power, infrastructure and processing time.

Predicting miRNA-Disease Associations Based On Multi-View Variational Graph Auto-Encoder With Matrix Factorization.

MicroRNAs (miRNAs) have been proved to play critical roles in diverse biological processes, including the human disease development process. Exploring the potential associations between miRNAs and diseases can help us better understand complex disease mechanisms. Given that traditional biological experiments are expensive and time-consuming, computational models can serve as efficient means to uncover potential miRNA-disease associations. This study presents a new computational model based on variational graph auto-encoder with matrix factorization (VGAMF) for miRNA-disease association prediction. More specifically, VGAMF first integrates four different types of information about miRNAs into an miRNA comprehensive similarity network and two types of information about diseases into a disease comprehensive similarity network, respectively. Then, VGAMF gets the non-linear representations of miRNAs and diseases, respectively, from those two comprehensive similarity networks with variational graph auto-encoders. Simultaneously, a non-negative matrix factorization is conducted on the miRNA-disease association matrix to get the linear representations of miRNAs and diseases. Finally, a fully connected neural network combines linear and non-linear representations of miRNAs and diseases to get the final predicted association score for all miRNA-disease pairs. In the 10-fold cross-validation experiments, VGAMF achieves an average AUC of 0.9280 on HMDD v2.0 and 0.9470 on HMDD v3.2, which outperforms other competing methods. Besides, the case studies on colon cancer and esophageal cancer further demonstrate the effectiveness of VGAMF in predicting novel miRNA-disease associations.

Multi-Modal Variational Graph Auto-Encoder for Recommendation Systems

Graph embedding based methods have been used in recommendation systems recently, owing to their advances in modeling nodes as embeddings in a low-dimensional space. By effective neighborhood aggregation, graph convolutional networks can exploit high-order connections of neighbors such that the learned embeddings could be more informative thus improve the recommendation performance. However, user and item representations learned by graph aggregation inherently contain uncertainty due to sparsity of user-item interactions and noise of item features. To address these challenges, we propose a multi-modal variational graph auto-encoder (MVGAE) method. Specifically, we design modality-specific variational encoders that learn a Gaussian variable for each node whereas the mean vector represents semantic information and the variance vector denotes the noise level of the corresponding modality. Moreover, with the conditional independence assumption, the modality-specific Gaussian node embeddings are fused according to the product-of-experts principle, where the semantic information in each modality is weighted based on the estimated uncertainty level. Extensive experiments on three public datasets, Amazon Movies, Amazon Electronics and AliShop-7 C, demonstrate that our proposed method achieves competitive performance when compared with the state-of-the-art algorithms.

Detailed analysis of Ethereum network on transaction behavior, community structure and link prediction.

Ethereum, the second-largest cryptocurrency after Bitcoin, has attracted wide attention in the last few years and accumulated significant transaction records. However, the underlying Ethereum network structure is still relatively unexplored. Also, very few attempts have been made to perform link predictability on the Ethereum transactions network. This paper presents a Detailed Analysis of the Ethereum Network on Transaction Behavior, Community Structure, and Link Prediction (DANET) framework to investigate various valuable aspects of the Ethereum network. Specifically, we explore the change in wealth distribution and accumulation on Ethereum Featured Transactional Network (EFTN) and further study its community structure. We further hunt for a suitable link predictability model on EFTN by employing state-of-the-art Variational Graph Auto-Encoders. The link prediction experimental results demonstrate the superiority of outstanding prediction accuracy on Ethereum networks. Moreover, the statistic usages of the Ethereum network are visualized and summarized through the experiments allowing us to formulate conjectures on the current use of this technology and future development.

Semisupervised Classification with High-Order Graph Learning Attention Neural Network

At present, the graph neural network has achieved good results in the semisupervised classification of graph structure data. However, the classification effect is greatly limited in those data without graph structure, incomplete graph structure, or noise. It has no high prediction accuracy and cannot solve the problem of the missing graph structure. Therefore, in this paper, we propose a high-order graph learning attention neural network (HGLAT) for semisupervised classification. First, a graph learning module based on the improved variational graph autoencoder is proposed, which can learn and optimize graph structures for data sets without topological graph structure and data sets with missing topological structure and perform regular constraints on the generated graph structure to make the optimized graph structure more reasonable. Then, in view of the shortcomings of graph attention neural network (GAT) that cannot make full use of the graph high-order topology structure for node classification and graph structure learning, we propose a graph classification module that extends the attention mechanism to high-order neighbors, in which attention decays according to the increase of neighbor order. HGLAT performs joint optimization on the two modules of graph learning and graph classification and performs semisupervised node classification while optimizing the graph structure, which improves the classification performance. On 5 real data sets, by comparing 8 classification methods, the experiment shows that HGLAT has achieved good classification results on both a data set with graph structure and a data set without graph structure.

CellVGAE: an unsupervised scRNA-seq analysis workflow with graph attention networks.

MotivationSingle-cell RNA sequencing allows high-resolution views of individual cells for libraries of up to millions of samples, thus motivating the use of deep learning for analysis. In this study, we introduce the use of graph neural networks for the unsupervised exploration of scRNA-seq data by developing a variational graph autoencoder architecture with graph attention layers that operates directly on the connectivity between cells, focusing on dimensionality reduction and clustering. With the help of several case studies, we show that our model, named CellVGAE, can be effectively used for exploratory analysis even on challenging datasets, by extracting meaningful features from the data and providing the means to visualize and interpret different aspects of the model.ResultsWe show that CellVGAE is more interpretable than existing scRNA-seq variational architectures by analysing the graph attention coefficients. By drawing parallels with other scRNA-seq studies on interpretability, we assess the validity of the relationships modelled by attention, and furthermore, we show that CellVGAE can intrinsically capture information such as pseudotime and NF-ĸB activation dynamics, the latter being a property that is not generally shared by existing neural alternatives. We then evaluate the dimensionality reduction and clustering performance on 9 difficult and well-annotated datasets by comparing with three leading neural and non-neural techniques, concluding that CellVGAE outperforms competing methods. Finally, we report a decrease in training times of up to × 20 on a dataset of 1.3 million cells compared to existing deep learning architectures.Availabilityand implementationThe CellVGAE code is available at https://github.com/davidbuterez/CellVGAE.Supplementary information Supplementary data are available at Bioinformatics online.

Graph2MDA: a multi-modal variational graph embedding model for predicting microbe-drug associations.

Accumulated clinical studies show that microbes living in humans interact closely with human hosts, and get involved in modulating drug efficacy and drug toxicity. Microbes have become novel targets for the development of antibacterial agents. Therefore, screening of microbe-drug associations can benefit greatly drug research and development. With the increase of microbial genomic and pharmacological datasets, we are greatly motivated to develop an effective computational method to identify new microbe-drug associations. In this article, we proposed a novel method, Graph2MDA, to predict microbe-drug associations by using variational graph autoencoder (VGAE). We constructed multi-modal attributed graphs based on multiple features of microbes and drugs, such as molecular structures, microbe genetic sequences and function annotations. Taking as input the multi-modal attribute graphs, VGAE was trained to learn the informative and interpretable latent representations of each node and the whole graph, and then a deep neural network classifier was used to predict microbe-drug associations. The hyperparameter analysis and model ablation studies showed the sensitivity and robustness of our model. We evaluated our method on three independent datasets and the experimental results showed that our proposed method outperformed six existing state-of-the-art methods. We also explored the meaning of the learned latent representations of drugs and found that the drugs show obvious clustering patterns that are significantly consistent with drug ATC classification. Moreover, we conducted case studies on two microbes and two drugs and found 75-95% predicted associations have been reported in PubMed literature. Our extensive performance evaluations validated the effectiveness of our proposed method. Source codes and preprocessed data are available at https://github.com/moen-hyb/Graph2MDA. Supplementary data are available at Bioinformatics online.

Local2Global: Unsupervised multi-view deep graph representation learning with Nearest Neighbor Constraint

Multi-view feature fusion is a vital phase for multi-view representation learning. Recently, most Graph Auto-Encoders (GAEs) and their variants focus on multi-view learning. However, most of them ignore deep representation fusion of features of each multi-view. Furthermore, there are scarcely unsupervised constraints guiding to enhance the graph representation capability in training process. In this paper, we propose a novel unsupervised Multi-view Deep Graph Representation Learning (MDGRL) framework on multi-view data which is based on the Graph Auto-Encoders (GAEs) for local feature leaning, a feature fusion module for producing global representation and a valid variant of Variational Graph Auto-Encoder (VGAE) for global deep graph representation learning. To fuse Nearest Neighbor Constraint (NNC) between the maximal degree nodes which represents the most close joining node and their adjacent nodes into VGAE, we propose a new Nearest Neighbor Constraint Variational Graph Auto-Encoder (NNC-VGAE) to enhance the global deep graph representation capability for multi-view data. In the training process of NNC-VGAE, NNC makes the adjacent nodes gradually close to the maximal degree node. Hence, the proposed MDGRL has excellent deep graph representation capability for multi-view data. Experiments on eight non-medical benchmark multi-view data sets and four medical data sets confirm the effectiveness of our MDGRL compared with other state-of-the-art methods for unsupervised clustering.

Station-level short-term demand forecast of carsharing system via station-embedding-based hybrid neural network

Station-based one-way carsharing system brings transformation to public mobility and spurs the growth of sharing economy. The accurate estimation of rental and return demand of carsharing stations to support vehicle relocation is essential, hence a station-level short-term demand forecasting method named as station-embedding-based hybrid neural network (SEHNN) integrated by variational graph auto-encoder (VGAE) and long short-term memory network (LSTM) is proposed. The VGAE module undertakes the functions of station feature extraction and embedding, while the LSTM module captures the time series regularity and forecast the station-level demand. The results from the real data of Lanzhou, China demonstrate that, compared with ElasticNet, ARIMA, LSTM, and ConvLSTM, the mean absolute error of the proposed model targeted at hourly demand forecasting is reduced by 56.5%, 47.2%, 38.7%, and 38.5%, respectively, and it also outperforms some widely used models among different intervals and scales including main stations and subset carsharing system.

Variational graph autoencoders for multiview canonical correlation analysis

We present a novel approach for multiview canonical correlation analysis based on a variational graph neural network model. We propose a nonlinear model which takes into account the available graph-based geometric constraints while being scalable to large-scale datasets with multiple views. This model combines the probabilistic interpretation of CCA with an autoencoder architecture based on graph convolutional neural network layers. Experiments with the proposed method are conducted on classification, clustering, and recommendation tasks on real datasets. The algorithm is competitive with state-of-the-art multiview representation learning techniques, in addition to being scalable and robust to instances with missing views.

Deep node clustering based on mutual information maximization

Variational Graph Autoencoders (VGAs) are generative models for unsupervised learning of node representations within graph data. While VGAs have been achieved state-of-the-art results for different predictive tasks on graph-structured data, they are susceptible to the over-pruning problem where only a small subset of the stochastic latent units are active. This can limit their modeling capacity and their ability to learn meaningful representations. In this paper, we present SOLI (Stacked auto-encoder for nOde cLusterIng), an information maximization approach for learning graph representations by leveraging maximal cliques. SOLI relies on aggregating useful representations by assigning clique-based weights to various edges in a neighborhood while maximizing mutual information. The learned representations are mindful of graph patches centered around each node, and can be used for a range of downstream tasks, and thus encouraging more active units. We demonstrate strong performance across three graph benchmark datasets.(Code is available at https://github.com/SoheilaMolaei/SOLI.)

A representation learning model based on variational inference and graph autoencoder for predicting lncRNA-disease associations

BackgroundNumerous studies have demonstrated that long non-coding RNAs are related to plenty of human diseases. Therefore, it is crucial to predict potential lncRNA-disease associations for disease prognosis, diagnosis and therapy. Dozens of machine learning and deep learning algorithms have been adopted to this problem, yet it is still challenging to learn efficient low-dimensional representations from high-dimensional features of lncRNAs and diseases to predict unknown lncRNA-disease associations accurately.ResultsWe proposed an end-to-end model, VGAELDA, which integrates variational inference and graph autoencoders for lncRNA-disease associations prediction. VGAELDA contains two kinds of graph autoencoders. Variational graph autoencoders (VGAE) infer representations from features of lncRNAs and diseases respectively, while graph autoencoders propagate labels via known lncRNA-disease associations. These two kinds of autoencoders are trained alternately by adopting variational expectation maximization algorithm. The integration of both the VGAE for graph representation learning, and the alternate training via variational inference, strengthens the capability of VGAELDA to capture efficient low-dimensional representations from high-dimensional features, and hence promotes the robustness and preciseness for predicting unknown lncRNA-disease associations. Further analysis illuminates that the designed co-training framework of lncRNA and disease for VGAELDA solves a geometric matrix completion problem for capturing efficient low-dimensional representations via a deep learning approach.ConclusionCross validations and numerical experiments illustrate that VGAELDA outperforms the current state-of-the-art methods in lncRNA-disease association prediction. Case studies indicate that VGAELDA is capable of detecting potential lncRNA-disease associations. The source code and data are available at https://github.com/zhanglabNKU/VGAELDA.

Joint sparsity-biased variational graph autoencoders

To bring the full benefits of machine learning to defense modeling and simulation, it is essential to first learn useful representations for sparse graphs consisting of both key entities (vertices) and their relationships (edges). Here, we present a new model, the Joint Sparsity-Biased Variational Graph AutoEncoder (JSBVGAE), capable of learning embedded representations of nodes from which both sparse network topologies and node features can be jointly and accurately reconstructed. We show that our model outperforms the previous state of the art on standard link-prediction and node-classification tasks, and achieves significantly higher whole-network reconstruction accuracy, while reducing the number of trained parameters.

Structural Adversarial Variational Auto-Encoder for Attributed Network Embedding

As most networks come with some content in each node, attributed network embedding has aroused much research interest. Most existing attributed network embedding methods aim at learning a fixed representation for each node encoding its local proximity. However, those methods usually neglect the global information between nodes distant from each other and distribution of the latent codes. We propose Structural Adversarial Variational Graph Auto-Encoder (SAVGAE), a novel framework which encodes the network structure and node content into low-dimensional embeddings. On one hand, our model captures the local proximity and proximities at any distance of a network by exploiting a high-order proximity indicator named Rooted Pagerank. On the other hand, our method learns the data distribution of each node representation while circumvents the side effect its sampling process causes on learning a robust embedding through adversarial training. On benchmark datasets, we demonstrate that our method performs competitively compared with state-of-the-art models.

Interpretable Variational Graph Autoencoder with Noninformative Prior

Variational graph autoencoder, which can encode structural information and attribute information in the graph into low-dimensional representations, has become a powerful method for studying graph-structured data. However, most existing methods based on variational (graph) autoencoder assume that the prior of latent variables obeys the standard normal distribution which encourages all nodes to gather around 0. That leads to the inability to fully utilize the latent space. Therefore, it becomes a challenge on how to choose a suitable prior without incorporating additional expert knowledge. Given this, we propose a novel noninformative prior-based interpretable variational graph autoencoder (NPIVGAE). Specifically, we exploit the noninformative prior as the prior distribution of latent variables. This prior enables the posterior distribution parameters to be almost learned from the sample data. Furthermore, we regard each dimension of a latent variable as the probability that the node belongs to each block, thereby improving the interpretability of the model. The correlation within and between blocks is described by a block–block correlation matrix. We compare our model with state-of-the-art methods on three real datasets, verifying its effectiveness and superiority.

Disentangled and Proportional Representation Learning for Multi-View Brain Connectomes.

Diffusion MRI-derived brain structural connectomes or brain networks are widely used in the brain research. However, constructing brain networks is highly dependent on various tractography algorithms, which leads to difficulties in deciding the optimal view concerning the downstream analysis. In this paper, we propose to learn a unified representation from multi-view brain networks. Particularly, we expect the learned representations to convey the information from different views fairly and in a disentangled sense. We achieve the disentanglement via an approach using unsupervised variational graph auto-encoders. We achieve the view-wise fairness, i.e. proportionality, via an alternative training routine. More specifically, we construct an analogy between training the deep network and the network flow problem. Based on the analogy, the fair representations learning is attained via a network scheduling algorithm aware of proportionality. The experimental results demonstrate that the learned representations fit various downstream tasks well. They also show that the proposed approach effectively preserves the proportionality.

EXPLORING THE DESIGN SPACE: HIGH-SPEED INVESTIGATION WITH GRAPH NEURAL PROCEDURES

Adders are a crucial component of microprocessors' data channel logic; therefore, their design has been at the forefront of VLSI research for quite some time. While EDA flow helps designers get closer to an optimal adder architecture, it isn't always enough. The design space is huge, which is why this is the case. A machine learning-based strategy was offered in earlier studies as a means to investigate the design space. Weak feature representations and an inefficient two-stage learning loop cause prefix adder structures to underperform. A multi-branch framework that combines a variational graph autoencoder and a neural process (NP) is first demonstrated; this is the graph neural process.