ABAE: Utilize Attention to Boost Graph Auto-Encoder

Abstract

Graph Auto-Encoder(GAE) emerged as a powerful node embedding method, has attracted extensive interests lately. GAE and most of its extensions rely on a series of encoding layers to learn effective node embeddings, while corresponding decoding layers trying to recover the original features. Promising performances on challenging tasks have demonstrated GAE’s powerful ability of representation. On the other hand, Subgraph Convolutional Networks(SCNs), as an extension of Graph Convolutional Networks(GCNs), can aggregate both tagged and local structural features in an artful way. In this paper, we show that SCNs can be improved (AttSCNs) by an attention mechanism to acquire better representational capability, which is competent for the duty of encoder. Then we develop inversed AttSCNs and propose a novel auto-encoder, i.e., Attention-Based Auto-Encoder(ABAE). This architecture utilizes attention mechanism to get insight of the data. We perform experiments on some challenging tasks to show the effectiveness of our models. Moreover, we construct AttSCNs for Node Classification. The results demonstrate that AttSCNs can produce considerable embeddings. Furthermore, we launch Link Prediction task for the proposed ABAE. Experimental results show that our ABAE has its fantastic power and achieves state-of-the-art in Link Prediction.