MIGP: Metapath Integrated Graph Prompt Neural Network

Abstract

Graph neural networks (GNNs) leveraging metapaths have garnered extensive utilization. Nevertheless, the escalating parameters and data corpus within graph pre-training models incur mounting training costs. Consequently, GNN models encounter hurdles including diminished generalization capacity and compromised performance amidst small sample datasets. Drawing inspiration from the efficacy demonstrated by self-supervised learning methodologies in natural language processing, we embark on an exploration. We endeavor to imbue graph data with augmentable, learnable prompt vectors targeting node representation enhancement to foster superior adaptability to downstream tasks. This paper proposes a novel approach, the Metapath Integrated Graph Prompt Neural Network (MIGP), which leverages learnable prompt vectors to enhance node representations within a pretrained model framework. By leveraging learnable prompt vectors, MIGP aims to address the limitations posed by mall sample datasets and improve GNNs’ model generalization. In the pretraining stage, we split symmetric metapaths in heterogeneous graphs into short metapaths and explicitly propagate information along the metapaths to update node representations. In the prompt-tuning stage, the parameters of the pretrained model are fixed, a set of independent basis vectors is introduced, and an attention mechanism is employed to generate task-specific learnable prompt vectors for each node. Another notable contribution of our work is the introduction of three patent datasets, which is a pioneering application in related fields. We will make these three patent datasets publicly available to facilitate further research on large-scale patent data analysis. Through comprehensive experiments conducted on three patent datasets and three other public datasets, i.e., ACM, IMDB, and DBLP, we demonstrate the superior performance of the MIGP model in enhancing model applicability and performance across a variety of downstream datasets. The source code and datasets are available in the website.11https://github.com/hzw-ai/MIGP, with password: MIGPNN2024.