Abstract

Text classification is a fundamental problem in natural language processing. Recent studies applied graph neural network (GNN) techniques to capture global word co-occurrence in a corpus. However, previous works are not scalable to large-sized corpus and ignore the heterogeneity of the text graph. To address these problems, we introduce a novel Transformer based heterogeneous graph neural network, namely Text Graph Transformer (TG-Transformer). Our model learns effective node representations by capturing structure and heterogeneity from the text graph. We propose a mini-batch text graph sampling method that significantly reduces computing and memory costs to handle large-sized corpus. Extensive experiments have been conducted on several benchmark datasets, and the results demonstrate that TG-Transformer outperforms state-of-the-art approaches on text classification task.

Highlights

  • Text classification is a widely studied problem in natural language processing and has been addressed in many real-world applications such as news filtering, spam detection, and health record systems (Kowsari et al, 2019; Che et al, 2015; Zhang et al, 2018)

  • Researchers have recently turned to Graph Neural Network (GNN) to exploit global features in text representation learning, which learns node embedding by aggregating information from neighbors through edges

  • The main contributions of this work are as follows: 1. We propose Text Graph Transformer, a heterogeneous graph neural network for text classification

Read more

Summary

Introduction

Text classification is a widely studied problem in natural language processing and has been addressed in many real-world applications such as news filtering, spam detection, and health record systems (Kowsari et al, 2019; Che et al, 2015; Zhang et al, 2018). Liu et al (2020) further improved classification accuracy by expanding the text graph with semantic and syntactic contextual information These GCN-based models on heterogeneous text graphs suffer from two practical issues. None of these models are scalable to largesized corpus due to high computation and memory costs. Instead of learning based on the full text graph, we propose a text graph sampling method that enables subgraph mini-batch training. We propose Text Graph Transformer, a heterogeneous graph neural network for text classification It is the first scalable graph-based method for the task to the best of our knowledge. 2. We propose a novel heterogeneous text graph sampling method that significantly reduces computing and memory costs. We perform experiments on several benchmark datasets, and the results demonstrate the effectiveness and efficiency of our model

Methodology
Text Graph Building
Text Graph Sampling
Text Graph Transformer
Experimental Setup
Experiment Results
Text Classification
Graph Neural Network
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call