Abstract
Text classification is a fundamental research direction, aims to assign tags to text units. Recently, graph neural networks (GNN) have exhibited some excellent properties in textual information processing. Furthermore, the pre-trained language model also realized promising effects in many tasks. However, many text processing methods cannot model a single text unit’s structure or ignore the semantic features. To solve these problems and comprehensively utilize the text’s structure information and semantic information, we propose a Bert-Enhanced text Graph Neural Network model (BEGNN). For each text, we construct a text graph separately according to the co-occurrence relationship of words and use GNN to extract text features. Moreover, we employ Bert to extract semantic features. The former part can take into account the structural information, and the latter can focus on modeling the semantic information. Finally, we interact and aggregate these two features of different granularity to get a more effective representation. Experiments on standard datasets demonstrate the effectiveness of BEGNN.
Highlights
Text classification is a fundamental task in natural language processing
We describe the structure of Bert-Enhanced text Graph Neural Network model (BEGNN) in detail
On the basis of using Bert to extract semantic features, and adding the structural features extracted by the graph neural network module, we hope that the two features can interact, rather than being separated from each other
Summary
Text classification is a fundamental task in natural language processing. It aims to assign labels to natural language text. Some works build homogeneous graphs or heterogeneous graphs from text data and perform graph neural network propagation such as convolution operations on the graphs [9,10] In this way, the model can take into account the structural information, which is of great significance for understanding the meaning of the text. (1) Our model can extract features of different granularities, from a pre-trained language model and graph neural networks for text representation It takes into account the semantic information, and the structural information, which improves the effect of the learned text representation. (2) In order to prevent the two features from being separated during the prediction process, we have designed and performed experiments on co-attention modules as well as different aggregation methods, which can consider the interaction of the two representations and make full use of them to achieve better classification capabilities. In the following paragraphs: Section 2 introduces researches about text classification methods related to our work, Section 3 illustrates the overall model we proposed, Section 4 shows the experimental results, and the conclusion
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
