TGCN-Bert Emoji Prediction in Information Systems Using TCN and GCN Fusing Features Based on BERT

Abstract

In recent studies, graph convolutional neural networks (GCNs) have been used to solve different natural language processing (NLP) tasks. However, few researches apply graph convolutional networks to short text classification. Emoji prediction, as a complex sentiment analysis task, has received even less attention. In this work, the authors propose TGCN-Bert which combines pre-trained BERT temporal convolutional networks (TCNs) and graph convolutional networks for short text classification and emoji prediction. They initialize the nodes with the help of BERT and define the edges in text graph based on the term frequency-inverse document frequency (TF-IDF) and positive point-wise mutual information (PPMI). They employ the model for emoji prediction task, and a metric based on emoji clustering is developed to better measure the validity of emoji prediction results. To validate the performance of TGCN-Bert, they compare it with other GCN variants on short text classification datasets and emoji prediction datasets; experiments show that TGCN-Bert achieves better performance.