A Joint Model for Text Classification with BERT-BiLSTM and GCN

Abstract

With the development of Graph Neural Network (GNN), a lot of GNN-based methods have been proposed in text classification. However, GNNs are difficult to capture the word relationship of the context in the document sequence. In addition, these models are common to use one-hot encoding to initialize the node features, which have high dimensions and sparse features. Therefore, it is difficult to represent the correlation between texts. To effectively learn the semantic features of contexts and the association between words, we propose a text classification method based on the BERT-BiLSTM word embedding model and the Graph Convolutional Network (GCN), which is named BBG. The BERT-BiLSTM can learn word embedding representations in different contexts and the semantic relationship of text contexts. The learned representation vectors are provided into GCN as the features of document nodes. The feature representations of the training data and unlabeled test data are learned by performing the label propagation in GCN. Finally, the classification results are output by utilizing the Softmax layer. Experiments on four benchmark datasets show that our method has better performances than existing text classification methods.