Compositional coding capsule network with k-means routing for text classification

Abstract

Text classification is a challenging problem which aims to identify the category of texts. In the process of training, word embeddings occupy a large part of parameters. Under the constraint of limited computing resources, it indirectly restricts the ability of subsequent network design. In order to reduce the number of parameters for constructing word embeddings, this paper explores compositional coding mechanism and proposes a compositional weighted coding method to replace the conventional embedding layer. Furthermore, inspired by the excellent performance of capsule network in image classification, we design a capsule network combined with our compositional weighted coding method for text classification. We also offer a new routing algorithm based on k-means clustering theory to thoroughly mine the relationship between capsules. Experiments conducted on eight challenging text classification datasets show that the proposed method achieves competitive accuracy compared to the state-of-the-art approach with significantly fewer parameters.