Deep Neural Semantic Network for Keywords Extraction on Short Text

Abstract

Keyword extraction is a branch of natural language processing, which plays an important role in many tasks, such as long text classification, automatic summary, machine translation, dialogue system, etc. All of them need to use high-quality keywords as a starting point. In this paper, we propose a deep learning network called deep neural semantic network (DNSN) to solve the problem of short text keyword extraction. It can map short text and words to the same semantic space, get the semantic vector of them at the same time, and then compute the similarity between short text and words to extract top-ranked words as keywords. The Bidirectional Encoder Representations from Transformers was first used to obtain the initial semantic feature vectors of short text and words, and then feed the initial semantic feature vectors to the residual network so as to obtain the final semantic vectors of short text and words at the same vector space. Finally, the keywords were extracted by calculating the similarity between short text and words. Compared with existed baseline models including Frequency, Term Frequency Inverse Document Frequency (TF-IDF) and Text-Rank, the model proposed is superior to the baseline models in Precision, Recall, and F-score on the same batch of test dataset. In addition, the precision, recall, and F-score are 6.79%, 5.67%, and 11.08% higher than the baseline model in the best case, respectively.