Simple methods to overcome the limitations of general word representations in natural language processing tasks

Abstract

Although general word representations (GWRs) by skip-gram or GloVe have been widely used in many natural language processing (NLP) tasks with considerable success, they require further improvement. First, a GWR only represents general information of a word, even though task-oriented information can be more useful in specific tasks. Second, a GWR cannot avoid the out-of-vocabulary (OOV) problem. Thus, some recent studies have proposed methods based on an additional complex model or deep knowledge of resources for each specific task. Although such methods have the potential for improved performance, we believe that the baseline systems of each NLP task are already expensive; hence, making them more complex would be problematic for real-world applications. Therefore, the objective of this study is to overcome the limitations of GWRs by developing simple but effective methods for task-specific word representations (TSWRs) and OOV representations (OOVRs). The proposed methods achieved state-of-the-art performance in four Korean NLP tasks, namely part-of-speech tagging, named entity recognition, dependency parsing, and semantic role labeling.