Xword: A Multi-lingual Framework for Expanding Words

Abstract

The word expansion task has applicability in information retrieval and question answering systems. It relieves the vocabulary mismatch problem leading to a higher recall. The recent word embedding models demonstrated merit for the word expansion task in comparison to the traditional n-gram models. However, to acquire quality embeddings in each language, the processes of corpus compilation, normalization and parameter tuning are time-consuming and challenging especially for poor resources languages such as Arabic. In this paper, we introduce Xword as an online multi-lingual framework for automatic word expansion. Xword relies on both pre-trained ad hoc word embedding models and n-gram models for the expansion task. Xword currently includes the two languages Arabic, and German. Xword represents the results of each model both individually and collectively. Additionally, Xword can filter out the result set based on sentiment and part of speech (POS) tag of every single word. Xword is available as a Web API along with the downloadable models and sufficient documentation on our public GitHub.