Arabic preprocessing for Statistical Machine Translation

Abstract

Arabic is a morphologically rich language. This poses some problems for statistical machine translation (SMT) approaches. In this chapter, we study the effect of different Arabic word-level preprocessing schemes and techniques on the quality of phrase-based SMT. We also present and evaluate different methods for combining preprocessing schemes. Our results show that given large training data sets, splitting off proclitics only performs best. However, for small training data sets, it is best to apply English-like tokenization using part-of-speech tags, and sophisticated morphological analysis and disambiguation. Moreover, choosing the appropriate preprocessing scheme produces a significant increase in BLEU score if there is a change in genre between training and test data. We also found that combining different preprocessing schemes leads to improved translation quality.