POS-based word alignment for small corpus

Abstract

A good quality word alignment system needs a high quality, domain specific and large size of the parallel corpus for training. Finding a high quality parallel corpus for a particular language pair in a specific domain is expensive and hard to find. Conversely, using a small corpus has some advantages like less training time and low memory requirement. It can be created or corrected manually. So this paper is an effort to achieve good quality word alignment with a small size of parallel corpus. This paper describes use of part of speech (POS) tag to improve the performance of statistical word alignment. This approach works well with small size of the corpus. Experiments were conducted on TDIL sample tourism corpus of 1000 sentences for English-Hindi language pair. Out of these 1000 sentences 950 sentences are used for training and 50 sentences are used for testing. F-measure is increased by approximately 4% and Alignment Error Rate (AER) decreased by approximately 4% in comparison to baseline system for word alignment GIZA++.