Twieng: A Multi-Domain Twi-English Parallel Corpus for Machine Translation of the Twi Language, A Low-Resource African Language

Abstract

A Twi-English parallel corpus is certainly an important resource for Machine Translation of Twi (ISO 639-3), a Low- Resource Language (LRL) which is mainly spoken in Ghana and Ivory Coast. Currently large-scale multidomain Twi- English parallel corpus is still unavailable partly due to the difficulties and the arduous efforts required in its design. A digital Twi lexicon curated purposely for linguistic research is also not available. In this paper, we present TWIENG – Twi English corpus, a large-scale multi-domain Twi-English parallel corpus and Twi lexicon, a digital Twi Dictionary. We discuss the data collection methodology, translation, alignment and compilation of the Twi-English parallel sentences and the technology we used to compile and host the corpus. Today’s parallel corpora are crawled from the web using web crawlers, the sentence pairs are processed, aligned, tokenized and compiled to create the corpus. We crawled English sentences from Ghanaian indigenous electronic news portals, Ghanaian Parliamentary Hansards, standard literature and also used crowdsourcing. The sentences are translated by professional translators and linguists, then aligned, tokenized and compiled. The corpus is curated using the sketch engine, a corpus manager and analysis software developed by Lexical Computing Limited. The corpus is manually evaluated by Twi professional linguists. The Corpus has 5,419 parallel sentences which were curled from local news portals, Ghana Parliament Hansard, The New Testament of the Twi Bible and through crowdsourcing via social media sites. CCS CONCEPTS • Computing Methodologies • Artificial Intelligence • Natural Language Processing