Abstract
We develop two techniques for analyzing the effect of porting a machine translation system to a new domain. One is a macro-level analysis that measures how domain shift affects corpus-level evaluation; the second is a micro-level analysis for word-level errors. We apply these methods to understand what happens when a Parliament-trained phrase-based machine translation system is applied in four very different domains: news, medical texts, scientific articles and movie subtitles. We present quantitative and qualitative experiments that highlight opportunities for future research in domain adaptation for machine translation.
Highlights
When building a statistical machine translation (SMT) system, the expected use case is often limited to a specific domain, genre and register ( “domain” refers to this set, in keeping with standard, imprecise, terminology), such as a particular type of legal or medical document
One important feature of our methodologies is that we focus on errors that could possibly be fixed given access to data from a new domain, rather than all errors that might arise because the particular translation model used is inadequate to capture the required
Adapting an SMT system from the Parliament domain to the news domain is not a representative adaptation task; there are a very small number of errors due to unseen words, which are minor in comparison to all other domains. (Despite the fact that most previous work focuses exclusively on using news as a “new” domain, §3). 2
Summary
When building a statistical machine translation (SMT) system, the expected use case is often limited to a specific domain, genre and register ( “domain” refers to this set, in keeping with standard, imprecise, terminology), such as a particular type of legal or medical document. It is expensive to obtain enough parallel data to reliably estimate translation models in a new domain. One can hope that large amounts of data from another, “old domain,” might be close enough to stand as a proxy. This is the defacto standard: we train SMT systems on Parliament proceedings, but use them to translate all sorts of new text. This results in significantly degraded translation quality. We show quantitative (§7.1) and qualitative (§7.2) results obtained from our methods on
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Transactions of the Association for Computational Linguistics
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
