Referential Translation Machines for Predicting Translation Performance

Abstract

Referential translation machines (RTMs) pioneer a language independent approach for predicting translation performance and to all similarity tasks with top performance in both bilingual and monolingual settings and remove the need to access any task or domain specific information or resource. RTMs achieve to become 1st in documentlevel, 4th system at sentence-level according to mean absolute error, and 4th in phrase-level prediction of translation quality in quality estimation task. 1 Referential Translation Machines Prediction of translation performance can help in estimating the effort required for correcting the translations during post-editing by human translators if needed. Referential translation machines achieve top performance in automatic and accurate prediction of machine translation performance independent of the language or domain of the prediction task. Each referential translation machine (RTM) model is a data translation prediction model between the instances in the training set and the test set and translation acts are indicators of the data transformation and translation. RTMs are powerful enough to be applicable in different domains and tasks while achieving top performance in both monolingual (Bicici and Way, 2015) and bilingual settings (Bicici et al., 2015b). Figure 1 depicts RTMs and explains the model building process (Bicici, 2016). RTMs use ParFDA (Bicici et al., 2015a) for selecting instances and interpretants, data close to the task instances for building prediction models and machine translation performance prediction system (MTPPS) (Bicici and Way, 2015) for generating features. We improve our RTM models (Bicici et Figure 1: RTM depiction: ParFDA selects interpretants close to the training and test data using parallel corpus in bilingual settings and monolingual corpus in the target language or just the monolingual target corpus in monolingual settings; an MTPPS uses interpretants and training data to generate training features and another uses interpretants and test data to generate test features in the same feature space; learning and prediction takes place taking these features as input. al., 2015b) with numeric expression identification using regular expressions and replace them with a label (Bicici, 2016). 2 RTM in the Quality Estimation Task We develop RTM models for all of the four subtasks of the quality estimation task (QET) in WMT16 (Bojar et al., 2016) (QET16), which include English to Spanish (en-es), English to German (en-de), and German to English (de-en) translation directions. The subtasks are: sentencelevel prediction (Task 1), word-level prediction (Task 2), phrase-level prediction (Task 2p), and document-level prediction (Task 3). Task 1 is about predicting HTER (human-targeted translation edit rate) (Snover et al., 2006) scores of sentence translations, Task 2 is about binary classification of word-level quality, Task 2p is about binary classification of phrase-level quality, and