Opera goer or Scrabble player: what makes a good translator?

Abstract

How should you select a person to carry out a translation? One approach is to request a sample translation and to evaluate it by hand. Quality Estimation addresses the problem of evaluation at least for Machine Translation output as a prediction task. This approach facilitates low-cost evaluation of MT outputs without expensive reference translations. However, the prediction of human translation in this way is difficult due to its subtlety of expression. We aimed to find out whether the qualifications, hobbies or personality traits of a person could predict their proficiency at translation. First, we gathered information about 82 participants; for each one we established the values of 146 attributes via a questionnaire. Second, we asked them to carry out some Japanese-to-English translations which we scored by hand. Third, we used the attributes as input and the translation scores as output to train the J48 decision-tree algorithm in order to predict the score of a translator from their attributes. This was then evaluated using ten-fold cross validation. When limiting to professional translators in Experiment 6, the best F-score was with Wrapper selection (0.775). The result was statistically significant ($$p < 0.05$$p<0.05). This classifier also showed the second highest Precision on Good (0.813). The second best F-score (0.737) has the highest Precision on Good (0.909), using Manual feature selection. Once again this was significant ($$p < 0.05$$p<0.05). The results suggest that certain attributes affect the prediction; in addition to experience and qualifications in translating into the target language, interest in going to the Opera, playing Scrabble or Contract Bridge, or enjoyment of cryptic crossword puzzles are important factors as well.