Machine-translated Text Research Articles

Estimating word-level quality of statistical machine translation output using monolingual information alone

AbstractVarious studies show that statistical machine translation (SMT) systems suffer from fluency errors, especially in the form of grammatical errors and errors related to idiomatic word choices. In this study, we investigate the effectiveness of using monolingual information contained in the machine-translated text to estimate word-level quality of SMT output. We propose a recurrent neural network architecture which uses morpho-syntactic features and word embeddings as word representations within surface and syntactic n-grams. We test the proposed method on two language pairs and for two tasks, namely detecting fluency errors and predicting overall post-editing effort. Our results show that this method is effective for capturing all types of fluency errors at once. Moreover, on the task of predicting post-editing effort, while solely relying on monolingual information, it achieves on-par results with the state-of-the-art quality estimation systems which use both bilingual and monolingual information.

Machine Translation in the Field of Law: A Study of the Translation of Italian Legal Texts into German

Abstract With the advent of the neural paradigm, machine translation has made another leap in quality. As a result, its use by trainee translators has increased considerably, which cannot be disregarded in translation pedagogy. However, since legal texts have features that pose major challenges to machine translation, the question arises as to what extent machine translation is now capable of translating legal texts or at least certain types of legal text into another legal language well enough so that the post-editing effort is limited, and, consequently, whether a targeted use in translation pedagogy can be considered. In order to answer this question, DeepL Translator, a machine translation system, and MateCat, a CAT system that integrates machine translation, were tested. The test, undertaken at different times and without specific translation memories, provided for the translation of several legal texts of different types utilising both systems, and was followed by systematisation of errors and evaluation of translation results. The evaluation was carried out according to the following criteria: 1) comprehensibility and meaningfulness of the target text; and 2) correspondence between source and target text in consideration of the specific translation situation. Overall, the results are considered insufficient to give post-editing of machine-translated legal texts a bigger place in translation pedagogy. As the evaluation of the correspondence between source and target text was fundamentally worse than with regard to the meaningfulness of the target text, translation pedagogy should respond by raising awareness about differences between machine translation output and human translation in this field, and by improving translation approach and strengthening legal expertise.

Function words in statistical machine-translated Chinese and original Chinese: A study into the translationese of machine translation systems

Abstract Statistical approaches have become the mainstream in machine translation (MT), for their potential in producing less rigid and more natural translations than rule-based approaches. However, on closer examination, the uses of function words between statistical machine-translated Chinese and the original Chinese are different, and such differences may be associated with translationese as discussed in translation studies. This article examines the distribution of Chinese function words in a comparable corpus consisting of MTs and the original Chinese texts extracted from Wikipedia. An attribute selection technique is used to investigate which types of function words are significant in discriminating between statistical machine-translated Chinese and the original texts. The results show that statistical MT overuses the most frequent function words, even when alternatives exist. To improve the quality of the end product, developers of MT should pay close attention to modelling Chinese conjunctions and adverbial function words. The results also suggest that machine-translated Chinese shares some characteristics with human-translated texts, including normalization and being influenced by the source language; however, machine-translated texts do not exhibit other characteristics of translationese such as explicitation.

No Longer Lost in Translation: Evidence that Google Translate Works for Comparative Bag-of-Words Text Applications

Automated text analysis allows researchers to analyze large quantities of text. Yet, comparative researchers are presented with a big challenge: across countries people speak different languages. To address this issue, some analysts have suggested using Google Translate to convert all texts into English before starting the analysis (Lucas et al. 2015). But in doing so, do we get lost in translation? This paper evaluates the usefulness of machine translation for bag-of-words models—such as topic models. We use the europarl dataset and compare term-document matrices (TDMs) as well as topic model results from gold standard translated text and machine-translated text. We evaluate results at both the document and the corpus level. We first find TDMs for both text corpora to be highly similar, with minor differences across languages. What is more, we find considerable overlap in the set of features generated from human-translated and machine-translated texts. With regard to LDA topic models, we find topical prevalence and topical content to be highly similar with again only small differences across languages. We conclude that Google Translate is a useful tool for comparative researchers when using bag-of-words text models.

Assessing complexity and difficulty levels of machine-translated texts

This paper addresses a proposal for assessing complexity and difficulty levels of machine-translated texts in Portuguese to be further post-edited without the support of the source text (monolingual post-editing) in an experimental setting. By using two objective standard parameters, namely readability indexes and word frequency, and by proposing post-editors’ perception of difficulty to comprehend and to post-edit machine-translated texts as a new parameter, we sought to select texts with similar textual complexity or difficulty levels. This selection was necessary to carry out an experiment with four monolingual post-editing tasks in Portuguese involving machine-translated texts from three different source languages (English, Spanish, and Chinese). The application of readability indexes in conjunction with word frequency based on a corpus to analyze machine-translated texts into Portuguese to be used in experiments showed to be consistent and adequate. This method can also be applied to select texts to be used in Portuguese language classrooms and to select Portuguese texts to be included in Portuguese language textbooks. The findings can also be applied to the translation classroom, in which teachers can use the same methodology to select texts to be translated or post-edited or encourage students to analyze the texts themselves before performing a task, so students can become aware of the potential effort to be invested on a task or the real effort invested on the task after performing it. Finally, post-editors’ perception proved to be a sound parameter to validate text selection.

Translate gene sequence into gene ontology terms based on statistical machine translation

This paper presents a novel method to predict the functions of amino acid sequences, based on statistical machine translation programs. To build the translation model, we use the “parallel corpus” concept. For instance, an English sentence “I love apples” and its corresponding French sentence “j’adore les pommes” are examples of a parallel corpus. Here we regard an amino acid sequence like “MTMDKSELVQKA” as one language, and treat its functional description as “0005737 0006605 0019904 (Gene Ontology terms)” as a sentence of another language. We select amino acid sequences and their corresponding functional descriptions in Gene Ontology terms to build the parallel corpus. Then we use a phrase-based translation model to build the “amino acid sequence” to “protein function” translation model. The Bilingual Evaluation Understudy (BLEU) score, an algorithm for measuring the quality of machine-translated text, of the proposed method reaches about 0.6 when neglecting the order of Gene Ontology words. Although its functional prediction performance is still not as accurate as search-based methods, it was able to give the function of amino acid sequences directly and was more efficient.

Structure–semantics interplay in complex networks and its effects on the predictability of similarity in texts

The classification of texts has become a major endeavor with so much electronic material available, for it is an essential task in several applications, including search engines and information retrieval. There are different ways to define similarity for grouping similar texts into clusters, as the concept of similarity may depend on the purpose of the task. For instance, in topic extraction similar texts mean those within the same semantic field, whereas in author recognition stylistic features should be considered. In this study, we introduce ways to classify texts employing concepts of complex networks, which may be able to capture syntactic, semantic and even pragmatic features. The interplay between various metrics of the complex networks is analyzed with three applications, namely identification of machine translation (MT) systems, evaluation of quality of machine translated texts and authorship recognition. We shall show that topological features of the networks representing texts can enhance the ability to identify MT systems in particular cases. For evaluating the quality of MT texts, on the other hand, high correlation was obtained with methods capable of capturing the semantics. This was expected because the golden standards used are themselves based on word co-occurrence. Notwithstanding, the Katz similarity, which involves semantic and structure in the comparison of texts, achieved the highest correlation with the NIST measurement, indicating that in some cases the combination of both approaches can improve the ability to quantify quality in MT. In authorship recognition, again the topological features were relevant in some contexts, though for the books and authors analyzed good results were obtained with semantic features as well. Because hybrid approaches encompassing semantic and topological features have not been extensively used, we believe that the methodology proposed here may be useful to enhance text classification considerably, as it combines well-established strategies.

Quiz-Based Evaluation of Machine Translation

Quiz-Based Evaluation of Machine Translation This paper proposes a new method of manual evaluation for statistical machine translation, the so-called quiz-based evaluation, estimating whether people are able to extract information from machine-translated texts reliably. We apply the method to two commercial and two experimental MT systems that participated in WMT 2010 in English-to-Czech translation. We report inter-annotator agreement for the evaluation as well as the outcomes of the individual systems. The quiz-based evaluation suggests rather different ranking of the systems compared to the WMT 2010 manual and automatic metrics. We also see that overall, MT quality is becoming acceptable for obtaining information from the text: about 80% of questions can be answered correctly given only machine-translated text.

Language Model Adaptation Using Machine-Translated Text for Resource-Deficient Languages

Text corpus size is an important issue when building a language model (LM). This is a particularly important issue for languages where little data is available. This paper introduces an LM adaptation technique to improve an LM built using a small amount of task-dependent text with the help of a machine-translated text corpus. Icelandic speech recognition experiments were performed using data, machine translated (MT) from English to Icelandic on a word-by-word and sentence-by-sentence basis. LM interpolation using the baseline LM and an LM built from either word-by-word or sentence-by-sentence translated text reduced the word error rate significantly when manually obtained utterances used as a baseline were very sparse.

An empirical investigation of temporal and technical post-editing effort

This article reports on a study designed to quantify temporal and technical post-editing effort in machine translated texts. The study involved two groups of English>German translators, one instructed to post-edit a machine-translated text and the other to translate the same text from English into German. In addition, text segments containing negative translatability indicators (NTIs) were isolated in order to compare the effort involved in post-editing and translating segments with such indicators and without. Results indicate that the post-editing effort for all segments was lower than the translation effort for those segments. Additional results suggest, among other things, that the removal of NTIs may reduce relative post-editing effort overall.

The Japanese government project for machine translation

The project is funded by a grant from the Agency of Science and Technology through the Special Coordination Funds for the Promotion of Science and Technology, and was started in fiscal 1982. The formal title of the project is "Research on Fast Information Services between Japanese and English for Scientific and Engineering Literature". The purpose is to demonstrate the feasibility of machine translation of abstracts of scientific and engineering papers between the two languages, and as a result, to establish a fast information exchange system for these papers. The project term was initially scheduled as three years from the fiscal year of 1982 with a budget of about seven hundred million yen, but, due to the present financial pressures on the government, the term has been extended to four years, up to 1986. The project is conducted by the close cooperation between four organizations. At Kyoto University, we have the responsibility of developing the software system for the core part of the machine translation process (grammar writing system and execution system); grammar systems for analysis, transfer and synthesis; detailed specification of what information is written in the word dictionaries (all the parts of speech in the analysis, transfer, and generation dictionaries), and the working manuals for constructing these dictionaries. The Electrotechnical Laboratories (ETL) are responsible for the machine translation text input and output, morphological analysis and synthesis, and the construction of the verb and adjective dictionaries based on the working manuals prepared at Kyoto. The Japan Information Center for Science and Technology (JICST) is in charge of the noun dictionary and the compiling of special technical terms in scientific and technical fields. The Research Information Processing System (RIPS) under the Agency of Engineer. # . mg Technology is responsible for completing the machine translation system, including the man-machine interfaces to the system developed at Kyoto, which allow preand post-editing, access to grammar rules, and dictionary maintenance. The project is not primarily concerned with the development of a final practical system; that will be developed by private industry using the results of this project. Technical know-how is already being transferred gradually to private enterprise through the participation in the project of people from industry. Software and linguistic data are also being transferred in part. Finally, complete technical transfer will be done under the proper conditions. The Japanese source texts being used are abstracts of scientific and technical papers published in the monthly JICST journal d Current Bibliography of Science and Technology. At present, the project is only processing texts in the electronics, electrical engineering, and computer science fields. English source texts will be abstracts from INSPEC in these f ields. . The sentence structures used in abstracts tend .to be complex compared to ordinary sentences, with long nominal compounds, noun-phrase conjunctions, mathematical and physical formulas, long embedded sentences, and so on. The analysis and translation of this type of sentence structure is far more difficult than ordinary sentence patterns. However, we have not included a pre-editing stage because we wanted to find the ultimate limitations on handling this type of complex sentence structure. Our system is based on the following concepts: 1. The use of all available linguistic information, both surface and syntactic. The writing of as detailed as possible syntactic rules. The development of a grammar writing system that can accept any future level of sophisticated linguistic theory. 2. The introduction of semantic information wherever necessary to enable the syntactic analysis to be as accurate as possible. The importance of semantic information not over-estimated; a well-balanced usage of both syntax and semantics. Heavily seman-