Professional Human Translation Research Articles

When minds meet machines: A pilot programme of using Chat-GPT to teach political speech translation in a Chinese-to-English translation classroom

This piece of reflection draws on a pilot programme of using Chat-GPT in the teaching of political speech translation in a Chinese-to-English translation classroom. The conduction of this pilot was driven by, among others, a growing concern by the third-year undergraduates doing Chinese-English translation as their major; they have aired worries that human translators might be replaced by large language models capable of doing the translation. The students observed and compared the human translation (by institutional interpreters) and the Chat-GPT translation of four political speeches delivered by Chinese government officials at the General Debate of the United Nations General Assembly. While acknowledging some strengths of Chat-GPT, they found that the GPT translation stayed too ‘close’ to the source text Chinese, potentially leading to issues at the lexical, sentence, and contextual levels. The author (also the course convenor) then reflected on the strengths and limitations of Chat-GPT in conjunction with pedagogical considerations regarding the ethical and practical implications of GPT-enabled machine translation. The paper concludes that GPT is not, and will not be, an adequate substitute for the language expertise, cultural awareness, ideological sensitivity, and creative abilities to ‘re-write’ on the part of professional human translators.

Exploring the Effectiveness of GPT-3 in Translating Specialized Religious Text from Arabic to English: A Comparative Study with Human Translation

In recent years, Natural Language Processing (NLP) models such as Generative Pre-trained Transformer 3 (GPT-3) have shown remarkable improvements in various language-related tasks, including machine translation. However, most studies that have evaluated the performance of NLP models in translation tasks have focused on general-purpose text, leaving the evaluation of their effectiveness in handling specialized text to be relatively unexplored. Therefore, this study aimed to evaluate the effectiveness of GPT-3 in translating specialized Arabic text to English and compare its performance to human translation. To achieve this goal, the study selected ten chapters from a specialized book written in Arabic, covering topics in specialized religious context. The chapters were translated by a professional human translator and by GPT-3 using its translation Application Programming Interface. The translation performance of GPT-3 to was compared to human translation using qualitative measures, specifically the Direct Assessment method. Additionally, the translations were evaluated using two different evaluation metrics, Bidirectional Encoder Representations from Transformers (BERT) score and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metric, which measure the similarity between the translated text and the reference text.The qualitative results show that GPT produced generally understandable translations but failed to capture nuances and cultural context. On the other hand, the quantitative results of the study showed that GPT-3 was able to achieve a relatively high level of accuracy in translating specialized religious text, with comparable scores to human translations in some cases. Specifically, the BERT score of GPT-3 translations was 0.83. The study also found that the Rouge score failed to fully reflect the capabilities of GPT-3 in translating specialized text.Overall, the findings of this study suggest that GPT-3 has promising potential as a translation tool for specialized religious text, but further research is needed to improve its capabilities and address its limitations.

Automated and Human Interaction in Written Discourse: A Contrastive Parallel Corpus-based Investigation of Metadiscourse Features in Machine-Human Translations

The rise of the internet has generated a need for fast online translations, which human translators cannot meet. Statistical tools such as Google and Baidu Translate provide automatic translation from one written language to another. This study reports the descriptive comparison of the machine-translation (MT) with human translation (HT), considering the metadiscoursal interactional features. The study uses a parallel corpus consisting of 79 texts translated from Chinese to English by professional human translators and machine translations (Baidu translate & Google translate) and a comparable reference corpus of non-translated English text. The statistical analysis revealed no statistically significant difference between Baidu and Google translate regarding all types of metadiscoursal indicators. However, the findings of this study demonstrate significant disparities in the interactional characteristics of various HT and MT groups. Compared to the metadiscourse features in non-translated English political texts, human translators were found to outperform machine translations in the use of attitude markers. In contrast, the distribution of directives in machine-translated texts is more native-like. In addition, MT and HT have utilized a significantly smaller number of hedges, self-mention, and readers than non-translated texts. Our results indicate that the MT systems, though still calling for further improvement, have shown tremendous growth potential and may complement human translators.

Traduction d’unités polylexicales du portugais en français par MT@EC et eTranslation

Translation of Multi-Word Units from Portuguese into French by MT@EC and eTranslation
 This paper aims to determine the extent to which the shift from statistical machine translation (SMT) to neural machine translation (NMT) improved the performance of European Union machine translation systems between 2015 and 2021 in terms of multi-word unit translation and domain coverage. To do so, we chose to test these systems on machine translation into French of multi-word units expressing quantitative and qualitative progression in Portuguese from Portugal. These units consist of the 2-gram ‘cada vez’ and a comparative adjective or adverb (cada vez COMP), and their word-for-word translation into French is not idiomatic (*chaque fois COMP). The most frequent translation into French is ‘de COMP en COMP’. This implies that these multi-word units must be translated ‘en bloc’, but their identification is not straightforward. On the one hand, COMP is not fixed and may include one (mais / plus, menos / moins, maior / plus grand, menor / plus petit, melhor / meilleur – mieux, pior / pire – plus mal) or several words (mais or menos N, ADJ, ADV). On the other hand, the 2-gram ‘cada vez’ can be part of other multi-word units expressing iteration (de cada vez (que)/(à) chaque fois (que)), or ‘dropper’ ([a certain quantity] de cada vez/ à la fois), This raises the challenge of ambiguity, well known to biotranslators and still often problematic for NMT. Moreover, units expressing quantitative or qualitative progression may raise other translation challenges when they are coordinate (with or without repetition of the 2-gram ‘cada vez’), when they are split (cada vez (…) COMP), or when they combine with verbs or nouns to form extended translation units whose translation into French can result in a more concise solution we refer to as ‘lexicalisation’. We established a biotranslation model based on a manually aligned French-Portuguese parallel literary corpus and online searchable French-Portuguese aligned corpora (translation memories). We selected a sample of occurrences of these multi-word units including several translation challenges. These occurrences were selected from a Portuguese journalistic corpus. They belong therefore to general language, whereas the EU's translation memories cover the domains dealt with by its institutions, which represents an additional challenge, considering the critical importance of domain coverage in the data to NMT performance quality. The selected occurrences were translated into French by the EU SMT system in 2015 (MT@EC) and 2019 (eTranslation Legacy) and by eTranslation (the EU NMT system) in 2019 and 2021. Firstly, MT output was analysed according to two general criteria: ‘non- literality’, that is translation into French without ‘chaque’, and acceptability from a semantic point of view, that is MT output without any false meaning, opposite meaning or nonsense. Then we looked at specific challenges, some of which could lead to original solutions, worthy of a professional human translator, such as lexicalisation, change of grammatical category or ‘recategorisation’ and ‘naturalisation’, that is phraseological or syntactic rearrangement that makes the target text more idiomatic. The results show that MT is improving, especially according to the criterion of non-literality. Original solutions are still rare, but they are diversifying in NMT output. Nevertheless, NMT remains imperfect, not least because of the inherent ambiguity of natural languages and the inevitable gaps in the data on which these systems are based. The results also demonstrate the importance of human intervention in the maintenance of the systems learning automatically, since the quality of SMT system’s output decreases between 2015 and 2019, when all efforts were focused on improving EU NMT system. Finally, results reveal the dangers of using English as a pivot language when translating from one Romance language into another, and the need to train future translators in NMT and post-editing.

Machine translation in society: insights from UK users

Machine translation (MT) tools like Google Translate can overcome language barriers and increase access to information. These tools also carry risks, and their societal role remains understudied. This article investigates typical uses and perceptions of MT based on a survey of 1200 United Kingdom residents who were representative of the national population in terms of age, sex, and ethnicity. We highlight three main findings from our analysis. First, participants often used MT for non-essential purposes that rarely justified professional human translations. Second, while they were highly satisfied with MT they also expressed desires for higher MT quality. These desires were usually motivated by expectations of perfection rather than fitness for purpose. Third, participants’ future vision for MT involved increasingly blurred boundaries between text and speech. The article calls for more MT research on the interface between written and spoken communication and on the ethical implications of rare but significant high-risk uses of the technology.

Automatische vertaling: een levensvatbare oplossing voor het recht?

The translation of legal information constitutes a special branch of translation practice, particularly due to the systemic nature of legal terminology, as well as complicated syntax and the use of fixed word associations and language templates. Professional human translators have always been the preferred way to translate legal documents because they have the linguistic expertise, a good understanding of the cultural context and capacity to make decisions when translating. As more and more international companies enter new markets, the need for translating legal texts increases, which often involves large amounts of data that need to be translated quickly. The possibilities and limits of machine translation (MT) are being frequently tested. Since the legal sector traditionally requires high-quality translations, the question arises whether machine translation can also work well in this field?

A Set of Recommendations for Assessing Human–Machine Parity in Language Translation

The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations. We reassess Hassan et al.'s 2018 investigation into Chinese to English news translation, showing that the finding of human–machine parity was owed to weaknesses in the evaluation design—which is currently considered best practice in the field. We show that the professional human translations contained significantly fewer errors, and that perceived quality in human evaluation depends on the choice of raters, the availability of linguistic context, and the creation of reference translations. Our results call for revisiting current best practices to assess strong machine translation systems in general and human–machine parity in particular, for which we offer a set of recommendations based on our empirical findings.

AccurIT: a prototype of a machine translation engine for English to Arabic translation

Current machine translators have reached an unprecedented level of sophistication in dealing with not only isolated words, but also longer sentences and paragraphs. Despite the advances achieved in this field, several challenges remain to be resolved for machine translation (MT) to be on par with professional human translation, including the quality of grammar and context accuracy, pragmatics, relevance, choice of vocabulary and ability to translate large files effectively based on this list's criteria. Another extremely problematic area that we have observed is incorrect literal translation of English phrases, proverbs, idioms, figurative speech and cliches, which proves to be an issue with most current translation programs, even ones built using a phrase-based approach. Therefore, this study's objective was to develop a prototype of an English-Arabic MT engine, AccurIT, to address MTs' English-to-Arabic translation-accuracy issues in general. We compared the results of our tool against Google and Azure Translation Hub based on some excerpts from the legal realm to demonstrate AccurIT's efficacy, and the results are promising.

A novel rule based machine translation scheme from Greek to Greek Sign Language: Production of different types of large corpora and Language Models evaluation

One of the aims of assistive technologies is to help people with disabilities to communicate with others and to provide means of access to information. As an aid to Deaf people, in this work we present a novel prototype Rule Based Machine Translation (RBMT) system for the creation of large and quality written Greek Sign Language (GSL) glossed corpora from Greek text. In particular, the proposed RBMT system assists the professional GSL translator in speeding up the production of different kinds of GSL glossed corpora. Then each glossed corpus is used for the production/creation of Language Model (LM) n-grams. With the GSL glossed corpus from Greek text, we can build, test and evaluate different kinds of Language Models for different kinds of glossed GSL corpora. Here, it should be noted that it does not require grammar knowledge of GSL but only very basic GSL phenomena covered by manual RBMT rules as it assists the professional human translator. Furthermore, it should also be stressed that Language Models for written GSL gloss are missing from the scientific literature, thus this work is pioneer in this field. Evaluation of the proposed scheme is carried out for the weather reports domain, where 20,284 tokens and 1000 sentences have been produced. By using the BiLingual Evaluation Understudy (BLEU) metric score, our prototype RBMT system achieves a relative score of 0.84 (84%) for 4-grams and 0.9 (90%) for 1-grams.

Machine Translation and Parallel Corpora: The Case of English-to-Chinese Translation of Relative Clauses

This paper compares translations done by professional human translators (as manifested in the Babel English-Chinese Parallel Corpus) and machine translations produced by Google Translate (with English as the source language and Simplified Chinese as the target language). Capitalizing on the branching direction of these two languages, we investigated translation strategies of restrictive relative clauses by focusing on which-relatives. The specific objectives of the study were to (a) extract relevant data from a parallel corpus; (b) assess machine translation quality; and (c) highlight the similarities and dissimilarities between corpus outputs and machine translation outputs. Of the 147 test materials, 115 Google Translate outputs (78.2%) were rated as ‘successful’ or ‘acceptable’. No significant differences were found between the degree of success in machine translations and linguistic factors (e.g., the active or passive voice in relative clauses). This finding confirms that linguistic knowledge is not required when using statistical machine translation (SMT) such as Google Translate. Another noteworthy finding was that Google Translate did not use the least frequently occurring translation strategy in the parallel corpus. This is not surprising given that SMT systems greatly rely on parallel corpora for training statistical translation models.

Evaluating the Accuracy of Google Translate for Diabetes Education Material.

BackgroundApproximately 21% of the US population speaks a language other than English at home; many of these individuals cannot effectively communicate in English. Hispanic and Chinese Americans, in particular, are the two largest minority groups having low health literacy in the United States. Fortunately, machine-generated translations represent a novel tool that non-English speakers can use to receive and relay health education information when human interpreters are not available.ObjectiveThe purpose of this study was to evaluate the accuracy of the Google Translate website when translating health information from English to Spanish and English to Chinese.MethodsThe pamphlet, “You are the heart of your family…take care of it,” is a health education sheet for diabetes patients that outlines six tips for behavior change. Two professional translators translated the original English sentences into Spanish and Chinese. We recruited 6 certified translators (3 Spanish and 3 Chinese) to conduct blinded evaluations of the following versions: (1) sentences translated by Google Translate, and (2) sentences translated by a professional human translator. Evaluators rated the sentences on four scales: fluency, adequacy, meaning, and severity. We performed descriptive analysis to examine differences between these two versions.ResultsCronbach's alpha values exhibited high degrees of agreement on the rating outcome of both evaluator groups: .919 for the Spanish evaluators and .972 for the Chinese evaluators. The readability of the sentences in this study ranged from 2.8 to 9.0 (mean 5.4, SD 2.7). The correlation coefficients between the grade level and translation accuracy for all sentences translated by Google were negative (eg, rMeaning=-.660), which indicates that Google provided accurate translation for simple sentences. However, the likelihood of incorrect translation increased when the original English sentences required higher grade levels to comprehend. The Chinese human translator provided more accurate translation compared to Google. The Spanish human translator, on the other hand, did not provide a significantly better translation compared to Google.ConclusionGoogle produced a more accurate translation from English to Spanish than English to Chinese. Some sentences translated by Google from English to Chinese exhibit the potential to result in delayed patient care. We recommend continuous training and credential practice standards for professional medical translators to enhance patient safety as well as providing health education information in multiple languages.

An Automatic Evaluation for Online Machine Translation: Holy Quran Case Study

The number of Free Online Machine Translation (FOMT) users witnessed a spectacular growth since 1994. FOMT systems change the aspects of machine translation (MT) and the mass translated materials using a wide range of natural languages and machine translation systems. Hundreds of millions of people use these FOMT systems to translate the holy Quran (Al-Qur?an) verses from the Arabic language to other natural languages, and vice versa. In this study, an automatic evaluation for the use of FOMT systems to translate Arabic Quranic text into English is conducted. The two well-known FOMT systems (Google and Bing Translators) are chosen to be evaluated in this study using a metric called Assessment of Text Essential Characteristics (ATEC). ATEC metric is one of the automatic evaluation metrics for machine translation systems. ATEC scores the correlation between the output of a machine translation system and professional human reference translation based on word choice, word orders and the similarity between MT output and the human reference translation. Extensive evaluation has been conducted on two well-known FOMT systems to translate Arabic Quranic text into English. This evaluation shows that Google translator performs better than Bing translator in translating Quranic text. It is noticed that the average ATEC score does not exceed 41% which indicates that FOMT systems are ineffective in translating Quranic texts accurately

Investigating the usefulness of machine translation for newcomers at the public library

This study investigates the potential of machine translation as an efficient and cost-effective means to translate sections of the Ottawa Public Library website into Spanish to better meet the linguistic needs of the Spanish-speaking newcomer community. One-hundred and fourteen community members participated in a recipient evaluation survey, in which they evaluated four different versions of a translated portion of the library’s website — a professional human translation, a maximally post-edited machine translation, a rapidly post-edited machine translation, and a raw machine translation. Participants also considered metadata such as the time and cost required to produce each version. Findings show that while machine translation cannot address every need, there are some instances for which the faster and cheaper post-edited versions are considered useful and acceptable to the community.

Performance of an online translation tool when applied to patient educational material

Language barriers may prevent clinicians from tailoring patient educational material to the needs of individuals with limited English proficiency. Online translation tools could fill this gap, but their accuracy is unknown. We evaluated the accuracy of an online translation tool for patient educational material. We selected 45 sentences from a pamphlet available in both English and Spanish, and translated it into Spanish using GoogleTranslate™ (GT). Three bilingual Spanish speakers then performed a blinded evaluation on these 45 sentences, comparing GT-translated sentences to those translated professionally, along four domains: fluency (grammatical correctness), adequacy (information preservation), meaning (connotation maintenance), and severity (perceived dangerousness of an error if present). In addition, evaluators indicated whether they had a preference for either the GT-translated or professionally translated sentences. The GT-translated sentences had significantly lower fluency scores compared to the professional translation (3.4 vs. 4.7, P < 0.001), but similar adequacy (4.2 vs. 4.5, P = 0.19) and meaning (4.5 vs. 4.8, P = 0.29) scores. The GT-translated sentences were more likely to have any error (39% vs. 22%, P = 0.05), but not statistically more likely to have a severe error (4% vs. 2%, P = 0.61). Evaluators preferred the professional translation for complex sentences, but not for simple ones. When applied to patient educational material, GT performed comparably to professional human translation in terms of preserving information and meaning, though it was slightly worse in preserving grammar. In situations where professional human translations are unavailable or impractical, online translation may someday fill an important niche.

A ranking method for example based machine translation results by learning from user feedback

Example-Based Machine Translation (EBMT) is a corpus based approach to Machine Translation (MT), that utilizes the translation by analogy concept. In our EBMT system, translation templates are extracted automatically from bilingual aligned corpora by substituting the similarities and differences in pairs of translation examples with variables. In the earlier versions of the discussed system, the translation results were solely ranked using confidence factors of the translation templates. In this study, we introduce an improved ranking mechanism that dynamically learns from user feedback. When a user, such as a professional human translator, submits his evaluation of the generated translation results, the system learns “context-dependent co-occurrence rules” from this feedback. The newly learned rules are later consulted, while ranking the results of the subsequent translations. Through successive translation-evaluation cycles, we expect that the output of the ranking mechanism complies better with user expectations, listing the more preferred results in higher ranks. We also present the evaluation of our ranking method which uses the precision values at top results and the BLEU metric.

Some Notes on The Proper Place of Men and Machines in Language Translation

Martin Kay's classic paper has hardly aged at all. Our society and our language have changed more than the relative status of people and machines. The title of the Kay paper would now be considered sexist in the USA, but it is still the case that the dream of MT "fires the imagination but it does not, except under very special circumstances, produce useful results," as he put it. There has been progress, however, in defining what those "very special circumstances" are. Two contrasts are useful in defining those circumstances. The first contrast is between controlled domain-specific language at one end of a spectrum of text types and dynamic general language at the other. A sublanguage could be considered to be a case of domain-specific language that is naturally rather than artificially controlled. MT techniques that work well for texts at the "domain" end of the spectrum do not work well for texts at the "general" end of the spectrum. Unfortunately for MT and fortunately for human translators, the majority of texts to be translated fall into a region of the spectrum where current linguistic theory and current techniques of computational linguistics do not fully apply and therefore do not result in high-quality raw output when these texts are translated by computer. This comment on translation quality brings us to another contrast that is relevant in defining the circumstances under which MT is useful: the contrast between highquality and "indicative" translation. It has often been assumed that for a translation to be useful it must be of sufficiently high quality to be comparable to the work of a professional human translator. Not so. Low-quality MT that is produced quickly and used only to get an indication of the content of the original text and which is then often discarded is sometimes called "indicative translation." Surprisingly, indicative translation is perhaps the fastest growing use for MT. However, it is important to point out that the increasing use of indicative translation at the European Union administrative centers and elsewhere has not reduced the need for high-quality translation by humans; it has at most slowed its growth. It should be remembered that when an indicative translation shows a document to be relevant, that document is often submitted for human translation and re-translated from the source text. Thus, indicative translation does not replace human translation; it complements it.

Machine-translated text: Is it comprehensible to proficient readears?

The purpose of the study was to investigate the possibility of using IBM-compatible microcomputers to produce comprehensible translations from English into Portuguese. A program was designed to store and process information about the English language, including lexical meanings, syntactic constraints and disambiguating rules. This information produced a data structure that was used to parse the English sentences and output a Portuguese text. The comprehensibility of the translated passages was tested with secondary school students by using both machine-translated passages and professional human translations. Each subject answered a reading comprehension test in both conditions. The different passages were balanced between conditions. The statistical analysis of the results indicated that the machine-translated passages, in spite of presenting some incorrect constructions, were understood by the subjects, both in terms of main ideas and details. The implications of the study are also discussed, mainly in terms of what kind of knowledge language learners should acquire to be able to interact meaningfully with authentic foreign language texts, as suggested by the findings of the investigation.