Evaluation Of Translation Quality Research Articles

The impact of contextual understanding on neural machine translation accuracy: A case study of Indonesian cultural idioms in English translation

The translation of Indonesian idiomatic expressions into English using Neural Machine Translation (NMT) systems presents significant challenges due to the intricate nature of idiomatic language. Idioms are culturally embedded constructs conveying meanings different from their literal interpretations. This study explores the effectiveness of NMT systems in capturing and translating Indonesian idiomatic expressions into English. Using a qualitative approach, 150 diverse Indonesian idioms were evaluated through Google Translate and DeepL. Each idiom was assessed for semantic accuracy, syntactic coherence, and contextual fidelity. Qualitative analysis provided a comprehensive evaluation of translation quality. Findings reveal significant challenges for NMT systems in translating idiomatic expressions. Both Google Translate and DeepL show strengths in some areas, but also have limitations. While they generally capture literal meanings, they often miss metaphorical nuances and cultural connotations. Syntactic errors such as incorrect word order and tense inconsistencies are common, especially in complex idioms. In addition, contextual fidelity analysis shows that NMT systems struggle with contextual appropriateness and pragmatic usage, resulting in translations lacking cultural sensitivity and relevance. These challenges highlight the need for improved algorithms to better interpret and translate idiomatic language across cultural contexts. Accurate idiom translation is crucial for language learners to understand cultural nuances and idiomatic usage. Enhancing the capabilities of NMT systems requires refining algorithms, developing context-aware models, and expanding training datasets with diverse idiomatic expressions. Overcoming these challenges will advance machine translation capabilities and improve cross-cultural communication and language learning experiences.

Evaluation of translation quality of English literary works based on big data

Evaluation of translation quality of English literary works based on big data

ASPECTS OF TRANSLATION QUALITY PROBLEM

Translation quality evaluation is generally referred to as one the mechanisms for translation industry development and the most important tool in the field of translator training, which determines the relevance of the stated topic. The purpose of this study is to review the problems and approaches to translation quality evaluation highlighting the differences in their treatment. The paper focuses on the study of the cognitive-terminological and system-activity aspects of translation evaluation, with special emphasis on the borderline between the process of quality evaluation and its result. An analysis of the term translation quality as used in meta-translation discourse demonstrated significant differences in the conceptualization of this notion. In translation theory, the term is generally used as an evaluative indicator of translation results, although in a broader sense it also refers to one of the main categories of translation studies that requires systematic research and description. In meta-translation discourse, translation quality may refer to a fairly high level of translation per se, but in combination with additional descriptive characteristics the term can also indicate different levels of quality, down to the lowest. To assess the results of translation using classical evaluative terms, namely equivalence and adequacy, it is generally necessary to clarify their conceptual content, which corresponds to a particular scientific paradigm. The existence of different and/or opposing points of view on the theory and practice of translation quality evaluation is accounted for by the complex and multidimensional nature of the phenomenon under consideration. The novelty of the conducted research lies in the distinction between translation evaluation as a result and translation evaluation as a process considered as a type of intellectual activity. Finally, we developed a model which reflects the structure of evaluation activities in the field of translation. The main components of this model are: the evaluation sphere, the evaluation actor, the object under evaluation (i.e. translation results), the purpose of evaluation activity, the evaluation method, the criteria and the parameters of evaluation. Following the system-activity approach one can take into account various facets of the problem of translation quality evaluation as well as the complex of motives and individual independent steps that determine the end result.

Data-driven Asian Adapted MQM Typology and Automation in Translation Quality Workflows

In this study we aim to test the impact of applying translation error taxonomies oriented towards European Languages in the annotation of Asian Languages. We aim to demonstrate how an error typology adapted for the latter languages can not only result in more linguistically accurate annotations, but also how this can be applied to automating and scaling translation quality evaluation.
 As such, we propose a Translation Errors Typology that aims to cover the shortcomings of the Multidimensional Quality Metrics (Lommel et al. 2014) framework (MQM) in what concerns the annotation of the East Asian Languages of Mandarin, Japanese and Korean. The effectiveness of the typology here proposed was tested by analysing the Inter-annotator agreement (IAA) scores obtained, in contrast with the typology proposed by Ye and Toral (2020) and the Unbabel Error Typology1. Finally, we propose a way of automating Translation Quality Workflows through a Quality Estimation (QE) technology that is able to predict the MQM scores of machine translation outputs at scale with a fair correlation with the human judgement produced by applying the East Asian Languages MQM module proposed in this study.

Employing Chatgpt in the Evaluation of Translation Quality

Employing Chatgpt in the Evaluation of Translation Quality

Strategies for Improving Accuracy and Fluency of Cultural Tourism Translations with the Help of Emerging Technologies

Abstract This paper proposes the preliminary architecture of the cultural tourism corpus in accordance with the requirements of cultural tourism translation accuracy and fluency and further optimizes the structure of the architecture by combining it with the cultural tourism translation quality evaluation standards. Using the simple Bayesian classifier and machine learning technology, respectively, the cultural tourism corpus data and text information are collected, inputted, organized, and classified sequentially. Using natural language processing technology, an encoder-decoder framework is constructed to semantically analyze the preprocessed corpus data information, and proposed algorithm performance evaluation criteria are proposed. Simulation experiments are set up to evaluate the model’s translation quality in conjunction with practical applications. The proposed model in this paper is superior to other models in terms of translation fluency, and the TER index of the model in this paper reaches 72.589%. The model proposed in this paper has a translation accuracy of above 90% after six practical application tests. To understand the cultural background, 52.1% of tourists are interested in using the model proposed in this paper.

Automatic Machine Translation Quality Evaluation Model under Feature Extraction Algorithm

Automatic Machine Translation Quality Evaluation Model under Feature Extraction Algorithm

원천텍스트 난이도와 기계번역 품질에 대한 고찰 - 구글 한영 법령번역 사례를 중심으로

This paper attempts to investigate the relationship between the source text difficulty and the quality of machine translation output. We examined the results of human evaluation of the Korean-English statutory translation by Google Translate and the source text difficulty measured in terms of lexical and terminological difficulty and syntactic complexity. The translations of two different statutes revealed varying correlation patterns examined in terms of fluency, accuracy, terminology and others. The results demonstrate that the two source texts’ syntactic complexity had a negative correlation with the machine translation quality evaluation. In other words, Google Translate was found to be weak at translating long and syntactically complex sentences. On the other hand, lexical and terminological difficulty did not have any correlation with the results of human evaluation despite the fact that statutes contain a lot of difficult words. Overall, the source text difficulty had a strong negative correlation with the human evaluation results, which was generally supported by the evaluators’ comments on fluency and accuracy errors.

ANALISIS KUALITAS TERJEMAHAN PADA LAMAN WEB KBRI BEIJING

This study aims to evaluate the accuracy of the Indonesian to Mandarin translation of a web page from the Indonesian Embassy in Beijing. The Nababan translation quality evaluation methodology, which was used in this study, divides translation product quality into three categories: accuracy, acceptability, and readability. This study has two objectives: 1) to analyze the level of accuracy, acceptability and readability of the Indonesian Embassy in Beijing's website and 2) to describe the quality of the translation of the Indonesian Embassy's website in Beijing. The text, which was broken up into 37 pieces of data, was sent to a competent rater to evaluate. The results revealed that the Indonesian Embassy in Beijing's website translation was of lower accuracy, acceptable quality, and had a moderate degree of readability, scoring an overall 2.48 out of 3 for correctness, acceptability, and readability. Although there are certain terms in the target text that require more than one reading to fully get the meaning, the target text is widely accepted and fairly simple for readers who are native speakers of it. It can be concluded that the content of the source text is not conveyed accurately to the target text

Особенности формирования у студентов навыка перевода с листа

Teaching sight translation is an integral part of translators' professional training, and it requires applying didactic measurements that increase students' motivation. The topic of didactic measurements in teaching sight translation is insufficiently developed in the contemporary international and domestic translation didactics and needs to be clarified. This paper aims to describe the didactic quantitative and qualitative measurements obtained and substantiated through applied theoretical and empirical research methods. The translation speed and evaluation of translation quality according to the scale of compliance with the norms of the target language are proposed as sight translation didactic measurements, which were identified and substantiated in the didactic experiment. The experiment involved 10 students in their fourth year of studying translation and interpretation at the Southern Federal University during the sixth and seventh semesters. The research describes relevant variables and results of the didactic experiment in sight translation training at the initial stage (4.2–4.3 printed characters per second). Translation speed was measured for translation from and into the foreign language in routine and stressful environments. Translation errors were analyzed, classified, and counted. The procedural difficulties of sight translation were observed and discussed with the students. Qualitative and quantitative measurements and their correlation in dynamics were compared. Diagnostic and boundary measurements of sight translation speed for the first year of translation training are proposed. Common and individual factors influencing the process of translation from the worksheets at the initial stage are found in the research; they include translation from or into the native language, the level of education, the speed of translation, the language pair, the stressful environment, the speed of reading, the density of the text, and the number of translation errors. It is proved that the speed and quality of translation in the course of the research grew unevenly, the speed of translation from a foreign language into Russian and the other way depends on the speed of reading in both languages.

Impacto da qualidade de textos de partida criados por utilizadores e agentes e a propagação de erros em sistemas de Tradução Automática

This paper proposes a typology concerning errors and linguistic structures found in the source text that have an impact on Machine Translation (MT). The main objectives of this project were firstly, to make a comparison between error typologies and analyze them according to their suitability; analyze annotated data and build a data-driven typology while adapting the previous existing typologies; make a distinction between the errors produced by users and agents in the online Customer Support domain; test the proposed typology with three case studies; methodize patterns in the errors found and verify their impact in MT systems; finally, create a typology ready for production for its particular field. At first, it was made a comparison between different typologies, whether they consider a bilingual or monolingual level (e.g. Unbabel Error Typology, MQM Typology (Lommel et al., 2014b) and SCATE MT Error Taxonomy (Tezcan et al., 2017). This comparison allowed us to verify the differences and similarities between them and, also, which issue types have been previously used. In order to build a data-driven typology, both sides of Customer Support were analyzed — user and agent — as they present different writing structures and are influenced by different factors. The results of that analysis were assessed through the annotation process with a bilingual error typology and were calculated with one of the most highly used manual evaluation metrics in translation quality evaluation — Multidimensional Quality Metrics (MQM), proposed in the QTLaunchPad project (2014), funded by the European Union. Through this analysis, it was then possible to build a data-driven typology — Source Typology. In order to aid future annotators of this typology, we provided guidelines concerning the annotation process and elaborate on the new additions of the typology. In the interest of confirming the reliability of this typology, three case studies were conducted in an internal pilot, with a total of 26,855 words, 2802 errors and 239 linguistic structures (represented in the ‘Neutral’ severity — associated with conversational markers, segmentation, emoticons, etc., characteristics of oral speech) annotated, with different purposes and taking into account several language pairs. In these studies, we verified the effectiveness of the new additions, as well as the transfer of source text errors to the target text. Besides that, it was also analyzed whether the linguistic structures annotated with the ‘Neutral’ severity had in fact any impact on the MT systems. This testing allowed us to confirm the effectiveness and reliability of the Source Typology, including what needs improvement.

Evaluation System of Mobile English Learning Platform by Using Deep Learning Algorithm

At present, China’s economic development continues to progress and foreign exchanges are also increasingly frequent. Learning to master the world’s universal pronunciation, English, has become a more important link. However, people’s living habits make it difficult to carry heavy desktop devices for a long time, so mobile English learning platform meets the development needs of people’s English teaching. Based on the existing English mobile learning platform, this paper puts forward the concept of integrating artificial intelligence and deep learning technology into it. Through deep learning, the learning status and learning situation of students in the process of English learning are extracted, so as to analyze the needs and learning interests of each student in English learning and then push corresponding English materials to each student, which improves the efficiency of English learning. In addition, deep learning can also model the data of students’ behaviors and build a language vector feature extraction mechanism and translation quality evaluation model, so as to carry out certain intelligent auxiliary correction and correction on students’ English grammatical expressions and spoken English. The research in this paper has achieved good results through practice. The practice results show that the integration of deep learning into the existing mobile English teaching platform can optimize the functions of the existing platform, provide more ideas for the development of online English learning, and has good theoretical and practical value.

The Quality Evaluation Method of Sci-Tech English Translation for Intercultural Communication

English for Science and Technology (EST), as a special language style, is widely used in the field of Science and Technology. For this kind of articles, the requirements of translation quality are relatively high. Therefore, this paper studies a quality evaluation method of Sci-Tech English translation for cross-cultural communication. As statistical machine translation has almost reached the limits of its capacity, neural machine translation is becoming the technology of the future. This paper also describes the evaluation of machine translation quality with and automatic evaluation process with machine learning technology. The evaluation index of EST translation quality is selected according to the selection principle and expert consultation method. Then, the weight of the index is calculated by using the analytic hierarchy process. Finally, the translation quality evaluation is given by using the fuzzy comprehensive evaluation, glass-box and black-box evaluation with machine learning method. The results show that under the application of the research method, the evaluation results are completely corresponding to the actual competition results of four competitors, which proves the effectiveness of the research method.

Evaluation of the Accuracy of Artificial Intelligence Translation Based on Deep Learning

A machine automatic translation quality evaluation model based on a deep learning algorithm is built with the aim of achieving an accurate evaluation of machine automatic translation quality. Deep learning-based automatic machine translation was made. During the unsupervised and supervised learning stages, language information extraction executes unsupervised learning and reconstructs automated translation samples of bilingual words using a noise lessening automatic coding machine. To improve the effect of language vector feature extraction, the machine optimizes the effect of language vector feature extraction by importing language vector function and machine automatic translation information into bilingual words. The machine automatic translation language vector function is imported into the translation quality evaluation model based on deep learning to realize the automatic machine translation quality evaluation. The experimental results show that the constructed model can accurately evaluate the quality of machine automatic translation, the pattern of translated sentences and the number of sentences do not adversely affect the evaluation performance of the model, and the model exerts a good effect is showing.

An Interpretive Study of Six Major Models towards Translation Quality Assessments

Juliane House, Katherina Reiss, Malcolm Williams, Xinxiang Wu, Hong’an Li, Xianzhu Si and Sanning He have been recognized as the most renowned scholars who’ve done profound and systematic researches on translation quality assessment. Through the interpretive study of these six evaluation models, we can grasp the advancement and effectiveness of the researches of translation quality evaluation at home and abroad, and bring some inspiration and suggestions to the future human-centered as well as computer-centered translation quality evaluation researches on the basis of the apprehension of the limitations of the researches in existence.

Some Translation Studies informed suggestions for further balancing methodologies for machine translation quality evaluation

Abstract This article intends to contribute to the current debate on the quality of neural machine translation (NMT) vs. (professional) human translation quality, where recently claims concerning (super)human performance of NMT systems have emerged. The article will critically analyse some current machine translation (MT) quality evaluation methodologies employed in studies claiming such performance of their MT systems. This analysis aims to identify areas where these methodologies are potentially biased in favour of MT and hence may overvalue MT performance while undervaluing human translation performance. Then, the article provides some Translation Studies informed suggestions for improving or debiasing these methodologies in order to arrive at a more balanced picture of MT vs. (professional) human translation quality.

An English Translation Quality Evaluation Model Integrating Knowledge Transfer and Wireless Network

Machine translation is to automatically translate one natural language text into another. It is one of the important applications of natural language processing. The research of machine translation is closely related to machine translation evaluation technology. Therefore, with the progress of machine translation technology in recent years, researchers are also committed to designing more reasonable evaluation methods. Manual evaluation is costly, time-consuming, and subjective, so it has been difficult to provide practical help for the development of machine translation systems. Finding a feasible automatic machine translation evaluation method has been a hot spot in machine translation evaluation research in recent years. Based on this, this paper proposes an English translation quality evaluation model integrating knowledge transfer and wireless network. Firstly, wireless network technology is used to coordinate multitasks, and then knowledge transfer optimization technology is used to construct an English translation quality evaluation model. In this paper, simulation experiments are designed to verify the effectiveness of the model. The experimental results show that the accuracy of the evaluation method based on fusion technology is significantly improved compared with the existing evaluation methods.

중국 관영매체 정치뉴스의 한국어 번역문에 대한 비평 및 제안 -House의 번역품질평가 모델을 기반으로

This paper examined the Korean translation of political news in Chinese state media. So far, relevant research has not achieved significant results, and most studies ignored the textual function of the original text, evaluated the translation text with subjective judgment and put forward translation strategies. Referring to House’s translation quality evaluation model, this paper criticized and analyzed the Korean translation of official media’s political news from the aspects of register (field, tenor, style) and genre, obtained the following conclusions. In terms of language field, the translation text failed to have the function of the original text and lost its authority. In terms of tenor, the translation text did not have the genre characteristics of news. In terms of style, the translation text failed to reach the level of written style. According to the evaluation results, the following improvement schemes and translation strategies were proposed. First, the official media should pay attention to the quality of translation and cultivate high-level translators; Second, the academic community should create a good research atmosphere; Third, translators should improve their professional quality and ability. The specific translation strategies are as follows: first, fully analyze the textual function of the original text; Secondly, explicit translation method and implicit translation method are determined; Third, fully consider cultural differences.

Japanese Translation Quality Evaluation System Based on Deep Neural Network Algorithm

The current translation quality evaluation system relies on the combination of manual and text comparison for evaluation, which has the defects of low efficiency and large evaluation errors. In order to optimize the defects of the current quality evaluation system, a Japanese translation quality evaluation system based on deep neural network algorithm will be designed. In order to improve the processing efficiency of the system, the USB3.0 communication module of the hardware system will be optimized. Based on the hardware design, the reference translation map is used to extend the reference translation of Japanese translation. The evaluation indexes of over- and under-translation are set, and the evaluation of Japanese translation quality is realized after the parameters are determined by training the deep neural network using the sample set. The system functional test results show that the average data transmission processing time of the system is improved by about 31.27%, and the evaluation error interval is smaller and the evaluation is more reliable.

Interdependence of expert categories and automated metrics applied to evaluate translation quality

The article evaluates the quality of translation; we consider the applied and pragmatic aspects of such evaluation in the conditions of the current rapid increase in the number of texts to be translated. The article summarizes a plethora of assessment principles, each having its merits and drawbacks, and examines the correlation between the categories of adequacy and equivalence as the basic assessment parameters. We conclude that equivalence is oriented towards the result of translation, whereas adequacy indicates whether translation as process corresponds to the communicative situation given. Adequacy is thus viewed as a discursive feature. The article also looks into the possibility of applying these assessment criteria to automated evaluation of translation quality, and compares the specific computer metrics or, algorithms: Cosine Similarity, Word Error Rate, BLEU, ROUGE, NIST, METEOR. We analyze the potential for applying these metrics to man-made rather than machine translated. The n-gram metric for evaluation of translation with explicit ordering Meteor is the most efficient algorithm; however, it can only be applied advantageously to formal standardized texts. The article points out that literary translation can hardly be evaluated on the basis of n-gram correspondences due to it characteristic feature: ample use of non-equivalent units even at the sentence level. The purely mathematical procedures overviewed in the article cannot serve as linguistic meta-models of assessment unless supplemented with syntactic and semantic algorithms.