Comparative Evaluation of Translation Memory (TM) and Machine Translation (MT) Systems in Translation between Arabic and English

Translation memory (TM) systems have become a key technology to help translators in dealing with a steadily growing demand for translation, and areknown to have a positive effect on productivity, quality, and cost of translation. However, certain restrictions of this technology prompt developers ofcomputer assisted translation (CAT) tools to improve and expand the functionality of TM systems. Therefore, machine translation (MT) is beingprogressively integrated into TM tools. The paper aims at analysing potential advantages of TM and MT integration, discusses the effects of MT on awork routine of professional translators, and presents opinions of translators towards the integration of MT into the process of translation.There are two ways of integrating MT into TM systems: MT systems are included into CAT software packages or translators are provided with apossibility to port their TM tools to commercial or free access on-line MT providers; according to the second scenario, TM systems are complementedwith statistical, example-based or rule-based MT technologies.The poor quality of MT output made a lot of professional translators reluctant to use MT; however, the data-driven MT approach is making a difference.There are studies that compare translation speed when post-editing MT suggestions, reviewing repetitions and correcting fuzzy matches from a TMdatabase. Results demonstrate that the difference is not so significant and the MT output when post-edited by a human translator can also reach thedesirable level of the quality. Thus, technological competence and post-editing skills are being emphasised as the most significant skills for a professionaltranslator.Although there are disagreements among translators towards the integration of MT into their workflow, potential productivity gains derived from theintegration of TM and MT technologies and changing requirements of the translation market with regard to speed and cost of translation are likely totrigger further developments in the automation of the process of translation. DOI: http://dx.doi.org/10.5755/j01.sal.0.23.5559

Introduction Translation from a source language into a target language has become a very important activity in recent years, both in official institutions (such as the United Nations and the EU, or in the parliaments of multilingual countries like Canada and Spain), as well as in the private sector (for example, to translate user's manuals or newspapers articles). Prestigious clients such as these cannot make do with approximate translations; for all kinds of reasons, ranging from the legal obligations to good marketing practice, they require target-language texts of the highest quality. The task of producing such high-quality translations is a demanding and time-consuming one that is generally conferred to expert human translators. The problem is that, with growing globalization, the demand for high-quality translation has been steadily increasing, to the point where there are just not enough qualified translators available today to satisfy it. This has dramatically raised the need for improved machine translation (MT) technologies. The field of MT has undergone something of a revolution over the last 15 years, with the adoption of empirical, data-driven techniques originally inspired by the success of automatic speech recognition. Given the requisite corpora, it is now possible to develop new MT systems in a fraction of the time and with much less effort than was previously required under the formerly dominant rule-based paradigm. As for the quality of the translations produced by this new generation of MT systems, there has also been considerable progress; generally speaking, however, it remains well below that of human translation. No one would seriously consider directly using the output of even the best of these systems to translate a CV or a corporate Web site, for example, without submitting the machine translation to a careful human revision. As a result, those who require publication-quality translation are forced to make a diffcult choice between systems that are fully automatic but whose output must be attentively post-edited, and computer-assisted translation systems (or CAT tools for short) that allow for high quality but to the detriment of full automation. Currently, the best known CAT tools are translation memory (TM) systems. These systems recycle sentences that have previously been translated, either within the current document or earlier in other documents. This is very useful for highly repetitive texts, but not of much help for the vast majority of texts composed of original materials. Since TM systems were first introduced, very few other types of CAT tools have been forthcoming. Notable exceptions are the TransType system and its successor TransType2 (TT2). These systems represent a novel rework-ing of the old idea of interactive machine translation (IMT). Initial efforts on TransType are described in detail in Foster; suffice it to say here the system's principal novelty lies in the fact the human-machine interaction focuses on the drafting of the target text, rather than on the disambiguation of the source text, as in all former IMT systems. In the TT2 project, this idea was further developed. A full-fledged MT engine was embedded in an interactive editing environment and used to generate suggested completions of each target sentence being translated. These completions may be accepted or amended by the translator; but once validated, they are exploited by the MT engine to produce further, hopefully improved suggestions. This is in marked contrast with traditional MT, where typically the system is first used to produce a complete draft translation of a source text, which is then post-edited (corrected) offline by a human translator. TT2's interactive approach offers a significant advantage over traditional post-editing. In the latter paradigm, there is no way for the system, which is off-line, to benefit from the user's corrections; in TransType, just the opposite is true. As soon as the user begins to revise an incorrect segment, the system immediately responds to that new information by proposing an alternative completion to the target segment, which is compatible with the prefix that the user has input. Another notable feature of the work described in this article is the importance accorded to a formal treatment of human-machine interaction, something that is seldom considered in the now-prevalent framework of statistical pattern recognition.

This paper addressees the problem of eliminating unsatisfactory outputs from machine translation (MT) systems. The authors intend to eliminate unsatisfactory MT outputs by using confidence measures. Confidence measures for MT outputs include the rank-sum-based confidence measure (RSCM) for statistical machine translation (SMT) systems. RSCM can be applied to non-SMT systems but does not always work well on them. This paper proposes an alternative RSCM that adopts a mixture of the N-best lists from multiple MT systems instead of a single-system's N-best list in the existing RSCM. In most cases, the proposed RSCM proved to work better than the existing RSCM on two non-SMT systems and to work as well as the existing RSCM on an SMT system.

Machine translation, which is less commonly referred as computer translation or automatic translation, is the generally accepted name for any system which uses a computer to transform a text in one language into some kind of text in another natural language. Fully automatic translation lies at one end of the scale and the work of the human translator armed with pencil and paper at the other. Between them are a number of possibilities for collaboration between man and computer which include word processing, terminology databases, voice recognition and translation memory systems. Machine translation (MT) and translation memory (TM) systems are frequently confused. Machine translation essentially involves the generation of target text from possibly unseen source text; translation memory programs are designed retrieve from pairs of matching strings stored in a database-a sophisticated electronic phrasebook, stocked with the translator's own work. MT developers have been looking at ways of incorporating full-sentence retrieval into their programs, whilst some TM vendors have introduced third-party translation engines handle unseen text. The future translation environment will probably exhibit both aspects by design rather than as add-ons. (5 pages)

Nowadays, in the age of global communications, the need for on-line, accurate, and cheap translation from one language into another has become absolutely pronounced. This situation can even turns critical when considering translation and interpreting in high risk technology domains. The discrepancy in Codes, Norms, and Standards in various countries as well as delayed information exchange gives rise to increasing disagreement in high-technology and dangerous fields of common engineering interest. The emphasis of this paper is the necessity of realizing the correlation between the main problems and achievements of both machine translation and translation memory systems as well as the possibilities of using such systems as a real tool (automatic workstation) for both analysts and translators. It means consideration of such system pragmatics which demands that we investigate different kinds of linguistic and extra-linguistic knowledge as well as interference of the authors’ mother tongues on the English special text generation. One of the solutions is to give due consideration of the text structure and to use translation memory principles when a single text is processed in a machine translation system.

Nowadays, in the age of global communications, the need for fast, accurate and cheap translation from one language into another has become very pronounced. This situation can turn critical when considering translation and interpreting in high risk technology domains. The discrepancy in Codes, Norms and Standards in various countries as well as delayed information exchange gives rise to increasing disagreement in high-technology and dangerous fields of common engineering interest. The main subject of this paper is the correlation between the main problems and achievements of the present-day machine translation and translation memory systems and the possibilities of using such systems as real tools for a translator. This means investigating different kinds of linguistic and extra-linguistic knowledge and taking into consideration interference from the authors’ mother tongues when they generate an English text in a special domain. One of the solutions is to give due consideration of the text structure and to use translation memory principles when a single text is processed in a machine translation system.

Examining the general impact of Controlled Language (CL) rules in the context of Machine Translation (MT) has been an area of research for many years. The present study focuses on the following question: how do CL rules impact MT output individually? By analysing a German corpus-based test suite of technical texts that have been translated into English by different MT systems, this study endeavours to answer this question at different levels: the general impact of CL rules (rule- and system-independent), their impact at rule level (system-independent) as well as at rule and system level. The results of five MT systems are analysed and contrasted: a rule-based system, a statistical system, two differently constructed hybrid systems, and a neural system. For this, a mixed-methods triangulation approach that includes error annotation, human evaluation, and automatic evaluation was applied. The data was analysed both qualitatively and quantitatively in terms of CL influence on the following parameters: number and type of MT errors, style and content quality, and scores of two automatic evaluation metrics. In line with many studies, the results show a general positive impact of the applied CL rules on the MT output. However, at rule level, only four rules proved to have positive effects on the aforementioned parameters; three rules had negative effects on the parameters; and two rules did not show any significant impact. At rule and system level, the rules affected the MT systems differently, as expected. Rules that had a positive impact on earlier MT approaches did not show the same impact on the neural MT approach. Furthermore, neural MT delivered distinctly better results than earlier MT approaches, namely the highest error-free, style and content quality rates both before and after applying the rules, which indicates that neural MT offers a promising solution that no longer requires CL rules for improving the MT output.

Professional translators of technical documents often use Translation Memory (TM) systems in order to capitalize on the repetitions frequently observed in these documents. TM systems typically exploit not only complete matches between the source sentence to be translated and some previously translated sentence, but also so-called fuzzy matches, where the source sentence has some substantial commonality with a previously translated sentence. These fuzzy matches can be very worthwhile as a starting point for the human translator, but the translator then needs to manually edit the associated TM-based translation to accommodate the differences with the source sentence to be translated. If part of this process could be automated, the cost of human translation could be significantly reduced. The paper proposes to perform this automation in the following way: a phrase-based Statistical Machine Translation (SMT) system (trained on a bilingual corpus in the same domain as the TM) is combined with the TM fuzzy match, by extracting from the fuzzy-match a large (possibly gapped) bi-phrase that is dynamically added to the usual set of "static" bi-phrases used for decoding the source. We report experiments that show significant improvements in terms of BLEU and NIST scores over both the translations produced by the stand-alone SMT system and the fuzzy-match translations proposed by the stand-alone TM system.

Translating idioms is one of the most difficult tasks for human translators and translation machines alike. The main problems consist in recognizing an idiom and in distinguishing idiomatic from non-idiomatic usage. Recognition is difficult since many idioms can be modified and others can be discontinuously spread over a clause. But with the help of systematic idiom collections and special rules the recognition of an idiom candidate is always possible. The distinction between idiomatic and non-idiomatic usage is more problematic. Sometimes this can be done by means of special words that are only used in an idiom. But in general this distinction is a question of semantics and pragmatics and therefore beyond the abilities of current translation systems. In this paper we investigate the requirements for automatically recognizing idioms and we check whether idiom recognition is possible within current translation systems, i.e. machine translation and translation memory systems. This is of current interest since the developers of translation systems have started to include huge idiom collections in their products.

Translation memory (TM) systems are among the most aggressively marketed and widely used computer-aided translation tools. Previous studies have focused on when and how TMs are used, but there is significantly less information available relating to translators' perceptions of and attitudes towards them. The goal of this thesis is to explore translators' unprompted opinions of the issues related to TM system usage. After analyzing postings on translators' discussion boards, I propose to compare current assumptions about TM systems and their use with what translators are expressing in their unprompted opinions. I believe that with a better understanding of different perspectives and attitudes, translators can evaluate and potentially adjust their own perceptions in light of others' experience, developers and vendors can respond more accurately to users' needs, clients can better comprehend translators' concerns, and researchers and trainers can properly address the issues currently surrounding TM system usage. This thesis is organized into three chapters. Following a general introduction, Chapter 1 explains the functioning of TM systems and the issues surrounding their use, and then explores what is known about the use of TM systems and attitudes towards them as these are expressed in scholarly research, vendor promotional materials, surveys of practicing translators, and analyses of mailing lists. Chapter 2 provides a description of the methodology used in this project to select a primary resource, extract TM-related information, and classify the data. Chapter 3 presents a summary and analysis of the data found in the corpus. Finally, the conclusion summarizes the findings of this research and their implications for translators, vendors, clients/agencies, translator trainers, and researchers, addresses areas requiring further investigation and research, and evaluates the methodology of the project.

Machine translation (MT) systems have been built using numerous different techniques for bridging the language barriers. These techniques are broadly categorized into approaches like Statistical Machine Translation (SMT) and Neural Machine Translation (NMT). End-to-end NMT systems significantly outperform SMT in translation quality on many language pairs, especially those with the adequate parallel corpus. We report comparative experiments on baseline MT systems for Assamese to other Indo-Aryan languages (in both translation directions) using the traditional Phrase-Based SMT as well as some more successful NMT architectures, namely basic sequence-to-sequence model with attention, Transformer, and finetuned Transformer. The results are evaluated using the most prominent and popular standard automatic metric BLEU (BiLingual Evaluation Understudy), as well as other well-known metrics for exploring the performance of different baseline MT systems, since this is the first such work involving Assamese. The evaluation scores are compared for SMT and NMT models for the effectiveness of bi-directional language pairs involving Assamese and other Indo-Aryan languages (Bangla, Gujarati, Hindi, Marathi, Odia, Sinhalese, and Urdu). The highest BLEU scores obtained are for Assamese to Sinhalese for SMT (35.63) and the Assamese to Bangla for NMT systems (seq2seq is 50.92, Transformer is 50.01, and finetuned Transformer is 50.19). We also try to relate the results with the language characteristics, distances, family trees, domains, data sizes, and sentence lengths. We find that the effect of the domain is the most important factor affecting the results for the given data domains and sizes. We compare our results with the only existing MT system for Assamese (Bing Translator) and also with pairs involving Hindi.

Corpus-based machine translation (MT) has been the main approach to developing and implementing MT systems in both academia and the industry over the last three decades. In this field, the type and size of the corpus used for training MT engines have presented problems for both statistical MT (SMT) systems as well as neural MT (NMT) systems, being the two dominant corpusbased approaches. Moreover, language pairs such as Turkish-English have been understudied within this framework. This article aims to evaluate the translation quality in Turkish-to-English custom MT systems that have been trained on different corpus sizes and types. Two NMT engines and two SMT engines were trained on the KantanMT platform using two different training corpus types with either only domain-specific cardiology corpus or this corpus plus a mixed-domain corpus. The study conducted both automatic evaluations with metrics including BLEU, F-Measure and TER, as well as a comprehensive human evaluation with metrics including fluency, A/B test, and adequacy. Lastly, the study realized a separate, subjective terminology evaluation in order to investigate how differently MT systems handle terminology, as this is a crucial aspect for specific-domain text types such as cardiology. While the automatic evaluation results suggest the SMT engines to perform better than NMT engines, all human evaluators rated the mixed-domain NMT engine as the highest performing one. However, the terminology evaluation task demonstrated SMT to still be able to perform better and to commit less terminology errors, despite the industry and academia shifting toward NMT engines.

This paper presents a hybrid machine translation (HMT) system that pursues syntactic analysis to acquire phrases of source sentences, translates the phrases using multiple online machine translation (MT) system application program interfaces (APIs) and generates output by combining translated chunks to obtain the best possible translation. The aim of this study is to improve translation quality of English – Latvian texts over each of the individual MT APIs. The selection of the best translation hypothesis is done by calculating the perplexity for each hypothesis using an n-gram language model. The result is a phrase-based multi-system machine translation system that allows to improve MT output compared to individual online MT systems. The proposed approach show improvement up to +1.48 points in BLEU and −0.015 in TER scores compared to the baselines and related research.

We address the problem of computing a consensus translation given the outputs from a set of machine translation (MT) systems. The translations from the MT systems are aligned with a multiple string alignment algorithm and the consensus translation is then computed. We describe the multiple string alignment algorithm and the consensus MT hypothesis computation. We report on the subjective and objective performance of the multilingual acquisition approach on a limited domain spoken language application. We evaluate five domain-independent off-the-shelf MT systems and show that the consensus-based translation performance is equal to or better than any of the given MT systems, in terms of both objective and subjective measures.

Many studies have shown that the application of controlled languages (CL) is an effective pre-editing technique to improve machine translation (MT) output. In this paper, we investigate whether this also holds true for neural machine translation (NMT). We compare the impact of applying nine CL rules on the quality of NMT output as opposed to that of rule-based, statistical, and hybrid MT by applying three methods: error annotation, human evaluation, and automatic evaluation. The analyzed data is a German corpus-based test suite of technical texts that have been translated into English by five MT systems (a neural, a rule-based, a statistical, and two hybrid MT systems). The comparison is conducted in terms of several quantitative parameters (number of errors, error types, quality ratings, and automatic evaluation metrics scores). The results show that CL rules positively affect rule-based, statistical, and hybrid MT systems. However, CL does not improve the results of the NMT system. The output of the neural system is mostly error-free both before and after CL application and has the highest quality in both scenarios among the analyzed MT systems showing a decrease in quality after applying the CL rules. The qualitative discussion of the NMT output sheds light on the problems that CL causes for this kind of MT architecture.