Translation Error Rate Research Articles

DeTELpy: Python package for high-throughput detection of amino acid substitutions in mass spectrometry datasets.

Errors in the processing of genetic information during protein synthesis can lead to phenotypic mutations, such as amino acid substitutions, e.g. by transcription or translation errors. While genetic mutations can be readily identified using DNA sequencing, and mutations due to transcription errors by RNA sequencing, translation errors can only be identified proteome-wide using mass spectrometry. Here, we provide a Python package implementation of a high-throughput pipeline to detect amino acid substitutions in mass spectrometry datasets. Our tools enable users to process hundreds of mass spectrometry datasets in batch mode to detect amino acid substitutions and calculate codon-specific and site-specific translation error rates. deTELpy will facilitate the systematic understanding of amino acid misincorporation rates (translation error rates), and the inference of error models across organisms and under stress conditions, such as drug treatment or disease conditions. deTELpy is implemented in Python 3 and is freely available with detailed documentation and practical examples at https://git.mpi-cbg.de/tothpetroczylab/detelpy and https://pypi.org/project/deTELpy/ and can be easily installed via pip install deTELpy.

On compositional generalization of transformer-based neural machine translation

Neural networks have been shown to have deficiencies in the ability of compositional generalization while existing work has generally targeted semantic parsing tasks. In this paper, we study the challenge of machine translation and introduce a dedicated Chinese→English dataset, CoGniTion (Compositional Generalization Machine Translation). The training, validation, and test sets have 216k, 10k, and 10k sentence pairs, respectively, which are simple but contain a variety of phrases. More importantly, we construct 2160 novel compounds(phrases) unseen in the training data and each compound is embedded into 5 distinct contexts, forming the CG-test set with 10,800 samples. We evaluate the performance of Transformer models based on conventional metrics like BLEU and the proposed compound translation error rates, demonstrating their difficulty in handling compositional generalization. For the Transformer trained from scratch, 27.42% of the test sentences are translated incorrectly, and 62.88% of the compounds fail to be translated consistently. The pretrained language models (PLM) including the large language models (LLM) still exhibit approximately 30% inconsistency. Moreover, we conduct an empirical analysis of various factors that impact compound translation performance, and show that the performance of PLMs may be overestimated due to unavoidable composition exposure in pertaining data.

The use of residual analysis to improve the error rate accuracy of machine translation

The aim of the study is to compare two different approaches to machine translation—statistical and neural—using automatic MT metrics of error rate and residuals. We examined four available online MT systems (statistical Google Translate, neural Google Translate, and two European commission’s MT tools—statistical mt@ec and neural eTranslation) through their products (MT outputs). We propose using residual analysis to improve the accuracy of machine translation error rate. Residuals represent a new approach to comparing the quality of statistical and neural MT outputs. The study provides new insights into evaluating machine translation quality from English and German into Slovak through automatic error rate metrics. In the category of prediction and syntactic-semantic correlativeness, statistical MT showed a significantly higher error rate than neural MT. Conversely, in the category of lexical semantics, neural MT showed a significantly higher error rate than statistical MT. The results indicate that relying solely on the reference when determining MT quality is insufficient. However, when combined with residuals, it offers a more objective view of MT quality and facilitates the comparison of statistical MT and neural MT.

Analysis of Feature Extraction Model for Machine Korean Translation Judging System

Language translation is becoming increasingly widespread and diversified as internationalization and information technology advance significantly. Nonetheless, there are cultural variations across different nations, and there will unavoidably be some variations brought on by translation software. While the machine translation system has attained a high level of development, the issues with the older machine translation methods still remain and cannot be fully resolved. The efficiency of Korean-Chinese translation must thus be improved by creating a more effective model for converting translation information. This has significant implications for the development of international relationships, commercial interactions, cultural development, and commerce between China and South Korea. The translation is the rendering of words into other languages so that they may be understood by both individuals. Nevertheless, there are several flaws and inconsistencies in the present Korean-to-Chinese translational information processing method. Thus, this study suggests a Diverse Deep Embedding Data Accumulation (DDEDA) algorithm for the translation framework. We utilized the Korean-Chinese corpus dataset for the research. Utilizing translation accuracy, precision, recall, error rate, and translation time, the performance of the suggested approach was examined and compared with previously employed methodologies. The findings demonstrate that the proposed method has taken 43s for translation.

Investigating Models for the Transcription of Mathematical Formulas in Images

The automated transcription of mathematical formulas represents a complex challenge that is of great importance for digital processing and comprehensibility of mathematical content. Consequently, our goal was to analyze state-of-the-art approaches for the transcription of printed mathematical formulas on images into spoken English text. We focused on two approaches: (1) The combination of mathematical expression recognition (MER) models and natural language processing (NLP) models to convert formula images first into LaTeX code and then into text, and (2) the direct conversion of formula images into text using vision-language (VL) models. Since no dataset with printed mathematical formulas and corresponding English transcriptions existed, we created a new dataset, Formula2Text, for fine-tuning and evaluating our systems. Our best system for (1) combines the MER model LaTeX-OCR and the NLP model BART-Base, achieving a translation error rate of 36.14% compared with our reference transcriptions. In the task of converting LaTeX code to text, BART-Base, T5-Base, and FLAN-T5-Base even outperformed ChatGPT, GPT-3.5 Turbo, and GPT-4. For (2), the best VL model, TrOCR, achieves a translation error rate of 42.09%. This demonstrates that VL models, predominantly employed for classical image captioning tasks, possess significant potential for the transcription of mathematical formulas in images.

Zur Erforschung von Korrelationen zwischen verschiedenen Fehlertypen bei der maschinellen Übersetzung aus dem Deutschen ins Slowakische

Abstract We focus on examining the impact of machine translation (MT) error rate on adequacy and fluency in machine-translated journalistic texts. German is the source language, with significant polysynthetic features in the formation of composites, and the target language is Slovak, with predominantly inflected features. We analyse twelve error categories, which are incorporated into the categorical framework for the analysis of MT errors and correspond to the four-member core MQM-DQF error typology. The results show that the most significant errors are in the categories of lexical semantics, syntactic-semantic correlativeness and grammar.

Framework for Handling Rare Word Problems in Neural Machine Translation System Using Multi-Word Expressions

Machine Translation (MT) systems are now being improved with the use of an ongoing methodology known as Neural Machine Translation (NMT). Natural language processing (NLP) researchers have shown that NMT systems are unable to deal with out-of-vocabulary (OOV) words and multi-word expressions (MWEs) in the text. OOV terms are those that are not currently included in the vocabulary that is used by the NMT system. MWEs are phrases that consist of a minimum of two terms but are treated as a single unit. MWEs have great importance in NLP, linguistic theory, and MT systems. In this article, OOV words and MWEs are handled for the Punjabi to English NMT system. A parallel corpus for Punjabi to English containing MWEs was developed and used to train the different models of NMT. Punjabi is a low-resource language as it lacks the availability of a large parallel corpus for building various NLP tools, and this is an attempt to improve the accuracy of Punjabi in the English NMT system by using named entities and MWEs in the corpus. The developed NMT models were assessed using human evaluation through adequacy, fluency and overall rating as well as automated assessment tools such as the bilingual evaluation study (BLEU) and translation error rate (TER) score. Results show that using word embedding (WE) and MWEs corpus increased the accuracy of translation for the Punjabi to English language pair. The best BLEU score obtained was 15.45 for the small test set, 43.32 for the medium test set, and 34.5 for the large test set, respectively. The best TER rate score obtained was 57.34% for the small test set, 37.29% for the medium test set, and 53.79% for the large test set, repectively.

Error-prone protein synthesis recapitulates early symptoms of Alzheimer disease in aging mice.

Age-related neurodegenerative diseases (NDDs) are associated with the aggregation and propagation of specific pathogenic protein species (e.g., Aβ, α-synuclein). However, whether disruption of synaptic homeostasis results from protein misfolding per se rather than accumulation of a specific rogue protein is an unexplored question. Here, we show that error-prone translation, with its frequent outcome of random protein misfolding, is sufficient to recapitulate many early features of NDDs, including perturbed Ca2+ signaling, neuronal hyperexcitability, and mitochondrial dysfunction. Mice expressing the ribosomal ambiguity mutation Rps9 D95N exhibited disrupted synaptic homeostasis resulting in behavioral changes reminiscent of early Alzheimer disease (AD), such as learning and memory deficits, maladaptive emotional responses, epileptiform discharges, suppressed circadian rhythmicity, and sleep fragmentation, accompanied by hippocampal NPY expression and cerebral glucose hypometabolism. Collectively, our findings suggest that random protein misfolding may contribute to the pathogenesis of age-related NDDs, providing an alternative framework for understanding the initiation of AD.

A study of an intelligent algorithm combining semantic environments for the translation of complex English sentences

Abstract In order to improve the translation quality of complex English sentences, this paper investigated unknown words. First, two baseline models, the recurrent neural machine translation (RNMT) model and the transformer model, were briefly introduced. Then, the unknown words were identified and replaced based on WordNet and the semantic environment and input to the neural machine translation (NMT) model for translation. Finally, experiments were conducted on several National Institute of Standards and Technology (NIST) datasets. It was found that the transformer model significantly outperformed the RNMT model, its average bilingual evaluation understudy (BLEU) value was 42.14, which was 6.96 higher than the RNMT model, and its translation error rate (TER) was also smaller. After combining the intelligent algorithm, the BLEU values of both models improved, and the TER became smaller; the average BLEU value of the transformer model combined with the intelligent algorithm was 43.7, and the average TER was 57.68. The experiment verifies that the transformer model combined with the intelligent algorithm is reliable in translating complex sentences and can obtain higher-quality translation results.

기계번역의 특허 전문용어 한영번역에서 나타나는 오류 양상 분석 – 구글 번역 결과를 중심으로

Terminology translation plays a key role in technical machine translation. This paper conducts quantitative and qualitative evaluation and presents a comprehensive error analysis on terminology translation from Korean to English. For this study, 637 terms are selected from a patent domain corpus. Among them, 425 terms are translated correctly while other 212 terms have one or more errors. Those identified errors are classified into seven error categories which include addition, omission, lexical errors, order errors, morphological errors, partial errors, and miscellanies. Among them, partial errors and morphological errors are the two top error categories, and lexical errors and omission cases are also high. The study also presents n-gram wise error rates. The calculation of n-gram wise terminology translation error rates shows that the curve for the Google Translation results goes upwards with the increasing size of n. This shows that neural machine translation can be vulnerable in translating a higher-order of n-gram terms. This study has implications for Korean-English machine translation research, in that it classifies error cases spotted in machine translation of terminology which is critical in securing accuracy of translations.

한국어 구어 문장 파파고 한-영 번역 오류 유형 고찰

This research seeks to evaluate the accuracy and usefulness of Papago Translator based on translation error rates, patterns, and frequency when used on spoken Korean utterances. The study analyzed a total of 777 spoken Korean sentences that were provided by 32 undergraduate students and seven graduate students majoring in English education, who were enrolled in the spring semester of 2021. We found that 434 (55.9%) sentences were translated correctly, and 343 (44.1%) were translated incorrectly. There were four categories of errors in general. The most frequent were inaccurate semantic errors, 286 (83.4%). This was followed by grammatical errors, 36 (10.5%), and omission errors, 19 (5.5%). Furthermore, there were 343 more detailed errors identified with 14 subcategories. Nighty-seven errors were due to differences in meaning between Korean and English, 57 involved Korean idioms that could not be pragmatically translated, and 38 were as a result of double meaning. Our findings suggest that it is necessary to carry out post-editing of machine translation results to ensure successful communication and proper use of translated English expressions.

Silencing of the ER and Integrative Stress Responses in the Liver of Mice with Error-Prone Translation.

Translational errors frequently arise during protein synthesis, producing misfolded and dysfunctional proteins. Chronic stress resulting from translation errors may be particularly relevant in tissues that must synthesize and secrete large amounts of secretory proteins. Here, we studied the proteostasis networks in the liver of mice that express the Rps2-A226Y ribosomal ambiguity (ram) mutation to increase the translation error rate across all proteins. We found that Rps2-A226Y mice lack activation of the eIF2 kinase/ATF4 pathway, the main component of the integrated stress response (ISR), as well as the IRE1 and ATF6 pathways of the ER unfolded protein response (ER-UPR). Instead, we found downregulation of chronic ER stress responses, as indicated by reduced gene expression for lipogenic pathways and acute phase proteins, possibly via upregulation of Sirtuin-1. In parallel, we observed activation of alternative proteostasis responses, including the proteasome and the formation of stress granules. Together, our results point to a concerted response to error-prone translation to alleviate ER stress in favor of activating alternative proteostasis mechanisms, most likely to avoid cell damage and apoptotic pathways, which would result from persistent activation of the ER and integrated stress responses.

Nucleolar TFIIE plays a role in ribosomal biogenesis and performance.

Ribosome biogenesis is a highly energy-demanding process in eukaryotes which requires the concerted action of all three RNA polymerases. In RNA polymerase II transcription, the general transcription factor TFIIH is recruited by TFIIE to the initiation site of protein-coding genes. Distinct mutations in TFIIH and TFIIE give rise to the degenerative disorder trichothiodystrophy (TTD). Here, we uncovered an unexpected role of TFIIE in ribosomal RNA synthesis by RNA polymerase I. With high resolution microscopy we detected TFIIE in the nucleolus where TFIIE binds to actively transcribed rDNA. Mutations in TFIIE affects gene-occupancy of RNA polymerase I, rRNA maturation, ribosomal assembly and performance. In consequence, the elevated translational error rate with imbalanced protein synthesis and turnover results in an increase in heat-sensitive proteins. Collectively, mutations in TFIIE—due to impaired ribosomal biogenesis and translational accuracy—lead to a loss of protein homeostasis (proteostasis) which can partly explain the clinical phenotype in TTD.

Random errors in protein synthesis activate an age-dependent program of muscle atrophy in mice

Random errors in protein synthesis are prevalent and ubiquitous, yet their effect on organismal health has remained enigmatic for over five decades. Here, we studied whether mice carrying the ribosomal ambiguity (ram) mutation Rps2-A226Y, recently shown to increase the inborn error rate of mammalian translation, if at all viable, present any specific, possibly aging-related, phenotype. We introduced Rps2-A226Y using a Cre/loxP strategy. Resulting transgenic mice were mosaic and showed a muscle-related phenotype with reduced grip strength. Analysis of gene expression in skeletal muscle using RNA-Seq revealed transcriptomic changes occurring in an age-dependent manner, involving an interplay of PGC1α, FOXO3, mTOR, and glucocorticoids as key signaling pathways, and finally resulting in activation of a muscle atrophy program. Our results highlight the relevance of translation accuracy, and show how disturbances thereof may contribute to age-related pathologies.

UPC: An Open Word-Sense Annotated Parallel Corpora for Machine Translation Study

Machine translation (MT) has recently attracted much research on various advanced techniques (i.e., statistical-based and deep learning-based) and achieved great results for popular languages. However, the research on it involving low-resource languages such as Korean often suffer from the lack of openly available bilingual language resources. In this research, we built the open extensive parallel corpora for training MT models, named Ulsan parallel corpora (UPC). Currently, UPC contains two parallel corpora consisting of Korean-English and Korean-Vietnamese datasets. The Korean-English dataset has over 969 thousand sentence pairs, and the Korean-Vietnamese parallel corpus consists of over 412 thousand sentence pairs. Furthermore, the high rate of homographs of Korean causes an ambiguous word issue in MT. To address this problem, we developed a powerful word-sense annotation system based on a combination of sub-word conditional probability and knowledge-based methods, named UTagger. We applied UTagger to UPC and used these corpora to train both statistical-based and deep learning-based neural MT systems. The experimental results demonstrated that using UPC, high-quality MT systems (in terms of the Bi-Lingual Evaluation Understudy (BLEU) and Translation Error Rate (TER) score) can be built. Both UPC and UTagger are available for free download and usage.

Development of English-Hindi Interactive Machine Translation

Machine Translation systems are still far from being perfect and to improve their performance the concept of Interactive Machine Translation (IMT) was introduced. This paper proposes an IMT system, which uses Statistical Machine Translation and a bilingual corpus on which several algorithms (Word error rate, Position Independent Error Rate, Translation Error Rate, n-grams) are implemented to translate text from English to Indian languages. The proposed system improves both the speed and productivity of the human translators as found through experiments.

Low-Resource Machine Transliteration Using Recurrent Neural Networks

Grapheme-to-phoneme models are key components in automatic speech recognition and text-to-speech systems. With low-resource language pairs that do not have available and well-developed pronunciation lexicons, grapheme-to-phoneme models are particularly useful. These models are based on initial alignments between grapheme source and phoneme target sequences. Inspired by sequence-to-sequence recurrent neural network--based translation methods, the current research presents an approach that applies an alignment representation for input sequences and pretrained source and target embeddings to overcome the transliteration problem for a low-resource languages pair. Evaluation and experiments involving French and Vietnamese showed that with only a small bilingual pronunciation dictionary available for training the transliteration models, promising results were obtained with a large increase in BLEU scores and a reduction in Translation Error Rate (TER) and Phoneme Error Rate (PER). Moreover, we compared our proposed neural network--based transliteration approach with a statistical one.

Novel Combinations of Agents Targeting Translation That Synergistically Inhibit Fungal Pathogens.

A range of fungicides or antifungals are currently deployed to control fungi in agriculture or medicine, but resistance to current agents is growing so new approaches and molecular targets are urgently needed. Recently, different aminoglycoside antibiotics combined with particular transport inhibitors were found to produce strong, synergistic growth-inhibition of fungi, by synergistically increasing the error rate of mRNA translation. Here, focusing on translation fidelity as a novel target for combinatorial antifungal treatment, we tested the hypothesis that alternative combinations of agents known to affect the availability of functional amino acids would synergistically inhibit growth of major fungal pathogens. We screened 172 novel combinations against three phytopathogens (Rhizoctonia solani, Zymoseptoria tritici, and Botrytis cinerea) and three human pathogens (Cryptococcus neoformans, Candida albicans, and Aspergillus fumigatus), showing that 48 combinations inhibited strongly the growth of the pathogens; the growth inhibition effect was significantly greater with the agents combined than by a simple product of their individual effects at the same doses. Of these, 23 combinations were effective against more than one pathogen, including combinations comprising food-and-drug approved compounds, e.g., quinine with bicarbonate, and quinine with hygromycin. These combinations [fractional inhibitory combination (FIC) index ≤0.5] gave up to 100% reduction of fungal growth yield at concentrations of agents which, individually, had negligible effect. No synergy was evident against bacterial, plant or mammalian cells, indicating specificity for fungi. Mode-of-action analyses for quinine + hygromycin indicated that synergistic mistranslation was the antifungal mechanism. That mechanism was not universal as bicarbonate exacerbated quinine action by increasing drug uptake. The study unveils chemical combinations and a target process with potential for control of diverse fungal pathogens, and suggests repurposing possibilities for several current therapeutics.

Interdependence, Reflexivity, Fidelity, Impedance Matching, and the Evolution of Genetic Coding.

Genetic coding is generally thought to have required ribozymes whose functions were taken over by polypeptide aminoacyl-tRNA synthetases (aaRS). Two discoveries about aaRS and their interactions with tRNA substrates now furnish a unifying rationale for the opposite conclusion: that the key processes of the Central Dogma of molecular biology emerged simultaneously and naturally from simple origins in a peptide•RNA partnership, eliminating the epistemological utility of a prior RNA world. First, the two aaRS classes likely arose from opposite strands of the same ancestral gene, implying a simple genetic alphabet. The resulting inversion symmetries in aaRS structural biology would have stabilized the initial and subsequent differentiation of coding specificities, rapidly promoting diversity in the proteome. Second, amino acid physical chemistry maps onto tRNA identity elements, establishing reflexive, nanoenvironmental sensing in protein aaRS. Bootstrapping of increasingly detailed coding is thus intrinsic to polypeptide aaRS, but impossible in an RNA world. These notions underline the following concepts that contradict gradual replacement of ribozymal aaRS by polypeptide aaRS: 1) aaRS enzymes must be interdependent; 2) reflexivity intrinsic to polypeptide aaRS production dynamics promotes bootstrapping; 3) takeover of RNA-catalyzed aminoacylation by enzymes will necessarily degrade specificity; and 4) the Central Dogma’s emergence is most probable when replication and translation error rates remain comparable. These characteristics are necessary and sufficient for the essentially de novo emergence of a coupled gene–replicase–translatase system of genetic coding that would have continuously preserved the functional meaning of genetically encoded protein genes whose phylogenetic relationships match those observed today.

Ambient intelligence framework for real-time speech-to-sign translation

ABSTRACTSign language can be used to facilitate communication with and between deaf or hard of hearing (Deaf/HH). With the advent of video streaming applications in smart TVs and mobile devices, it is now possible to use sign language to communicate over worldwide networks. In this article, we develop a prototype assistive device for real-time speech-to-sign translation. The proposed device aims at enabling Deaf/HH people to access and understand materials delivered in mobile streaming videos through the applications of pipelined and parallel processing for real-time translation, and the application of eye-tracking based user-satisfaction detection to support dynamic learning to improve speech-to-signing translation. We conduct two experiments to evaluate the performance and usability of the proposed assistive device. Nine deaf people participated in these experiments. Our real-time performance evaluation shows the addition of viewer’s attention-based feedback reduced translation error rates by 16% (per the sign error rate [SER] metric) and increased translation accuracy by 5.4% (per the bilingual evaluation understudy [BLEU] metric) when compared to a non-real-time baseline system without these features. The usability study results indicate that our assistive device was also pleasant and satisfying to deaf users, and it may contribute to greater engagement of deaf people in day-to-day activities.