Simultaneous Machine Translation Research Articles

Dap-SiMT: divergence-based adaptive policy for simultaneous machine translation

Dap-SiMT: divergence-based adaptive policy for simultaneous machine translation

Prefix Alignment for Training Simultaneous Machine Translation

Prefix Alignment for Training Simultaneous Machine Translation

Multi-modal simultaneous machine translation fusion of image information

Simultaneous translation is to translate a sentence before people finish it, to understand the speaker's intention in real-time. At present, simultaneous machine translation still relies on text-to-text data resources. However, the output information from the encoder side is used for the decoder as the input data recourse in the pure text translation system. This information is only derived from the text content, and the input information is single, causing a shortage of decoding information at the decoder and the vocabulary is missed in translation. The translator will also visually capture the information of the surrounding scenes to assist himself in the translation work, based on this feature, we propose a multi-modal simultaneous machine translation of fusion image information. We extract information from the image, add the information to the decoder side of the translation system, increase the input data resource of the decoder, and help the system improve the translation quality. We use the Multi30K dataset for experimental verification. Compared with the translation system of plain text, the method we propose can translate more complete sentences, richer content, and better translation results.

Supervised Visual Attention for Simultaneous Multimodal Machine Translation

There has been a surge in research in multimodal machine translation (MMT), where additional modalities such as images are used to improve translation quality of textual systems. A particular use for such multimodal systems is the task of simultaneous machine translation, where visual context has been shown to complement the partial information provided by the source sentence, especially in the early phases of translation. In this paper, we propose the first Transformer-based simultaneous MMT architecture, which has not been previously explored in simultaneous translation. Additionally, we extend this model with an auxiliary supervision signal that guides the visual attention mechanism using labelled phrase-region alignments. We perform comprehensive experiments on three language directions and conduct thorough quantitative and qualitative analyses using both automatic metrics and manual inspection. Our results show that (i) supervised visual attention consistently improves the translation quality of the simultaneous MMT models, and (ii) fine-tuning the MMT with supervision loss enabled leads to better performance than training the MMT from scratch. Compared to the state-of-the-art, our proposed model achieves improvements of up to 2.3 BLEU and 3.5 METEOR points.

Streaming cascade-based speech translation leveraged by a direct segmentation model

The cascade approach to Speech Translation (ST) is based on a pipeline that concatenates an Automatic Speech Recognition (ASR) system followed by a Machine Translation (MT) system. Nowadays, state-of-the-art ST systems are populated with deep neural networks that are conceived to work in an offline setup in which the audio input to be translated is fully available in advance. However, a streaming setup defines a completely different picture, in which an unbounded audio input gradually becomes available and at the same time the translation needs to be generated under real-time constraints. In this work, we present a state-of-the-art streaming ST system in which neural-based models integrated in the ASR and MT components are carefully adapted in terms of their training and decoding procedures in order to run under a streaming setup. In addition, a direct segmentation model that adapts the continuous ASR output to the capacity of simultaneous MT systems trained at the sentence level is introduced to guarantee low latency while preserving the translation quality of the complete ST system. The resulting ST system is thoroughly evaluated on the real-life streaming Europarl-ST benchmark to gauge the trade-off between quality and latency for each component individually as well as for the complete ST system.

Digital economy: communication using neural networks in international communication

The article considers theoretical and practical issues of development and implementation, as well as potential development of technological solutions for voice communication of people speaking foreign languages using simultaneous machine translation, as well as potential directions of these programs for international conferences, seminars, business meetings, education in foreign universities. The article deals with the concrete experience of implementing a scientific and technological project on software development and application development carried out within the framework of the innovation project of the Russian Foundation for Innovations Development. The potential of development of this type of technologies and products in Russia in conditions of digital economy creation is also analyzed.

Technological disruption in foreign language teaching: The rise of simultaneous machine translation

The fear of technology replacing jobs can be traced back to Aristotle, who, before great technological advances existed, ventured that machines may one day end the need for human labor (Campa 2014). In the current era, there is overwhelming evidence of technological unemployment. This evidence comes in the form of jobs that were once common, but have largely been replaced by technology such as switchboard operators, travel agents, booth cashiers, bank tellers, and typists. These jobs still exist, but their numbers have declined sharply because they were easily replaced by technology. Statistical models indicate future job losses in these areas will continue with booth cashiers at an 84% risk of losing their jobs, travel agents at a 10% risk, and typists at an 81% risk (Frey & Osborne 2013). These, generally, entry level positions do not require specialized training or advanced degrees, which may explain some of the job losses. However, current trends indicate that training and advanced degrees do not necessarily offer protection against technological unemployment, with most analysts predicting that technology will soon replace lawyers (Markoff 2011), pharmacy technicians, and accountants (Frey & Osborne 2013). Unemployment in career sectors such as these will have adverse effects not only on the workers, but also on the systems that support them. When the need for lawyers, pharmacists, and accountants collapses, what will happen to law schools, colleges of pharmacy, and accounting departments that train specialists in these fields? What will happen to the support systems that depend on these jobs or the scholars that move these fields forward through research activities?