Spoken Language Translation System Research Articles

Creating a ground truth multilingual dataset of news and talk show transcriptions through crowdsourcing

This paper describes the development of a multilingual and multigenre manually annotated speech dataset, freely available to the research community as ground truth for the evaluation of automatic transcription systems and spoken language translation systems. The dataset includes two video genres--television broadcast news and talk-shows--and covers Flemish, English, German, and Italian, for a total of about 35 h of television speech. Besides segmentation and orthographic transcription, we added a very rich annotation on the audio signal, both at the linguistic level (e.g. filled pauses, pronunciation errors, disfluencies, speech in a foreign language) and at the acoustic level (e.g. background noise and different types of non-speech events). Furthermore, a subset of the transcriptions is translated in four directions, namely Flemish to English, German to English, German to Italian and English to Italian. The development of this dataset was organized in several phases, relying on expert transcribers as well as involving non-expert contributors through crowdsourcing. We first conducted a feasibility study to test and compare two methods for crowdsourcing speech transcription on broadcast news data. These methods are based on different transcription processes (i.e. parallel vs. iterative) and incorporate two different quality control mechanisms. With both methods, we achieved near-expert transcription quality--in terms of word error rate--for English, German and Italian data. Instead, for Flemish data we were not able to get a sufficient response from the crowd to complete the offered transcription tasks. The results obtained demonstrate that the viability of methods for crowdsourcing speech transcription significantly depends on the target language. This paper provides a detailed comparison of the results obtained with the two crowdsourcing methods tested, describes the main characteristics of the final ground truth resource created as well as the methodology adopted, and the guidelines prepared for its development.

Vollautomatisches Dolmetschen – Möglichkeiten und Grenzen

This article describes how technologies needed to fully automate inter- pretation work. It assesses the performance of existing technologies and compa- res them to the work of human conference interpreters. The article goes on to evaluate the problems to be encountered at every step of a spoken language translation system: speech recognition, machine translation and speech synthe- sis. The article closes with an overview of existing products and tries to define areas where current technology can improve interlingual communication. This article argues that a 'universal translator' as depicted in science fiction will be hard to implement with statistical machine translation at its core

Lightly supervised word-sense translation-error detection and resolution in an interactive conversational spoken language translation system

Lexical ambiguity can cause critical failure in conversational spoken language translation (CSLT) systems that rely on statistical machine translation (SMT) if the wrong sense is presented in the target language. Interactive CSLT systems offer the capability to detect and pre-empt such word-sense translation errors (WSTEs) by engaging the human operators in a precise clarification dialogue aimed at resolving the problem. This paper presents an end-to-end framework for accurate detection and interactive resolution of WSTEs to minimize communication errors due to ambiguous source words. We propose (a) a novel, extensible, two-level classification architecture for identifying potential WSTEs in SMT hypotheses; (b) a constrained phrase-pair clustering mechanism for identifying the translated sense of ambiguous source words in SMT hypotheses; and (c) an interactive strategy that integrates this information to request specific clarifying information from the operator. By leveraging unsupervised and lightly supervised learning techniques, our approach minimizes the need for expensive human annotation in developing each component of this framework. Each component, as well as the overall framework, was evaluated in the context of an interactive English-to-Iraqi Arabic CSLT system.

An overview of the development of resources, techniques, and, systems for Indian spoken languages

India possesses a large variety of languages and dialects spoken in different parts of the country. These languages possess some unique linguistic, phonological, and phonetic properties different from European languages. Research is being done in several of Indian languages—such as Hindi, Bangla, etc. to study the articulatory, acoustic—phonetic and prosodic nature for the purpose of creating standards of phonetic representation of phonemes and Pronunciation Lexicon in Indian Languages. Comprehensive and task specific language corpora, speech databases in laboratory as well as in mobile communication situation and the tools/technics required for processing of speech signals are being developed. The emphasis is on developing multilingual human-machine interaction systems. Some of the recently developed systems include multi-lingual speech recognition system for voice enabled services, multilingual text to speech synthesis system, speaker and language identification system for general purpose and forensic applications. Recognition of emotions in spoken speech, spoken language translation system, etc. The paper presents an overview of such studies conducted in various laboratories, academic institutions, and industries in India pertaining to these areas. The technologies used for data collection, processing, and recognition/ synthesis, etc., utilized and status of the development have been mentioned.

A-STAR: Toward translating Asian spoken languages

This paper outlines the first Asian network-based speech-to-speech translation system developed by the Asian Speech Translation Advanced Research (A-STAR) consortium. Eight research groups comprising the A-STAR members participated in the experiments, covering nine languages, i.e., eight Asian languages (Hindi, Indonesian, Japanese, Korean, Malay, Thai, Vietnamese, and Chinese) and English. Each A-STAR member contributed one or more of the following spoken language technologies: automatic speech recognition, machine translation, and text-to-speech through Web servers. The system was designed to translate common spoken utterances of travel conversations from a given source language into multiple target languages in order to facilitate multiparty travel conversations between people speaking different Asian languages. It covers travel expressions including proper nouns that are names of famous places or attractions in Asian countries. In this paper, we describe the issues of developing spoken language technologies for Asian languages, and discuss the difficulties involved in connecting different heterogeneous spoken language translation systems through Web servers. This paper also presents speech-translation results including subjective evaluation, from the first A-STAR field testing which was carried out in July 2009.

An ARM-Based System-on-a-Programmable-Chip Architecture for Spoken Language Translation

Previous research shows that there are two architectures for spoken language translation (SLT) system implementation. One is client-server based systems that should be built on the server computer but unreliability of the remote connection. The other is to build portable stand-alone devices but it lacks realtime performance. In this brief, a system-on-a-programmable-chip (SoPC) solution is proposed by realizing the entire SLT system within a single chip. This SoPC is characterized by small size, low cost, real-time operation, and high portability. This entire design was implemented on ALTERA EPXA10 device. Performance for English-to-Mandarin translation process can be completed within 1 s at a 46.22-MHz clock frequency with 3000 translation patterns. The total logic usage of the EPXA10 device is 50% (about 19318 logic cells).

Paraphrasing spoken Chinese using a paraphrase corpus

One of the key issues in spoken-language translation is how to deal with unrestricted expressions in spontaneous utterances. We have developed a paraphraser for use as part of a translation system, and in this paper we describe the implementation of a Chinese paraphraser for a Chinese-Japanese spoken-language translation system. When an input sentence cannot be translated by the transfer engine, the paraphraser automatically transforms the sentence into alternative expressions until one of these alternatives can be translated by the transfer engine. Two primary issues must be dealt with in paraphrasing: how to determine new expressions, and how to retain the meaning of the input sentence. We use a pattern-based approach in which the meaning is retained to the greatest possible extent without deep parsing. The paraphrase patterns are acquired from a paraphrase corpus and human experience. The paraphrase instances are automatically extracted and then generalized into paraphrase patterns. A total of 1719 paraphrase patterns obtained using this method and an implemented paraphraser were used in a paraphrasing experiment. The results showed that the implemented paraphraser generated 1.7 paraphrases on average for each test sentence and achieved an accuracy of 88%.

Toward Practical Spoken Language Translation

This paper argues that the time is now right to field practical Spoken Language Translation (SLT) systems. Several sorts of practical systems can be built over the next few years if system builders recognize that, at the present state of the art, users must cooperate and compromise with the programs. Further, SLT systems can be arranged on a scale, in terms of the degree of cooperation or compromise they require from users. In general, the broader the intended linguistic or topical coverage of a system, the more user cooperation or compromise it will presently require. The paper briefly discusses the component technologies of SLT systems as they relate to user cooperation and accommodation ("human factors engineering"), with examples from the authors' work. It describes three classes of "cooperative" SLT systems which could be put into practical use during the next few years.

Analyzing inflectional morphology in a spoken language translation system

At least one speech input is received and at least one token is generated from speech input. Morphemes of the tokens are reduced to at least one feature. Furthermore, an inflection type of the token is identified. At least one dictionary is searched for entries comprising features that match the features reduced from the morphemes. At least one lexical feature structure is generated for the token by inserting at least one morphological feature associated with the inflection type into the entry feature. An output is provided comprising at least one lexical feature structure.

Detection of sentence accents in a speech recognition system.

Speech recognition systems do not usually utilize prosodic information, i.e., information signaled by segmental duration and the fundamental frequency contour of the speech signal. The acoustic manifestation of prosody is, more often than not, considered as a disturbance in current statistical approaches to the speech recognition problem. The detection and transformation of sentence accent in, e.g., spoken language translation systems, will enable stress on a certain word in one language to be transformed into a suitable representation of corresponding constituents in the other language and satisfy the same semantic goal. In this study, a system for automatic detection of sentence accents to be used in speech recognition systems, is presented. The fundamental frequency is extracted from the speech signal and an estimated frequency declination is subtracted from the actual fundamental frequency in order to give a normalized representation of the variations. These fundamental frequency variations are given in musical intervals. The interpretations of sentence accents are carried out from this normalized manifestation of the fundamental frequency. Both the system architecture and some preliminary results will be shown.

Opportunities for advanced speech processing in military computer-based systems

The author presents a study of military applications of advanced speech processing technology which includes three major elements: (1) review and assessment of current efforts in military applications of speech technology. (2) identification of opportunities for future military applications of advanced speech technology, and (3) identification of problem areas where research in speech processing is needed to meet application requirements, and of current research thrusts which appear promising. Opportunities for advanced applications are identified by means of descriptions of several generic systems which would be possible with advances in speech technology and in system integration. These generic systems include an integrated multirate voice/data communications terminal, an interactive speech enhancement system, a voice-controlled pilot's associate system, advanced air traffic control training systems, a battle management command and control support system with spoken natural language interface, and a spoken language translation system. In the applications discussed, various spectral analysis and digitization techniques are teamed up with artificial intelligence schemes to allow a machine to recognize continuous-speech sentences with vocabularies running to thousands of words. >