Speech Processing Technology Research Articles

Music Perception in Children With Cochlear Implants

AbstractAs a result of the widespread use of cochlear implants, individuals with profound hearing loss now are able to hear sounds ranging from a syllable to a symphony. This form of “electric hearing” has been remarkably successful in providing sound to the deaf population and at least 100,000 implantation procedures have been performed worldwide in more than 80 countries (Clark, 2008). Today, it is routine for post-lingual deafened individuals (one who lost their hearing after normal childhood language acquisition) to achieve high performance on language tests following implantation (Lalwani, Larky, Wareing, Kwast, & Schindler, 1998). Deaf children implanted at an early age with a CI usually develop excellent spoken language skills, with placement into mainstream educational schooling (Francis, Koch, Wyatt, & Niparko, 1999). The overwhelming emphasis on language perception in CI users has led to relative neglect of non-linguistic sound perception. Yet, the auditory world consists of many other sounds besides those of spoken language. Of all non-linguistic sounds, perception of music—particularly pitch and timbre—represents the greatest challenge for implant-mediated listening (Limb, 2006). High-level perception of music rarely is attained through conventional speech processing technology in adults or children. Recent technological advances, however, have increased the processing capabilities of modern CIs and hold great promise for music perception and quality of life for children with cochlear implants (Lassaletta et al., 2007).

Analytical Study on Fundamental Frequency Contours of Thai Expressive Speech Using Fujisaki's Model

Problem statement: In spontaneous speech communication, prosody is an important factor that must be taken into account, since the prosody effects on not only the naturalness but also the intelligibility of speech. Focusing on synthesis of Thai expressive speech, a number of systems has been developed for years. However, the expressive speech with various speaking styles has not been accomplished. To achieve the generation of expressive speech, we need to model the fundamental frequency (F0) contours accurately to preserve the speech prosody. Approach: Therefore this study proposes an analysis of model parameters for Thai speech prosody with three speaking styles and two genders which is a preliminary work for speech synthesis. Fujisaki's modeling; a powerful tool to model the F0 contour has been adopted, while the speaking styles of happiness, sadness and reading have been considered. Seven derived parameters from the Fujisaki's model are as follows. The first parameter is baseline frequency which is the lowest level of F0 contour. The second and third parameters are the numbers of phrase commands and tone commands which reflect the frequencies of surges of the utterance in global and local levels, respectively. The fourth and fifth parameters are phrase command and tone command durations which reflect the speed of speaking and the length of a syllable, respectively. The sixth and seventh parameters are amplitudes of phrase command and tone command which reflect the energy of the global speech and the energy of local syllable. Results: In the experiments, each speaking style includes 200 samples of one sentence with male and female speech. Therefore our speech database contains 1200 utterances in total. The results show that most of the proposed parameters can distinguish three kinds of speaking styles explicitly. Conclusion: From the finding, it is a strong evidence to further apply the successful parameters in the speech synthesis systems or other speech processing technologies.

The Enduring Spoken Word

In his Perspective (“Unlocking the potential of the spoken word,” 26 September 2008, p. [1787][1]), D. W. Oard describes how writing caused a landslide in humanity's cultural landscape, in large part because it was a findable, permanent record. He suggests that today's speech recognition and recording technologies may mean that the comeback of the spoken word is upon us. However, Oard's argument suggests a radical turn where there is none, for the simple reason that speech has never left our side. The invention of writing allowed information to be stored reliably in a medium other than human memory. Speech processing technology is just a variation on that theme. Oard highlights the potential of the spoken word for information retrieval purposes. In no sense does this bring us to “the threshold of a new era”; it merely dusts off the worn-out view of the spoken word as mere vehicle for transporting ideas ([1][2]). The full potential of the spoken word has always been more complex than the words themselves; the speaker and listener jointly construct meaning guided by common ground, social relationships, gestures, body language, and facial expressions as much as by the auditory signal ([2][3]–[5][4]). Discourse over the past 50,000 years has encompassed a lot more than neat ready to be data-mined. Unlocking its full potential requires a richer and more dynamic view of language than that espoused by Oard. 1. 1.[↵][5] 1. A. Ortony 1. M. J. Reddy , in Metaphor and Thought, A. Ortony, Ed. (Cambridge Univ. Press, Cambridge, 1979), pp. 284-297. 2. 2.[↵][6] 1. D. Tedlock, 2. B. Mannheim , The Dialogic Emergence of Culture (Univ. of Illinois Press, Champaign, IL, 1995). 3. 3. 1. D. Tannen , Talking Voices: Repetition, Dialogue, and Imagery in Conversational Discourse, ed. 2 (Cambridge Univ. Press, Cambridge, 2007). 4. 4. 1. H. H. Clark , Using Language (Cambridge Univ. Press, Cambridge, 1996). 5. 5.[↵][7] 1. N. J. Enfield, 2. S. C. Levinson , Roots of Human Sociality: Culture, Cognition, and Human Interaction (Berg, Oxford, 2006). # Response {#article-title-2} Rather than arguing that speech would overcome writing in another radical cultural shift, my intent was to suggest that speech would reemerge to stand side by side with writing as a conduit for transporting ideas with permanence and findability. As M. Dingemanse observes, speech can be so much more than a mere conduit, and I would agree that we are far from being able to build machines that can reasonably model the full richness of human expression, whether spoken or written. Many of our most widely used machines for processing language (such as search engines and translation systems) rely on fairly shallow representations of meaning, and predicting fundamental changes in that situation would seem to me highly speculative. Machines are merely tools, however—it is we, not our machines, who must ultimately make sense of what we see, hear, and read. But we should not underestimate the importance of having machines that can help us to find what we need. Dingemanse's critique reminds us that change and continuity coexist, and that although permanence and findability can help us to use speech in new ways, many of the ways speech presently pervades our lives will surely also remain with us. [1]: /lookup/doi/10.1126/science.1157353 [2]: #ref-1 [3]: #ref-2 [4]: #ref-5 [5]: #xref-ref-1-1 View reference 1. in text [6]: #xref-ref-2-1 View reference 2. in text [7]: #xref-ref-5-1 View reference 5. in text

Audiovisual automatic speech recognition: Progress and challenges

The paper overviews recent progress and challenges in a number of audiovisual speech processing technologies with main emphasis on the problem of automatic speech recognition. It is well known that visual channel information can improve automatic speech processing for human-computer interaction. To automatically process and incorporate such information into automatic systems, a number of steps are required that are surprisingly similar accross speech technologies. Crucial above all is the issue of feature representation of visual speech and its robust extraction. In addition, appropriate integration of the audio and visual representations is required, in order to ensure improved performance of the bimodal systems over audio-only baselines. These topics are discussed in detail in the talk, with main emphasis on their application to the speech recognition problem in the challenging environments of automobiles and smart rooms.

Thai speech processing technology: A review

This paper presents a comprehensive review of Thai speech technology, from its impetus in the early 1960s to 2005. Thai is the official language of Thailand, and is spoken by over 60 million people worldwide. As with Chinese, it is a tonal language. It has a spelling system using a Thai alphabet, but has no explicit word boundaries, similar to several Asian languages, such as Japanese and Chinese. It does have explicit marks for tones, as in the languages of the neighboring countries, Laos and Vietnam. Therefore, with these unique characteristics, research and development of language and speech processing specifically for Thai is necessary and quite challenging. This paper reviews the progress of Thai speech technology in five areas of research: fundamental analyses and tools, text-to-speech synthesis (TTS), automatic speech recognition (ASR), speech applications, and language resources. At the end of the paper, the progress and focus of Thai speech research, as measured by the number of publications in each research area, is reviewed and possible directions for future research are suggested.

Computer-assisted translation using speech recognition

Current machine translation systems are far from being perfect. However, such systems can be used in computer-assisted translation to increase the productivity of the (human) translation process. The idea is to use a text-to-text translation system to produce portions of target language text that can be accepted or amended by a human translator using text or speech. These user-validated portions are then used by the text-to-text translation system to produce further, hopefully improved suggestions. There are different alternatives of using speech in a computer-assisted translation system: From pure dictated translation to simple determination of acceptable partial translations by reading parts of the suggestions made by the system. In all the cases, information from the text to be translated can be used to constrain the speech decoding search space. While pure dictation seems to be among the most attractive settings, unfortunately perfect speech decoding does not seem possible with the current speech processing technology and human error-correcting would still be required. Therefore, approaches that allow for higher speech recognition accuracy by using increasingly constrained models in the speech recognition process are explored here. All these approaches are presented under the statistical framework. Empirical results support the potential usefulness of using speech within the computer-assisted translation paradigm.

A probabilistic framework for dialog simulation and optimal strategy learning

The design of Spoken Dialog Systems cannot be considered as the simple combination of speech processing technologies. Indeed, speech-based interface design has been an expert job for a long time. It necessitates good skills in speech technologies and low-level programming. Moreover, rapid development and reusability of previously designed systems remains uneasy. This makes optimality and objective evaluation of design very difficult. The design process is therefore a cyclic process composed of prototype releases, user satisfaction surveys, bug reports and refinements. It is well known that human intervention for testing is time-consuming and above all very expensive. This is one of the reasons for the recent interest in dialog simulation for evaluation as well as for design automation and optimization. In this paper we expose a probabilistic framework for a realistic simulation of spoken dialogs in which the major components of a dialog system are modeled and parameterized thanks to independent data or expert knowledge. Especially, an Automatic Speech Recognition (ASR) system model and a User Model (UM) have been developed. The ASR model, based on articulatory similarities in language models, provides task-adaptive performance prediction and Confidence Level (CL) distribution estimation. The user model relies on the Bayesian Networks (BN) paradigm and is used both for user behavior modeling and Natural Language Understanding (NLU) modeling. The complete simulation framework has been used to train a reinforcement-learning agent on two different tasks. These experiments helped to point out several potentially problematic dialog scenarios.

A Speech-Centric Perspective for Human-Computer Interface: A Case Study

Speech technology has been playing a central role in enhancing human-machine interactions, especially for small devices for which graphical user interface has obvious limitations. The speech-centric perspective for human-computer interface advanced in this paper derives from the view that speech is the only natural and expressive modality to enable people to access information from and to interact with any device. In this paper, we describe some recent work conducted at Microsoft Research, aimed at the development of enabling technologies for speech-centric multimodal human-computer interaction. In particular, we present a case study of a prototype system, called MapPointS, which is a speech-centric multimodal map-query application for North America. This prototype navigation system provides rich functionalities that allow users to obtain map-related information through speech, text, and pointing devices. Users can verbally query for state maps, city maps, directions, places, nearby businesses and other useful information within North America. They can also verbally control applications such as changing the map size and panning the map moving interactively through speech. In the current system, the results of the queries are presented back to users through graphical user interface. An overview and major components of the MapPointS system will be presented in detail first. This will be followed by software design engineering principles and considerations adopted in developing the MapPointS system, and by a description of some key robust speech processing technologies underlying general speech-centric human-computer interaction systems.

음성인식모듈을 이용한 선박조타용 임베디드 시스템 개발

최근 국내외적으로 소형선박의 효율적인 운용을 위해서 많은 자동화 기술의 발전을 이루었으나, 이에 따른 선박 운용은 더욱 복잡해져 각종 계기와 선박 기기를 조작해야하는 번거로움이 있다. 이러한 문제를 해결하기 위하여 휴먼 인터페이스 기술의 하나인 음성정보처리기술을 선박 운용시스템에 적용한 연구 사례가 있으나 아직까지 구체적인 시스템 구현이 미비하다고 할 수 있다. 따라서, 본 논문에서는 선박운항자의 및 부담경감 및 효율적인 선박운용을 위하여 음성인식 모델을 이용한 선박조타용 임베디드 시스템을 구현하고자 한다. 구체적인 연구방법으로는, 화자독립방식의 음성인식 모듈(VR-33R)을 이용하여 선박의 조타기를 제어할 수 있는 임베디드 시스템을 설계 및 구현하고, 실제 선박시스템을 축소한 모형선박의 조타시스템에 적용하여 그 효용성을 확인하고, 실용화를 위한 방안을 모색하고자 한다. Recently, various studies had been made for automatic control system of small ships, in order to improve maneuvering and to reduce labor and working on board. To achieve efficient operation of small ships, it had been accomplished to rapid development of automatic technique, but the ship operation had been more complicated because of the need to handle various gauges and instruments. To solve these problems, there are examples to be applied to the speech information processing technologies which is one of the human interface methods in the system operation of ship, but the implementation of definite system is still incomplete. Therefore, the purpose of this paper is to implement the control system for ship steering using the voice recognition module.

Robots that can hear, understand and talk

In this survey paper we analytically examine the state of the art in speech and natural language processing technologies, and one of their most promising applications in the robotics world as a user interface to facilitate human-robot interaction/communication and robot control by spoken natural language. Theoretical aspects of spoken language technology and the main bottlenecks in developing a conversational interface for a robot have been presented in depth with results found while searching the literature related to the major breakthroughs made in this field. In this study, we present a brief technical introduction to talk-active robots, and to discuss related future technical challenges and technical approaches used. Efforts have been made to highlight the limitations and missing directions of the research and development in the spoken language technology which are creating hurdles in the development of voice-active robots for real-world applications.

Future directions in speech information processing

Speech processing technologies, including speech recognition, synthesis, and coding are expected to play important roles in an advanced multimedia society with user-friendly human–machine interfaces. Speech recognition systems include not only those that recognize messages but also those that recognize the identity of the speaker. This paper predicts future directions in speech processing. It describes the most important research problems and tries to forecast where progress will be made in the near future and what applications will become commonplace as a result of the increased capabilities. The most promising application area is telecommunications. To solve various fundamental problems, a unified approach across speech recognition, synthesis, and coding is indispensable. Handling the common phenomenon of voice individuality from different aspects are: research on speaker adaptation in speech recognition, automatic speaker verification, voice conversion in speech synthesis, and the problems of very low-bit-rate speech coding quality variation from speaker to speaker. The methods that essentially solve such problems should be based on a common mathematical model. Research on the mechanism of speech information processing in our human brains, that is, how meanings of speech are understood and how speech is produced, is also crucial for epoch-making technological development in conversational speech understanding and natural speech synthesis.

A source–filter model of humpback whale (Megaptera novaeangliae) vocal sound production

Acoustic waveforms produced by vocalizing humpback whales are well described by a source–filter model. This model characterizes vocalizations as the output of a time-varying signal generator (the source) passed through a time-varying resonator (the filter). For most humpback whale vocalizations, the source can be modeled as either a quasiperiodic impulse train, white noise, or a combination of these two signal types; the resonator can be modeled as an all-pole filter. The source–filter model is useful because it (a) provides a way to quantitatively characterize humpback whale vocalizations that can be automated and that allows for the ‘‘resynthesis’’ of sounds; (b) facilitates the application of advanced speech processing technologies to the analysis of humpback vocalizations; and (c) can potentially provide clues about the actual mechanisms humpback whales use to produce sounds. Currently, it is not known how humpback whales internally produce sounds, although structures in the larynx are thought to be involved. Source–filter-based analysis of humpback vocalizations revealed characteristic regions of frequency enhancement (formants) that may correspond to resonant frequencies of a physiological ‘‘filter.’’ Distribution of formants varied with pitch in a manner that appeared similar to pitch-related formant variation in human singers, suggesting that both species use analogous production mechanisms.

Splitting the unit delay [FIR/all pass filters design

A fractional delay filter is a device for bandlimited interpolation between samples. It finds applications in numerous fields of signal processing, including communications, array processing, speech processing, and music technology. We present a comprehensive review of FIR and allpass filter design techniques for bandlimited approximation of a fractional digital delay. Emphasis is on simple and efficient methods that are well suited for fast coefficient update or continuous control of the delay value. Various new approaches are proposed and several examples are provided to illustrate the performance of the methods. We also discuss the implementation complexity of the algorithms. We focus on four applications where fractional delay filters are needed: synchronization of digital modems, incommensurate sampling rate conversion, high-resolution pitch prediction, and sound synthesis of musical instruments.

Speech and audio processing technology for Internet.

Speech and audio processing technology for Internet.

Deployment of human-machine dialogue systems.

The deployment of systems for human-to-machine communication by voice requires overcoming a variety of obstacles that affect the speech-processing technologies. Problems encountered in the field might include variation in speaking style, acoustic noise, ambiguity of language, or confusion on the part of the speaker. The diversity of these practical problems encountered in the "real world" leads to the perceived gap between laboratory and "real-world" performance. To answer the question "What applications can speech technology support today?" the concept of the "degree of difficulty" of an application is introduced. The degree of difficulty depends not only on the demands placed on the speech recognition and speech synthesis technologies but also on the expectations of the user of the system. Experience has shown that deployment of effective speech communication systems requires an iterative process. This paper discusses general deployment principles, which are illustrated by several examples of human-machine communication systems.

Opportunities for advanced speech processing in military computer-based systems

The author presents a study of military applications of advanced speech processing technology which includes three major elements: (1) review and assessment of current efforts in military applications of speech technology. (2) identification of opportunities for future military applications of advanced speech technology, and (3) identification of problem areas where research in speech processing is needed to meet application requirements, and of current research thrusts which appear promising. Opportunities for advanced applications are identified by means of descriptions of several generic systems which would be possible with advances in speech technology and in system integration. These generic systems include an integrated multirate voice/data communications terminal, an interactive speech enhancement system, a voice-controlled pilot's associate system, advanced air traffic control training systems, a battle management command and control support system with spoken natural language interface, and a spoken language translation system. In the applications discussed, various spectral analysis and digitization techniques are teamed up with artificial intelligence schemes to allow a machine to recognize continuous-speech sentences with vocabularies running to thousands of words. >

HuMaNet: An experimental human-machine communications network based on ISDN wideband audio

A human's sensory capacity to assimilate, perceive, and react to information is much smaller than the capacity of modern transport facilities that convey the information. Moreover, the volume of information and the complexity of the terminal used to access the information can overwhelm people. The challenge, then, is to match an information system's capabilities to those of our senses. Recent advances in speech-processing technology have made natural voice — our preferred means for information exchange — feasible for human-machine communications. We describe an experimental network, called HuMaNet, that is implemented on commercial ISDN transport. The HuMaNet system uses speech-processing technology to make communications easier and more natural. Spoken commands control the system, which combines image and audio compression, database management, hands-free teleconferencing, and text-to-speech synthesis. Although HuMaNet is only in its initial phase, it has proved a remarkably habitable environment for human control of a complex computer and communications system.

Customization of a DSP integrated circuit for a consumer product

The design of a toy doll's speech recognition and response circuitry is summarized. The digital signal processing (DSP) system used as the core of this audio processing circuitry is discussed. Hardware used and the basic DSP architecture for the audio processing are reviewed. Two systems considered (DSP with codec and custom DSP chip) are briefly outlined. The speech processing, synthesis, and recognition technologies needed for this toy are detailed. Several DSP algorithms for use with this system are reviewed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Automation in Training and Education

Computer-based training and education are becoming increasingly prevalent and important. This article discusses the history of automated instruction, current applications, issues and problems, and future prospects. Current applications include such traditional uses of computers as testing, drills, tutorials, games, simulations, and student management. New applications include embedded training, computer literacy, interactive videodisc, and electronic lectures. The most important issue in automated instruction at present is the time and costs associated with the development of courseware. Other important problems include the difficulty of implementing individualized instruction in organizations accustomed to classroom teaching, the scarcity of educators who are computer literate, and the evaluation of courseware. Microcomputers have greatly accelerated the growth of computer-based instruction in all domains. Intelligent CAI, authoring systems, hand-held computers, speech processing, and telecommunication technologies such as videotex are seen as shaping the future direction of automated instruction.