Distant Talking Research Articles

Environmentally robust ASR front-end for deep neural network acoustic models

This paper examines the individual and combined impacts of various front-end approaches on the performance of deep neural network (DNN) based speech recognition systems in distant talking situations, where acoustic environmental distortion degrades the recognition performance. Training of a DNN-based acoustic model consists of generation of state alignments followed by learning the network parameters. This paper first shows that the network parameters are more sensitive to the speech quality than the alignments and thus this stage requires improvement. Then, various front-end robustness approaches to addressing this problem are categorised based on functionality. The degree to which each class of approaches impacts the performance of DNN-based acoustic models is examined experimentally. Based on the results, a front-end processing pipeline is proposed for efficiently combining different classes of approaches. Using this front-end, the combined effects of different classes of approaches are further evaluated in a single distant microphone-based meeting transcription task with both speaker independent (SI) and speaker adaptive training (SAT) set-ups. By combining multiple speech enhancement results, multiple types of features, and feature transformation, the front-end shows relative performance gains of 7.24% and 9.83% in the SI and SAT scenarios, respectively, over competitive DNN-based systems using log mel-filter bank features.

Making Machines Understand Us in Reverberant Rooms: Robustness Against Reverberation for Automatic Speech Recognition

Speech recognition technology has left the research laboratory and is increasingly coming into practical use, enabling a wide spectrum of innovative and exciting voice-driven applications that are radically changing our way of accessing digital services and information. Most of today's applications still require a microphone located near the talker. However, almost all of these applications would benefit from distant-talking speech capturing, where talkers are able to speak at some distance from the microphones without the encumbrance of handheld or body-worn equipment [1]. For example, applications such as meeting speech recognition, automatic annotation of consumer-generated videos, speech-to-speech translation in teleconferencing, and hands-free interfaces for controlling consumer-products, like interactive TV, will greatly benefit from distant-talking operation. Furthermore, for a number of unexplored but important applications, distant microphones are a prerequisite. This means that distant talking speech recognition technology is essential for extending the availability of speech recognizers as well as enhancing the convenience of existing speech recognition applications.

에너지 기반 가중치를 이용한 음성 특징의 자동회귀 이동평균 필터링

In this paper, a robust feature compensation method to deal with the environmental mismatch is proposed. The proposed method applies energy based weights according to the degree of speech presence to the Mean subtraction, Variance normalization, and ARMA filtering (MVA) processing. The weights are further smoothed by the moving average and maximum filters. The proposed feature compensation algorithm is evaluated on AURORA 2 task and distant talking experiment using the robot platform, and we obtain error rate reduction of 14.4 % and 44.9 % by using the proposed algorithm comparing with MVA processing on AURORA 2 task and distant talking experiment, respectively.

Robust Speech Recognition by Model Adaptation and Normalization Using Pre-Observed Noise

Users require speech recognition systems that offer rapid response and high accuracy concurrently. Speech recognition accuracy is degraded by additive noise, imposed by ambient noise, and convolutional noise, created by space transfer characteristics, especially in distant talking situations. Against each type of noise, existing model adaptation techniques achieve robustness by using HMM-composition and CMN (cepstral mean normalization). Since they need an additive noise sample as well as a user speech sample to generate the models required, they can not achieve rapid response, though it may be possible to catch just the additive noise in a previous step. In the previous step, the technique proposed herein uses just the additive noise to generate an adapted and normalized model against both types of noise. When the user's speech sample is captured, only online-CMN need be performed to start the recognition processing, so the technique offers rapid response. In addition, to cover the unpredictable S/N values possible in real applications, the technique creates several S/N HMMs. Simulations using artificial speech data show that the proposed technique increased the character correct rate by 11.62% compared to CMN.

Realistic application of acoustic blind source separation

Blind source separation in an acoustic environment is a challenging problem. The reverberation times in realistic rooms require long separation filters. They have to be estimated in very short times as the room responses change drastically with user location. In addition, convolutive blind separation has a number of indeterminations that make the identification of filter coefficients difficult. Decorrelation of changing cross-power spectra of nonstationary signal has been proposed as a separation criterion by various authors. We have presented in particular a frequency domain on-line algorithm that seeks to decorrelate cross-power spectra estimated over multiple times. We will report the performance of the algorithm in a realistic environment. The results will be compared to more conventional adaptive beamforming algorithms. Furthermore, speech recognition results for a command and control application with a distant talking user will be presented.

Hands-free speech recognition by a microphone array and HMM composition

Hands-free speech interface is one of the final goals of human–machine interface. This paper introduces two methods for distant talking speech recognition in noisy and reverberant rooms. The first method is speech recognition using a microphone array. The microphone array enables one to enhance a speech signal using spatial phase differences even in environments where unstationary noises exist. The proposed method is composed of two modules: (1) a high SNR signal retrieval by a delay-and-sum beamformer and (2) localization and trace of the speaker’s direction by extracting a signal power and pitch harmonics. The second method is speech recognition based on the HMM composition. The proposed HMM composition is obtained by extending the HMM composition method of an additive noise to that of the convolutional acoustical transfer function. The HMMs are prepared beforehand for clean speech, noise, and acoustical transfer function. Then the HMM composition is conducted twice for a speech HMM and an acoustical transfer function HMM in the cepstrum domain and for the distorted speech HMMs and noise HMM in a linear spectral domain. The speaker-dependent/independent word recognition experiments using tied-mixture monophone HMMs are carried out and have clarified the effectiveness of the proposed methods. Furthermore, an effective coupling of these methods is also discussed.

A Study of Talking Distance and Related Parameters in Hands-Free Telephony

This paper outlines the problems of providing satisfactory hands-free operation of the telephone and discusses various methods which can be applied to their solution. Particular attention is given to acoustic environment, proximity talking (as applied to hands-free operation of the telephone) and voice switching. Preference indications were obtained for 18 subjects in 18 different locations for proximity and distant talking without voice switching, and for the same number of subjects and locations for proximity and distant talking with voice switching. In the lotter tests, supplementary data were also obtained without voice switching. Regardless of the type of circuitry, the preference decision was considerably affected by the amount of reverberation at the hands-free location; proximity operation was generally favored under conditions of moderate to high reverberation, nonproximity operation under conditions of low reverberation (high room constant). Other factors which affected the preference for hands-free proximity and distant talking are discussed.

General aspects of hands-free telephony

The use of a simple hands-free telephone set with distant talking and loudspeaker reception introduces transmission degradation not encountered with hand-set usage. Proper usage and good acoustic surroundings reduce these impairments to a point where satisfactory transmission is obtained on a large percentage of calls in today's plant which is designed to have transmission margin on a good proportion of all calls. There are several potentialities for improving the hands-free telephone set but it is not yet clear whether or not the gain will justify the added complexity or the added restrictions placed on the user. It is to be expected that very excellent performance could be obtained from a unit such as a booth which combines optimum instrument design with favorable environment.