Speech Source Separation Research Articles

Joint algorithm for permutation problem in frequency-domain blind speech source separation

A new method for solving the permutation problem in the frequency-domain Blind Source Separation(BSS) was presented.This new method divided the whole frequency-domain into three sections and applied different permutation algorithms to different sections.The neighboring frequency correlation and harmonic frequency correlation coefficient of signal amplitudes were used together for the low-frequency section,the direction of arrival estimation for speech sources for the mid-frequency section,and the combination of correlation coefficient comparison and direction estimation for the high-frequency section,respectively.Experimental results show that the new method provides a more robust and precise solution to the permutation problem than the algorithm only with interfrequency correlation coefficient of signal amplitudes or only with direction of arrival estimation for sources.

A nonlinear frequency‐domain beamformer for underdetermined speech mixtures

Extraction of a target speech source from among multiple interfering speech sources is challenging when there are fewer microphones than sources (the underdetermined case). Existing speech source separation techniques often suffer from artifacts as well as performance deterioration in reverberant environments, and in some cases also need to estimate the number of sources present. This paper introduces a frequency‐domain non‐linear beamformer that can perform speech source separation of underdetermined mixtures, is reasonably artifact free and does not require prior knowledge of the number of speakers. Our approach models the data via a Gaussian mixture distribution in the observation domain, which can be learned using the expectation maximization (EM) algorithm. A non‐linear distortionless beamformer is then developed, based on this model. Simulations of the non‐linear beamformer in underdetermined mixtures with room reverberation confirm its capability to successfully separate speech sources with virtually no artifacts.

Separation of Multiple Speech Signals by usingTriangular Microphone Array

Speech source separation has been an important topic to realize speech-based human-machine interfaces or high quality hand-free communication with machines. For source separation, Independent Component Analysis (ICA) and time-frequency masking are powerful methods as a tool of Blind Source Separation (BSS) of speech mixtures. The latter method is based on the assumption called \W-Disjoint Orthogonality" which implies the cell component sparsity of speech in the time-frequency domain. One of the topics treated in this article is to introduce the time-frequency masking scheme is applied to the equilateral triangular array where the three delay estimates from each microphone pairs are obtained. In addition, it is used to improve histogram-mapping algorithm by integrate and coordinate transformation of three delay estimates. Some experiments in real environment for separating multiple sources are performed to verify the effectiveness.

Fast Noise Compensation and Adaptive Enhancement for Speech Separation

We propose a novel approach to improve adaptive decorrelation filtering- (ADF-) based speech source separation in diffuse noise. The effects of noise on system adaptation and separation outputs are handled separately. First, fast noise compensation (NC) is developed for adaptation of separation filters, forcing ADF to focus on source separation; next, output noises are suppressed by speech enhancement. By tracking noise components in output cross-correlation functions, the bias effect of noise on the system adaptation objective function is compensated, and by adaptively estimating output noise autocorrelations, the speech separation output is enhanced. For fast noise compensation, a blockwise fast ADF (FADF) is implemented. Experiments were conducted on real and simulated diffuse noises. Speech mixtures were generated by convolving TIMIT speech sources with acoustic path impulse responses measured in a real room with reverberation time T60 = 0.3 second. The proposed techniques significantly improved separation performance and phone recognition accuracy of ADF outputs.

Speech intelligibility improvement using convolutive blind source separation assisted by denoising algorithms

The present study is concerned with the blind source separation (BSS) of speech and speech-shaped noise sources. All recordings were carried out in an anechoic chamber using a dummy head (two microphones, one in each ear). The program which implements the algorithm for BSS of convolutive mixtures introduced by Parra and Spence [Parra, L., Spence, C., 2000a. Convolutive blind source separation of non-stationary sources. IEEE Trans. Speech Audio Process. 8(3), 320–327 (US Patent US6167417)] was used to separate out the signals. In the postprocessing phase two different denoising algorithms were used. The first was based on a minimum mean-square error log-spectral amplitude estimator [Ephraim, E., Malah, D., 1985. Speech enhancement using a minimum mean-square error log-spectral amplitude estimator. IEEE Trans. Speech Audio Process. ASSP-33(2), 443–445], while the second one was based on Wiener filter in which the concept of an a priori signal-to-noise estimation presented by Ephraim (as mentioned above) was applied [Scalart, P., Filho, J.V., 1996. Speech enhancement based on a priori signal to noise estimation. IEEE Internat. Conf. Acoust. Speech Signal Process. 1, 629–632]. Non-sense word tests were used as a target speech in both cases while one or two disturbing sources were used as interferences. The speech intelligibility before and after the BSS was measured for three subjects with audiologically normal hearing. Next the speech signal after BSS was denoised and presented to the same listeners. The results revealed some ambiguities caused by the insufficient number of microphones compared to the number of sound sources. For one disturbance only, the intelligibility improvement was significant. However, when there were two disturbances and the target speech, the separation was much poorer. The additional denoising, as could be expected, raises the intelligibility slightly. Although the BSS method requires more research on optimization, the results of the investigation imply that it may be applied to hearing aids in the future.

Visual voice activity detection as a help for speech source separation from convolutive mixtures

Audio–visual speech source separation consists in mixing visual speech processing techniques (e.g., lip parameters tracking) with source separation methods to improve the extraction of a speech source of interest from a mixture of acoustic signals. In this paper, we present a new approach that combines visual information with separation methods based on the sparseness of speech: visual information is used as a voice activity detector (VAD) which is combined with a new geometric method of separation. The proposed audio–visual method is shown to be efficient to extract a real spontaneous speech utterance in the difficult case of convolutive mixtures even if the competing sources are highly non-stationary. Typical gains of 18–20 dB in signal to interference ratios are obtained for a wide range of (2 × 2) and (3 × 3) mixtures. Moreover, the overall process is computationally quite simpler than previously proposed audio–visual separation schemes.

Blind source separation of speech sources in wavelet packet domains using Laplacianmixture model expectation maximization estimation in overcomplete cases

Speech processing has benefited a great deal from wavelet transforms. Wavelet packetsdecompose signals into broader components using linear spectral bisecting. Themixing matrix is the key issue in the blind source separation literature especially inunderdetermined cases (more sources than sensors). In this paper, algorithms are proposedfor estimating the mixing matrix and separation of speech signals from noise free linearmixtures in overcomplete cases. Mixtures of speech signals are decomposed using waveletpackets, the phase difference between the two mixtures is defined and used in the waveletdomain, and histograms of phase differences are obtained for every wavelet packet. In ourmethod, the Laplacian mixture model is considered in the wavelet packet domain, and isapplied to each histogram of packets. An expectation maximization algorithm is used totrain the model and calculate the model parameters. We also propose a novelmethod for obtaining the best wavelet packet node for finding source directions inscatter plots using variance calculations. A comparison is made to evaluate theperformances of different mother wavelets in the estimation of the mixing matrixand the best wavelet has been chosen. On the basis of the geometrical model, atwo-step adaptive algorithm is proposed for separating sources from mixtures.

Speech Source Separation in Convolutive Environments Using Space-Time-Frequency Analysis

We propose a new method for speech source separation that is based on directionally-disjoint estimation of the transfer functions between microphones and sources at different frequencies and at multiple times. The spatial transfer functions are estimated from eigenvectors of the microphones' correlation matrix. Smoothing and association of transfer function parameters across different frequencies are performed by simultaneous extended Kalman filtering of the amplitude and phase estimates. This approach allows transfer function estimation even if the number of sources is greater than the number of microphones, and it can operate for both wideband and narrowband sources. The performance of the proposed method was studied via simulations and the results show good performance.

A blind channel identification-based two-stage approach to separation and dereverberation of speech signals in a reverberant environment

Blind separation of independent speech sources from their convolutive mixtures in a reverberant acoustic environment is a difficult problem and the state-of-the-art blind source separation techniques are still unsatisfactory. The challenge lies in the coexistence of spatial interference from competing sources and temporal echoes due to room reverberation in the observed mixtures. Focusing only on optimizing the signal-to-interference ratio is inadequate for most if not all speech processing systems. In this paper, we deduce that spatial interference and temporal echoes can be separated and an M/spl times/N MIMO system will be converted into M SIMO systems that are free of spatial interference. Furthermore we show that the channel matrices of these SIMO systems are irreducible if the channels from the same source in the MIMO system do not share common zeros. Thereafter we can apply the Bezout theorem to remove reverberation in those SIMO systems. Such a two-stage procedure leads to a novel sequential source separation and speech dereverberation algorithm based on blind multichannel identification. Simulations with measurements obtained in the varechoic chamber at Bell Labs demonstrate the success and robustness of the proposed algorithm in highly reverberant acoustic environments.

Developing an audio-visual speech source separation algorithm

Looking at the speaker’s face is useful to hear better a speech signal and extract it from competing sources before identification. This might result in elaborating new speech enhancement or extraction techniques exploiting the audio-visual coherence of speech stimuli. In this paper, a novel algorithm plugging audio-visual coherence estimated by statistical tools on classical blind source separation algorithms is presented, and its assessment is described. We show, in the case of additive mixtures, that this algorithm performs better than classical blind tools both when there are as many sensors as sources, and when there are less sensors than sources. Audio-visual coherence enables a focus on the speech source to extract. It may also be used at the output of a classical source separation algorithm, to select the “best” sensor with reference to a target source.

Convolutive blind source separation of speech signals based on amplitude modulation decorrelation

The problem of blind separation of a convolutive mixture of speech signals is considered. Signal separation is performed in the frequency domain. The amplitude modulation decorrelation algorithm for convolutive blind source separation of speech is described, the first ideas of which were presented by Anemüller at the joint ASA/EAA/DAGA meeting, Berlin, 1999. The algorithm is based on the fact that the frequency-specific envelopes of speech signals exhibit correlation across different frequencies. This feature can be used to solve the ‘‘permutation problem’’ of frequency-domain-based blind source separation algorithms. Furthermore, it leads to separation of good quality since it results in a high number of constraints which must be fulfilled by the unmixed signals. Results for the separation of speech signals are presented for different mixing scenarios, including real-room reverberant mixing.

A cocktail party effect in the median plane?

Recent research has been conducted on the effects of spatialized audio on a listener’s ability to detect and identify a target speech signal when presented among nontarget speech signals in the horizontal plane [W. T. Nelson et al., Proceedings of the 1998 IMAGE Conference (1998), pp. 159–166]. However, the existence of a ‘‘cocktail party effect’’ in the median plane has not been addressed. The purpose of the present investigation was to determine whether or not the spatial separation of multiple simultaneous speech sources in the median plane leads to improved detection and intelligibility. Independent variables include the number and angular separation of the speech signals, the sex of the target talker, and the presence or absence of head motion cues. All speech signals—phrases from a modified Coordinate Response Measure [T. J. Moore, AGARD Conference Proceedings 311: Aural Communication in Aviation (1981), pp. 2.1–2.6]—were digitally filtered via nonindividualized HRTFs and presented over headphones. Results will be compared with those obtained in the horizontal plane. Inplications for the design of auditory displays will be discussed.

New Computerized Finnish Speech in Noise Test and Binaural Hearing

Aided and unaided binaural versus monaural recognition of speech has been studied with more or less laborious methods when speech and noise sources are spatially separated. The efficacy of the new Finnish speech in noise test to show binaural advantage was evaluated in free field conditions. Thirteen subjects with normal hearing were tested measuring speech recognition thresholds in noise (SRTN) in anechoic chambre with different situations of spatial separation of speech and noise sources monaurally and binaurally. The noise source was fixed above the head (treating both ears similarly) and equidistant speech was delivered at different azimuths. SRTNs with mixed speech and noise without spatial separation served as normalizing reference. Differencies in SRTNs consistently followed the changing head and pinna shadow effects on the speech-to-noise ratio with a smaller binaural advantage. This new test seems to give quite economically the same results as those of previous more sophisticated investigations on speech recognition in free field situations.