Sources In Reverberant Environments Research Articles

Separation of Multiple Speech Sources in Reverberant Environments Based on Sparse Component Enhancement

Separation of Multiple Speech Sources in Reverberant Environments Based on Sparse Component Enhancement

Multi-Source DOA Estimation in Reverberant Environments by Jointing Detection and Modeling of Time-Frequency Points

In this article, the direction of arrival (DOA) estimation of multiple speech sources in reverberant environments is investigated based on the recording of a soundfield microphone. First, the recordings are analyzed in the time-frequency (T-F) domain to detect both “points” (single T-F points) and “regions” (multiple, adjacent T-F points) corresponding to a single source with low reverberation (known as low-reverberant-single-source (LRSS) points). Then, a LRSS point detection algorithm is proposed based on a joint dominance measure and instantaneous single-source point (SSP) identification. Following this, initial DOA estimates obtained for the detected LRSS points are analyzed using a Gaussian Mixture Model (GMM) derived by the Expectation-Maximization (EM) algorithm to cluster components into sources or outliers using a rule-based method. Finally, the DOA of each actual source is obtained from the estimated source components. Experiments on both simulated data and data recorded in an actual acoustic chamber demonstrate that the proposed algorithm exhibits improved performance for the DOA estimation in reverberant environments when compared to several existing approaches.

Masker–Signal Interaction in a Localization Task with a Moving Sound Source under the Influence of Precedence Effect

The precedence effect refers to a group of auditory phenomena related to the ability to localize sound sources in reverberant environment. The precedence effect was investigated using a localization task in the free sound field. The signal consisted of a direct sound (lead) and a moving target sound (lag). Lead–lag delays ranged from 1 to 40 ms. The lead was always stationary (+15°). The moving lag traveled to or from the lead along two paths situated at two locations (from –86° to –52° and from –52° to –86°). The subjects marked the start and end points of the trajectory of the moving sounds. Localization of the start and end points of the lag was compared with localization of the single source. It was found that the lead affected the start and end points of the lag trajectory. The shift depended on the direction of movement. The start points were always shifted in the direction of motion of the lag. The end points were shifted in the opposite direction. The shifts were decreasing as the delay was increasing. The effect of the lag on the lead localization was small.

The Role of Reverberation and Magnitude Spectra of Direct Parts in Contralateral and Ipsilateral Ear Signals on Perceived Externalization

Several studies show that the reverberation and spectral details in direct sounds are two essential cues for perceived externalization of virtual sound sources in reverberant environments. The present study investigated the role of these two cues in contralateral and ipsilateral ear signals on perceived externalization of headphone-reproduced binaural sound images at different azimuth angles. For this purpose, seven pairs of non-individual binaural room impulse responses (BRIRs) were measured at azimuth angles of −90°, −60°, −30°, 0°, 30°, 60°, and 90° in a listening room. The magnitude spectra of direct parts were smoothed, and the reverberation was removed, either in left or right ear BRIRs. Such modified BRIRs were convolved with a speech signal, and the resulting binaural sounds were presented over headphones. Subjects were asked to assess the degree of perceived externalization for the presented stimuli. The result of the subjective listening experiment revealed that the magnitude spectra of direct parts in ipsilateral ear signals and the reverberation in contralateral ear signals are important for perceived externalization of virtual lateral sound sources.

Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localization of Multiple Sources in Reverberant Environments

This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localisation of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localisation of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localisation model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian mixture model (GMM) based system that employs interaural time differences (ITDs) and ILDs as localisation features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localisation accuracies under challenging acoustic scenarios in which multiple talkers and room reverberation are present.

Acoustic source localization in strong reverberant environment by parametric Bayesian dictionary learning

Sparse representation techniques have become increasingly promising for localizing the sound source in reverberant environment, where the multipath channel effects can be accurately characterized by the image model. In this paper, a dictionary is constructed by discretizing the inner space of the enclosure, which is parameterized by the unknown energy reflective ratio. More specifically, each atom of the dictionary can characterize a specific source-to-microphone multipath channel. Subsequently, source localization can be reformulated as a joint sparse signal recovery and parametric dictionary learning problem. In particular, a sparse Bayesian framework is utilized for modeling, where its solution can be obtained by variational Bayesian expectation maximization technique. Moreover, the joint sparsity in frequency domain is exploited to improve the dictionary learning performances. A remarkably advantage of this approach is that no laborious parameter tuning procedure is required and statistical information can be provided. Numerical simulation results have shown that the proposed algorithm achieves high source localization accuracy, low sidelobes and high robustness for multiple sources with low computational complexity in strong reverberant environments, compared with other state-of-the-art methods.

The precedence effect for signals moving in the horizontal plane

The precedence effect refers to a group of auditory phenomena related to the ability to locate sound sources in reverberant environments. In the present study, this phenomenon was investigated using two moving signals. The first signal was direct (lead) and the other was delayed (lag). The motion of the sound source was created by successive switching of ten loudspeakers. The continuity of the motion was created by simultaneously attenuating the stimulus in the previous loudspeaker and enhancing it in the next one. The length of the path of the lead and lag was 34°. The lead moved from 34° to 0° (to the right) and the lag moved –52° to –86° (to the left). The duration of the lead and the lag was 1 s. Lead–lag delays ranged from 1 to 40 ms. Subjects had to indicate the location of the lag. The results indicate that the lead signal dominated in the sound localization at short delay durations (up to 18 ms). In spite of the instructions, all the subjects pointed at the lead, which suggests that they perceived the lag in this location. Two distinct sounds were perceived at the longest delays. The mean echo threshold and its standard deviation in eight subjects was 9.6 ± 4.5 ms.

Stimulus coherence influences sound-field localization and fusion/segregation of leading and lagging sounds.

The ability to localize sound sources in reverberant environments is dependent upon first-arriving information, an outcome commonly termed "the precedence effect." For example, in laboratory experiments, the combination of a leading (direct) sound followed by a lagging (reflected) sound is localized in the direction of the leading sound. This study was designed to measure the degree to which stimulus compactness/diffuseness (i.e., coherence as represented by interaural cross correlation) of leading and lagging sounds influences performance. The compactness/diffuseness of leading or lagging sounds was varied by either presenting a sound from a single loudspeaker or by presenting mutually uncorrelated versions of similar sounds from nine adjacent loudspeakers. In separate experiments, the listener's task was to point to the perceived location of leading and lagging 10-ms long low-pass filtered white noises or 2-s long tokens of speech. The leading and lagging stimuli were presented either from speakers located directly in front of the listeners or from speakers located ±45° to the right or left. The results indicate that leading compact (coherent) sounds influence perceived location more so than do leading diffuse (incoherent) sounds. This was true independent of whether the sounds were Gaussian noises or tokens of speech.

Coherence influences sound-field localization dominance in a precedence paradigm

The ability to localize sound sources in reverberant environments is dependent upon first-arriving information, an outcome commonly termed “the precedence effect.” For example, a combination of a leading (direct) sound followed by a lagging (reflected) sound is localized in the direction of the leading sound. We measured how the compactness/diffuseness (i.e., the interaural coherence) of leading and lagging sounds, respectively, influences performance. The compactness/diffuseness of leading or lagging sounds was varied by either presenting a sound from a single loudspeaker or by presenting mutually uncorrelated versions of similar sounds from nine adjacent loudspeakers. The listener pointed to the perceived location of leading and lagging sources of sounds which were 10-ms long, low-pass filtered white noises or 2-second long tokens of speech. The sounds were presented either from speakers located directly in front of the listener or from speakers located 45° to the right or left. Leading compact/coherent...

Direction-of-arrival estimation of passive acoustic sources in reverberant environments based on the Householder transformation.

This paper presents an approach to the direction-of-arrival (DOA) estimation problem in acoustic environments using microphone arrays. It works in the short-time Fourier transform (STFT) domain. It first transforms the noisy speech signals received at the array into the STFT domain. A Householder transformation is then constructed and applied to the multichannel STFT coefficients in each subband. This transformation converts the multichannel STFT coefficients into two components: one is a single coefficient that is dominated by the signal of interest and the other consists of the M - 1 coefficient that is dominated by noise (or even consists of noise-only if there is no reverberation), where M is the number of sensors. A cost function is then formed from the outputs of the Householder transformation and the DOA information can subsequently be obtained by searching the extremum value of this cost function in the angle range between 0° and 180°. Simulation results are provided to illustrate the performance of this approach.

Classification of reverberant audio signals using clustered ad hoc distributed microphones

In a real world scenario, the automatic classification of audio signals constitutes a difficult problem. Often, reverberation and interfering sounds reduce the quality of a target source signal. This results in a mismatch between test and training data when a classifier is trained on clean and anechoic data. To classify disturbed signals more accurately we make use of the spatial distribution of microphones from ad hoc microphone arrays. In the proposed algorithm clusters of microphones that either are dominated by one of the sources in an acoustic scenario or contain mainly signal mixtures and reverberation are estimated in the audio feature domain. Information is shared within and in between these clusters to create one feature vector for each cluster to classify the source dominating this cluster. We evaluate the algorithm using simultaneously active sound sources and different ad hoc microphone arrays in simulated reverberant scenarios and multichannel recordings of an ad hoc microphone setup in a real environment. The cluster based classification accuracy is higher than the accuracy based on single microphone signals and allows for a robust classification of simultaneously active sources in reverberant environments.

Improved estimation of direction of arrival of sound sources for hearing aids using gyroscopic information

Determining the direction of arrival (DOA) of a sound source is important in spatial audio signal processing, as it can lead to substantial improvement in noise reduction performance. Techniques such as generalized cross correlation with phase transform (GCC-PHAT) and adaptive eigenvalue decomposition (AED) perform optimally when the measurement microphones are fixed in place. However, hearing-aid microphones move with the listener’s head movements, which can result in momentarily inaccurate directional estimates and noise artifacts in the output signal. Techniques such as GCC-PHAT experience degraded short-term performance in the presence of multiple signals and noise. The system presented measures instantaneous head movement velocity using a micro-electromechanical systems (MEMS) gyroscope attached to binaurally communicating hearing aids. Estimates of DOA for physically stationary sources are shifted based on the gyroscope's head-movement information. Using GCC-PHAT with gyroscopic input can produce robust in situ DOA estimates for several sources in reverberant environments. In addition, the gyroscope allows an adaptive beamformer to be steered to a target direction, compensating for head movements on a very short timescale during DOA estimation. Results show improved localization performance over a standard GCC-PHAT system during head movements.

An ICA-SCT-PHD Filter Approach for Tracking and Separation of Unknown Time-Varying Number of Sources

In this paper we present a solution to the problem of tracking and separation of a mixture of concurrent sources in a reverberant environment where the number of sources is unknown and varies with time: new sources can appear and existing sources can disappear or undergo silence periods. In order to deal with this challenging problem, we synergistically combine two key ideas, one in the front end and the other at the back end. In the front end we employ independent component analysis (ICA) to demix the mixtures and the state coherence transform (SCT) to represent the signals in a direction of arrival (DOA) detection framework. By exploiting the frequency sparsity of the sources, ICA/SCT is even effective when the number of simultaneous sources is greater than the number of sensors therefore allowing for minimal number of sensors to be used. At the back end, the probability hypothesis density (PHD) filter is incorporated in order to track the multiple DOAs and determine the number of sources. The PHD filter is based on random finite sets (RFS) where the multi-target states and the number of targets are integrated to form a set-valued variable with uncertainty in the number of sources. A Gaussian mixture implementation of the PHD filter (GM-PHD) is utilized that solves the data association problem intrinsically, hence providing distinct DOA tracks. The distinct tracks also make the separation task possible by going back and rearranging the outputs of the ICA stage. The tracking and separation capabilities of the proposed method is demonstrated using simulations of multiple sources in reverberant environments.

Solving permutations in frequency-domain for blind separation of an arbitrary number of speech sources

Blind separation of speech sources in reverberant environments is usually performed in the time-frequency domain, which gives rise to the permutation problem: the different ordering of estimated sources for different frequency components. A two-stage method to solve permutations with an arbitrary number of sources is proposed. The suggested procedure is based on the spectral consistency of the sources. At the first stage frequency bins are compared with each other, while at the second stage the neighboring frequencies are emphasized. Experiments for perfect separation situations and for live recordings show that the proposed method improves the results of existing approaches.

Low latency localization of multiple sound sources in reverberant environments

Sound source localization algorithms determine the physical position of a sound source in respect to a listener. For practical applications, a localization algorithm design has to take into account real world conditions like multiple active sources, reverberation, and noise. The application can impose additional constraints on the algorithm, e.g., a requirement for low latency. This work defines the most important constraints for practical applications, introduces an algorithm, which tries to fulfill all requirements as good as possible, and compares it to state-of-the-art sound source localization approaches.

Multiple Reverberant Sound Localization Based on Rigorous Zero-Crossing-Based ITD Selection

This letter presents a multiple sound localization method for non-stationary speech sources in reverberant environments. Although zero-crossing-based interaural time differences (ITDs) were robust to diffuse noise and energy-based onset detection tackled reverberant speech , the onset detection method was sensitive to parameters. The proposed method employs echo-free onset detection , and this letter adds reverberation time estimation to obtain accurate echoes and signal-to-noise ratio estimation to assist the onset detection based on typical properties of acoustic reverberation in selecting ITDs corresponding to source locations. Experimental results indicate the effectiveness of the proposed method in reverberant environments.

Detection of the break in interaural correlation is affected by interaural delay, aging, and center frequency

This study investigated whether interaural integration is affected by introducing an interaural delay. In Experiment 1, both younger adults with normal hearing and older adults in the early stages of presbycusis were able to detect a transient break in interaural correlation (BIC) in the temporal middle of interaurally correlated wideband noises. However, their duration thresholds for detecting the BIC became larger with increasing interaural time difference (ITD) from 0 to 6 ms, and the threshold increase for older participants was larger than that for younger participants. In Experiment 2, to investigate whether the effect of changing ITD on the BIC detection is frequency-dependent, 1/3-octave narrowband noises with various center frequencies were used as stimuli. Results show that the duration threshold for detecting the BIC was higher for high-frequency noises than for low-frequency noises. Also, with increasing ITD from 0 to 4 ms, the threshold increase was larger for high-frequency noises than for low-frequency noises. The results suggest that there are age- and frequency-related temporal declines in maintaining fine-structure signals for interaural integration. These declines may affect the recognition of sound sources in reverberant environments.

Relative Transfer Function Identification Using Convolutive Transfer Function Approximation

In this paper, we present a relative transfer function (RTF) identification method for speech sources in reverberant environments. The proposed method is based on the convolutive transfer function (CTF) approximation, which enables to represent a linear convolution in the time domain as a linear convolution in the short-time Fourier transform (STFT) domain. Unlike the restrictive and commonly used multiplicative transfer function (MTF) approximation, which becomes more accurate when the length of a time frame increases relative to the length of the impulse response, the CTF approximation enables representation of long impulse responses using short time frames. We develop an unbiased RTF estimator that exploits the nonstationarity and presence probability of the speech signal and derive an analytic expression for the estimator variance. Experimental results show that the proposed method is advantageous compared to common RTF identification methods in various acoustic environments, especially when identifying long RTFs typical to real rooms.

Disambiguation of TDOA Estimation for Multiple Sources in Reverberant Environments

This paper presents a novel approach to estimate the time difference of arrival (TDOA) for multiple sources in reverberant environments. It resolves ambiguities in TDOA estimation caused by multipath propagation and multiple sources. By exploiting two TDOA constraints, the raster condition and the zero cyclic sum condition, we are able to identify and reject the echo path TDOAs and to assign the direct path TDOAs correctly to different sources. For the latter purpose, an efficient algorithm for the synthesis of approximately consistent TDOA graphs has been developed. A real experiment demonstrates the superior performance of our algorithms.

Investigation of the relationship among three common measures of precedence: fusion, localization dominance, and discrimination suppression.

Listeners have a remarkable ability to localize and identify sound sources in reverberant environments. The term "precedence effect" (PE; also known as the "Haas effect," "law of the first wavefront," and "echo suppression") refers to a group of auditory phenomena that is thought to be related to this ability. Traditionally, three measures have been used to quantify the PE: (1) Fusion: at short delays (1-5 ms for clicks) the lead and lag perceptually fuse into one auditory event; (2) Localization dominance: the perceived location of the leading source dominates that of the lagging source; and (3) Discrimination suppression: at short delays, changes in the location or interaural parameters of the lag are difficult to discriminate compared with changes in characteristics of the lead. Little is known about the relation among these aspects of the PE, since they are rarely studied in the same listeners. In the present study, extensive measurements of these phenomena were made for six normal-hearing listeners using 1-ms noise bursts. The results suggest that, for clicks, fusion lasts 1-5 ms; by 5 ms most listeners hear two sounds on a majority of trials. However, localization dominance and discrimination suppression remain potent for delays of 10 ms or longer. Results are consistent with a simple model in which information from the lead and lag interacts perceptually and in which the strength of this interaction decreases with spatiotemporal separation of the lead and lag. At short delays, lead and lag both contribute to spatial perception, but the lead dominates (to the extent that only one position is ever heard). At the longest delays tested, two distinct sounds are perceived (as measured in a fusion task), but they are not always heard at independent spatial locations (as measured in a localization dominance task). These results suggest that directional cues from the lag are not necessarily salient for all conditions in which the lag is subjectively heard as a separate event.