Multichannel Acoustic Echo Cancellation Research Articles

Multichannel Acoustic Echo Cancellation With Beamforming in Dynamic Environments

Multichannel Acoustic Echo Cancellation With Beamforming in Dynamic Environments

Neural Cascade Architecture for Multi-Channel Acoustic Echo Suppression

Traditional acoustic echo cancellation (AEC) works by identifying an acoustic impulse response using adaptive algorithms. This paper proposes a neural cascade architecture for joint acoustic echo and noise suppression to address both single-channel and multi-channel AEC (MCAEC) problems. The proposed cascade architecture consists of two modules. A convolutional recurrent network (CRN) is employed in the first module for complex spectral mapping. Its output is fed as an additional input to the second module, where a long short-term memory network (LSTM) is utilized for magnitude mask estimation. The entire architecture is trained in an end-to-end manner with the two modules optimized jointly using a single loss function. The final output is generated using the enhanced phase and magnitude obtained from the first and the second module, respectively. The cascade architecture enables the proposed method to obtain robust magnitude estimation as well as phase enhancement. The proposed method is investigated under different AEC setups. We find that the deep learning based approach avoids the no-uniqueness problem in traditional MCAEC. For MCAEC setups with multiple microphones, combining deep MCAEC with supervised beamforming further improves the system performance. Evaluation results show that the proposed approach effectively suppresses acoustic echo and noise while preserving speech quality, and consistently outperforms related methods under different setups.

Speech Processing for Digital Home Assistants: Combining Signal Processing With Deep-Learning Techniques

Once a popular theme of futuristic science fiction or far-fetched technology forecasts, digital home assistants with a spoken language interface have become a ubiquitous commodity today. This success has been made possible by major advancements in signal processing and machine learning for so-called far-field speech recognition, where the commands are spoken at a distance from the sound-capturing device. The challenges encountered are quite unique and different from many other use cases of automatic speech recognition (ASR). The purpose of this article is to describe, in a way that is amenable to the nonspecialist, the key speech processing algorithms that enable reliable, fully hands-free speech interaction with digital home assistants. These technologies include multichannel acoustic echo cancellation (MAEC), microphone array processing and dereverberation techniques for signal enhancement, reliable wake-up word and end-of-interaction detection, and high-quality speech synthesis as well as sophisticated statistical models for speech and language, learned from large amounts of heterogeneous training data. In all of these fields, deep learning (DL) has played a critical role.

Efficient multichannel acoustic echo cancellation using constrained tap selection schemes in the subband domain

Acoustic echo cancellation (AEC) is a key speech enhancement technology in speech communication and voice-enabled devices. AEC systems employ adaptive filters to estimate the acoustic echo paths between the loudspeakers and the microphone(s). In applications involving surround sound, the computational complexity of an AEC system may become demanding due to the multiple loudspeaker channels and the necessity of using long filters in reverberant environments. In order to reduce the computational complexity, the approach of partially updating the AEC filters is considered in this paper. In particular, we investigate tap selection schemes which exploit the sparsity present in the loudspeaker channels for partially updating subband AEC filters. The potential for exploiting signal sparsity across three dimensions, namely time, frequency, and channels, is analyzed. A thorough analysis of different state-of-the-art tap selection schemes is performed and insights about their limitations are gained. A novel tap selection scheme is proposed which overcomes these limitations by exploiting signal sparsity while not ignoring any filters for update in the different subbands and channels. Extensive simulation results using both artificial as well as real-world multichannel signals show that the proposed tap selection scheme outperforms state-of-the-art tap selection schemes in terms of echo cancellation performance. In addition, it yields almost identical echo cancellation performance as compared to updating all filter taps at a significantly reduced computational cost.

Multichannel Acoustic Echo Cancellation in the Wave Domain With Increased Robustness to Nonuniqueness

Acoustic echo cancellation (AEC) is significantly more challenging in the case of multichannel reproduction compared to the single-channel case. This is mainly due to the so-called nonuniqueness problem, which becomes especially severe for massive multichannel reproduction systems like, e.g., wave field synthesis systems or higher-order ambisonics, where the different loudspeaker signals are often strongly correlated. In this paper, a multichannel wave-domain AEC structure is proposed that maintains robustness without altering the loudspeaker signals, as is commonly required to preserve the reproduction quality. The proposed method exploits the dominance of certain couplings in the wave-domain representation of the loudspeaker-enclosure-microphone system. In this representation, preferring a strong weight of the dominant couplings in the identified system improves the system identification performance. A modification of the well-known generalized frequency-domain adaptive filtering algorithm is then presented to implement the proposed approach, including a self-contained description of all necessary components. An experimental evaluation compares the proposed method to competing approaches and shows that the proposed method compares well in terms of convergence behavior while avoiding the performance degradation, which is intrinsic to other approaches.

Multi-channel Acoustic Echo Cancellation Using Adaptive Fuzzy Fusion Algorithm

Multi-channel Acoustic Echo Cancellation Using Adaptive Fuzzy Fusion Algorithm

Inter-Channel Decorrelation by Sub-Band Resampling for Multi-Channel Acoustic Echo Cancellation

An inter-channel decorrelation procedure is highly recommended for multi-channel acoustic echo cancellation (AEC) to directly assist adaptive filtering algorithms in overcoming the so-called “non-uniqueness” problem. Although various methods have been proposed in the past to mitigate the problem and lower the misalignment of the adaptive filter, the introduced audible distortion often limits the effectiveness of those algorithms. In this paper, we investigate in detail the decorrelation by resampling technique and a proper design strategy for sub-band resampling (SBR), which permits finely tuned, frequency specific control of the trade-off between decorrelation, measured in terms of the (magnitude-squared) coherence, and audio quality, evaluated in terms of objective speech quality measures and a subjective listening test. Our most recent study provides a deep analysis of the performance bounds of the resampling procedure by analyzing the relationship between resampling, coherence, and achievable steady-state misalignment. We provide a novel, theoretically justifiable, and perceptually motivated strategy for SBR to achieve a fast multi-channel AEC convergence rate while maintaining the highest audio quality when compared to other conventional decorrelation procedures.

An efficient post-processing filter for multi-channel acoustic echo cancellation system

The acoustic echo usually cannot be totally removed by the adaptive filter due to the insufficient filter taps, nonlinearity of the echo path transfer function, and the background noise. A post-processing filter, which aims to suppress the residual echo, thus plays an important role in the acoustic echo cancellation system. In this paper, an efficient post filter is proposed based on the coherence between the error signal and the echo estimation of the adaptive filter. The proposed method can be seamlessly combined with the multi-delay frequency-domain adaptive filter. Furthermore, it can be easily extended to multi-channel processing. The instrumental evaluation and listening test both demonstrate the superiority of the proposed method.

Recursive Bayesian Control of Multichannel Acoustic Echo Cancellation

We present a novel recursive Bayesian method in the DFT-domain to address the multichannel acoustic echo cancellation problem. We model the echo paths between the loudspeakers and the near-end microphone as a multichannel random variable with a first-order Markov property. The incorporation of the near-end observation noise, in conjunction with the multichannel Markov model, leads to a multichannel state-space model. We derive a recursive Bayesian solution to the multichannel state-space model, which turns out to be well suited for input signals that are not only auto-correlated but also cross-correlated. We show that the resulting multichannel state-space frequency-domain adaptive filter (MCSSFDAF) can be efficiently implemented due to the submatrix-diagonality of the state-error covariance. The filter offers optimal tracking and robust adaptation in the presence of near-end noise and echo path variability.

Spatial audio and room acoustics.

Schroeder’s interest in the behavior of sound in small enclosures eventually lead to the use of statistical methods of digital signal processing to estimate and track a room’s response for echo control. He showed that a small frequency shift (5 Hz) in a loudspeaker-enclosure-microphone system could be used to mitigate the coupling feedback in public address systems. He also employed frequency shifting for echo and feedback control in the first stereo teleconferencing system. We use many of the results from the early work on stereo teleconferencing in developing solutions for multichannel, immersive systems. Schroeder’s work on simulating room acoustics form the fundamentals of how we simulate and test new algorithms. His work on stereo reproduction and perception forms the basis for our motivation to explore multichannel systems (specifically, the spatial release from masking effects). The work on frequency shifting in public address systems was a first attempt at a problem that research is still focused on, multichannel acoustic echo cancellation, which addresses the howling feedback and echo that results from a loudspeaker-enclosure-microphone system.

Batch-Online Semi-Blind Source Separation Applied to Multi-Channel Acoustic Echo Cancellation

Semi-blind source separation (SBSS) is a special case of the well-known blind source separation (BSS) when some partial knowledge of the source signals is available to the system. In particular, a batch adaptation in the frequency domain based on independent component analysis (ICA) can be effectively used to jointly perform source separation and multichannel acoustic echo cancellation (MCAEC) through SBSS without double-talk detection. Many issues related to the implementation of an SBSS system are discussed in this paper. After a deep analysis of the structure of the SBSS adaptation, we propose a constrained batch-online implementation that stabilizes the convergence behavior even in the worst case scenario of a single far-end talker along with the non-uniqueness condition on the far-end mixing system. Specifically, a matrix constraint is proposed to reduce the effect of the non-uniqueness problem caused by highly correlated far-end reference signals during MCAEC. Experimental results show that high echo cancellation can be achieved just as the misalignment remains relatively low without any preprocessing procedure to decorrelate the far-end signals even for the single far-end talker case.

WOZ acoustic data collection for interactive TV

This paper describes a multichannel acoustic data collection recorded under the European DICIT project, during Wizard of Oz (WOZ) experiments carried out at FAU and FBK-irst laboratories. The application of interest in DICIT is a distant-talking interface for control of interactive TV working in a typical living room, with many interfering devices. The objective of the experiments was to collect a database supporting efficient development and tuning of acoustic processing algorithms for signal enhancement. In DICIT, techniques for sound source localization, multichannel acoustic echo cancellation, blind source separation, speech activity detection, speaker identification and verification as well as beamforming are combined to achieve a maximum possible reduction of the user speech impairments typical of distant-talking interfaces. The collected database permitted to simulate at preliminary stage a realistic scenario and to tailor the involved algorithms to the observed user behaviors. In order to match the project requirements, the WOZ experiments were recorded in three languages: English, German and Italian. Besides the user inputs, the database also contains non-speech related acoustic events, room impulse response measurements and video data, the latter used to compute three-dimensional positions of each subject. Sessions were manually transcribed and segmented at word level, introducing also specific labels for acoustic events.

Considering modal aliasing in the implementation of an acoustic echo canceller in the wave domain.

Traditional multichannel acoustic echo cancellation applied to wave field synthesis systems is computationally extremely expensive due to the large number of channels to be identified. By using wave domain adaptive filtering (WDAF) one is able to choose an approximation which neglects the interaction (exchange of energy) between different modes. This reduces the number of necessary adaptive filter coefficients drastically. Furthermore, it allows to use computationally relatively inexpensive single-channel filter adaptation algorithms. Then, however, modal aliasing due to the limited resolution of the used microphone array becomes a significant problem since it constitutes an interaction between different modes with respect to the microphone signals. The effects of modal aliasing on the adaptation behavior of a WDAF-based echo cancellation system are analyzed and a modified wave domain representation of the synthesized wave field is proposed which reflects the limited resolution of the microphone array.

Surround sound echo cancellation in the spherical harmonic domain

The problem of creating a multiuser hands‐free immersive telecommunications environment poses many challenges for acoustic signal processing. The most pressing is the creation of a fast multichannel acoustic echo canceller (MCAEC) to eliminate acoustic feedback created in the speaker‐microphone loop. Traditional multichannel adaptive algorithms for echo cancellation are not fast enough to work in systems with highly correlated multichannel signals like those found in the traditional telecommunications setup. This work presents analysis of a proposed acoustic echo cancellation system that operates in the spherical harmonic domain. Transforming the microphone and speaker signals into the spherical harmonic domain reduces both the number of channels and the correlation between channels for faster convergence of the adaptive MCAEC filters. This method also has the added benefit of being independent of the microphone and speaker array configuration, resulting in a hybrid mono/stereo/multichannel acoustic echo canceller. This work is funded by NSF Grant IIS‐0534221.

Some current challenges in multichannel acoustic signal processing

Hands-free human/machine communication involves major challenges for today’s state of the art signal processing if the humans should not be forced to carry or wear any technical gear and should be free to move. The spatial domain sampled by multiple loudspeakers for reproduction and multiple microphones for signal acquisition allows for spatialized listening experiences and for spatially selective acoustic scene analysis, respectively. Along with the possibility to serve several human communication partners simultaneously, such systems also introduce the need for localization algorithms. Three main types of impairment have to be addressed by signal processing algorithms to fulfill the promises of the general concept: first, acoustic feedback from the loudspeakers into the microphones; second, ambient noise and undesired pointlike interferers; and, finally, echoes and reverberation as picked up by the human ear when listening to the loudspeaker signals or as recorded by the microphones that actually should record only the direct-path component of the source signal. Accordingly, some of the current challenges in multichannel acoustic echo cancellation, in spatially selective noise and interference suppression, in source separation and localization, and in dereverberation are investigated.

A novel framework of multi-channel acoustic echo cancellation

Abstract Conventionally, multi-channel acoustic echo cancellation (AEC) achieves the goal by estimating the impulse responses of the local room. However, generally, conventional AEC methods have no unique solutions. Due to the strong correlation of the input signals, conventional methods are with many disadvantages. To overcome this problems, a new framework is proposed in this paper based on SIMO (single input multiple output) blind deconvolution. Under the new framework, we achieve the goal by identifying the impulse responses of distant room and avoiding the disadvantages of the conventional methods. *Supported by National Natural Science foundation of China (Grand Nos. 60325310, 60505005), Guangdong Province Science Foundation for Program of Research Team (Grant No. 04205783), Guangdong Province Natural Science Foundation (Grant Nos. 05103553,05006508) and the Specialized Prophasic Basic Research Projects of Ministry of science and Technology, china (Grant No. 2005CCA04100)

Generalized multichannel frequency-domain adaptive filtering: efficient realization and application to hands-free speech communication

In unknown environments where we need to identify, model, or track unknown and time-varying channels, adaptive filtering has been proven to be an effective tool. In this contribution, we focus on multichannel algorithms in the frequency domain that are especially well suited for input signals which are not only auto-correlated but also highly cross-correlated among the channels. These properties are particularly important for applications like multichannel acoustic echo cancellation. Most frequency-domain algorithms, as they are well known from the single-channel case, are derived from existing time-domain algorithms and are based on different heuristic strategies, e.g, for stepsize normalization. Here, we present a new rigorous derivation of a whole class of multichannel adaptive filtering algorithms in the frequency domain based on a recursive least-squares criterion. Then, from the normal equation, we derive a generic adaptive algorithm in the frequency domain. Due to the rigorous approach, the proposed framework inherently takes the coherence between all input signal channels into account. An analysis of this multichannel algorithm shows that the mean-squared error convergence is independent of the input signal statistics (i.e., both auto-correlation and cross-correlation). A useful approximation provides interesting links between some well-known algorithms for the single-channel case and the general multichannel framework. We also give design rules for important parameters to optimize the performance in practice. The computational complexity is kept low by introducing several new techniques, such as a robust recursive Kalman gain computation in the frequency domain and efficient fast Fourier transform (FFT) computation tailored to overlapping data blocks. Simulation results and real-time performance for applications such as multichannel acoustic echo cancellation show the high efficiency of the approach.

Multichannel signal processing at Bell Labs Acoustics Research—Sampled by a postdoc

In the mid 1980’s, the first large microphone arrays for audio capture were designed and realized by Jim Flanagan and Gary Elko. After the author joined Bell Labs in 1989, the first real-time digital beamformer for teleconferencing applications was implemented and formed a starting point for the development of several novel beamforming techniques. In parallel, multichannel loudspeaker systems were already investigated and research on acoustic echo cancellation, small-aperture directional microphones, and sensor technology complemented the research scenario aiming at seamless hands-free acoustic communication. Arrays of many sensors and loudspeakers for sampling the spatial domain combined with advanced signal processing sparked new concepts that are still fueling ongoing research around the world—including the author’s research group. Here, robust adaptive beamforming has found its way from large-scale arrays into many applications using smaller apertures. Blind source separation algorithms allow for effective spatial filtering without a priori information on source positions. Full-duplex communication using multiple channels for both reproduction and recording is enabled by multichannel acoustic echo cancellation combined with beamforming. Recently, wave domain adaptive filtering, a new concept for handling many sensors and many loudspeakers, has been verified for arrays that may well remind some observers of former Bell Labs projects.

Audio recording and reproduction in CARROUSO: Getting closer to perfection?

State-of-the-art systems for spatial audio reproduction utilize two to six discrete playback channels. A problem inherent to these systems is the relatively small area where the listener is able to experience a true 3-D sound sensation. This so-called ‘‘sweet spot’’ can be significantly enlarged by using loudspeaker arrays in combination with wave field synthesis (WFS) technology, initially developed at Delft University. By following this approach, actual sonic spaces can be reproduced in their entirety and not only discrete multichannel representations thereof. While loudspeaker arrays can be used to reproduce sound fields, microphone arrays can be used for sound field capture and analysis. Having high-quality audio reproduction in mind, microphone array designs are presented that need to fulfill stricter requirements than what has been traditionally considered for microphone array applications. Information on acoustic source position is essential for WFS-based rendering techniques. As will be shown, joint audio–video object tracking proves to be efficient for this task. Moreover, full-duplex applications based on WFS technology, like high-quality teleconferencing or remote music teaching, call for sophisticated multichannel acoustic echo cancellation algorithms. The European project ‘‘CARROUSO’’ aims at developing, integrating, and building a real-time system that embraces all previously described technologies in an MPEG-4 context.

A multichannel acoustic echo canceler double‐talk detector based on a normalized cross‐correlation matrix

AbstractMultichannel acoustic echo cancellation is basically composed of two parts. One part is a multichannel system identification problem which is nontrivial to solve. The other part, which is also nontrivial, is the so‐called double‐talk detection problem. Near‐end speech detection is based on a test statistic. Recently, a new double‐talk test statistic based on the normalized cross‐correlation vector was proposed For the single‐channel case. Obviously, in the multichannel case, there are several solutions, but what should be the optimal one is not yet known. Then, a fundamental question arises: how do we deal optimally with multiple channels? In this paper, we generalize the idea of normalized cross‐correlation vector (single‐channel) to the matrix case (multichannel), derive a frequency‐domain version, and show how to combine both the multichannel frequency‐domain adaptive filter and the multichannel double‐talk detector.