Distributed Microphone Arrays Research Articles

Reverberation-Robust Self-Calibration and Synchronization of Distributed Microphone Arrays by Mitigating Heteroscedasticity and Outlier Occurrence in TDoA Measurements.

The network of distributed microphone arrays is usually established in an ad hoc manner; hence, network parameters such as the mutual positioning and rotation of the arrays, positions of sources, and synchronization of their recording onset times are initially unknown. In this article, we consider the problem of passively jointly self-calibrating and synchronizing distributed arrays in reverberant rooms. We use a typical two-step approach where, initially, the relative geometry of the network is estimated using Direction of Arrival (DoA) measurements. Subsequently, the absolute scale and synchronization parameters are estimated using Time Difference of Arrival (TDoA) measurements. This article presents methods to improve the robustness and accuracy of estimation of the absolute geometric scaling and synchronization parameters in reverberant conditions, in which TDoA measurements do not follow a normal distribution; furthermore, outliers often occur. To remedy these issues, we propose a Weighted Least Squares (WLS) estimator and schema for weighting the TDoA measurements to increase the estimation accuracy from heteroscedastic TDoA measurements. In addition, we propose an iterative reweighing algorithm with a binary weight to detect and reject TDoA outliers, which exploits the residuals of the parametric model in the least absolute value minimization. A numerical evaluation shows significant improvements in the proposed method over the state of the art in terms of the relative scaling error and mean absolute value of the synchronization parameters.

A study on the influence of reflective surfaces on sound source localization using distributed acoustic measurement equipment

This research aims to develop a system that will detect sound source localization in a broad area using cross-correlation functions for signals from distributed microphone arrays, which is difficult with conventional methods that use microphone arrays. For the localization of sound sources, four microphones were distributed in an anechoic room. Based on the result, the system was able to estimate the sound source position within a plus or minus 20 mm of accuracy. To verify the accuracy of the system, a reflective surface was set up in an anechoic room to confirm the effect of reflected and diffracted sound. No significant increase in error was observed when the reflective surface was behind the microphone as viewed from the sound source. However, the estimation error increased when diffracted sound was present.

Acoustic Source Localization Using Kernel-based Extreme Learning Machine in Distributed Microphone Array

Acoustic source localization using distributed microphone array is a challenging task due to the influences of noise and reverberation. In this paper, acoustic source localization using kernel-based extreme learning machine in distributed microphone array is proposed. Specifically, the space of interest is divided into some labeled positions, and the candidate generalized cross correlation function in each node is treated as the feature mapped into the hidden nodes of extreme learning machine. During the training phase, by the implementation of kernel function, the output weights of the classifier are calculated and do not need to be tuned. After the kernel-based extreme learning machine (K-ELM) is well trained, the measured generalized cross correlation data are fed into the K-ELM classifier, and the output is the estimated acoustic source position. The proposed method needs less human intervention for both training and testing and it does not need to calibrate the node in advance. Simulation and real-world experimental results reveal that the proposed method has extremely fast training and testing speeds, and can obtain better localization performance than steered response power, K-nearest neighbor, and support vector machine methods.

Group conversation enhancement using distributed microphone arrays with adaptive binauralization

Hearing aids and other listening devices perform poorly in noisy, reverberant venues like restaurants and conference centers with numerous active sound sources. Microphone arrays can use array processing techniques like beamforming to isolate talkers from a specific region in the room while attenuating undesired sound sources. However, beamforming often removes spatial cues and is typically restricted to isolating a single talker at a time. Previous work has shown the effectiveness of remote microphones worn by talkers and adapting the signal at the earpiece to improve the intelligibility of group conversations. Due to the increase in hybrid meetings and classrooms, many spaces are equipped with high throughput, low latency devices including large microphone arrays. In this work, we present a system that aggregates information collected by microphone arrays distributed in a room to enhance the intelligibility of talkers in a group conversation. The beamformed signal from the microphone arrays is adapted to match the magnitude and phase of the earpiece microphones. The filters are continuously updated in order to track motion of both the listeners and talkers.

Large-scale simulation of bird localization systems in forests with distributed microphone arrays

Acoustic wildlife monitoring systems are important tools for capturing information about animal habitation in ecosystems. Previous work has demonstrated the effectiveness of audio-based bird localization techniques. However, few studies have investigated the performance and robustness of distributed systems in large forests. Here, the performance of distributed microphone arrays for localizing birds is examined by simulating forest scenes with added reverberation, ambient noise, and measurement errors. The simulation revealed the importance of the signal-to-noise ratio and the spectral weighting in the localization algorithm. These results may guide the design of large-scale wildlife monitoring systems and suggest promising directions for further improvements.

Method for Direct Localization of Multiple Impulse Acoustic Sources in Outdoor Environment

A method for the direct outdoor localization of multiple impulse acoustic sources by a distributed microphone array is proposed. This localization problem is of great interest for gunshot, firecracker and explosion detection localization in a civil environment, as well as for gun, mortar, small arms, artillery, sniper detection localization in military battlefield monitoring systems. Such a kind of localization is a complicated technical problem in many aspects. In such a scenario, the permutation of impulse arrivals on distributed microphones occurs, so the application of classical two-step localization methods, such as time-of-arrival (TOA), time-difference-of-arrival (TDOA), angle-of-arrival (AOA), fingerprint methods, etc., is faced with the so-called association problem, which is difficult to solve. The association problem does not exist in the proposed method for direct (one-step) localization, so the proposed method is more suitable for localization in a given acoustic scenario than the mentioned two-step localization methods. Furthermore, in the proposed method, direct localization is performed impulse by impulse. The observation interval used for the localization could not be arbitrarily chosen; it is limited by the duration of impulses. In the mathematical model formulated in the paper, atmospheric factors in acoustic signal propagation (temperature, pressure, etc.) are included. The results of simulations show that by using the proposed method, centimeter localization accuracy can be achieved.

Acoustic camera system for measuring ultrasound communication in mice

SummaryTo investigate biological mechanisms underlying social behaviors and their deficits, social communication via ultrasonic vocalizations (USVs) in mice has received considerable attention as a powerful experimental model. The advances in sound localization technology have facilitated the analysis of vocal interactions between multiple mice. However, existing sound localization systems are built around distributed-microphone arrays, which require a special recording arena and long processing time. Here, we report a novel acoustic camera system, USVCAM, which enables simpler and faster USV localization and assignment. The system comprises recently developed USV segmentation algorithms with a modification for overlapping vocalizations that results in high accuracy. Using USVCAM, we analyzed USV communications in a conventional home cage, and demonstrated novel vocal interactions in female ICR mice under a resident-intruder paradigm. The extended applicability and usability of USVCAM may facilitate future studies investigating typical and atypical vocal communication and social behaviors, as well as the underlying mechanisms.

Sound source localization using multiple ad hoc distributed microphone arrays.

Sound source localization is crucial for communication and sound scene analysis. This study uses direction-of-arrival estimates of multiple ad hoc distributed microphone arrays to localize sound sources in a room. An affine mapping between the independent array estimates and the source coordinates is derived from a set of calibration points. Experiments show that the affine model is sufficient to locate a source and can be calibrated to physical dimensions. A projection of the local array estimates increases localization accuracy, particularly further away from the calibrated region. Localization tests in three dimensions compare the affine approach to a nonlinear neural network.

Evaluating railway noise sources using distributed microphone array and graph neural networks

Identifying and evaluating railway noise sources is beneficial for establishing more targeted suppression and mitigation strategies. This paper proposes a novel approach to evaluate dominant railway noise sources for all bogie regions, by using an array of microphones and a physics-informed graph neural networks (GNN) model. The GNN model encodes not only the acoustic data collected by the microphone array but also the spatial relationship between noise sources and microphones, and it also considers the physical background of sound in terms of Doppler effect and acoustic attenuation . In-situ measurements of railway noises have been taken on an operating metro line to train and validate the GNN model which exhibits satisfactory performance in evaluating the noise sources of interest. The proposed approach enables a more flexible and cost-effective microphone array implementation strategy in comparison to the commonly adopted commercialized acoustic beamforming system.

Distributed Sound Transmission and Smart City Planning Management Based on Convolutional Neural Network

In this article, we provide a detection method based on a convolutional neural network algorithm. Preamble detection is very important in underwater acoustic communication. Only the preamble signal can trigger the receiving operation to effectively process the acoustic data received underwater. The prerequisite for specifying the joint position of the distributed acoustic multiarray is the mutual calibration calculation of each array. This paper designs and develops a prototype system for microphone array sound source positioning and provides an optimized system to obtain calibration source mapping points for distributed sound transmission mutual calibration, and for mutual distributed acoustic microphone arrays, use epipolar geometry and normalization algorithm for calibration to provide a basis for a mutual calibration calculation model and functional module design. Finally, perform numerical simulation verification. It provides a design plan and implementation method for the Internet-based smart city education system, aiming to achieve professional education and educational needs related to the Internet of Things. First, we will take Internet application technology as an example to analyze the functional requirements of the education system, including the positioning of human resource development, integrated sensors, C/C#/Java application development technology, and other technical means to realize the smart city education system. Based on the theory of business management, this paper studies the business planning issues of smart city construction. The main content is divided into three parts: business composition and management and planning management. This is an important part of business. It plays a decisive role in the overall planning and implementation stage and protects the implementation progress of the project and ensures the quality of the project by including important content.

3D Multiple Sound Source Localization by Proposed T-Shaped Circular Distributed Microphone Arrays in Combination with GEVD and Adaptive GCC-PHAT/ML Algorithms.

Multiple simultaneous sound source localization (SSL) is one of the most important applications in the speech signal processing. The one-step algorithms with the advantage of low computational complexity (and low accuracy), and the two-step methods with high accuracy (and high computational complexity) are proposed for multiple SSL. In this article, a combination of one-step-based method based on the generalized eigenvalue decomposition (GEVD), and a two-step-based method based on the adaptive generalized cross-correlation (GCC) by using the phase transform/maximum likelihood (PHAT/ML) filters along with a novel T-shaped circular distributed microphone array (TCDMA) is proposed for 3D multiple simultaneous SSL. In addition, the low computational complexity advantage of the GCC algorithm is considered in combination with the high accuracy of the GEVD method by using the distributed microphone array to eliminate spatial aliasing and thus obtain more appropriate information. The proposed T-shaped circular distributed microphone array-based adaptive GEVD and GCC-PHAT/ML algorithms (TCDMA-AGGPM) is compared with hierarchical grid refinement (HiGRID), temporal extension of multiple response model of sparse Bayesian learning with spherical harmonic (SH) extension (SH-TMSBL), sound field morphological component analysis (SF-MCA), and time-frequency mixture weight Bayesian nonparametric acoustical holography beamforming (TF-MW-BNP-AHB) methods based on the mean absolute estimation error (MAEE) criteria in noisy and reverberant environments on simulated and real data. The superiority of the proposed method is presented by showing the high accuracy and low computational complexity for 3D multiple simultaneous SSL.

Distributed Microphones Speech Separation by Learning Spatial Information With Recurrent Neural Network

With the development of deep neural networks, multi-channel speech separation techniques with fixed array geometries have achieved remarkable performance. However, distributed microphone array processing remains a challenging problem because it requires the network to be able to process inputs with varying dimensions. To address this problem, we propose a triple-path recurrent neural network (TPRNN) with multi-scale aggregation blocks for distributed microphone array multi-channel speech separation. First, we extend the single-channel dual-path recurrent neural network by additionally adding multi-scale aggregation blocks and adaptive feature fusion blocks. Next, a third path along the spatial dimension is introduced to model spatial information. By this means, TPRNN can alternately and iteratively perform inter-channel, intra-chunk, and inter-chunk modeling. The experimental results show that the proposed approach outperforms other advanced baselines for multi-channel speech separation and enhancement tasks using spatially distributed microphones.

Multiple Sound Source Localization in Three Dimensions Using Convolutional Neural Networks and Clustering Based Post-Processing

Sound source localization methods are successfully applied for various estimation tasks, such as tracking and detecting objects, aiming cameras, and navigating robots. However, large and usually complex distributed microphone arrays are used for three-dimensional acoustic source localization. This study proposes a convolutional neural network architecture for three-dimensional sound source localization using a single tetrahedral microphone array. A spectrum phase component of a microphone array signal was designed as the input of the model, while the output represents a three-dimensional space. The paper provides extensive experimental results of the given method on a semi-synthetic audio data set and a real-world microphone array. Furthermore, cluster-based post-processing has been shown to increase the accuracy of three-dimensional localization by more than 30%. The experimental results on a synthesized data set using the image source method showed 1.08 m localization uncertainties. The estimate of the investigated sound sources had a mean absolute error of 18.97° and elevation error of 48.49°. An additional advantage of the proposed method is the ability to predict the location of the sound source from a single signal analysis frame. This gives instant localization and is in line with many alternative applications. The proposed solution does not require intensive preprocessing of the audio signals and can be used as a video camera pointing system based on a microphone array. In the future, it would be relevant to investigate the localization performance of more than two sound sources, and the variable acoustic conditions could also be assessed.

A feature-based data association method for multiple acoustic source localization in a distributed microphone array

Multisource localization using time difference of arrival (TDOA) is challenging because the correct combination of TDOA estimates across different microphone pairs, corresponding to the same source, is usually unknown, which is termed as the data association problem. Moreover, many existing multisource localization techniques are originally demonstrated in two dimensions, and their extensions to three dimensions (3D) are not straightforward and would lead to much higher computational complexity. In this paper, we propose an efficient, feature-based approach to tackle the data association problem and achieve multisource localization in 3D in a distributed microphone array. The features are generated by using interchannel phase difference (IPD) information, which indicates the number of times each frequency bin across all time frames has been assigned to sources. Based on such features, the data association problem is addressed by correlating most similar features across different microphone pairs, which is executed by solving a two-dimensional assignment problem successively. Thereafter, the locations of multiple sources can be obtained by imposing a single-source location estimator on the resulting TDOA combinations. The proposed approach is evaluated using both simulated data and real-world recordings.

A Joint Diagonalization Based Efficient Approach to Underdetermined Blind Audio Source Separation Using the Multichannel Wiener Filter

Blind source separation (BSS) of audio signals aims to separate original source signals from their mixtures recorded by microphones. The applications include automatic speech recognition in a noisy/multi-speaker environment, hearing aids, and music analysis. Independent component analysis (ICA) can perform BSS efficiently, but it is basically inapplicable to the underdetermined case—the number of sources $\boldsymbol{>}$ the number of microphones. In contrast, a BSS approach using the multichannel Wiener filter (MWF) is applicable even to the underdetermined case, but conventional methods based on this approach—including full-rank spatial covariance analysis (FCA)—are highly inefficient. This is because these methods require massive numbers of matrix inversions to design the MWF. To obtain the best of both worlds, we take a joint diagonalization approach: We restrict spatial covariance matrices of all sources to the class of jointly diagonalizable matrices. This enables the above matrix inversions to be replaced by mere scalar inversions of the diagonal elements of diagonal matrices. Based on this, we present FastFCA and FastMNMF—efficient methods for underdetermined BSS. In an experiment, FastFCA was several orders of magnitude faster than FCA without sacrificing separation performance. We also present a unified framework for underdetermined and determined BSS, which highlights theoretical connections between various methods including ours. The efficiency of our BSS methods makes them suitable for large data (e.g., data augmentation for machine learning) or limited computational resources encountered in, e.g., hearing aids, distributed microphone arrays, and online BSS.

Multiple Acoustic Source Localization in Microphone Array Networks

The problem of multiple acoustic source localization using observations from a microphone array network is investigated in this article. Multiple source signals are assumed to be window-disjoint-orthogonal (WDO) on the time-frequency (TF) domain and time delay of arrival (TDOA) measurements are extracted at each TF bin. A Bayesian network model is then proposed to jointly assign the measurements to different sources and estimate the acoustic source locations. Considering that the WDO assumption is usually violated under reverberant and noisy environments, we construct a relational network by coding the distance information between the distributed microphone arrays such that adjacent arrays have higher probabilities of observing the same acoustic source, which is able to mitigate the miss detection issues in adverse environments. A Laplace approximate variational inference method is introduced to estimate the hidden variables in the proposed Bayesian network model. Both simulations and real data experiments are performed. The results show that our proposed method is able to achieve better source localization accuracy than existing methods.

Distributed multiple speaker tracking based on time delay estimation in microphone array network

Multiple speaker tracking in distributed microphone array (DMA) network is a challenging task. A critical issue for multiple speaker scenarios is to distinguish the ambiguous observation and associate it to the corresponding speaker, especially under reverberant and noisy environments. To address the problem, a distributed multiple speaker tracking method based on time delay estimation in DMA is proposed in this study. Specifically, the time delay estimated by the generalised cross-correlation function is treated as an observation. In order to distinguish the observation for each speaker, the possible time delays, refer to as candidates, are extracted based on data association technique. Considering the ambient influence, a time delay estimation strategy is designed to calculate the time delay for each speaker from the candidates. Finally, only the reliable time delays in DMA are propagated throughout the whole network by diffusion fusion algorithm and used for updating the speakers' state within the distributed Kalman filter framework. The proposed approach can track multiple speakers successfully in a non-centralised manner under reverberant and noisy environments. Simulation results indicate that, compared with other methods, the proposed method can achieve a smaller root mean square error for multiple speaker tracking, especially in adverse conditions.

Battlefield acoustics: Applications and challenges

Some primary objectives of research and development for battlefield acoustic sensing include the optimization of sensors, arrays, and signal processing algorithms for the detection, localization, and classification of diverse broadband and transient threats in complex environments. Time-difference-of-arrival and frequency domain signal processing techniques using distributed microphone arrays allow sound source localization and tracking through triangulation in the 3-D space. This presentation will introduce general applications of broadband acoustic and infrasonic microphone arrays, acoustic particle velocity sensors, geophones, propagation environment effects and wind mitigation methods.

Comparison of beamforming algorithms for microphone arrays in Matlab

Subject of Research. The paper considers the main quantitative characteristics of microphone arrays in their performance analysis and beamforming algorithms applied for their study. Methods. The principal characteristics, calculation approaches and evaluations were examined. The most widespread beamforming algorithms were analyzed, such as: Delay-and-Sum (DAS), minimum variance distortionless response (MVDR), Frost’s algorithm or linearly constrained minimum variance (LCMV), generalized sidelobe canceller (GSC). The calculations and comparative analysis of the algorithms were performed in Matlab simulation environment. The following quantitative characteristics were obtained: signal-to-noise ratio, signal-interference-noise ratio, directivity index, directivity characteristics, and beamwidth. Main Results. Algorithm comparison results are presented on the example of a linear microphone array. The linearly constrained minimum variance algorithm has shown the most satisfactory results. The simulation results must be clarified by real experiments. Practical Relevance. The results of the work can be used for analysis of more complex microphone arrays and acoustic structures, such as microphone arrays with “floating” geometry and distributed microphone arrays.

Multi-Hypothesis Square-Root Cubature Kalman Particle Filter for Speaker Tracking in Noisy and Reverberant Environments

In this paper, a multi-hypothesis square-root cubature Kalman particle filter (MH-SRCKPF) is proposed for speaker tracking in noisy and reverberant environments with distributed microphone arrays. The conventional cubature Kalman particle filter (CKPF) uses the cubature Kalman filter (CKF) to generate its proposal for particle sampling. Such a proposal incorporates only one observation from a certain localization function for the state estimation, which is vulnerable to noise or reverberation, yielding the degraded tracking performance. To tackle the problem, by incorporating multiple possible observations into CKF for the proposal, a multi-hypothesis CKPF (MH-CKPF) algorithm is first developed. Furthermore, to improve the numerical stability, an MH-SRCKPF algorithm is developed, where the state estimate and the square root of the error covariance are propagated at each time. Finally, the MH-SRCKPF is applied to the speaker tracking problems in distributed microphone arrays. Experimental results demonstrate that the proposed MH-SRCKPF outperforms the competing methods in the presence of noise and reverberation. Meanwhile, by propagating the square root of the state covariance, the proposed method exhibits attractive numerical characteristics.