Microphone Array Data Research Articles

MIRACLE—a microphone array impulse response dataset for acoustic learning

This work introduces a large dataset comprising impulse responses of spatially distributed sources within a plane parallel to a planar microphone array. The dataset, named MIRACLE, encompasses 856,128 single-channel impulse responses and includes four different measurement scenarios. Three measurement scenarios were conducted under anechoic conditions. The fourth scenario includes an additional specular reflection from a reflective panel. The source positions were obtained by uniformly discretizing a rectangular source plane parallel to the microphone for each scenario. The dataset contains three scenarios with a spatial resolution of 23mm\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$23\\,\ extrm{mm}$$\\end{document} at two different source-plane-to-array distances, as well as a scenario with a resolution of 5mm\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5\\,\ extrm{mm}$$\\end{document} for the shorter distance. In contrast to existing room impulse response datasets, the accuracy of the provided source location labels is assessed and additional metadata, such as the directivity of the loudspeaker used for excitation, is provided. The MIRACLE dataset can be used as a benchmark for data-driven modelling and interpolation methods as well as for various acoustic machine learning tasks, such as source separation, localization, and characterization. Two timely applications of the dataset are presented in this work: the generation of microphone array data for data-driven source localization and characterization tasks and data-driven model order reduction.

Precise acoustic drone localization and tracking via drone noise: Steered response power - phase transform around harmonics

This study introduces a new method for precise acoustic drone localization and tracking via the noise generated by the drone. Drone noises include harmonic frequency components which are related to the rotational speed of the propeller and the number of blades. This study integrates utilization of the frequency components around harmonics into Steered Response Power - Phase Transform (SRP-PHAT). First, a custom discrete Fourier transform (namely DFT-Harmonics) is defined which concentrates only on the vicinities of harmonics to capture the frequency components possibly related to the drone sound. Then, DFT-Harmonics is integrated into SRP-PHAT, which is named SRP-Harmonics. The benefits of SRP-Harmonics are explained and illustrated via SRP maps and videos. Experiments with real microphone array data show that SRP-Harmonics is precise in localizing and tracking a drone, while the ordinary SRP-PHAT can not be reasonably successful. Moreover, SRP-Harmonics after Kaiser window can maintain its performance even when significant level of artificial white noise or natural wind noise exists in the data.

Noise measurement of a quadrotor drone in an anechoic chamber

Operation of drones near densely populated urban areas can cause significant noise pollution issues. Drone noise measurements in controllable environments are essential for noise certification. This study presents experimental measurement results of the noise radiated from a quadrotor drone at various working conditions. The experiment was conducted in an anechoic chamber. The noise was measured by microphone arrays with a total of 15 free-field microphones. The instantaneous position of the drone was recorded by an optical motion capture system. A criterion on the measurement distance to ensure acoustic far-field condition is proposed and justified by the measurement results. Noise directivity maps on a spherical surface were reconstructed using the microphone array data at different observer angles. Correlations were found between the directivity pattern and the airframe geometry. Additionally, the effect of loading on noise was analyzed through parametric tests in which the drone was loaded with additional mass. The unbalanced loading case was also considered. For each test condition, multiple tests were conducted to reduce the statistical errors and uncertainties of the measurements, and the results present high repeatability.

Recent research advances on high-speed railway noise

Around 42,000 km high-speed railways are in operation in China at speeds up to 350 km/h. 400 km/h trains are also under development. Huge resources have been put into research on high-speed railway noise. This paper outlines advances achieved in several key projects by covering the following aspects: measurement results showing typical characteristics of high-speed railway noise; analyses on source contribution to pass-by noise based microphone array data; modelling of high-speed rolling, aerodynamic and viaduct bridge noise; prediction of the vibro-acoustic behaviour of car-body with poro-elastic media; and finally, analyses of vibration transmission in the suspension/bogie system.

A framework for generating large-scale microphone array data for machine learning

The use of machine learning for localization of sound sources from microphone array data has increased rapidly in recent years. Newly developed methods are of great value for hearing aids, speech technologies, smart home systems or engineering acoustics. The existence of openly available data is crucial for the comparability and development of new data-driven methods. However, the literature review reveals a lack of openly available datasets, especially for large microphone arrays. This contribution introduces a framework for generation of acoustic data for machine learning. It implements tools for the reproducible random sampling of virtual measurement scenarios. The framework allows computations on multiple machines, which significantly speeds up the process of data generation. Using the framework, an example of a development dataset for sound source characterization with a 64-channel array is given. A containerized environment running the simulation source code is openly available. The presented approach enables the user to calculate large datasets, to store only the features necessary for training, and to share the source code which is needed to reproduce datasets instead of sharing the data itself. This avoids the problem of distributing large datasets and enables reproducible research.

Indoor sound source localization under reverberation by extracting the features of sample covariance

Three dimensional indoor sound source localization is well known as a challenging task due to the complicated mechanics of reverberation. The conventional model-based methods generally require the prior knowledge of the microphone array geometry, the environment’s parameters, and the statistics of the signal and noise. On the other hand, the data-based methods could work in the absence of all/part of the aforementioned prior knowledge, while the localization performance depends on the established features of the data and the nature of neural network. In this work, a feature vector constructed from the sample covariance matrix of the microphone array data is fed to a designed deep neural network. Numerical simulation results show that the proposed method outperforms [21] in both ranging and direction finding, hence locating a human-speech source in a three-dimensional reverberant room space. An average relative localization error below 3% can be reached, which is robust to the room reverberation level of T60 from 0 to 600 ms when the signal-to-noise-ratio is not below 0 dB. Moreover, the robustness of the proposed method against different scenarios is verified: the increment of relative localization error is below 5% for various room dimensions, below 10% for different speech signals, and below 3% for microphone array offsets.

Fast grid-free strength mapping of multiple sound sources from microphone array data using a Transformer architecture.

Conventional microphone array methods for the characterization of sound sources that require a focus-grid are, depending on the grid resolution, either computationally demanding or limited in reconstruction accuracy. This paper presents a deep learning method for grid-free source characterization using a Transformer architecture that is exclusively trained with simulated data. Unlike previous grid-free model architectures, the presented approach requires a single model to characterize an unknown number of ground-truth sources. The model predicts a set of source components, spatially arranged in clusters. Integration over the predicted cluster components allows for the determination of the strength for each ground-truth source individually. Fast and accurate source mapping performance of up to ten sources at different frequencies is demonstrated and strategies to reduce the training effort at neighboring frequencies are given. A comparison with the established grid-based CLEAN-SC and a probabilistic sparse Bayesian learning method on experimental data emphasizes the validity of the approach.

Binaural reproduction of dummy head and spherical microphone array data-A perceptual study on the minimum required spatial resolution.

Dynamic binaural synthesis requires binaural room impulse responses (BRIRs) for each head orientation of the listener. Such BRIRs can either be measured with a dummy head or calculated from the spherical microphone array (SMA) data. Because the dense dummy head measurements require enormous effort, alternatively sparse measurements can be performed and then interpolated in the spherical harmonics domain. The real-world SMAs, on the other hand, have a limited number of microphones, resulting in spatial undersampling artifacts. For both of the methods, the spatial order N of the underlying sampling grid influences the reproduction quality. This paper presents two listening experiments to determine the minimum spatial order for the direct sound, early reflections, and reverberation of the dummy head or SMA measurements required to generate the horizontally head-tracked binaural synthesis perceptually indistinguishable from a high-resolution reference. The results indicate that for direct sound, N = 9-13 is required for the dummy head BRIRs, but significantly higher orders of N = 17-20 are required for the SMA BRIRs. Furthermore, significantly lower orders are required for the late parts with N = 4-5 for the early reflections and reverberation of the dummy head BRIRs but N = 12-13 for the early reflections and N = 6-9 for the reverberation of the SMA BRIRs.

Efficient binaural rendering of spherical microphone array data by linear filtering

High-quality rendering of spatial sound fields in real-time is becoming increasingly important with the steadily growing interest in virtual and augmented reality technologies. Typically, a spherical microphone array (SMA) is used to capture a spatial sound field. The captured sound field can be reproduced over headphones in real-time using binaural rendering, virtually placing a single listener in the sound field. Common methods for binaural rendering first spatially encode the sound field by transforming it to the spherical harmonics domain and then decode the sound field binaurally by combining it with head-related transfer functions (HRTFs). However, these rendering methods are computationally demanding, especially for high-order SMAs, and require implementing quite sophisticated real-time signal processing. This paper presents a computationally more efficient method for real-time binaural rendering of SMA signals by linear filtering. The proposed method allows representing any common rendering chain as a set of precomputed finite impulse response filters, which are then applied to the SMA signals in real-time using fast convolution to produce the binaural signals. Results of the technical evaluation show that the presented approach is equivalent to conventional rendering methods while being computationally less demanding and easier to implement using any real-time convolution system. However, the lower computational complexity goes along with lower flexibility. On the one hand, encoding and decoding are no longer decoupled, and on the other hand, sound field transformations in the SH domain can no longer be performed. Consequently, in the proposed method, a filter set must be precomputed and stored for each possible head orientation of the listener, leading to higher memory requirements than the conventional methods. As such, the approach is particularly well suited for efficient real-time binaural rendering of SMA signals in a fixed setup where usually a limited range of head orientations is sufficient, such as live concert streaming or VR teleconferencing.

Direction-of-arrival estimation with blind surface impedance compensation for spherical microphone array.

The direction-of-arrival (DoA) estimation from spherical microphone array data requires mode strength compensation for removing the dependency on the array surface impedance. However, in practice, mismatches exist in the actual and modeled surface impedances and degrade the DoA estimation performance. Herein, a non-parametric beamformer that can accurately estimate DoAs under the mode strength mismatch is proposed. The technique utilizes a block-diagonal covariance matrix and order-dependent normalization of spherical harmonic coefficients. The proposed method tested for various surface impedance mismatches and DoAs shows robust estimation performance with reduced dependency on the array surface impedance.

Audio-Visual Multi-Channel Integration and Recognition of Overlapped Speech

Automatic speech recognition (ASR) technologies have been significantly advanced in the past few decades. However, recognition of overlapped speech remains a highly challenging task to date. To this end, multi-channel microphone array data are widely used in current ASR systems. Motivated by the invariance of visual modality to acoustic signal corruption and the additional cues they provide to separate the target speaker from the interfering sound sources, this paper presents an audio-visual multi-channel based recognition system for overlapped speech. It benefits from a tight integration between a speech separation front-end and recognition back-end, both of which incorporate additional video input. A series of audio-visual multi-channel speech separation front-end components based on TF masking , Filter&Sum and mask-based MVDR neural channel integration approaches are developed. To reduce the error cost mismatch between the separation and the recognition components, the entire system is jointly fine-tuned using a multi-task criterion interpolation of the scale-invariant signal to noise ratio (Si-SNR) with either the connectionist temporal classification (CTC), or lattice-free maximum mutual information (LF-MMI) loss function. Experiments suggest that: the proposed audio-visual multi-channel recognition system outperforms the baseline audio-only multi-channel ASR system by up to 8.04% (31.68% relative) and 22.86% (58.51% relative) absolute WER reduction on overlapped speech constructed using either simulation or replaying of the LRS2 dataset respectively. Consistent performance improvements are also obtained using the proposed audio-visual multi-channel recognition system when using occluded video input with the lip region randomly covered up to 60%.

Separation of rail and wheel contributions to pass-by noise with sparse regularization methods

This paper proposes a method for separating rail and wheel noise contributions via sparse regularization of microphone array data. The underlying idea is to promote sparse solutions, which jointly approximate the two noise contributions with few non-zero coefficients. The main hypothesis is that the acoustic field radiated by the rail is sparse in a dictionary of plane waves, that the acoustic field radiated by the wheels is sparse in a dictionary of moving sources, and that both acoustic fields are dense in the opposite dictionaries. How well this happens is studied with the coherence between the plane waves and the moving sources. The strength of the proposed approach is that it does not require static tests prior to the pass-by. The separation is performed in three main steps, executed in the time-frequency domain. First, the rail contribution is separated from the total pass-by noise using matching pursuit optimization, promoting solutions with a limited number of plane waves per frequency. Second, the residual between the total pass-by noise and the estimated rail noise is calculated. And third, the wheel contribution is separated from the residual via ℓ1-norm minimization, promoting solutions that are row-sparse at all frequencies. The separation performance is investigated with synthetic data, and validated with experimental data against reference predictions with the TWINS software, for pass-by noise measurements of trains running at 40, 80 and 160 km/h.

Perceptual Evaluation of Mitigation Approaches of Impairments due to Spatial Undersampling in Binaural Rendering of Spherical Microphone Array Data

Spherical microphone arrays (SMAs) are widely used to capture spatial sound fields that can then be rendered in various ways as a virtual acoustic environment (VAE) including headphone-based binaural synthesis. Several practical limitations have a significant impact on the fidelity of the rendered VAE. The finite number of microphones of SMAs leads to spatial undersampling of the captured sound field, which, on the one hand, induces spatial aliasing artifacts and, on the other hand, limits the order of the spherical harmonics (SH) representation. Several approaches have been presented in the literature that aim to mitigate the perceptual impairments due to these limitations. In this article, we present a listening experiment evaluating the perceptual improvements of binaural rendering of undersampled SMA data that can be achieved using state-of-the-art mitigation approaches. In particular, we examined the Magnitude Least-Squares algorithm, the Bandwidth Extraction Algorithm for Microphone Arrays, Spherical Head Filters, SH Tapering, and a newly proposed equalization filter. In the experiment, subjects rated the perceived differences between a dummy head and the corresponding SMA auralization. We found that most mitigation approaches lead to significant perceptual improvements, even though audible differences to the reference remain.

Extended Vector-Based EB-ESPRIT Method

The estimation of direction of arrivals (DoAs) from spherical microphone array data is one of the key issues in extracting source information from all-around audio recordings. One such technique is the eigenbeam estimation of signal parameters via the rotational invariance technique (EB-ESPRIT), which separates the signal subspace related to the stationary sound field and then directly estimates DoAs of multiple sound sources. EB-ESPRIT has been evolved in many different ways by involving different types of recurrence relations of spherical harmonics, all of which are able to identify DoAs of a limited number of sources that are noticeably smaller than the number of finite-order spherical harmonic coefficients recorded. In this work, we report that it is possible to go beyond the known limits of detectable sources. The proposed formula is also based on conventional recurrence relations and probably permits to reach the ultimate limit by additional constraints of the signal parameters that can better exploit the highest-order coefficients. Monte-Carlo simulations conducted with various source positions and signal-to-noise ratios (SNRs) reveal that the proposed technique can detect more sources with insignificant loss in estimation performance and robustness.

A deep learning method for grid-free localization and quantification of sound sources.

In this contribution it is examined whether the use of deep neural networks can lead to an accurate characterization of single point sources from microphone array data. Based on conventional beamforming maps, the proposed method aims at estimating the source coordinates and the strength. The residual network architecture, a well-established model in the field of image recognition, is successfully applied to this task. The investigation reveals a method that fast and accurately renders the position and strength of an unknown source. Moreover, the accuracy of the position estimation is higher than the grid resolution of the beamforming map.

Multiple sound-source localization and identification with a spherical microphone array and lavalier microphone data

Spherical microphone arrays have attained a considerable amount of interest for their dynamic beamforming and tracking capabilities. Although many robust algorithms have been developed to accurately localize individual sources, multiple source localization, especially for concurrent or colocated sources, has proved to be a challenging topic. This project demonstrates a technique to isolate individual concurrent sources by using lavalier microphone data to determine the relative difference in signal energy between sources. This data is then incorporated into the source-tracking algorithm governing the beamforming spherical array, a particle filter, by allowing the rejection of frames containing concurrent source information. This improves the tracking ability of the system. [Work supported by Rensselaer HASS Fellowship, NSF No. 1631674, and CISL.]

Perception and prediction of apparent source width and listener envelopment in binaural spherical microphone array auralizations.

This article deals with the assessment and prediction of the reproduction quality when binaurally auralizing spherical microphone array data for room simulation applications. The auralization is perceptually assessed in a listening experiment using the two attributes, apparent source width (ASW) and listener envelopment (LEV), for spatial quality description, whereas the technical analysis employs a psychoacoustically motivated model for room acoustical perception (RAP) which is specifically designed to estimate ASW and LEV. Both analyses focus on the array configuration in terms of varying modal resolutions and its influence on the spatial reproduction quality. The auralizations comprise three simulated environments, i.e., free-field sound fields as well as a dry and a reverberant room. Ten different audio signals are used as test material. Perceptual results show that the array configuration strongly influences the perception of ASW and LEV which also depends on the reflection properties of the simulated room. The ASW and LEV predictions by the RAP model correlate well with the results from the listening experiment.

Application of synthetic array techniques for improved simulations of hot supersonic jet noise

The synthetic array technique has previously been used to provide insight for developing improved microphone array data processing techniques. In recent work, however, it has also been successfully applied to diagnose the source of unexpected (and nonphysical) tones which arose in unsteady computational fluid dynamics (CFD) simulations of hot supersonic jets. The insight obtained from this method has allowed the authors to rapidly arrive at a strategy that greatly improves the results. This success opens up additional areas where the synthetic array technique can find application.

Open-source software for the application of microphone array methods

The open-source Python library Acoular is aimed at the processing of microphone array data. It features a number of algorithms for acoustic source characterization in time domain and frequency domain. The modular, object-oriented architecture allows for flexible programming and a multitude of applications. This includes the processing of measured array data, the mapping of sources, the filtering of subcomponent noise, and the generation of synthetic data for test purposes. Several examples illustrating its versatility are given, as well as one example for implementing a new algorithm into the package.

Multiple Frequency and Source Angle Estimation by Gaussian Mixture Model with Modified Microphone Array Data Model

Multiple Frequency and Source Angle Estimation by Gaussian Mixture Model with Modified Microphone Array Data Model