Microphone Array Signal Research Articles

Time-Aligned Spatial Upsampling of Spherical Microphone Array Recordings

With the Spherical Array Interpolation by Time Alignment (SARITA) method, the authors introduced an approach for spatial upsampling of spherical microphone array (SMA) signals (T. Lübeck, J. M. Arend, and C. Pörschmann, IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1163–1174 [2023]). The basic idea of this method is to perform the interpolation after time-aligning adjacent microphone signals. The upsampled SMA signals can be represented as spherical harmonic coefficients of much higher spatial order than is possible with the sparsely measured signals. Instead of impulse responses, the method is now applied to SMA recordings. Binaural decoding of upsampled SMA recordings is compared technically and perceptually to a baseline Ambisonics decoding. The results show that the SARITA method can also be applied to low-order array recordings and significantly improves their binaural reproduction.

Online adaptation of fourier series-based acoustic transfer function model and its application to sound source localization and separation

In this paper, we propose an online adaptation method for Fourier series-based acoustic transfer function (FS-ATF) models for robot audition systems using microphone array signal processing. The ATF represents the characteristics of signal propagation from a sound source to a microphone, which is essential for sound source localization (SSL) and sound source separation (SSS). For practical applications in dynamically changing real-world environments, ATF models must satisfy two criteria: (1) they must be adaptable to changes in the acoustic environment; and (2) they must be lightweight to be suitable for resource-constrained systems, such as robots with limited memory and computational capacity. The proposed method addresses these challenges using Fourier series expansions for interpolation, which reduces the memory footprint of the ATF model and facilitates online adaptation to acoustic environmental changes. The experimental results demonstrate that the proposed online adaptation method both improves the SSL and SSS performance while reducing the size of the ATF model, which represents a significant improvement over existing online ATF adaptation methods.

Drone auralization model with statistical synthesis of amplitude and frequency modulations

This paper presents a drone auralization model that reproduces the spectro-temporal and spatial characteristics of a drone during flight. Focusing on perceptual plausibility, the time-variant processes are modeled by taking into account the statistical amplitude and frequency modulation distributions of a reference drone sound. For completeness, the far-field directivity is extracted based on time-variant wave backpropagation from microphone array signals. Both components consider a combined level calibration with regard to the reconstructed sound pressure on a spherical surface around the source. With regard to reproducibility, this paper is accompanied by supplemental data to present a synthesis model including the oscillator and digital filter coefficients for procedural audio synthesis. From evaluation, the model shows good agreement by comparison of psychoacoustic measures of the synthesized drone to a recorded reference. The drone auralization model can be applied in future research on urban soundscapes where Unmanned Aerial Vehicles (UAV) may appear in a great variety of use cases. Furthermore, it can deliver input data for simulation tools where the spatial radiation characteristics of a drone should be included, such as the development of array-based drone detection.

Physics-informed neural network assisted spherical microphone array signal processing

Thanks to their rotational symmetry that facilitates three-dimensional signal processing, spherical microphone arrays are the common array apertures used for spatial audio and acoustic applications. However, practical implementations of spherical microphone arrays suffer from two issues. First, at high frequency range, a large number of sensors are needed to accurately capture a sound field. Second, the accompanying signal processing algorithm, i.e., the spherical harmonic decomposition method, requires a variable radius array or a rigid surface array to circumvent the spherical Bessel function nulls. Such arrays are hard to design and introduce a scattering field. To address these issues, this paper proposes to assist a spherical microphone array with a physics-informed neural network (PINN) for three-dimensional signal processing. The PINN models the sound field around the array based on the sensor measurements and the acoustic wave equation, augmenting the sound field information captured by the array through prediction. This makes it possible to analyze a high frequency sound field with a reduced number of sensors and avoid the spherical Bessel function nulls with a simple single radius open-sphere microphone array.

Microphone Array Speech Enhancement Via Beamforming Based Deep Learning Network

In general, in-car speech enhancement is an application of the microphone array speech enhancement in particular acoustic environments. Speech enhancement inside the moving cars is always an interesting topic and the researchers work to create some modules to increase the quality of speech and intelligibility of speech in cars. The passenger dialogue inside the car, the sound of other equipment, and a wide range of interference effects are major challenges in the task of speech separation in-car environment. To overcome this issue, a novel Beamforming based Deep learning Network (Bf-DLN) has been proposed for speech enhancement. Initially, the captured microphone array signals are pre-processed using an Adaptive beamforming technique named Least Constrained Minimum Variance (LCMV). Consequently, the proposed method uses a time-frequency representation to transform the pre-processed data into an image. The smoothed pseudo-Wigner-Ville distribution (SPWVD) is used for converting time-domain speech inputs into images. Convolutional deep belief network (CDBN) is used to extract the most pertinent features from these transformed images. Enhanced Elephant Heard Algorithm (EEHA) is used for selecting the desired source by eliminating the interference source. The experimental result demonstrates the effectiveness of the proposed strategy in removing background noise from the original speech signal. The proposed strategy outperforms existing methods in terms of PESQ, STOI, SSNRI, and SNR. The PESQ of the proposed Bf-DLN has a maximum PESQ of 1.98, whereas existing models like Two-stage Bi-LSTM has 1.82, DNN-C has 1.75 and GCN has 1.68 respectively. The PESQ of the proposed method is 1.75%, 3.15%, and 4.22% better than the existing GCN, DNN-C, and Bi-LSTM techniques. The efficacy of the proposed method is then validated by experiments.

A fast data-driven method for inverse microphone array signal processing

Microphone arrays have long been used to characterize and locate sound sources. However, existing algorithms for processing the signals are computationally expensive and, consequently, different methods need to be explored. Recently, the trained iterative soft thresholding algorithm (TISTA), a data-driven solver for inverse problems, was shown to improve on existing approaches. Here, a more in-depth analysis of its robustness and frequency dependence is provided using synthesized as well as real measurement data. It is demonstrated that TISTA yields favorable results in comparison to a covariance matrix fitting inverse method, especially for large numbers of sources.

Synthesizing coherence loss by atmospheric turbulence in virtual microphone array signals.

Phased microphone array methods are increasingly used to localize and quantify noise sources of aircraft under flight condition. However, beamforming results suffer from loss of image resolution and corruption of sound levels due to atmospheric turbulence causing coherence loss between microphones. A synthesis method is presented that reproduces these effects in a virtual environment. Sound propagation through turbulent atmosphere is described by models by Ostashev and Wilson and by von Kármán turbulence spectra. Spatial coherence is calculated based on the parabolic equation for statistically inhomogeneous, isotropic turbulence. Decorrelation of signals is achieved by time-varying mixing of mutually independent signals with identical PSD based on coherence factors. The concept of auralization is employed to account for propagation delay, geometrical spreading, Doppler effect, air absorption, and ground effect. The application is demonstrated for a virtual 56 m aperture microphone array. The impact of different meteorological conditions on the beamforming and deconvoluted results are presented. For increasing turbulence strength, the results show decreasing sound levels and increasingly blurred images. The proposed method allows us to reproduce the effects of turbulence-induced coherence loss in phased microphone array measurements and to optimize array designs and algorithms in a virtual, controllable environment.

Spatial Upsampling of Sparse Spherical Microphone Array Signals

We present a method for spatial upsampling of signals captured with spherical microphone arrays with a limited number of microphones. The upsampling is performed by adding virtual microphone signals using interpolation between the measured array signals. For interpolation, the differences in amplitude and time of neighbored signals are weighted separately. For this, the time differences are determined with a broadband cross-correlation. After the upsampling, the array signals can be represented as spherical harmonics (SH) coefficients at significantly higher orders. This work introduces the proposed upsampling procedure and presents a technical evaluation based on simulated and measured sound fields. Thereby, we focus on the binaural decoding of the upsampled SH signals. For perceptual evaluation, we present the results of a listening experiment evaluating the binaural reproduction of the upsampled signals compared to a baseline Ambisonics decoding and the state-of-the-art decoding with Magnitude Least-Squares optimization. Both the technical and perceptual evaluations consider only array impulse responses, but the algorithm could generally be applied to array streams. The evaluation shows that the proposed method significantly improves the binaural reproduction of low-order spherical microphone array captures compared to established methods.

Perceptual evaluation of approaches for binaural reproduction of non-spherical microphone array signals

Microphone arrays consisting of sensors mounted on the surface of a rigid, spherical scatterer are popular tools for the capture and binaural reproduction of spatial sound scenes. However, microphone arrays with a perfectly spherical body and uniformly distributed microphones are often impractical for the consumer sector, in which microphone arrays are generally mounted on mobile and wearable devices of arbitrary geometries. Therefore, the binaural reproduction of sound fields captured with arbitrarily shaped microphone arrays has become an important field of research. In this work, we present a comparison of methods for the binaural reproduction of sound fields captured with non-spherical microphone arrays. First, we evaluated equatorial microphone arrays (EMAs), where the microphones are distributed on an equatorial contour of a rigid, spherical 1. Second, we evaluated a microphone array with six microphones mounted on a pair of glasses. Using these two arrays, we conducted two listening experiments comparing four rendering methods based on acoustic scenes captured in different rooms2. The evaluation includes a microphone-based stereo approach (sAB stereo), a beamforming-based stereo approach (sXY stereo), beamforming-based binaural reproduction (BFBR), and BFBR with binaural signal matching (BSM). Additionally, the perceptual evaluation included binaural Ambisonics renderings, which were based on measurements with spherical microphone arrays. In the EMA experiment we included a fourth-order Ambisonics rendering, while in the glasses array experiment we included a second-order Ambisonics rendering. In both listening experiments in which participants compared all approaches with a dummy head recording we applied non-head-tracked binaural synthesis, with sound sources only in the horizontal plane. The perceived differences were rated separately for the attributes timbre and spaciousness. Results suggest that most approaches perform similarly to the Ambisonics rendering. Overall, BSM, and microphone-based stereo were rated the best for EMAs, and BFBR and microphone-based stereo for the glasses array.

Multiple Sound Source Localization, Separation, and Reconstruction by Microphone Array: A DNN-Based Approach

Synchronistical localization, separation, and reconstruction for multiple sound sources are usually necessary in various situations, such as in conference rooms, living rooms, and supermarkets. To improve the intelligibility of speech signals, the application of deep neural networks (DNNs) has achieved considerable success in the area of time-domain signal separation and reconstruction. In this paper, we propose a hybrid microphone array signal processing approach for the nearfield scenario that combines the beamforming technique and DNN. Using this method, the challenge of identifying both the sound source location and content can be overcome. Moreover, the use of a sequenced virtual sound field reconstruction process enables the proposed approach to be quite suitable for a sound field which contains a dominant, stronger sound source and masked, weaker sound sources. Using this strategy, all traceable, mainly sound, sources can be discovered by loops in a given sound field. The operational duration and accuracy of localization are further improved by substituting the broadband weighted multiple signal classification (BW-MUSIC) method for the conventional delay-and-sum (DAS) beamforming algorithm. The effectiveness of the proposed method for localizing and reconstructing speech signals was validated by simulations and experiments with promising results. The localization results were accurate, while the similarity and correlation between the reconstructed and original signals was high.

Multiple Sound Source Localization in Three Dimensions Using Convolutional Neural Networks and Clustering Based Post-Processing

Sound source localization methods are successfully applied for various estimation tasks, such as tracking and detecting objects, aiming cameras, and navigating robots. However, large and usually complex distributed microphone arrays are used for three-dimensional acoustic source localization. This study proposes a convolutional neural network architecture for three-dimensional sound source localization using a single tetrahedral microphone array. A spectrum phase component of a microphone array signal was designed as the input of the model, while the output represents a three-dimensional space. The paper provides extensive experimental results of the given method on a semi-synthetic audio data set and a real-world microphone array. Furthermore, cluster-based post-processing has been shown to increase the accuracy of three-dimensional localization by more than 30%. The experimental results on a synthesized data set using the image source method showed 1.08 m localization uncertainties. The estimate of the investigated sound sources had a mean absolute error of 18.97° and elevation error of 48.49°. An additional advantage of the proposed method is the ability to predict the location of the sound source from a single signal analysis frame. This gives instant localization and is in line with many alternative applications. The proposed solution does not require intensive preprocessing of the audio signals and can be used as a video camera pointing system based on a microphone array. In the future, it would be relevant to investigate the localization performance of more than two sound sources, and the variable acoustic conditions could also be assessed.

Parametric Ambisonic Encoding of Arbitrary Microphone Arrays

This article proposes a parametric signal-dependent method for the task of encoding microphone array signals into Ambisonic signals. The proposed method is presented and evaluated in the context of encoding a simulated seven-sensor microphone array, which is mounted on an augmented reality headset device. Given the inherent flexibility of the Ambisonics format, and its popularity within the context of such devices, this array configuration represents a potential future use case for Ambisonic recording. However, due to its irregular geometry and non-uniform sensor placement, conventional signal-independent Ambisonic encoding is particularly limited. The primary aims of the proposed method are to obtain Ambisonic signals over a wider frequency band-width, and at a higher spatial resolution, than would otherwise be possible through conventional signal-independent encoding. The proposed method is based on a multi-source sound-field model and employs spatial filtering to divide the captured sound-field into its individual source and directional ambient components, which are subsequently encoded into the Ambisonics format at an arbitrary order. It is demonstrated through both objective and perceptual evaluations that the proposed parametric method outperforms conventional signal-independent encoding in the majority of cases.

Binaural rendering from microphone array signals of arbitrary geometry.

A method of binaural rendering from microphone array signals of arbitrary geometry is proposed. To reproduce binaural signals from microphone array recordings at a remote location, a spherical microphone array is generally used for capturing a soundfield. However, owing to the lack of flexibility in the microphone arrangement, the single spherical array is sometimes impractical for estimating a large region of a soundfield. We propose a method based on harmonic analysis of infinite order, which allows the use of arbitrarily placed microphones. In the synthesis of the estimated soundfield, a spherical-wave-decomposition-based binaural rendering is also formulated to take into consideration the distance in measuring head-related transfer functions. We develop and evaluate a composite microphone array consisting of multiple small arrays. Experimental results including those of listening tests indicate that our proposed method is robust against change in listening position in the recording area.

Binaural Auralization of Microphone Array Room Impulse Responses Using Causal Wiener Filtering

Binaural room auralization involves Binaural Room Impulse Responses (BRIRs). Dynamic binaural synthesis (i.e., head-tracked presentation) requires BRIRs for multiple head poses. Artificial heads can be used to measure BRIRs, but BRIR modeling from microphone array room impulse responses (RIRs) is becoming popular since personalized BRIRs can be obtained for any head pose with low extra effort. We present a novel framework for estimating a binaural signal from microphone array signals, using causal Wiener filtering and polynomial matrix formalism. The formulation places no explicit constraints on the geometry of the microphone array and enables directional weighting of the estimation error. A microphone noise model is used for regularization and to balance filter performance and noise gain. A complete procedure for BRIR modeling from microphone array RIRs is also presented, employing the proposed Wiener filtering framework. An application example illustrates the modeling procedure using a 19-channel spherical microphone array. Direct and reflected sound segments are modeled separately. The modeled BRIRs are compared to measured BRIRs and are shown to be waveform-accurate up to at least 1.5 kHz. At higher frequencies, correct statistical properties of diffuse sound field components are aimed for. A listening test indicates small perceptual differences to measured BRIRs. The presented method facilitates fast BRIR data set acquisition for use in dynamic binaural synthesis and is a viable alternative to Ambisonics-based binaural room auralization.

Low Complexity Modeling of Cross-Spectral Matrix and Its Application in the Non-Synchronous Measurements of Microphones Array

The resolution related with the image quality of acoustic imaging using a microphone array is limited by the size and density of the array. However, non-synchronous measurements can exceed the constraints defined by measurements with a single fixed array. The non-synchronous measurements can achieve a larger and denser aperture array by moving the single array. The key problem of archiving the non-synchronous measurements of a microphone array is the matrix completion of a block diagonal spectral matrix. A comprehensive investigation of the block diagonal spectral matrix completion is still missing in the microphone array signal processing literature since it is usually considered as a difficult problem if without the integrated physical information of the sound field. In this paper, the mathematical relation between the max-norm and the nuclear-norm is pointed out, and the max-norm minimization of the spectral matrix is discussed, which is an effective model with integrated spatial representation of the sound field for the spectral matrix completion. First, the max-norm is utilized for the low complexity modeling and the corresponding Proximal-Point method for Max-norm based Spectral matrix completion (PPMSMC) is proposed; Second, the rank, nuclear-norm and max-norm have been investigated as complexity measures in the context of spectral matrix completion, and their performances are compared. The proposed low-rank methods have constructed a concrete theorem foundation for the non-synchronous measurements of the microphone array.

Effects of Additive Noise in Binaural Rendering of Spherical Microphone Array Signals

Additive noise produced by the recording hardware will contribute to streamed signals from spherical microphone arrays under practical conditions. For the application of binaural reproduction and under the assumption that the noise is uncorrelated between the array channels, the spectral properties and the overall level of the rendered noise in the ear signals have been shown to be strongly influenced by the configuration of the array as well as of the processing pipeline. In a previous investigation, we determined the audibility thresholds for changes in the rendered noise due to listener head rotations as a function of the differences in noise level of individual array channels. In this article, we calibrate the instrumental metric of Composite Loudness Level to the perceptual data and predict audibility of changes in the additive noise due to head rotations for a broad set of array configurations and distributions of the noise levels across array channels. We demonstrate that some types of microphone layouts can produce audible variations even if the noise level is equal in all channels. This is particularly the case for sampling grids that exhibit negative quadrature weights such as the Lebedev and Fliege-Maier grids for some spherical harmonic orders. The analysis of configurations with unevenly distributed noise contributions show that the influence of the noise from individual array channels is determined by the proximity of their virtual location to the relative trajectory of the ears.

Acoustic source imaging using densely connected convolutional networks

The localization of acoustic sources using acoustic imaging methods such as acoustic beamforming can be limited by the Rayleigh resolution limit at relatively low frequencies and the output of these methods may also produce spatial aliasing images referred to as sidelobes, particularly at high frequencies. To date, there are very few Deep Neural Network (DNN) applications for acoustic imaging to help alleviate some of these issues. In this study, several DNN models were developed with a training strategy specifically designed for an acoustic imaging task. This proposed DNN-based method is examined for various acoustic source conditions and compared with other classic acoustic imaging methods. The DNN method is able to recognize the pattern behind microphone array signals for different source positions, by using the real-component of the cross-spectral matrix of the received pressure vectors at the microphone positions. This input feature allows the DNN model to accurately locate and quantify the source strengths of complicated distributed acoustic sources, even at low frequencies that typically challenge acoustic beamforming deconvolution methods. The loss function of the DNN model is based on the difference between the estimated and true acoustic source maps, that is used to iteratively improve weighting functions within the DNN hidden layers. DNN models with both a fixed and random number of input sources are simulated for a range of specific frequencies. The DNN model with a random number of input sources is tested against conventional beamforming, CLEAN-SC and DAMAS, revealing a far improved source localization and source strength estimation. These DNN models represent a very promising proof-of-concept for the use of DNN models in the field of acoustic imaging.

Hybrid microphone array signal processing approach for faulty wheel identification and ground impedance estimation in wheel/rail system

This study investigates a microphone array signal processing approach in a theoretical and experimental manner for faulty wheel identification and localisation and ground impedance estimation in a wheel/rail system. The sound source location during wheel rotation is estimated by a broadband weighted multiple signal classification (BW-MUSIC) method, while the impedance of the corresponding ground surface is estimated by the Levenberg–Marquardt and Crank Nicolson (LM–CN) method. The accurate location of the faulty wheel is determined by the kurtosis beamformer. As an acoustics-based noncontact diagnosis method, this technique overcomes the challenge presented by contact between the sensors and the examined structures, and it is more applicable for impulsive signals with broadband features, such as impact noise generated from faults on the wheel surface. The accuracy of the location and impedance estimations is also examined in this study, and the results obtained from the numerical and experimental analyses are observed to be in good agreement. With multiple sound sources and the interference of ground reflection, the BW-MUSIC method can provide a separate and distinct localisation result in the sound map, while the LM–CN method can provide a preferable estimation result of the ground impedance, allowing the faulty wheel to be detected accurately.

Microphone and Loudspeaker Array Signal Processing Steps towards a “Radiation Keyboard” for Authentic Samplers

To date electric pianos and samplers tend to concentrate on authenticity in terms of temporal and spectral aspects of sound. However, they barely recreate the original sound radiation characteristics, which contribute to the perception of width and depth, vividness and voice separation, especially for instrumentalists, who are located near the instrument. To achieve this, a number of sound field measurement and synthesis techniques need to be applied and adequately combined. In this paper we present the theoretic foundation to combine so far isolated and fragmented sound field analysis and synthesis methods to realize a radiation keyboard, an electric harpsichord that approximates the sound of a real harpsichord precisely in time, frequency, and space domain. Potential applications for such a radiation keyboard are conservation of historic musical instruments, music performance, and psychoacoustic measurements for instrument and synthesizer building and for studies of music perception, cognition, and embodiment.

Multipole orthogonal beamforming combined with an inverse method for coexisting multipoles with various radiation patterns

Identification of sources with non-uniform radiation patterns is attracting considerable attention these years. A multipole beamforming algorithm combined with an inverse method, is proposed to detect coexisting multipoles with various radiation patterns. Positions and strengths of sources are estimated first. The sound field of each source is represented by the eigenvector of the cross-spectral matrix of microphone array signals. Source directivity is removed by the normalization and square operations on elements of the eigenvector. Each source is represented by spherical harmonics after source localization. Types and orientations of sources can be inferred from their reconstructed radiation patterns. The proposed algorithm is verified by numerical simulations and experiments using a loudspeaker and vortex shedding from a cylinder. Compared to existing multipole beamforming algorithms, it reduces the condition number of the transfer matrix and removes the spurious sources on source maps. Positions, types and orientations of sources are estimated more accurately. The resolution of source maps is comparable to that of CLEAN-SC.