Spherical Harmonic Domain Research Articles

Diagonal Unloading Beamforming in the Spherical Harmonic Domain for Acoustic Source Localization in Reverberant Environments

Spherical microphone arrays allow the sound field analysis in three dimensions with the advantage of having the same resolution in all directions. By considering the frequency-independent character of the steering vectors in the spherical harmonic (SH) domain, we propose a very low-complexity SH diagonal unloading (DU) beamforming with a novel frequency smoothing power transform (FSPT) of the covariance matrices. We consider the direction of arrival (DOA) estimation problem of acoustic sources in reverberant conditions. The DU beamforming provides high resolution directional response since it exploits the subspace orthogonality property of the covariance matrix by the removal or the attenuation of the signal subspaces, obtained through the subtraction of an opportune diagonal matrix from the covariance matrix. The FSPT aims at smoothing the narrowband covariance matrices of the entire set of frequency domain components, and it pursues this goal by minimizing the narrowband error contributions due to reverberation in the broadband frequency smoothing covariance matrix. We analyze the DOA estimation performance using speech signals with simulations and real acoustic data in reverberant conditions. The results show that the proposed SH-DU-FSPT has a DOA estimation performance comparable to that of high resolution state-of-the-art methods with a significant reduction of the computational cost, since the steering directional responses are computed on the broadband frequency smoothing covariance matrix.

Speech Enhancement Using Masking for Binaural Reproduction of Ambisonics Signals

Speech enhancement in a single channel has been well studied in the literature in applications such as speech communication systems. However, in emerging applications such as virtual reality and spatial audio, in addition to attenuating undesired signals, the ability to preserve the spatial information of the desired signal captured in a noisy environment is of great importance. Nevertheless, there are only a few studies in the literature that propose solutions to this challenge. Most of these studies present solutions that attenuate the undesired signals, while preserving only limited spatial information regarding the desired signal, such as the direction of arrival (DOA). Methods that preserve complete spatial information have only recently been suggested, and have not been studied comprehensively. In this paper, two such methods based on time-frequency masking are investigated with the aim of attenuating the undesired signal, while preserving the spatial components of the desired signal. The first is referred to as spatial masking and is based on masking in the plane wave density (PWD) domain, and the second on masking in the spherical harmonics (SH) domain. The two methods are compared with a reference method, based on beamforming followed by single-channel time-frequency masking. Objective analysis and two listening tests were conducted in order to evaluate the performance of these methods for speech enhancement. It was shown that the spatial masking based method better preserves the desired component of the sound field, while the performance of the SH based method more strongly depends on the sources' distances. On the other hand, the SH based method better preserves the DOA of the residual noise, while the DOA of the residual noise under the spatial masking based method is strongly affected by the undesired signal.

Semi-Supervised Multiple Source Localization Using Relative Harmonic Coefficients Under Noisy and Reverberant Environments

This article develops a semi-supervised algorithm to address the challenging multi-source localization problem in a noisy and reverberant environment, using a spherical harmonics domain source feature of the relative harmonic coefficients. We present a comprehensive research of this source feature, including (i) an illustration confirming its sole dependence on the source position, (ii) a feature estimator in the presence of noise, (iii) a feature selector exploiting its inherent directivity over space. Source features at varied spherical harmonic modes, representing unique characterization of the soundfield, are fused by the Multi-Mode Gaussian Process modeling. Based on the unifying model, we then formulate the mapping function revealing the underlying relationship between the source feature(s) and position(s) using a Bayesian inference approach. Another issue of the overlapped components is addressed by a pre-processing technique performing overlapped frame detection, which in turn reduces this challenging problem to a single source localization. It is highlighted that this data-driven method has a strong potential to be implemented in practice because only a limited number of labeled measurements is required. We evaluate this proposed algorithm using simulated recordings between multiple speakers in diverse environments, and extensive results confirm improved performance in comparison with the state-of-art methods. Additional assessments using real-life recordings further prove the effectiveness of the method, even at unfavorable circumstances with severe source overlapping.

An order-aware scheme for robust direction of arrival estimation in the spherical harmonic domain.

Direction of arrival (DOA) estimation of sound sources using a spherical microphone array is usually performed in the spherical harmonic (SH) domain. In a non-noisy environment, it suffices to use only the zeroth- and first-order spherical harmonic beams (SHBs) in the SH domain for DOA estimation. One such method is based on the pseudo-intensity vector (PIV), which is attractive due to its low computational complexity. To improve the performance of the PIV method in reverberant environments, some methods have been proposed recently to further exploit high-order SHBs. However, these methods ignore the effect of noise on high-order SHBs, which may lead to poor performance in low signal-to-noise ratio (SNR) environments. To address the problem, this paper proposes an order-aware scheme that is able to select the high-order SHBs reliable for robust DOA estimation of multiple speech sources. Simulation and real-world experimental results demonstrate that the order-aware scheme based methods outperform their existing counterparts with less computational complexity in terms of both accuracy and robustness of DOA estimation. Moreover, the performance improvement is more significant in low SNR environment and in a scenario with small angular separation of sources.

Near-Field Acoustic Source Localization Using Spherical Harmonic Features

Near-field acoustic source localization and beamforming has hitherto not been investigated extensively in the spherical harmonic domain under reverberant conditions. In this paper, a novel method for the near-field direction of arrival (DOA) and range estimation using signal invariant and direction independent spherical harmonic features is proposed. A spatial pressure interpolation method that effectively captures the acoustic energy on the surface of the sphere is first developed in the spherical harmonic domain. Near-field DOA estimates are then computed using this pressure distribution. Spherical harmonic features that are signal invariant and direction independent are then extracted using the near field DOA estimates. Signal invariant features are obtained by normalizing spherical harmonic coefficients with a component that is proportional to the source signal strength. Direction independent features are obtained using two methods. Rotation of spherical harmonic functions over a sphere is performed using Wigner-D functions in one method, whereas in the other, the effect of DOA is compensated by spherical harmonic normalization. Using the signal invariant and direction independent features, a learning-based framework which utilizes a convolutional neural network and voicing activity detection is also developed to compute the range of the near-field source. Experiments are conducted both on simulated and real speech data for evaluating the performance of the proposed spherical harmonic features in the context of near-field localization as well as beamforming. Root mean square error of both near-field DOA and source range estimates are obtained. Objective evaluation of near-field beamformed acoustic outputs is also performed. Results obtained are motivating enough for the method to be used in practical near-field beamforming applications.

Reciprocity gap functional in spherical harmonic domain for gridless sound field decomposition

A sound field decomposition method based on the reciprocity gap functional (RGF) in the spherical harmonic domain is proposed. To estimate and reconstruct a continuous sound field including sources by using multiple microphones, an intuitive and powerful strategy is to decompose the sound field into Green’s functions. Sparse-representation algorithms have been applied to this decomposition problem; however, it requires the discretization of the target region into grid points to construct a dictionary matrix. Discretization-based methods lead to decomposition errors of off-grid sources and high computational cost of sparse representation. We apply the RGF to sparse sound field decomposition, which makes it possible to decompose the sound field as a closed-form solution without discretization. In addition, the formulation in the spherical harmonic domain enables the flexible arrangement of microphones under the assumption of the spherical target region. Numerical simulation results indicated that high decomposition and reconstruction accuracies can be achieved by the proposed method, especially at low frequencies, with a low computational cost.

Compressive Spherical Beamforming for Acoustic Source Identification

This study examines a compressive spherical beamforming (CSB) method, using a rigid spherical microphone array to localize and quantify the acoustic contribution of sources. The method relies on the array signal model in the spherical harmonics domain that can be represented as a spatially sparse problem. This makes it possible to use compressive sensing to solve an underdetermined problem via promoting sparsity. The estimation of the angular position of sources with respect to the microphone array, as well as the three-dimensional localization over a volume are investigated. Several sparse recovery algorithms [orthogonal matching pursuit (OMP), generalized OMP, ϱ1-norm minimization, and reweighted ϱ1-norm minimization] are examined for this purpose. The numerical and experimental results indicate that sparse recovery methods outperform conventional spherical harmonics beamforming. Reweighted ϱ1-norm has good adaptability to noise, improving the robustness of CSB. The method can successfully localize the angular position of sources, and quantify their relative pressure contribution. The method is promising to localize sources in a three-dimensional domain of interest. However, the three-dimensional localization is more sensitive to noise, source distance, and properties of the sensing matrix than the two-dimensional localization.

Learning SO(3) Equivariant Representations with Spherical CNNs

We address the problem of 3D rotation equivariance in convolutional neural networks. 3D rotations have been a challenging nuisance in 3D classification tasks requiring higher capacity and extended data augmentation in order to tackle it. We model 3D data with multi-valued spherical functions and we propose a novel spherical convolutional network that implements exact convolutions on the sphere by realizing them in the spherical harmonic domain. Resulting filters have local symmetry and are localized by enforcing smooth spectra. We apply a novel pooling on the spectral domain and our operations are independent of the underlying spherical resolution throughout the network. We show that networks with much lower capacity and without requiring data augmentation can exhibit performance comparable to the state of the art in standard 3D shape retrieval and classification benchmarks.

Stable recurrent calculation of isotropic Gaussian filter coefficients

Filtering in the spherical harmonic domain has become a common practice for the post-processing of temporal gravity field solutions derived from the Gravity Recovery and Climate Experiment (GRACE) satellite mission. The most frequently used filtering technique to suppress high-frequency noise in GRACE data is the isotropic Gaussian averaging filter. The implementation of the isotropic Gaussian filter in the spherical harmonic domain requires the calculation of filter coefficients, which is performed using either a forward recurrence relation, or numerical integration. Both methods show limitations, namely, the forward recurrence relation is numerically unstable and the numerical integration suffers from low accuracy. The objective of this study is to investigate these open problems more thoroughly and to provide alternative expressions for the calculation of isotropic Gaussian filter coefficients. We test three methods (i.e., backward recurrence, continued fraction and sweep algorithms) for the stable evaluation of three-term recurrence relations and examine their properties. All methods successfully solve the instability issues of the forward recurrence relation. They are also more accurate and more computationally efficient than numerical integration.

Direction of Arrival Estimation for Reverberant Speech Based on Enhanced Decomposition of the Direct Sound

Direction of arrival (DOA) estimation for speech sources is an important task in audio signal processing. This task becomes a challenge in reverberant environments, which are typical to real scenarios. Several methods of DOA estimation for speech sources have been developed recently, in an attempt to overcome the effect of reverberation. One effective approach aims to identify time-frequency bins in the short time Fourier transform domain that are dominated by the direct sound. This approach was shown to be particularly adequate for spherical arrays, with processing in the spherical harmonics domain. The direct-path dominance (DPD) test, and a method which is based on the directivity of the sound field are recent examples. While these methods seem to perform well, high reverberation conditions may degrade their performance. In this paper, the structure of the spatial correlation matrix is comprehensively studied, showing that under some well-defined conditions, the DOA of the direct sound can be correctly extracted from its dominant eigenvector, even when contaminated by reflections. This new insight leads to the development of a new test, performing an enhanced decomposition of the direct sound (EDS), denoted the DPD-EDS test. The proposed test is compared to previous DPD tests, and to other recently proposed reverberation-robust methods, using computer simulations and an experimental study, demonstrating its potential advantage. The studies include multiple speakers in highly reverberant environments, therefore representing challenging real-life acoustics scenes.

Spatial sound intensity vectors in spherical harmonic domain.

Sound intensity is a fundamental quantity describing acoustic wave fields and it contains both energy and directivity information. It is used in a variety of applications such as source localization, reproduction, and power measurement. Until now, intensity is defined at a point in space, however given sound propagates over space, knowing its spatial distribution could be more powerful. This paper formulates spatial sound intensity vectors in spherical harmonic domain such that the vectors contain energy and directivity information over continuous spatial regions. These representations are derived with finite sets of closed form coefficients enabling ease of implementation.

Spherical Harmonic Atomic Norm and its Application to DOA Estimation

Signal processing in the spherical harmonic (SH) domain has the advantages of analyzing a signal on the sphere with equal resolution in the whole space and of decomposite the frequency- and location-dependent components of the signal. Therefore, it finds recent applications in signal recovery and localization. In this paper, we consider the gridless sparse signal recovery problem in the SH domain with atomic norm minimization (ANM). Due to the absence of Vandermonde structure for spherical harmonics, the Vandermonde decomposition theorem, which is the mathematic foundation of conventional ANM approaches, is not applicable in the SH domain. To address this issue, a low-dimensional semidefinite programming (SDP) method to implement the spherical harmonic atomic norm minimization (SH-ANM) approach is proposed. This method does not rely on the Vandermonde decomposition and can recover the atomic decomposition in the SH domain directly. As an application, we develop the direction-of-arrival estimation approach based on the proposed SH-ANM method, and computer simulations demonstrate that its performance is superior to the state-of-the-art counterparts. Furthermore, we validate the results in real-life acoustics scenes for multiple speakers localization using measured data in LACATA challenge.

The role of direct sound spherical harmonics representation in externalization using binaural reproduction

The importance of the information in the direct sound to human perception of spatial sound sources is an ongoing research topic. The classification between direct sound and diffuse or reverberant sound forms the basis of numerous studies in the field of spatial audio. In particular, parametric spatial audio representation methods use this classification and employ signal processing in order to enhance the audio quality at reproduction. However, current literature does not provide information concerning the impact of ideal direct sound representation on externalization, in the context of Ambisonics. This paper aims to assess the importance of the spatial information in the direct sound in the externalization of a sound field when using binaural reproduction. This is done in the spherical harmonics (SH) domain, where an ideal direct sound representation within an otherwise Ambisonics signal is simulated, and its perceived externalization is evaluated in a formal listening test. This investigation leads to the conclusion that externalization of a first order Ambisonics signal may be significantly improved by enhancing the direct sound component, up to a level similar to a third order Ambisonics signal.

Mitigating wind noise with a spherical microphone array.

This paper utilizes a rigid spherical microphone array to reduce wind noise. In the experiments conducted, a loudspeaker is used to reproduce the desired sound signal and an axial fan is employed to generate wind noise in an anechoic chamber. The sound signal and wind noise are measured separately with the spherical microphone array and analyzed in the spherical harmonic domain. The wind noise is found to be irregularly distributed in the spherical harmonic domain, distinct from the sound signal which is concentrated in the first few spherical harmonic modes. This difference is utilized to reduce wind noise without degrading the desired sound pressure level (SPL) by use of a low pass filter method in the spherical harmonic domain. Experimental results with both single-tonal and multi-tonal sound signals demonstrate that the proposed method can reduce wind noise by more than 10 dB in the frequency range below 500 Hz. The SPL of the desired sound signal can be extracted from wind noise with an error within 1.0 dB, even when the sound level is 8 dB lower than wind noise.

Spherical harmonics based generalized image source method for simulating room acoustics.

Allen and Berkley's image source method (ISM) is proven to be a very useful and popular technique for simulating the acoustic room transfer function (RTF) in reverberant rooms. It is based on the assumption that the source and receiver of interest are both omnidirectional. With the inherent directional nature of practical loudspeakers and the increasing use of directional microphones, the above assumption is often invalid. The main objective of this paper is to generalize the frequency domain ISM in the spherical harmonics domain such that it could simulate the RTF between practical transducers with higher-order directivity. This is achieved by decomposing transducer directivity patterns in terms of spherical harmonics and by applying the concept of image sources in spherical harmonics based propagation patterns. Therefore, from now on, any transducer can be modeled in the spherical harmonics domain with a realistic directivity pattern and incorporated with the proposed method to simulate room acoustics more accurately. We show that the proposed generalization also has an alternate use in terms of enabling RTF simulations for moving point-transducers inside pre-defined source and receiver regions.

Two-Step Spherical Harmonics ESPRIT-Type Algorithms and Performance Analysis

Spherical arrays have been widely used in direction-of-arrival (DOA) estimation in recent years, and the high-resolution estimation of signal parameter via rotational invariance technique (ESPRIT) was developed in the spherical harmonics domain. However, the spherical harmonics ESPRIT (SHESPRIT) cannot estimate the DOA when the elevation approaches 90°. To solve this problem, we present a two-step SHESPRIT (TS-SHESPRIT) based on two new recurrence relations of complex spherical harmonics. Furthermore, we develop a real-valued two-step SHESPRIT (RTS-SHESPRIT) that exploits a unitary matrix to obtain a real-valued relation between the signal subspace and the steering matrix to further reduce the computational complexity. However, the number of sources that are estimated by RTS-SHESPRIT is limited. Therefore, we propose the semi-RTS-SHESPRIT method, which reduces the computational complexity associated with eigenvalue decomposition (EVD) and avoids the limitations of RTS-SHESPRIT. Relative to SHESPRIT and TS-SHESPRIT, RTS-SHESPRIT and semi-RTS-SHESPRIT reduce the computational burden by 75% during EVD. Furthermore, we derive the mean square errors (MSEs) of the above algorithms and significantly simplify the MSE expressions. Different expressions for the MSEs are due to different recurrence relations used by different SHESPRIT-type algorithms. All proposed two-step SHESPRIT-type algorithms have higher accuracy than traditional SHESPRIT. The simulation results demonstrate the satisfactory performance of our methods.

PSD Estimation and Source Separation in a Noisy Reverberant Environment Using a Spherical Microphone Array

In this paper, we propose an efficient technique for estimating individual power spectral density (PSD) components, i.e., PSD of each desired sound source as well as of noise and reverberation, in a multi-source reverberant sound scene with coherent background noise. We formulate the problem in the spherical harmonics domain to take the advantage of the inherent orthogonality of the spherical harmonics basis functions and extract the PSD components from the cross-correlation between the different sound field modes. We also investigate an implementation issue that occurs at the nulls of the Bessel functions and offer an engineering solution. The performance evaluation takes place in a practical environment with a commercial microphone array in order to measure the robustness of the proposed algorithm against all the deviations incurred in practice. We also exhibit an application of the proposed PSD estimator through a source septation algorithm and compare the performance with a contemporary method in terms of different objective measures.

Performance analysis of Low-complexity MVDR beamformer in spherical harmonics domain

Abstract Finite sample size usually has a significant impact on the performance of the minimum variance distortionless response (MVDR) beamformer. Here, statistical analysis of a low-complexity spherical harmonics MVDR (LC-SHMVDR) beamformer is conducted using two unitary matrices. Based on the first unitary matrix, the true covariance matrix becomes block-centrohermitian. This block-centrohermitian property is utilized to obtain the probability distribution function (PDF) of the array output. Then, the PDFs of the estimated covariance matrix and the weight vector become available. After the second unitary transformation, the steering vector and forward-backward (FB) averaged covariance matrix become real-valued. With these exact PDFs, we derive some explicit expressions in terms of the variance of the weight vector, the output signal-interference-noise ratio (SINR) and the mean-square error (MSE) to measure the effects of finite sample and real-valued processing. Compared with the traditional MVDR beamformer, the proposed method requires less computational complexity and performs better as verified by theoretical analysis and simulation results.

Sparse plane wave decomposition of a low frequency sound field within a cylindrical cavity using spherical microphone arrays

Conventionally plane wave decomposition (PWD) of a low frequency sound field using a spherical microphone array (SMA) would suffer from low spatial resolution. Although compressive sensing (CS) has been employed to estimate a sparse set of plane waves when formulated in the spherical harmonics domain, its performance at the low frequency is still not fully discussed, particularly if the plane waves are densely discretized, the columns of the sensing matrix will become highly correlated. To address these problems, a two-step l1-norm minimization method for the PWD is developed. First, a sufficient set of sound field coefficients in the spherical harmonics domain is solved using CS, which is equivalent to the sparse spherical harmonics decomposition (SHD), however, with the sparsity constraint imposed on the plane-wave basis instead of the coefficients vector. With the estimated coefficients, a sparse set of plane waves can then be recovered using CS by requiring that the truncated order is sufficiently high. By means of a scan-based measurement with a feasible SMA, and with the sparsity constraint imposed on the plane-wave basis, the proposed method proved effective in improving spatial resolution with less measurements through both simulations and experiments within a cylindrical cavity.

Translations of spherical harmonics expansion coefficients for a sound field using plane wave expansions.

A translation method for the spherical harmonics expansion coefficients of a sound field using plane wave expansions is proposed. It is based on the decomposition of a plane wave in the spherical harmonics domain and without the use of the spherical harmonics addition theorem, thus is very computationally efficient. Simulations are conducted for validation. The stabilities of the translations are compared with the conventional method in terms of matrix condition numbers. The proposed method is demonstrated to be more robust as the frequency increases and when upscaling the coefficients. Besides, the computation is much faster when high truncated orders are solved.