Microphone Array Beamforming Research Articles

An IoT-based contactless neonatal respiratory monitoring system for neonatal care assistance in postpartum center

According to previous studies, one of the major causes of 20 % to 25 % of neonatal deaths is respiratory distress syndrome (RDS). Early identification, progressive monitoring, and treatment and/or management of neonatal RDS can substantially increase the rate of survival in neonates. However, global research indicates frequent shortages and burnout among nursing staff, especially in postpartum units, contributing to the difficulty in early identification of RDS in neonates. Clinicians currently use breathing sounds and frequency as key criteria in the Neonatal Resuscitation Program (NRP) for identifying and treating RDS. In practice, the monitoring of respiratory signal abnormalities relies on sensor patches, which frequently detach from the neonates’ slippery skin, leading to potential skin injuries and unstable signal reception. This paper presents an Internet of Things (IoT)-based contactless neonatal respiratory monitoring system that integrates computer vision (CV), beamforming microphone array (BFMA), and millimeter Wave (mmWave) radar, all connected to a cloud platform. Clinical trials revealed that CV-based neonatal feature identification achieved over 96 % accuracy within 40 cm to 120 cm. The neonatal breathing sound strengthening, utilized CV and BFMA, achieved an average sound-to-noise ratio (SNR) of 5.07 dB, and CV with mmWave radar reduced chest displacement signal error from 0.66 to 0.26 BPM. Additionally, survey results showed that doctors and clinical personnel were satisfied with the system's functionality and usability. This demonstrates the system's ability to assist in monitoring respiratory signals of swaddled neonates and in the early identification of neonatal RDS, with further applications in neonatal care at postpartum centers.

Designing a smart classroom with advanced Audiovisual equipment and soft collaboration capabilities for Higher Education

This is a technology-enabled classroom design which reflects a thoughtful integration of cutting-edge audiovisual solutions to enhance the learning environment. The inclusion of automated audio-visual solutions indicates a commitment to creating a seamless and efficient learning experience through technology. The incorporation of a voice lift application ensures clear and audible communication for the instructor, addressing potential challenges in large classroom settings. The use of Dante-based communication over Ethernet signifies a high-quality and reliable audio distribution system, enabling efficient communication within the classroom. The inclusion of features catering to individuals with disabilities demonstrates a commitment to creating an inclusive and accessible learning environment. The beamforming microphone array system with multiple elements is a sophisticated technology that allows for effective tracking of the active participant. This ensures that the audio focus adapts to the speaker's location. The automatic and steerable coverage type enhances the adaptability of the microphone array system, automatically focusing on the speaker. This feature contributes to an immersive and dynamic learning experience. The support for Power over Ethernet (PoE) simplifies the installation and management of AV devices in the classroom. It reduces the need for additional power outlets and cables, streamlining the overall setup. Overall, this design prioritizes advanced audiovisual technologies to create a unique, state-of-the-art, and inclusive technology-enabled classroom. The focus on clear communication, adaptability, and ease of use demonstrates a commitment to providing an optimal learning environment for both faculty and students

A quantity to identify turbulence related sound generation on surfaces

This contribution addresses the problem of identifying sources of sound on aerodynamic surfaces subject to turbulent flow. A quantity is proposed, which identifies the location of the sound generation and serves as well to quantify these sources. The suggested concept reduces fluctuating surface pressure data to the part of the pressure which is actually relevant for sound generation (representing only a very small fraction of the former). The focus is put on the interpretation of surface pressures rather than prediction, i.e. data is assumed known, typically obtained from scale-resolving numerical simulations of the turbulent flow around the surfaces. The idea is to generate knowledge from this data, as a basis for designing more silent aero shapes. The localization of sources of sound generated aerodynamically is usually accomplished by phased microphone array (numerical) beamforming techniques in various variants involving acoustic propagation of source data to the acoustic farfield and subsequent data processing. This “detour” to the acoustic farfield indeed enables the separation of sound and aerodynamic (nearfield) data. The proposed approach explicitly uses nearfield information of the surface pressure field to identify and quantify aerodynamic sources of sound on surfaces without the need to introduce beamforming. The main hypothesis of this approach states that sound is generated where the mirror principle of flow quantities, ideally satisfied at plane hard surfaces, is violated. It turns out that the hypothesis is equivalent to the statement that aerosound is produced by diffraction of the pressure nearfield incident to a surface and as such intimately related to curvature. A physical quantity is proposed to quantify and thereby localize this source of sound. The method to determine the quantity is successfully validated for the cases of leading and trailing edge sound generation at an airfoil.

Further Development of Rotating Beamforming Techniques Using Asynchronous Measurements

When rotating noise sources, such as turbomachinery, are investigated using phased microphone array measurements and beamforming, sidelobes appear on the resulting beamforming maps. Sidelobes can be decreased by increasing the number of microphones. However, if the investigated phenomenon is steady, then there is a cost-effective alternative: performing asynchronous measurements using phased arrays having a limited number of microphones. The single beamforming maps can be combined in order to arrive at results that are superior in resolution and sidelobe levels. This technique has been investigated in the literature, but according to the authors’ best knowledge, has not yet been applied to turbomachinery. This article introduces a means for applying the asynchronous measurement technique and the combination methods for rotating noise sources. The combination methods are demonstrated on two rotating point sources (both in simulations and measurements), and then on an axial flow fan test case. In the case of the two rotating point sources, the achievable improvement in resolution, average-, and maximum sidelobe levels are shown as compared to the single results. In the case of the axial flow fan, it is demonstrated that the combination methods provide more reliable noise source locations and reveal further noise sources.

Acoustic imaging with spherical microphone array and Kriging

Acoustic imaging can be performed using a spherical microphone array (SMA) and conventional beamforming (CBF) or spherical harmonic beamforming (SHB). At low frequencies, the mainlobe width depends on the SMA radius for CBF and on the order of the spherical harmonics expansion for SHB, which is related to the number of microphones. In this letter, Kriging is used to virtually increase the SMA radius and/or the number of microphones. Numerical and experimental investigations show the effectiveness of Kriging to reduce the mainlobe width and thus improve the acoustic images obtained with a SMA and CBF or SHB.

A new sixth-order Eigenmike® spherical microphone array for spatial sound field recording

A new spherical microphone array capable of up to 6-th order spatial sound field recording is described. Spherical microphone array beamforming is normally accomplished by judicially combining multiple acoustic pressure microphones appropriately mounted on the surface of a rigid sphere to form a set of orthonormal spherical harmonic modal beams or “Eigenbeams” (also referred to as Higher-Order-Ambisonics). Eigenbeamformer processing exploits the diffraction and scattering of the incident acoustic waves onto the rigid sphere. General beamformers are then created by properly combining the orthonormal Eigenbeams to form and steer a desired set of beampatterns up to the decomposition order limit. One advantage of using the modal decomposition approach to beamforming is that it is computationally compact, and results in an elegant scalable beamformer architecture. Steering beams formed by the spherical array is accomplished by a computationally simple matrix multiply. Another benefit is that beampatterns formed using spherical array processing maintain their design spatial responses independent of the steering direction. Beampatterns can also be frequency independent. In this talk, we will describe a new 64-element spherical microphone array and associated software capable of up to 6-th order performance, as well as present some real-world measurements.

Deep learning based cough detection camera using enhanced features

Coughing is a typical symptom of COVID-19. To detect and localize coughing sounds remotely, a convolutional neural network (CNN) based deep learning model was developed in this work and integrated with a sound camera for the visualization of the cough sounds. The cough detection model is a binary classifier of which the input is a two second acoustic feature and the output is one of two inferences (Cough or Others). Data augmentation was performed on the collected audio files to alleviate class imbalance and reflect various background noises in practical environments. For effective featuring of the cough sound, conventional features such as spectrograms, mel-scaled spectrograms, and mel-frequency cepstral coefficients (MFCC) were reinforced by utilizing their velocity (V) and acceleration (A) maps in this work. VGGNet, GoogLeNet, and ResNet were simplified to binary classifiers, and were named V-net, G-net, and R-net, respectively. To find the best combination of features and networks, training was performed for a total of 39 cases and the performance was confirmed using the test F1 score. Finally, a test F1 score of 91.9% (test accuracy of 97.2%) was achieved from G-net with the MFCC-V-A feature (named Spectroflow), an acoustic feature effective for use in cough detection. The trained cough detection model was integrated with a sound camera (i.e., one that visualizes sound sources using a beamforming microphone array). In a pilot test, the cough detection camera detected coughing sounds with an F1 score of 90.0% (accuracy of 96.0%), and the cough location in the camera image was tracked in real time.

Performance Enhancement of Functional Delay and Sum Beamforming for Spherical Microphone Arrays

Functional delay and sum (FDAS) beamforming for spherical microphone arrays can achieve 360° panoramic acoustic source identification, thus having broad application prospects for identifying interior noise sources. However, its acoustic imaging suffers from severe sidelobe contamination under a low signal-to-noise ratio (SNR), which deteriorates the sound source identification performance. In order to overcome this issue, the cross-spectral matrix (CSM) of the measured sound pressure signal is reconstructed with diagonal reconstruction (DRec), robust principal component analysis (RPCA), and probabilistic factor analysis (PFA). Correspondingly, three enhanced FDAS methods, namely EFDAS-DRec, EFDAS-RPCA, and EFDAS-PFA, are established. Simulations show that the three methods can significantly enhance the sound source identification performance of FDAS under low SNRs. Compared with FDAS at SNR = 0 dB and the number of snapshots = 1000, the average maximum sidelobe levels of EFDAS-DRec, EFDAS-RPCA, and EFDAS-PFA are reduced by 6.4 dB, 21.6 dB, and 53.1 dB, respectively, and the mainlobes of sound sources are shrunk by 43.5%, 69.0%, and 80.0%, respectively. Moreover, when the number of snapshots is sufficient, the three EFDAS methods can improve both the quantification accuracy and the weak source localization capability. Among the three EFDAS methods, EFDAS-DRec has the highest quantification accuracy, and EFDAS-PFA has the best localization ability for weak sources. The effectiveness of the established methods and the correctness of the simulation conclusions are verified by the acoustic source identification experiment in an ordinary room, and the findings provide a more advanced test and analysis tool for noise source identification in low-SNR cabin environments.

Microphone Array Beamforming With High Flexible Interference Attenuation and Noise Reduction

This paper studies the problem of microphone array beamforming to enhance a speech signal of interest in adverse acoustic environments, where interference and additive background noise coexist. The problem is formulated as one of convex optimization whose solution under a specified level of interference attenuation leads to an interference controlled maximum noise reduction (ICMR) beamformer, which can be expressed as a linear combination of two MVDR beamformers: one attempts to extract the desired source signal while the other attempts to extract the interference. The combination coefficients are functions of the array manifold vectors, noise coherence matrix, and the specified interference attenuation factor. By tuning the interference attenuation factor, the ICMR beamformer can be implemented to achieve aggressive interference attenuation or even eliminate interference completely; but this may lead to less additive noise suppression or even noise amplification. To control the maximum sacrifice in gain (SG) of the signal-to-noise ratio (SNR) that is acceptable for additive reduction, a variant of ICMR is derived, which is named as the ICMR-SG beamformer. Simulations are performed and the results show that the ICMR beamformer is able to control the amount of interference attenuation. In comparison, ICMR-SG controls the maximum SG of SNR while achieving the optimal possible level of interference attenuation.

Circular microphone array beamforming to improve speech recognition for virtual agents

Under funding from an NIH NIDCD SBIR phase 1 and phase 2 grant, mh acoustics built a 16-element concentric circular microphone array using both analog and digital MEMS microphones. The circular microphone array design was trademarked as the EigenEar® microphone array and the unique beamforming processing is covered by U.S. patent 8903106. The presentation aims at showing that a circular third-order differential beamformer is capable of improving the recognition rate of popular voice recognition services used by a number of commercially available virtual assistants. These services offer an API to a client allowing speech data to be uploaded, analyzed, and returned in text form to the client. Experiments were done to compare the recognition accuracy from two separate audio streams. The EigenEar microphone array has two output channels and one channel contained the output of one omnidirectional microphone in the array and the other channel contained the output of the beamformer. The two audio streams are time synchronous so both streams presented to the virtual assistant are time aligned. The talk will describe the EigenEar circular microphone array and how the experiment was setup and the ASR results obtained from the virtual agents.

Damage identification of wind turbine blades using an adaptive method for compressive beamforming based on the generalized minimax-concave penalty function

Wind turbine blades are critical components in wind energy generation, and blade health management is a challenging issue for the operation and maintenance of wind turbines. In this paper, an adaptive method is developed to identify blade damages based on the microphone array and compressive beamforming, and global and remote health assessment can be accomplished. In this method, the generalized minimax-concave penalty function is employed to enhance sparse recovery capacities, and step-sizes in computation processes are adjusted adaptively to adapt to variational conditions. Besides, potential damage locations are extracted in coarse acoustic maps to improve convergence rates. Numerical simulations show that high spatial resolutions can be achieved by the proposed method, and the computation time for solving acoustic inverse problems is less than using existing algorithms, especially with low-frequency sources. Moreover, experiments are conducted with a small-scale wind turbine. Results demonstrate that several damages in operating blades can be precisely recognized with high efficiencies, and the deterioration of acoustic maps induced by improper step-sizes can be avoided. The proposed method provides a promising way for in-situ health monitoring of wind turbine blades.

On microphone array beamforming and insights into the underlying signal models in the short-time-Fourier-transform domain.

This paper studies signal models for microphone array beamforming in the short-time-Fourier-transform (STFT) domain with long acoustic impulse responses. The major contributions are as follows. First, the signal modeling problem is investigated in the STFT domain and a general decomposition is proposed for the convolved source signal. Second, new insights into the array manifold are presented: the STFT of the windowed acoustic impulse response from the source to the sensors. Third, the structure of the reference signal is analyzed: it can be viewed as the output of a beamformer without considering the noise in the observation signal. Fourth, based on the new perspectives and decomposition, a signal model is derived based on the use of the superdirective beamformer. Finally, three performance measures are defined, based on which three optimal/suboptimal signal models are derived and their performance is assessed under different acoustic environments and analysis window lengths. The performance of the well-known minimum variance distortionless response (MVDR) beamformer is evaluated, which justifies the properties of the developed signal models.

On the Robustness of the Superdirective Beamformer

In microphone array beamforming, a high directional gain is always desired for acoustic noise and reverberation suppression; as a result, the superdirective beamformer has been of great interest in many applications. However, this beamformer is well known to be very sensitive to array imperfections. While much effort has been made to improve its robustness, it is still a major problem. This paper is essentially devoted to the study of the robustness of the superdirective beamformer and derivation of better ways to deal with this important issue. We first prove that any distortionless fixed beamformer can be written as the sum of two orthogonal beamformers, i.e., the sum of the classical delay-and-sum (DS) beamformer and a reduced-rank beamformer. Based on this property, different kinds of robust superdirective beamformers are then developed. We also show that the robust design problem can be transformed into a quadratic eigenvalue problem (QEP), which leads to a solution that achieves the maximum possible directivity factor (DF) while meets the white noise gain (WNG) constraint over a frequency band of interest.

On the Design of Differential Kronecker Product Beamformers

In this paper, we present a generalized approach for differential microphone array (DMA) beamforming in the short-time Fourier transform (STFT) domain. We propose a multistage beamforming approach, which considers a Kronecker product (KP) decomposition of the global beamformer into two independent sub-beamformers. We derive differential KP beamformers according to different criteria and analyze their performances, which are tuned by three design parameters. These parameters allow a high beamforming design flexibility; in particular, non-differential or non-KP beamformers may be obtained as special cases. Depending on the selection of parameters, we demonstrate a preferable performance with the new approach with respect to the white noise gain and directivity factor measures. In addition, we consider the task of speech enhancement. We show that differential KP beamformers perform better than non-differential and non-KP beamformers in terms of the quality and intelligibility of their respective time-domain enhanced signals, particularly in moderately reverberant environments.

Beamforming with Cube Microphone Arrays Via Kronecker Product Decompositions

Microphone arrays combined with beamforming have been widely used to solve many important acoustic problems in a wide range of applications. Much effort has been devoted in the literature to microphone array beamforming, among which the Kronecker product beamforming method developed recently has demonstrated some interesting properties. Generally, this method decomposes the global beamforming filter into a Kronecker product of a number of sub-beamforming filters, each of which corresponds to a virtual subarray and can be designed individually. This decomposition not only reduces significantly the number of beamforming coefficients, but also can be explored to improve the robustness and flexibility of beamforming. This paper extends Kronecker product beamforming from two-dimensional arrays into three-dimensional cube arrays. We consider two decompositions, i.e., fully and partially separable ones. The former decomposes the entire array into three linear subarrays while the latter decomposes the entire array into a linear subarray and a planar one. Then, for each case, we derive the Kronecker product maximum white noise gain beamformer, the Kronecker product approximate maximum directivity factor (DF) beamformer, the Kronecker product null-steering beamformer, and the Kronecker product iterative maximum DF beamformer. Simulation results demonstrate the properties and advantages of the proposed beamformers.

Microphone Array Geometries for Horizontal Spatial Audio Object Capture With Beamforming

Microphone array beamforming can be used to enhance and separate sound sources, with applications in the capture of object-based audio. Many beamforming methods have been proposed and assessed against each other. However, the effects of compact microphone array design on beamforming performance have not been studied for this kind of application. This study investigates how to maximize the quality of audio objects extracted from a horizontal sound field by filter-and-sum beamforming, through appropriate choice of microphone array design. Eight uniform geometries with practical constraints of a limited number of microphones and maximum array size are evaluated over a range of physical metrics. Results show that baffled circular arrays outperform the other geometries in terms of perceptually relevant frequency range, spatial resolution, directivity and robustness. Moreover, a subjective evaluation of microphone arrays and beamformers is conducted with regards to the quality of the target sound, interference suppression and overall quality of simulated music performance recordings. Baffled circular arrays achieve higher target quality and interference suppression than alternative geometries with wideband signals. Furthermore, subjective scores of beamformers regarding target quality and interference suppression agree well with beamformer onaxis and off-axis responses; with wideband signals the superdirective beamformer achieves the highest overall quality.

Design and Implementation of an Active Noise Control Headphone With Directional Hear-Through Capability

This article presents the design and implementation of an active noise control (ANC) headphone system with a directional hear-through capability and compares the performance of this system to that of a standard hear-through headphone system. The directional hear-through ANC headphones are a novel integration of microphone array beamforming and ANC technologies into a pair of headphones, which provide the consumer with additional functionality and new, digitally augmented ways to interact with their acoustic environment. As the microphone array is necessarily compact, superdirective beamforming is utilised to increase its low and mid frequency directional performance. In this unique integration of two current consumer technologies, first, the ANC subsystem attempts to maximise the attenuation and then the beamformer output is added to the control signal and reproduced by the headphones’ loudspeakers, with the appropriate compensation to avoid self-cancellation. The experimental study demonstrates that the proposed spatially selective ANC headphones provide a hear-through capability in the look direction, whilst reducing ambient noise and enabling the wearer to experience reduced noise communication in a noisy environment. The proposed system thus offers the consumer the potential for an electronically enhanced acoustic experience, allowing a selective reduction in environmental noise whilst desired exterior noise remains audible.

Classification, positioning, and tracking of drones by HMM using acoustic circular microphone array beamforming

This paper addresses issues with monitoring systems that identify and track illegal drones. The development of drone technologies promotes the widespread commercial application of drones. However, the ability of a drone to carry explosives and other destructive materials may pose serious threats to public safety. In order to reduce these threats, we propose an acoustic-based scheme for positioning and tracking of illegal drones. Our proposed scheme has three main focal points. First, we scan the sky with switched beamforming to find sound sources and record the sounds using a microphone array; second, we perform classification with a hidden Markov model (HMM) in order to know whether the sound is a drone or something else. Finally, if the sound source is a drone, we use its recorded sound as a reference signal for tracking based on adaptive beamforming. Simulations are conducted under both ideal conditions (without background noise and interference sounds) and non-ideal conditions (with background noise and interference sounds), and we evaluate the performance when tracking illegal drones.

Benefits of Beamforming With Local Spatial-Cue Preservation for Speech Localization and Segregation.

A study was conducted to examine the benefits afforded by a signal-processing strategy that imposes the binaural cues present in a natural signal, calculated locally in time and frequency, on the output of a beamforming microphone array. Such a strategy has the potential to combine the signal-to-noise ratio advantage of beamforming with the perceptual benefit of spatialization to enhance performance in multitalker mixtures. Participants with normal hearing and with hearing loss were tested on both speech localization and speech-on-speech masking tasks. Performance for the spatialized beamformer was compared with that for three other conditions: a reference condition with no processing, a beamformer with no spatialization, and a hybrid beamformer that operates only in the high frequencies to preserve natural binaural cues in the low frequencies. Beamforming with full-bandwidth spatialization supported speech localization and produced better speech reception thresholds than the other conditions.

Beamforming of phased microphone array for rotating sound source localization

Phased microphone array beamforming has become a standard technique to localize sound sources. For localization of rotating sound sources at free-space conditions in the frequency domain, a number of algorithms have been investigated. Two most famous algorithms were proposed by Pannert and Maier (J. Sound Vib., 333, 2014) and Herold and Sarradj (Noise Control Eng. J., 63, 2015). The relationship between the cross-spectral matrixes used in these two algorithms is still unclear. This paper investigated their relationship by proposing a new approach to calculate the cross-spectral matrix. Their relationship can then be interpreted from different interpolations used to calculate the sound pressures at virtual rotating array microphones from pressures at real stationary microphones. The former uses Fourier interpolation, while the latter uses linear interpolation. Compared with the latter algorithm, the cross-spectral matrix in the former algorithm has lower computational efficiency, nearly equal sound source location precision, and better sound source strength precision at high frequencies due to its better spectrum reconstruction capability. Additionally, the steering vector based on numerically solving the transcendental equation is proposed as an alternative with higher computational efficiency to the steering vector in the former algorithm.