Minimum Variance Distortionless Response Beamformer Research Articles

Iteratively Refined Multi-Channel Speech Separation

The combination of neural networks and beamforming has proven very effective in multi-channel speech separation, but its performance faces a challenge in complex environments. In this paper, an iteratively refined multi-channel speech separation method is proposed to meet this challenge. The proposed method is composed of initial separation and iterative separation. In the initial separation, a time–frequency domain dual-path recurrent neural network (TFDPRNN), minimum variance distortionless response (MVDR) beamformer, and post-separation are cascaded to obtain the first additional input in the iterative separation process. In iterative separation, the MVDR beamformer and post-separation are iteratively used, where the output of the MVDR beamformer is used as an additional input to the post-separation network and the final output comes from the post-separation module. This iteration of the beamformer and post-separation is fully employed for promoting their optimization, which ultimately improves the overall performance. Experiments on the spatialized version of the WSJ0-2mix corpus showed that our proposed method achieved a signal-to-distortion ratio (SDR) improvement of 24.17 dB, which was significantly better than the current popular methods. In addition, the method also achieved an SDR of 20.2 dB on joint separation and dereverberation tasks. These results indicate our method’s effectiveness and significance in the multi-channel speech separation field.

A unified beamforming and source separation model for static and dynamic human-robot interaction.

This paper presents a unified model for combining beamforming and blind source separation (BSS). The validity of the model's assumptions is confirmed by recovering target speech information in noise accurately using Oracle information. Using real static human-robot interaction (HRI) data, the proposed combination of BSS with the minimum-variance distortionless response beamformer provides a greater signal-to-noise ratio (SNR) than previous parallel and cascade systems that combine BSS and beamforming. In the difficult-to-model HRI dynamic environment, the system provides a SNR gain that was 2.8 dB greater than the results obtained with the cascade combination, where the parallel combination is infeasible.

A distortionless convolution beamformer design method based on the weighted minimum mean square error for joint dereverberation and denoising

This paper designs a weighted minimum mean square error (WMMSE) based distortionless convolution beamformer (DCBF) for joint dereverberation and denoising. By effectively using WMMSE with the constraint of distortionless, a DCBF is deduced, where the outputs of the weighted prediction error (WPE) filter and the WPE-based minimum variance distortionless response (MVDR) beamformer are combined to initialize target signal for balancing signal distortion, residual reverberation and residual noise. In addition, two optimization factors are introduced to reduce the reverberation and noise when the initialized target signal is used for the solution of beamformer. As a result, the designed beamformer is presented as a linear combination of the WMMSE-based convolution beamformer (CBF) and weighted power minimization distortionless response (WPD) filter. The experimental results demonstrate the superior performance of the designed beamformer for joint dereverberation and denoising compared to the reference methods.

Sound field visualization system by MVDR beamforming with augmented reality and point clouds

In previous research, we proposed an augmented reality (AR) and mixed reality (MR) system visualizing the sound field data measured by the beamforming method. Our system visualized the sound pressure as a colormap on the AR/MR device in real-time. Free viewpoint and visualization of a colormap along the spatial geometry observed by the AR/MR sensor enabled us to understand the location of the sound source. On the other hand, the spacial geometry information of the measurement space was used only for display purposes and not in the processing of sound source location estimation. And then the previous system displaying a flat colormap projected on the spatial geometry has not been able to make measurements that precisely in space. In this paper, we propose a sound field measurement and AR/MR visualization system using point cloud data acquired by light detection and ranging (LiDAR) sensor. The MVDR (minimum variance distortionless response) beamforming calculates the beamforming output by using point cloud data. The sound sources are assumed to be located on point clouds, and the system allows fast localization of a large number of sources. In addition, it is possible to interpret the sound field in various ways based on the scattered sound pressure information. We show some basic experiments conducted in an anechoic chamber.

Robust Adaptive Beamformer Based on Constant Modulus Penalty Criteria

In this study, a robust adaptive beamformer based on constant modulus (CM) criteria is developed to improve the robustness of the array beamforming, which is a reconstructing minimal optimization for solving the mismatch problem of weight vector caused by steering vector mismatch. In the global positioning system (GPS) L1 band, firstly, a GPS array signal is modelled by designing a dual-polarized antenna array. Secondly, the distortion problem of beamforming is formulated in the traditional minimum variance distortionless response (MVDR) beamformer. For repairing the weight vector mismatch problem, a novel beamformer based on the CM envelope response is proposed to reconstruct MVDR beamforming in the array processing. Besides, min-max penalty criteria are used to enable the beamformer to allocate more degrees of freedom (DOFs) when penalizing the MVDR beamformer responses. Finally, an auxiliary two-element real variable is designed to plan the proposed beamformer. But it is still a nonconvex quadratic programming problem, so an alternating direction method of multipliers (ADMM) is employed to transform the objective function into several subproblems. Illustrative numerical simulation results are provided for validating the effectiveness of the proposed beamformer by comparing it with other existing approaches.

Identification of Damage in Beams by Modal Curvatures Using Acoustic Beamformers

This paper presents an approach to damage identification in beams by modal curvatures based on the use of beamforming algorithms. These processors have been successfully used in acoustics for the last thirty years to solve the inverse problems encountered in source recognition and image reconstruction, based on ultrasonic waves. In addition, beamformers apply to a broader range of problems in which the forward solutions are computable and measurable. This especially concerns the field of structural vibrations, where the use of such estimators has not received attention to date. In this paper, modal curvatures will play the role of replica vectors of the imaging field. The choice to use modal curvatures is motivated by means of numerical studies and experimental tests on a steel beam. Furthermore, we compare the performance of the Bartlett and minimum variance distortionless response (MVDR) beamformers with an estimator based on the simple minimization of the difference between model and measured data. The results suggest that the application of the MVDR beamformer is highly effective, especially in cases of slight damage between two sensors. MVDR enables both damage localization and quantification.

Deep Learning Approach to Source Localization of Electromagnetic Waves in the Presence of Various Sources and Noise

In this paper, the 3D localization and signal enhancement problem of a source in a noisy environment is addressed using an antenna array to ensure symmetry in communication engineering. The use of machine-learning-dependent convolutional recurrent neural networks (CRNN) and a minimum variance distortionless response (MVDR) beamformer for the localization of the source is developed. Furthermore, to ensure the adaptability of the signal enhancement module during deployment in a new environment or in new conditions, the training of a meta-learning model is conducted. At first, during the localization, the direction of arrival (DoA) estimation in both azimuth and elevation angles is generated. This is generated in a noisy three-dimensional plane and multi-source signal. Employing the DoA estimates, the MVDR is used for the enhancement of the signal source. Verifying the proposed method in the presence of mutual coupling, the two scenarios in communication engineering were simulated using a ray-tracing tool in the form of a real-world problem towards enhancing a signal source in a noisy environment and in the presence of various sources. The results obtained demonstrate how the proposed method outperforms the machine learning and parametric methods. In addition, the trained meta-learning model is employed to demonstrate how the proposed method is adaptable to any environment and still maintains an appreciable quality performance index after retraining with few data. Finally, the results obtained are motivating enough for the practical application of the proposed method.

SATCOM Earth Station Arrays Anti-Jamming Based on MVDR Algorithm

In modern life, a large amount of information is transmitted through satellites. The anti-jamming ability of satellite communication earth stations is the basis for ensuring the smooth flow of information. In the case that multiple small earth stations cannot communicate due to jamming, the minimum variance distortionless response (MVDR) algorithm based on reflector antenna array is proposed. Firstly, the gain expression of satellite communication earth station antenna is derived, and the mathematical model of the received signal of the reflector antenna array is established. Then, the MVDR beamforming algorithm based on the reflector antenna is proposed and applied to the satellite communication earth station reflector antenna array, and the anti-jamming capability of the reflector antenna array using this algorithm is analyzed through simulation. The simulation results show that the earth station reflector antenna array using MVDR algorithm can effectively achieve array gain of the direction of the satellite signal, suppress the jamming signals, increase the output signal to interference plus noise ratio (SINR) of the reflector antenna array, and increase the output signal to jamming plus noise ratio with the increase in the number of snapshots.

Design of a robust MVDR beamforming method with Low-Latency by reconstructing covariance matrix for speech enhancement

Aiming at solving the problems of the conventional minimum variance distortionless response (MVDR) beamformer in practical applications, such as the sensibility of the steering vector mismatch and beampattern distortion, a robust broadband MVDR beamforming method with low-latency by reconstructing covariance matrix is proposed and applied to speech enhancement with a linear microphone array in this paper. In this work, some important steps are optimized, and the main contribution is to consider the problem of correlation terms generated by the low latency. Firstly, the direction of arrival (DOA) is corrected and the steering vector is estimated based on the sparsity of the DOAs corresponding to the sound sources, which improves the ability of anti-mismatches in the steering vector. Secondly, the correlation terms between the sound sources and noise are estimated and eliminated by the Capon power within the eigen-subspace, and the indirect dominant method is used to eliminate the correlation terms between the sound sources, so that the covariance matrix is reconstructed to obtain a more robust MVDR beamformer. Thirdly, the problem of white noise amplification at low frequency bins is analyzed, and a white noise gain (WNG) modification method is proposed to obtain a compromise between the interference suppression and WNG. In the experiments, the TIMIT corpus is used to generate the multi-channel speech data set, and the performance of the proposed method is evaluated with different DOAs and input signal to interference plus noise ratios (SINRs). The experimental results show that the proposed method can effectively suppress the interferences and reduce the noise with strong robustness.

Three-stage hybrid neural beamformer for multi-channel speech enhancement.

This paper proposes a hybrid neural beamformer for multi-channel speech enhancement, which comprises three stages, i.e., beamforming, post-filtering, and distortion compensation, called TriU-Net. The TriU-Net first estimates a set of masks to be used within a minimum variance distortionless response beamformer. A deep neural network (DNN)-based post-filter is then utilized to suppress the residual noise. Finally, a DNN-based distortion compensator is followed to further improve speech quality. To characterize the long-range temporal dependencies more efficiently, a network topology, gated convolutional attention network, is proposed and utilized in the TriU-Net. The advantage of the proposed model is that the speech distortion compensation is explicitly considered, yielding higher speech quality and intelligibility. The proposed model achieved an average 2.854 wb-PESQ score and 92.57% ESTOI on the CHiME-3 dataset. In addition, extensive experiments conducted on the synthetic data and real recordings confirm the effectiveness of the proposed method in noisy reverberant environments.

Minimum Variance Distortionless Response—Hanbury Brown and Twiss Sound Source Localization

Sound source target localization is an extremely useful technique that is currently utilized in many fields. The Hanbury Brown and Twiss (HBT) interference target localization method based on sound fields is not accurate enough for localization at low signal-to-noise ratios (below 0 dB). To address this problem, this paper introduces Minimum Variance Distortionless Response (MVDR) beamforming and proposes a new MVDR-HBT algorithm. Specifically, for narrowband signals, the inverse of the correlation matrix of the sound signal is calculated, and a guiding vector is constructed to compute the MVDR direction weights. These direction weights are then used to weight the correlation function of the HBT algorithm. Subsequently, the MVDR-HBT algorithm is extended from narrowband signals to broadband signals. As a result, the directivity of the HBT algorithm is optimized for wide- and narrowband signals, resulting in improved localization accuracy. Finally, the target localization accuracy of the MVDR-HBT algorithm is analyzed through simulation and localization experiments. The results show that the MVDR-HBT algorithm can accurately determine the direction of a sound source, with localization errors at different positions that are smaller than those produced by HBT. The localization performance of MVDR-HBT is considerably better than that of HBT, further verifying the simulation results. This study provides a new idea for target localization within an acoustic propagation medium (air).

Adaptive Beamforming with Hydrophone Arrays Based on Oblique Projection in the Presence of the Steering Vector Mismatch

In sonar systems, the performance of adaptive beamformers severely degrades when mismatches occur between the actual and presumed steering vectors of the desired signal, mainly due to hydrophone position errors, amplitude-phase errors, and the scattered effect of arrays. Similarly, an inadequate number of “training” samples can lead to performance degradations similar to those caused by mismatches. In this paper, an adaptive beamforming algorithm based on oblique projection (OP-ABF) mismatch compensation is proposed to remove the degradation caused by the arbitrary-type steering vector mismatch of the desired signal. The proposed algorithm is motivated by the fact that the weight vector of adaptive beamforming can be represented as a linear combination of the optimal one and the oblique projection (OP) vector, which is generated by the steering vector mismatch and does not exist without this. Our algorithm was developed by constructing the oblique projection mismatch compensation vector (OPMCV) to provide the minimum variance distortionless response (MVDR) beamformer. Then, the algorithm could be implemented by the solution of the OP matrix with the formulation of the covariance matrix loading (CML). The simulation results of a uniform linear array (ULA) and a half-cylindrical conformal array (HCCA) show that the OP-ABF can optimize the original weight vector as much as possible without sacrificing the output signal-to-interference-plus-noise ratio (SINR) under different conditions. Experimental results for the HCCA also confirm the effectiveness of this algorithm.

A distributed approach to robust minimum variance distortionless response beamforming in large‐scale arrays

AbstractRobust beamforming is a commonly‐used solution to various array processing problems. Recently, with the inevitable and rapid increasing of array size, the complexity issue arising from high‐dimension matrix computations has become an urgent concern in algorithm design. In this paper, an efficient distributed robust beamforming algorithm is developed for large‐scale arrays. Specifically, the problem of robust minimum variance distortionless response (MVDR) beamforming with sidelobe level control is considered. By fitting the worst‐case reformulation of robust MVDR beamforming into the framework of alternating direction method of multipliers, the problem can be iteratively solved in a distributed manner, with each iteration being computed in closed form. Numerical results demonstrate that this design is more efficient than many related approaches.

A double zero MVDR beamformer that is universal over number of second-order notches

Adaptive beamformers, like the minimum variance distortionless response (MVDR) beamformer, suffer from interferer direction mismatch due to interferer motion. This mismatch reduces interferer suppression and degrades the beamformer’s performance. Applying a flatter and broader notch near the interferer location effectively suppresses moving interferers. The double zero MVDR (DZ MVDR) adaptive beamformer [Tuladhar & Buck, JASA, 2015] produces second-order notches in the interferer’s directions by squaring the array polynomial of a subarray SMI MVDR beamformer. The performance of the DZ MVDR beamformer depends on the chosen number of second-order notches by applying different subarray aperture sizes. If the number of interferers in a dynamic environment is known, then the ideal number of second-order notches can be implemented. In practice, the number of interferers is an unknown and often time-varying parameter. To address this challenge, we propose a universal DZ (UDZ) MVDR beamformer that is universal over the number of second-order notches. This universal beamformer requires fewer degrees of freedom than competing beamformers to suppress the interferers, thus providing a better white noise gain in the array output. We present simulation results illustrating the benefits of the UDZ MVDR beamformer in the presence of multiple moving interferers. [Work supported by ONR 321US.]

A binaural heterophasic adaptive beamformer and its deep learning assisted implementation

Beamforming is one of the most effective approaches to distant sound acquisition in complex acoustic environments, where noise, reverberation, and interference coexist; as a result, a significant number of efforts have been devoted to it over the last few decades. However, conventional beamformers produce a monaural output or colinear outputs, which are not optimal from the perception perspective. To take advantage of the human binaural hearing properties, a new type of fixed beamforming methods were developed recently, which attempt not only to attenuate noise but also render the signal of interest and residual noise into different perceptual regions, thereby achieving higher speech intelligibility. This work extends the principle of fixed binaural beamforming and develops a binaural heterophasic minimum variance distortionless response (MVDR) beamformer. A deep neural network (DNN) based noise estimation method is used to assist the implementation of this heterophasic MVDR beamformer, which is advantageous over the traditional one as it renders the desired source signal and residual noise to different perceptual regions, thereby yielding higher intelligibility. In comparison with the fixed binaural heterophasic beamformers, it can take advantage of the statistics of the noise to achieve better array performance. Results of simulations and listening tests validate the properties of the proposed technique.

Implementation of the Digital QS-SVM-Based Beamformer on an FPGA Platform.

To address practical challenges in establishing and maintaining robust wireless connectivity such as multi-path effects, low latency, size reduction, and high data rate, we have deployed the digital beamformer, as a spatial filter, by using the hybrid antenna array at an operating frequency of 10 GHz. The proposed digital beamformer utilizes a combination of the two well-established beamforming techniques of minimum variance distortionless response (MVDR) and linearly constrained minimum variance (LCMV). In this case, the MVDR beamforming method updates weight vectors on the FPGA board, while the LCMV beamforming technique performs nullsteering in directions of interference signals in the real environment. The most well-established machine learning technique of support vector machine (SVM) for the Direction of Arrival (DoA) estimation is limited to problems with linearly-separable datasets. To overcome the aforementioned constraint, the quadratic surface support vector machine (QS-SVM) classifier with a small regularizer has been used in the proposed beamformer for the DoA estimation in addition to the two beamforming techniques of LCMV and MVDR. In this work, we have assumed that five hybrid array antennas and three sources are available, at which one of the sources transmits the signal of interest. The QS-SVM-based beamformer has been deployed on the FPGA board for spatially filtering two signals from undesired directions and passing only one of the signals from the desired direction. The simulation results have verified the strong performance of the QS-SVM-based beamformer in suppressing interference signals, which are accompanied by placing deep nulls with powers less than -10 dB in directions of interference signals, and transferring the desired signal. Furthermore, we have verified that the performance of the QS-SVM-based beamformer yields other advantages including average latency time in the order of milliseconds, performance efficiency of more than 90%, and throughput of nearly 100%.

Passive source localization based on multipath arrival angles with a vertical line array using sparse Bayesian learning.

In deep water, multipath time delays or frequency-domain interference periods of the acoustic intensity combined with multipath arrival angles are typically used for source localization. However, depth estimate is hard to achieve for a narrowband source at a remote part of the direct arrival zone as the required bandwidth increases with the source range. In this paper, a passive source localization method with a vertical line array, suitable for both broadband and narrowband sources, is proposed. Based on the variation trends of multipath angles with source range and depth, source localization is achieved by only matching the measured angles of the direct path and surface-reflected path with model-based values of a predefined grid of potential source locations. Considering the angle resolution limited by the array aperture and the presence of coherent multipath, sparse Bayesian learning is used and compared with the conventional beamforming and the minimum-variance distortionless-response beamforming to resolve and estimate the multipath angles. Simulations and experimental data of explosive sources collected by a vertical line array in the South China Sea are carried out to illustrate the method and demonstrate the performance.

A Pre-Separation and All-Neural Beamformer Framework for Multi-Channel Speech Separation

Thanks to the use of deep neural networks (DNNs), microphone array speech separation methods have achieved impressive performance. However, most existing neural beamforming methods explicitly follow traditional beamformer formulas, which possibly causes sub-optimal performance. In this study, a pre-separation and all-neural beamformer framework is proposed for multi-channel speech separation without following the solutions of the conventional beamformers, such as the minimum variance distortionless response (MVDR) beamformer. More specifically, the proposed framework includes two modules, namely the pre-separation module and the all-neural beamforming module. The pre-separation module is used to obtain pre-separated speech and interference, which are further utilized by the all-neural beamforming module to obtain frame-level beamforming weights without computing the spatial covariance matrices. The evaluation results of the multi-channel speech separation tasks, including speech enhancement subtasks and speaker separation subtasks, demonstrate that the proposed method is more effective than several advanced baselines. Furthermore, this method can be used for symmetrical stereo speech.

A Full Loading-Based MVDR Beamforming Method by Backward Correction of the Steering Vector and Reconstruction of the Covariance Matrix

In order to improve the performance of the diagonal loading-based minimum variance distortionless response (MVDR) beamformer, a full loading-based MVDR beamforming method is proposed in this paper. Different from the conventional diagonal loading methods, the proposed method combines the backward correction of the steering vector of the target source and the reconstruction of the covariance matrix. Firstly, based on the linear combination, an appropriate full loading matrix was constructed to correct the steering vector of the target source backward. Secondly, based on the spatial sparsity of the sound sources, an appropriate loading matrix was constructed to further suppress interferences. Thirdly, the spatial response power was utilized to derive a more accurate direction of arrival (DOA) of the target source, which is helpful for obtaining a more accurate steering vector of the target source and a more effective covariance matrix iteratively. The simulation results show that the proposed method can effectively suppress interferences and noise.

Simultaneous Execution of Dereverberation, Denoising, and Speaker Separation Using a Neural Beamformer for Adapting Robots to Real Environments

It remains challenging for robots to accurately perform sound source localization and speech recognition in a real environment with reverberation, noise, and the voices of multiple speakers. Accordingly, we propose “U-TasNet-Beam,” a speech extraction method for extracting only the target speaker’s voice from all ambient sounds in a real environment. U-TasNet-Beam is a neural beamformer comprising three elements: a neural network for removing reverberation and noise, a second neural network for separating the voices of multiple speakers, and a minimum variance distortionless response (MVDR) beamformer. Experiments with simulated data and recorded data show that the proposed U-TasNet-Beam can improve the accuracy of sound source localization and speech recognition in robots compared to the conventional methods in a noisy, reverberant, and multi-speaker environment. In addition, we propose the spatial correlation matrix loss (SCM loss) as a loss function for the neural network learning the spatial information of the sound. By using the SCM loss, we can improve the speech extraction performance of the neural beamformer.