Microphone Signals Research Articles

DOANet: a deep dilated convolutional neural network approach for search and rescue with drone-embedded sound source localization

Drone-embedded sound source localization (SSL) has interesting application perspective in challenging search and rescue scenarios due to bad lighting conditions or occlusions. However, the problem gets complicated by severe drone ego-noise that may result in negative signal-to-noise ratios in the recorded microphone signals. In this paper, we present our work on drone-embedded SSL using recordings from an 8-channel cube-shaped microphone array embedded in an unmanned aerial vehicle (UAV). We use angular spectrum-based TDOA (time difference of arrival) estimation methods such as generalized cross-correlation phase-transform (GCC-PHAT), minimum-variance-distortion-less-response (MVDR) as baseline, which are state-of-the-art techniques for SSL. Though we improve the baseline method by reducing ego-noise using speed correlated harmonics cancellation (SCHC) technique, our main focus is to utilize deep learning techniques to solve this challenging problem. Here, we propose an end-to-end deep learning model, called DOANet, for SSL. DOANet is based on a one-dimensional dilated convolutional neural network that computes the azimuth and elevation angles of the target sound source from the raw audio signal. The advantage of using DOANet is that it does not require any hand-crafted audio features or ego-noise reduction for DOA estimation. We then evaluate the SSL performance using the proposed and baseline methods and find that the DOANet shows promising results compared to both the angular spectrum methods with and without SCHC. To evaluate the different methods, we also introduce a well-known parameter—area under the curve (AUC) of cumulative histogram plots of angular deviations—as a performance indicator which, to our knowledge, has not been used as a performance indicator for this sort of problem before.

Simple and Efficient Double-Talk-Detector for Acoustic Echo Cancellation

Acoustic Echo Cancellation (AEC) is a topic that has received a great interest in recent years. However, a significant challenge remains with the problem of double-talk especially when the adaptive filter has a fast convergence rate. In this case, the double-talk detector (DTD) must reply in early stage and halt updating of the adaptive filter in order to avoid filter coefficients divergence. Indeed, a complex and inappropriate DTD can seriously affect the convergence rate of the adaptive filter and global performances of the AEC system. In this paper, an implementation of a simple and efficient DTD based on a recursive estimation of the decision variable which is resulting from the level comparison between far-end and microphone signals is proposed. The presented algorithm is then compared with the normalized cross-correlation (NCC) method which is taken as a reference in this work. In the simulation tests, the recursive least squares (RLS) algorithm is used to update the adaptive filter coefficients. The speech signals used in the tests are taken from the TIMIT database.

Experimental Wind Characterization with the SuperCam Microphone under a Simulated martian Atmosphere

Located on top of the mast of the Mars 2020 Perseverance rover, the SuperCam instrument suite includes a microphone to record audible sounds from 100Hz to 10kHz on the surface of Mars. It will support SuperCam’s Laser-Induced Breakdown Spectroscopy investigation by recording laser-induced shock-waves but it will also record aeroacoustic noise generated by wind flowing past the microphone. This experimental study was conducted in the Aarhus planetary wind-tunnel under low CO2 pressure with wind generated at several velocities. It focused on understanding the wind-induced acoustic signal measured by microphones instrumented in a real scale model of the rover mast as a function of the wind speed and wind orientation. Acoustic spectra recorded under a wind flow show that the low-frequency range of the microphone signal is mainly influenced by the wind velocity. In contrast, the higher frequency range is seen to depend on the wind direction relative to the microphone. On the one hand, for the wind conditions tested inside the tunnel, it is shown that the Root Mean Square of the pressure, computed over the 100Hz to 500Hz frequency range, is proportional to the dynamic pressure. Therefore, the SuperCam microphone will be able to estimate the wind speed, considering an in situ cross-calibration with the Mars Environmental Dynamic Analyzer. On the other hand, for a given wind speed, it is observed that the root mean square of the pressure, computed over the 500Hz to 2000Hz frequency range, is at its minimum when the microphone is facing the wind whereas it is at its maximum when the microphone is pointing downwind. Hence, a full 360∘ rotation of the mast in azimuth in parallel with sound recording can be used to retrieve the wind direction.We demonstrate that the SuperCam Microphone has a priori the potential to determine both the speed and the direction of the wind on Mars, thus contributing to atmospheric science investigations.

Motor data-regularized nonnegative matrix factorization for ego-noise suppression

Ego-noise, i.e., the noise a robot causes by its own motions, significantly corrupts the microphone signal and severely impairs the robot’s capability to interact seamlessly with its environment. Therefore, suitable ego-noise suppression techniques are required. For this, it is intuitive to use also motor data collected by proprioceptors mounted to the joints of the robot since it describes the physical state of the robot and provides additional information about the ego-noise sources. In this paper, we use a dictionary-based approach for ego-noise suppression in a semi-supervised manner: first, an ego-noise dictionary is learned and subsequently used to estimate the ego-noise components of a mixture by computing a weighted sum of dictionary entries. The estimation of the weights is very sensitive against other signals beside ego-noise contained in the mixture. For increased robustness, we therefore propose to incorporate knowledge about the physical state of the robot to the estimation of the weights. This is achieved by introducing a motor data-based regularization term to the estimation problem which promotes similar weights for similar physical states. The regularization is derived by representing the motor data as a graph and imprints the intrinsic structure of the motor data space onto the dictionary model. We analyze the proposed method and evaluate its ego-noise suppression performance for a large variety of different movements and demonstrate the superiority of the proposed method compared to an approach without using motor data.

Residual Echo Suppression Considering Harmonic Distortion and Temporal Correlation

In acoustic echo cancellation, a certain level of residual echo resides in the output of the linear echo canceller because of the nonlinearity of the power amplifier, loudspeaker, and acoustic transfer function in addition to the estimation error of the linear echo canceller. The residual echo in the current frame is correlated not only to the linear echo estimates for the harmonically-related frequency bins in the current frame, but also with linear echo estimates, residual echo estimates, and microphone signals in adjacent frames. In this paper, we propose a residual echo suppression scheme considering harmonic distortion and temporal correlation in the short-time Fourier transform domain. To exploit residual echo estimates and microphone signals in past frames without the adverse effect of the near-end speech and noise, we adopt a double-talk detector which is tuned to have a low false rejection rate of double-talks. Experimental results show that the proposed method outperformed the conventional approach in terms of the echo return loss enhancement during single-talk periods and the perceptual evaluation of speech quality scores during double-talk periods.

Automated Relative Fundamental Frequency Algorithms for Use With Neck-Surface Accelerometer Signals

Relative fundamental frequency (RFF) has been suggested as a potential acoustic measure of vocal effort. However, current clinical standards for RFF measures require time-consuming manual markings. Previous semi-automated algorithms have been developed to calculate RFF from microphone signals. The current study aimed to develop fully automated algorithms to calculate RFF from neck-surface accelerometer signals for ecological momentary assessment and ambulatory monitoring of voice. Training a set of 2646 /vowel-fricative-vowel/ utterances from 317 unique speakers, with and without voice disorders, was used to develop automated algorithms to calculate RFF values from neck-surface accelerometer signals. The algorithms first rejected utterances with poor vowel-to-noise ratios, then identified fricative locations, then used signal features to determine voicing boundary cycles, and finally calculated corresponding RFF values. These automated RFF values were compared to the clinical gold-standard of manual RFF calculated from simultaneously collected microphone signals in a novel test set of 639 utterances from 77 unique speakers. Automated accelerometer-based RFF values resulted in an average mean bias error (MBE) across all cycles of 0.027 ST, with an MBE of 0.152 ST and -0.252 ST in the offset and onset cycles closest to the fricative, respectively. All MBE values were smaller than the expected changes in RFF values following successful voice therapy, suggesting that the current algorithms could be used for ecological momentary assessment and ambulatory monitoring via neck-surface accelerometer signals.

A Real-Time Speech Separation Method Based on Camera and Microphone Array Sensors Fusion Approach.

In the context of assisted human, identifying and enhancing non-stationary speech targets speech in various noise environments, such as a cocktail party, is an important issue for real-time speech separation. Previous studies mostly used microphone signal processing to perform target speech separation and analysis, such as feature recognition through a large amount of training data and supervised machine learning. The method was suitable for stationary noise suppression, but relatively limited for non-stationary noise and difficult to meet the real-time processing requirement. In this study, we propose a real-time speech separation method based on an approach that combines an optical camera and a microphone array. The method was divided into two stages. Stage 1 used computer vision technology with the camera to detect and identify interest targets and evaluate source angles and distance. Stage 2 used beamforming technology with microphone array to enhance and separate the target speech sound. The asynchronous update function was utilized to integrate the beamforming control and speech processing to reduce the effect of the processing delay. The experimental results show that the noise reduction in various stationary and non-stationary noise environments were 6.1 dB and 5.2 dB respectively. The response time of speech processing was less than 10ms, which meets the requirements of a real-time system. The proposed method has high potential to be applied in auxiliary listening systems or machine language processing like intelligent personal assistant.

Multiresolution Speech Enhancement Based on Proposed Circular Nested Microphone Array in Combination with Sub-Band Affine Projection Algorithm

Speech enhancement is one of the most important fields in audio and speech signal processing. The speech enhancement methods are divided into the single and multi-channel algorithms. The multi-channel methods increase the speech enhancement performance by providing more information with the use of more microphones. In addition, spatial aliasing is one of the destructive factors in speech enhancement strategies. In this article, we first propose a uniform circular nested microphone array (CNMA) for data recording. The microphone array increases the accuracy of the speech processing methods by increasing the information. Moreover, the proposed nested structure eliminates the spatial aliasing between microphone signals. The circular shape in the proposed nested microphone array implements the speech enhancement algorithm with the same probability for the speakers in all directions. In addition, the speech signal information is different in frequency bands, where the sub-band processing is proposed by the use of the analysis filter bank. The frequency resolution is increased in low frequency components by implementing the proposed filter bank. Then, the affine projection algorithm (APA) is implemented as an adaptive filter on sub-bands that were obtained by the proposed nested microphone array and analysis filter bank. This algorithm adaptively enhances the noisy speech signal. Next, the synthesis filters are implemented for reconstructing the enhanced speech signal. The proposed circular nested microphone array in combination with the sub-band affine projection algorithm (CNMA-SBAPA) is compared with the least mean square (LMS), recursive least square (RLS), traditional APA, distributed multichannel Wiener filter (DB-MWF), and multichannel nonnegative matrix factorization-minimum variance distortionless response (MNMF-MVDR) in terms of the segmental signal-to-noise ratio (SegSNR), perceptual evaluation of speech quality (PESQ), mean opinion score (MOS), short-time objective intelligibility (STOI), and speed of convergence on real and simulated data for white and colored noises. In all scenarios, the proposed method has high accuracy at different levels and noise types by the lower distortion in comparison with other works and, furthermore, the speed of convergence is higher than the compared researches.

IMAGING DIAGNOSTICS OF COMBUSTION INSTABILITY IN PREMIXED SWIRLING COMBUSTION

An experimental study on combustion instability is presented with focus on propane-air premixed swirling flames. Swirling flames under self-excited oscillation are studied by imaging of visible light and OH* chemiluminescence filter under several typical conditions. The dynamical characteristics of swirling flames were analysed by Dynamic Mode Decomposition (DMD) method. Three types of unstable modes in the combustor system were observed, which correspond to typical acoustic resonant modes (LF mode, C1/4 mode and P1/2 mode) of the combustor system. The combustion instability is in the longitudinal mode. Furthermore, the structure of downstream hot burnt gas under stable combustion and unstable combustion is studied by imaging of visible light and near-infrared light. Results show that there is a significant difference in the downstream flow under stable combustion and unstable combustion. The DMD spectrum of the flame and the downstream hot burnt gas obtained is the same, which is close to the characteristic frequency of acoustic pressure captured by the microphone signal. The visible light and near-infrared light imaging observation method adopted in this paper provides a new imaging method for the investigation of thermo-acoustic instability.

Beamforming correction for the singular problem in identifying rotating sources with non-uniform directivity.

Identification of rotating sources with non-uniform directivity has been paid much attention in the field of aeroacoustic measurements over recent years. Singularities may be produced on the source map by using the rotating source identifier based on the multipole model due to zeros of the directivity function. A correction method is proposed to remove the influence of source directivity on source imaging and restrain the singular problem. De-Dopplerized microphone signals are transformed to the frequency domain and deconvolved with transfer functions to compensate for directivity functions. Numerical simulations, as well as experiments using rotating dipole loudspeakers, were conducted to verify the proposed method. It is demonstrated that the method is suitable for rotating sources with arbitrary orientation and works well under a high level of background noise. Positions and strengths of sources are estimated more accurately than traditional algorithms.

Beamforming Method for Extracting the Broadband Noise Sources of Counter-Rotating Open Rotors

Counter-rotating open rotors (CRORs) are known to have advantageous propulsive efficiency properties but, at the same time, many noise emission issues related to this technology still need to be investigated. The noise they generate consists of tonal and broadband components, of which the tonal components often appear in narrow frequency bands with large amplitudes. During beamforming investigations, this often makes sorting them out from among other noise sources in the same frequency bin rather straightforward. On the other hand, the broadband noise sources can be characterized as having small amplitudes: in general, they are not the dominant noise sources in many frequency bins; but, overall, a significant noise component that needs to be investigated. The literature has provided a single microphone signal preprocessing method for removing the tonal components, hence isolating the broadband noise of a CROR: the signal of which has been shown to be appropriate for the investigation of broadband spectra. In this paper, this preprocessing method is further developed into a beamforming method that can be used in order to localize the broadband noise sources of CRORs. The resulting beamforming maps have been compared to the results of earlier broadband noise CROR studies, showing the validity of the new method and the advantages of using it as compared to methods used in other investigations.

DESIGN OF DETECTING ACOUSTIC WAVES AT THE EXERCISE SMART MINE USING ACOUSTIC SENSOR

The Indonesian Navy is a unit of defense of the Republic of Indonesia (NKRI). Sea mines are explosive devices placed in waters to destroy ships. Acoustics is a branch of physics that deals with all mechanical waves in liquids, gases and solids, such as vibration, sound and ultrasonic. Arduino has made microcontroller technology more accessible and easier for even novice users. Because of this, there are special functions in audio or acoustic processing techniques aimed at using Arduino. The application of acoustic signals in the military field of the Navy, in the identification of vessels caught by the microphone receiver. The movement of objects that are caught by the microphone will be analyzed to conclude the object. From the results of the sound characteristics raised by the object, it will give a different picture for different variations of the object. Processing acoustic signals using DSP will make it easier to infer the captured acoustic signal. The circuit board used is a microphone signal amplifier and signal conditioner, for processing the audio signal obtained from the microphone and to produce output data which is the result of the process using Arduino. Analog-to-digital converter (ADC) of the microcontroller works in the voltage range from 0.0 V to +3.3 V or +5.0 V. Toachieve good sampling results, that the signal peak is close to the maximum value of the ADC. Additionally, voltage amplitudes above this threshold can damage the controller input port or produce unwanted harmonics. Therefore, a signal pre-amp circuit is needed to guarantee the appropriate audio sample signal as well as to protect the system. The test results obtained from the system analyzer frequency and signal meters can be used to identify acoustic signals.
 Keyword: Singnal Acoustic/michophone, controller, Arduino Uno, pre-amp frequency analyzer

Microphone and Loudspeaker Array Signal Processing Steps towards a “Radiation Keyboard” for Authentic Samplers

To date electric pianos and samplers tend to concentrate on authenticity in terms of temporal and spectral aspects of sound. However, they barely recreate the original sound radiation characteristics, which contribute to the perception of width and depth, vividness and voice separation, especially for instrumentalists, who are located near the instrument. To achieve this, a number of sound field measurement and synthesis techniques need to be applied and adequately combined. In this paper we present the theoretic foundation to combine so far isolated and fragmented sound field analysis and synthesis methods to realize a radiation keyboard, an electric harpsichord that approximates the sound of a real harpsichord precisely in time, frequency, and space domain. Potential applications for such a radiation keyboard are conservation of historic musical instruments, music performance, and psychoacoustic measurements for instrument and synthesizer building and for studies of music perception, cognition, and embodiment.

Spatial Coherence-Aware Multi-Channel Wind Noise Reduction

Outdoor recording is particularly challenging in the presence of wind, which induces highly non-stationary noise in the microphone signals. To enhance a desired signal, e.g., speech, a dedicated noise reduction processing is required. The reduction is usually performed by estimating an unknown set of parameters, e.g., the noise and the speech power spectral densities (PSDs). In contrast to the commonly used assumption of uncorrelated wind noise in multi-channel recordings, we assume the spatial correlation of wind noise contributions to be non-zero at low frequencies when closely-spaced microphones are employed. In our earlier work, we assumed that the spatial coherence was known, i.e., given by a model which depends on the free-field speed and direction of the air stream. As these quantities are unknown in practice, we propose in this work a method to recursively estimate the spatial coherence matrix based on the microphone observations. In addition, we prove the equivalence of two recently developed noise PSD estimation methods when uncorrelated wind noise is assumed, and we propose an approximation of both estimators which is independent of the propagation vector of the speech source at sufficiently low frequency and for a small inter-microphone distance. An evaluation in terms of improvements in speech quality, signal-to-noise ratio and intelligibility is carried out using both simulated and measured wind noise samples.

Recursive Least-Squares Lattice Algorithm Combined With Secondary-Path Innovation and Lattice-Order Decision Algorithms for Active Noise Control

This paper proposes a new active noise control (ANC) system based on a recursive least-squares lattice (RLSL) algorithm by designing secondary-path innovation (SPI) and lattice-order decision (LOD) algorithms. The SPI algorithm associated with the power spectral factorization of the secondary path is designed to apply the RLSL algorithm to the ANC system without the filtered-input structure. The SPI algorithm whitens the error microphone signals into the virtual error signals just ahead of the secondary path to construct the virtual desired signals corresponding to the outputs of the lattice filter. The LOD algorithm is designed to operate the ANC system without knowing the tap length of the optimal finite impulse response (FIR) filter. The decision algorithm using the initial and final estimation errors of the lattice filter estimates the tap length of the optimal FIR filter by increasing the lattice order one-by-one until the final estimation error reaches below a threshold value. Even in environments where the tap length of the optimal FIR filter is unknown, the proposed ANC system with low computational complexity shows almost the same performance as the ANC system based on a filtered-input recursive least squares algorithm.

Square Root-Based Multi-Source Early PSD Estimation and Recursive RETF Update in Reverberant Environments by Means of the Orthogonal Procrustes Problem

Multi-channel short-time Fourier transform (STFT) domain-based processing of reverberant microphone signals commonly relies on power-spectral-density (PSD) estimates of early source images, where early refers to reflections contained within the same STFT frame. State-of-the-art approaches to multi-source early PSD estimation, given an estimate of the associated relative early transfer functions (RETFs), conventionally minimize the approximation error defined with respect to the early correlation matrix, requiring non-negative inequality constraints on the PSDs. Instead, we here propose to factorize the early correlation matrix and minimize the approximation error defined with respect to the early-correlation-matrix square root. The proposed minimization problem -- constituting a generalization of the so-called orthogonal Procrustes problem -- seeks a unitary matrix and the square roots of the early PSDs up to an arbitrary complex argument, making non-negative inequality constraints redundant. A solution is obtained iteratively, requiring one singular value decomposition (SVD) per iteration. The estimated unitary matrix and early PSD square roots further allow to recursively update the RETF estimate, which is not inherently possible in the conventional approach. An estimate of the said early-correlation-matrix square root itself is obtained by means of the generalized eigenvalue decomposition (GEVD), where we further propose to restore non-stationarities by desmoothing the generalized eigenvalues in order to compensate for inevitable recursive averaging. Simulation results indicate fast convergence of the proposed multi-source early PSD estimation approach in only one iteration if initialized appropriately, and better performance as compared to the conventional approach.

Synthesis of real world drone signals based on lab recordings

There is a great interest in the generation of plausible drone signals in various applications, e.g. for auralization purposes or the compilation of training data for detection algorithms. Here, a methodology is presented which synthesises realistic immission signals based on laboratory recordings and subsequent signal processing. The transformation of a lab drone signal into a virtual field microphone signal has to consider a constant pitch shift to adjust for the manoeuvre specific rotational speed and the corresponding frequency dependent emission strength correction, a random pitch shift variation to account for turbulence induced rotational speed variations in the field, Doppler frequency shift and time and frequency dependent amplitude adjustments according to the different propagation effects. By evaluation of lab and field measurements, the relevant synthesizer parameters were determined. It was found that for the investigated set of drone types, the vertical radiation characteristics can be successfully described by a generic frequency dependent directivity pattern. The proposed method is applied to different drone models with a total weight between 800 g and 3.4 kg and is discussed with respect to its abilities and limitations comparing both, recordings taken in the lab and the field.

Perceptual Evaluation of Binaural MVDR-Based Algorithms to Preserve the Interaural Coherence of Diffuse Noise Fields.

Besides improving speech intelligibility in background noise, another important objective of noise reduction algorithms for binaural hearing devices is preserving the spatial impression for the listener. In this study, we evaluate the performance of several recently proposed noise reduction algorithms based on the binaural minimum-variance-distortionless-response (MVDR) beamformer, which trade-off between noise reduction performance and preservation of the interaural coherence (IC) for diffuse noise fields. Aiming at a perceptually optimized result, this trade-off is determined based on the IC discrimination ability of the human auditory system. The algorithms are evaluated with normal-hearing participants for an anechoic scenario and a reverberant cafeteria scenario, in terms of both speech intelligibility using a matrix sentence test and spatial quality using a MUlti Stimulus test with Hidden Reference and Anchor (MUSHRA). The results show that all the binaural noise reduction algorithms are able to improve speech intelligibility compared with the unprocessed microphone signals, where partially preserving the IC of the diffuse noise field leads to a significant improvement in perceived spatial quality compared with the binaural MVDR beamformer while hardly affecting speech intelligibility.

Integrated Sidelobe Cancellation and Linear Prediction Kalman Filter for Joint Multi-Microphone Speech Dereverberation, Interfering Speech Cancellation, and Noise Reduction

In multi-microphone speech enhancement, reverberation as well as additive noise and/or interfering speech are commonly suppressed by deconvolution and spatial filtering, e.g., using multi-channel linear prediction (MCLP) on the one hand and beamforming, e.g., a generalized sidelobe canceler (GSC), on the other hand. In this paper, we consider several reverberant speech components, whereof some are to be dereverberated and others to be canceled, as well as a diffuse (e.g., babble) noise component to be suppressed. In order to perform both deconvolution and spatial filtering, we integrate MCLP and the GSC into a novel architecture referred to as integrated sidelobe cancellation and linear prediction (ISCLP), where the sidelobe-cancellation (SC) filter and the linear prediction (LP) filter operate in parallel, but on different microphone signal frames. Within ISCLP, we estimate both filters jointly by means of a single Kalman filter. We further propose a spectral Wiener gain post-processor, which is shown to relate to the Kalman filter's posterior state estimate. The presented ISCLP Kalman filter is benchmarked against two state-of-the-art approaches, namely first a pair of alternating Kalman filters respectively performing dereverberation and noise reduction, and second an MCLP+GSC Kalman filter cascade. While the ISCLP Kalman filter is roughly $M^2$ times less expensive than both reference algorithms, where $M$ denotes the number of microphones, it is shown to perform similarly as compared to the former, and to outperform the latter.

Binaural LCMV Beamforming With Partial Noise Estimation

Besides reducing undesired sources (interfering sources and background noise), another important objective of a binaural beamforming algorithm is to preserve the spatial impression of the acoustic scene, which can be achieved by preserving the binaural cues of all sound sources. While the binaural minimum variance distortionless response (BMVDR) beamformer provides a good noise reduction performance and preserves the binaural cues of the desired source, it does not allow to control the reduction of the interfering sources and distorts the binaural cues of the interfering sources and the background noise. Hence, several extensions have been proposed. First, the binaural linearly constrained minimum variance (BLCMV) beamformer uses additional constraints, enabling to control the reduction of the interfering sources while preserving their binaural cues. Second, the BMVDR with partial noise estimation (BMVDR-N) mixes the output signals of the BMVDR with the noisy reference microphone signals, enabling to control the binaural cues of the background noise. Merging the advantages of both extensions, in this paper we propose the BLCMV with partial noise estimation (BLCMV-N). We show that the output signals of the BLCMV-N can be interpreted as a mixture of the noisy reference microphone signals and the output signals of a BLCMV using an adjusted interference scaling parameter. We provide a theoretical comparison between the BMVDR, the BLCMV, the BMVDR-N and the proposed BLCMV-N in terms of noise and interference reduction performance and binaural cue preservation. Experimental results using recorded signals as well as the results of a perceptual listening test show that the BLCMV-N is able to preserve the binaural cues of an interfering source (like the BLCMV), while enabling to trade off between noise reduction performance and binaural cue preservation of the background noise (like the BMVDR-N).