Generalized Cross Correlation Phase Transform Research Articles

Комбинированная модель локализации акустических источников с применением технологии глубокого обучения

This paper presents a deep learning model for sound source localization, which considers localization as a source direction classification problem. An approach is proposed based on the integration of sound intensity features and GCC-PHAT (Generalized cross-correlation - phase transform) features as input data for convolutional neural networks. Taking into account the modeling conditions, datasets were created for the purpose of training, validating and testing the model with spatial resolutions of 10º and 2 º. Simulation results demonstrated the effectiveness of the proposed model in localizing the source with high accuracy in a closed environment and in the presence of reverberation. The proposed model with a resolution of 10º outperformed the model fed with only sound intensity features as input features, achieving improvement in accuracy by 6,57% and in prediction accuracy by 2,86%, while the model with a resolution of 2º achieved an improvement in accuracy by 15,57% and in prediction accuracy by 2,04%.

Sound source localization based on residual network and channel attention module

This paper presents a sound source localization (SSL) model based on residual network and channel attention mechanism. The method takes the combination of log-Mel spectrogram and generalized cross-correlation phase transform (GCC-PHAT) as the input features, and extracts the time–frequency information by using the residual structure and channel attention mechanism, thus obtaining a better localizing performance. The residual blocks are introduced to extract deeper features, which can stack more layers for high-level features and avoid gradient vanishing or exploding at the same time. The attention mechanism is taken into account for the feature extraction stage in the proposed SSL model, which can focus on the most important information on the input features. We use the signals collected by microphone array to explore the performance of the model under different features, and find the most suitable input features of the proposed method. We compare our method with other models on public dataset. Experience results show a quite substantial improvement of sound source localizing performance.

Deep learning assisted sound source localization using two orthogonal first-order differential microphone arrays.

Sound source localization in noisy and reverberant rooms using microphone arrays remains a challenging task, especially for small-sized arrays. Recent years have seen promising advances on deep learning assisted approaches by reformulating the sound localization problem as a classification one. A key to the deep learning-based approaches lies in extracting sound location features effectively in noisy and reverberant conditions. The popularly adopted features are based on the well-established generalized cross correlation phase transform (GCC-PHAT), which is known to be helpful in combating room reverberation. However, the GCC-PHAT features may not be applicable to small-sized arrays. This paper proposes a deep learning assisted sound localization method using a small-sized microphone array constructed by two orthogonal first-order differential microphone arrays. An improved feature extraction scheme based on sound intensity estimation is also proposed by decoupling the correlation between sound pressure and particle velocity components in the whitening weighting construction to enhance the robustness of the time-frequency bin-wise sound intensity features. Simulation and real-world experimental results show that the proposed deep learning assisted approach can achieve higher spatial resolution and is superior to its state-of-the-art counterparts using the GCC-PHAT or sound intensity features for small-sized arrays in noisy and reverberant environments.

DOANet: a deep dilated convolutional neural network approach for search and rescue with drone-embedded sound source localization

Drone-embedded sound source localization (SSL) has interesting application perspective in challenging search and rescue scenarios due to bad lighting conditions or occlusions. However, the problem gets complicated by severe drone ego-noise that may result in negative signal-to-noise ratios in the recorded microphone signals. In this paper, we present our work on drone-embedded SSL using recordings from an 8-channel cube-shaped microphone array embedded in an unmanned aerial vehicle (UAV). We use angular spectrum-based TDOA (time difference of arrival) estimation methods such as generalized cross-correlation phase-transform (GCC-PHAT), minimum-variance-distortion-less-response (MVDR) as baseline, which are state-of-the-art techniques for SSL. Though we improve the baseline method by reducing ego-noise using speed correlated harmonics cancellation (SCHC) technique, our main focus is to utilize deep learning techniques to solve this challenging problem. Here, we propose an end-to-end deep learning model, called DOANet, for SSL. DOANet is based on a one-dimensional dilated convolutional neural network that computes the azimuth and elevation angles of the target sound source from the raw audio signal. The advantage of using DOANet is that it does not require any hand-crafted audio features or ego-noise reduction for DOA estimation. We then evaluate the SSL performance using the proposed and baseline methods and find that the DOANet shows promising results compared to both the angular spectrum methods with and without SCHC. To evaluate the different methods, we also introduce a well-known parameter—area under the curve (AUC) of cumulative histogram plots of angular deviations—as a performance indicator which, to our knowledge, has not been used as a performance indicator for this sort of problem before.

Localizing speakers in multiple rooms by using Deep Neural Networks

In the field of human speech capturing systems, a fundamental role is played by the source localization algorithms. In this paper a Speaker Localization algorithm (SLOC) based on Deep Neural Networks (DNN) is evaluated and compared with state-of-the art approaches. The speaker position in the room under analysis is directly determined by the DNN, leading the proposed algorithm to be fully data-driven. Two different neural network architectures are investigated: the Multi Layer Perceptron (MLP) and Convolutional Neural Networks (CNN). GCC-PHAT (Generalized Cross Correlation-PHAse Transform) Patterns, computed from the audio signals captured by the microphone are used as input features for the DNN. In particular, a multi-room case study is dealt with, where the acoustic scene of each room is influenced by sounds emitted in the other rooms. The algorithm is tested by means of the home recorded DIRHA dataset, characterized by multiple wall and ceiling microphone signals for each room. In detail, the focus goes to speaker localization task in two distinct neighboring rooms.As term of comparison, two algorithms proposed in literature for the addressed applicative context are evaluated, the Crosspower Spectrum Phase Speaker Localization (CSP-SLOC) and the Steered Response Power using the Phase Transform speaker localization (SRP-SLOC). Besides providing an extensive analysis of the proposed method, the article shows how DNN-based algorithm significantly outperforms the state-of-the-art approaches evaluated on the DIRHA dataset, providing an average localization error, expressed in terms of Root Mean Square Error (RMSE), equal to 324 mm and 367 mm, respectively, for the Simulated and the Real subsets.

Moving-Target Position Estimation Using GPU-Based Particle Filter for IoT Sensing Applications

A particle filter (PF) has been introduced for effective position estimation of moving targets for non-Gaussian and nonlinear systems. The time difference of arrival (TDOA) method using acoustic sensor array has normally been used to for estimation by concealing the location of a moving target, especially underwater. In this paper, we propose a GPU -based acceleration of target position estimation using a PF and propose an efficient system and software architecture. The proposed graphic processing unit (GPU)-based algorithm has more advantages in applying PF signal processing to a target system, which consists of large-scale Internet of Things (IoT)-driven sensors because of the parallelization which is scalable. For the TDOA measurement from the acoustic sensor array, we use the generalized cross correlation phase transform (GCC-PHAT) method to obtain the correlation coefficient of the signal using Fast Fourier Transform (FFT), and we try to accelerate the calculations of GCC-PHAT based TDOA measurements using FFT with GPU compute unified device architecture (CUDA). The proposed approach utilizes a parallelization method in the target position estimation algorithm using GPU-based PF processing. In addition, it could efficiently estimate sudden movement change of the target using GPU-based parallel computing which also can be used for multiple target tracking. It also provides scalability in extending the detection algorithm according to the increase of the number of sensors. Therefore, the proposed architecture can be applied in IoT sensing applications with a large number of sensors. The target estimation algorithm was verified using MATLAB and implemented using GPU CUDA. We implemented the proposed signal processing acceleration system using target GPU to analyze in terms of execution time. The execution time of the algorithm is reduced by 55% from to the CPU standalone operation in target embedded board, NVIDIA Jetson TX1. Also, to apply large-scaled IoT sensing applications, we use NVIDIA Tesla K40c as target GPU. The execution time of the proposed multi-state-space model-based algorithm is similar to the one-state-space model algorithm because of GPU-based parallel computing. Experimental results show that the proposed architecture is a feasible solution in terms of high-performance and area-efficient architecture.

2차원 상의 음원위치 추정을 위한 효율적인 영역분할방법

음원의 위치를 찾는 SSL(Sound Source Localization)은 로봇과의 인터페이스, 화상회의, 스마트 자동차 등 여러 분야에서 꼭 필요한 기술이다. 일반적으로 음원의 위치 정보를 활용하는 기술들은 주로 측정 장치에 대한 음원의 각도 정보를 찾아서 이용하고 있다. 그러나 음원의 위치에 대한 각도를 추정할 때 이용하는 사인 역함수의 비선형적인 특성으로 추정된 각도에 오차가 발생하며, 이에 대한 방안으로 마이크가 담당하는 영역을 분할하는 방법이 제안되었다. 본 논문에서는 마이크 어레이 패턴에 따른 영역분할 방법을 제안하고 음원의 위치를 2차원상의 평면 좌표로 특정하는 방법으로 위치 추정 성능을 평가하였다. 실험에서 잡음에 강인한 GCC-PHAT(Generalized Cross Correlation Phase Transform) 방법을 사용했으며, 마이크 어레이의 패턴은 마이크 3개와 4개로 삼각형과 사각형 두 종류로 구성하였으며, 100개의 음성 데이터로 실험한 결과 실제 환경에서는 3개의 마이크 어레이를 사용해서는 영역 분할 해상도가 낮아서 음원의 위치를 정해진 특정 범위내로 추정하는데 실패했으나, 4개 마이크를 이용하여 해상도를 높였더니 위치추정 성공률이 67 %로 크게 향상됨을 확인할 수 있었다. SSL (Sound Source Localization) has been applied to several applications such as man-machine interface, video conference system, smart car and so on. But in the process of sound source localization, angle estimation error is occurred mainly due to the non-linear characteristics of the sine inverse function. So an approach was proposed to decrease the effect of this non-linear characteristics, which divides the microphone's covering space into narrow regions. In this paper, we proposed an optimal space dividing way according to the pattern of microphone array. In addition, sound source's 2-dimensional position is estimated in order to evaluate the performance of this dividing method. In the experiment, GCC-PHAT (Generalized Cross Correlation PHAse Transform) method that is known to be robust with noisy environments is adopted and triangular pattern of 3 microphones and rectangular pattern of 4 microphones are tested with 100 speech data respectively. The experimental results show that triangular pattern can't estimate the correct position due to the lower space area resolution, but performance of rectangular pattern is dramatically improved with correct estimation rate of 67 %.

An environment aware ML estimation of acoustic radiation pattern with distributed microphone pairs

This paper presents a parametric approach to classify the radiation pattern of an acoustic source given the signals captured by multiple microphones. The radiation pattern influences the way the acoustic waves propagate within an enclosure, with direct implications on the behavior of most audio processing algorithms. In particular, the Generalized Cross-Correlation PHAse Transform is affected by the emission pattern as well as by the orientation of the source. A Maximum Likelihood estimator is introduced by using descriptors of the acoustic characteristics of the environment, e.g. wall absorption coefficients and room dimensions, from which models of the observed Generalized Cross-Correlation PHAse Transform are derived for a specific emission pattern. A generic unimodal source directivity is modeled using a parameterized cardioid function. A sub-band implementation is proposed to account for the frequency dependence of the source emission pattern. Experiments on simulated and real data show that the acoustic radiation pattern can be estimated in an effective way under noisy and reverberant conditions.

Sound source localization based on discrimination of cross-correlation functions

Sound source localization plays a crucial role in many microphone arrays application, ranging from speech enhancement to human–computer interface in a reverberant noisy environment. The steered response power (SRP) using the phase transform (SRP-PHAT) method is one of the most popular modern localization algorithms. The SRP-based source localizers have been proved robust, however, the methods may fail to locate the sound source in adverse noise and reverberation conditions, especially when the direct paths to the microphones are unavailable. This paper proposes a localization algorithm based on discrimination of cross-correlation functions. The cross-correlation functions are calculated by the generalized cross-correlation phase transform (GCC-PHAT) method. Using cross-correlation functions, sound source location is estimated by one of the two classifiers: Naive-Bayes classifier and Euclidean distance classifier. Simulation results have demonstrated that the proposed algorithms provide higher localization accuracy than the SRP-PHAT algorithm in reverberant noisy environment.

Multimodal Speaker Segmentation and Identification in Presence of Overlapped Speech Segments

We describe a multimodal algorithm for speaker segmentation and identification with two main contributions: First, we propose a hidden Markov model architecture that performs fusion of three information sources: a multicamera system for participant localization, a microphone array for speaker localization, and a speaker identification system. Second, we present a novel likelihood model for the microphone array observations for dealing with overlapped speech. We propose a modification of the Steered Power Response Generalized Cross Correlation Phase Transform (SPR-GCC-PHAT) function that takes into account the possible microphone occlusions and use its local maxima as microphone array observations. The likelihood of the extracted local maxima given positions of active speakers is modeled using the Joint Probabilistic Data Association (JPDA) framework. The state in the proposed hidden Markov model is a vector of the speaker activity indicators of present participants, and the unknown parameter is the mapping of participants’ locations to the set of all possible participants’ identities. We present and compare two ways for the joint estimation of the states and the unknown parameter: the first, a forward Bayesian filter that performs sequential estimate updates as new observations arrive and the second, a batch decoding using the Viterbi algorithm. Results show that, for both decoding algorithms, the proposed method outperforms standard speaker segmentation systems based on (a) speaker identification and (b) microphone array processing, for dataset with significant portion (27.4%) of overlapped speech and scores as high as 94.4% on the F-measure scale.

반향음과 잡음 환경을 고려한 실시간 소리 추적 시스템

소리를 이용한 위치 추적은 마이크로폰을 이용하여 신호를 수집하고 수집된 신호로 부터 마이크로폰 간의 신호 도달 시간차를 추정한 뒤 추정된 시간차를 이용하여 소리의 발생 위치를 추정하는 과정을 거치게 된다. 실내 환경에서 이를 활용하기 위해서는 잡음과 반향음에 대한 강건성을 확보해야만 하는 제약이 따른다. 특히 실시간으로 구현하기 위해서는 계산의 효율성까지 고려되어야 한다. 본 논문에서는 네 개의 저가 콘덴서 마이크로폰을 이용하여 비용적인 측면과 계산량에서의 효율성을 모두 추구하였다. 네 개의 마이크로폰을 이용하여 마이크로폰 간의 소리 도달 시간차를 구하는 계산량을 줄였고 GCC-PHAT(Generalized Cross Correlation-Phase Transform) 알고리즘을 이용해서 강건성을 높였으며 iterative least square 방식을 이용하여 높은 정확도의 위치 데이터를 얻을 수 있었다. Sound localization algorithm usually adapts three step process: sampling sound signals, estimating time difference of arrival between microphones, estimate location of sound source. To apply this process in indoor environment, sound localization algorithm must be strong enough against reverberant and noisy condition. Additionally, calculation efficiency must be considered in implementing real-time sound localization system. To implement real-time robust sound localization system we adapt four low cost condenser microphones which reduce the cost and total calculation load. And to get TDOA(Time Differences of Arrival) of microphones we adapt GCC-PHAT(Generalized Cross Correlation-Phase Transform) which is robust algorithm to the reverberant and noise environment. The position of sound source was calculated by using iterative least square algorithm which produce highly accurate position data.

Sound source localization with the aid of excitation source information in home robot environments

This paper is concerned with multiple microphone-based sound source localization in home robot environments. For this purpose, we use the excitation source information to determine the time-delay between each two microphones from speech source when robot's name is called. Furthermore, we present a novel method to estimate the reliable localization angle from the obtained time-delay values. Thus, it can be used as a core technique in conjunction with human-robot interaction that can naturally interact between human and robots in home robot applications. The experimental results on sound localization database revealed that the presented method showed a good performance in comparison with the conventional methods such as time difference of arrival (TDOA) and generalized cross correlation- phase transform (GCC-PHAT) in distant-varying, noise and reverberant environments.