Speech Enhancement Performance Research Articles

Pitch pattern matching based speech enhancement

The problem of speech enhancement in diverse noisy conditions has historically focused on the vocal tract spectral magnitude. However, studies have shown that improved quality, speaking style and speaker identity are all impacted by reliable prosody/F0 (pitch) information for human listening. In this study, we propose a speech enhancement algorithm based on pitch pattern matching. It can be considered as an example-based method since we attempt to replace the speech segment in the noisy speech with corresponding detected components from a dictionary which contains the clean speech signals. The average pitch value as well as overall pitch dynamic trends are used as features for pitch pattern matching. The speech segment in the dictionary with the best matched pitch pattern feature will be used to assist in the computation of an enhanced speech segment in the noisy speech sample. Here, Wiener filter is used for obtaining the target baseline speech from the noisy speech. The experimental results show that the pitch pattern feature is more computational efficient than a spectral based feature alone for speech enhancement, while obtaining similar speech enhancement performance in terms of both speech intelligibility and speech quality.

Blind Sampling Rate Offset Estimation for Wireless Acoustic Sensor Networks Through Weighted Least-Squares Coherence Drift Estimation

Microphone arrays allow to exploit the spatial coherence between simultaneously recorded microphone signals, e.g., to perform speech enhancement, i.e., to extract a speech signal and reduce background noise. However, in systems where the microphones are not sampled in a synchronous fashion, as it is often the case in wireless acoustic sensor networks, a sampling rate offset (SRO) exists between signals recorded in different nodes, which severely affects the speech enhancement performance. To avoid this performance reduction, the SRO should be estimated and compensated for. In this paper, we propose a new approach to blind SRO estimation for an asynchronous wireless acoustic sensor network, which exploits the phase drift of the coherence between the asynchronous microphones signals. We utilize the fact that the SRO causes a linearly increasing time delay between two signals and hence a linearly increasing phase-shift in the short-time Fourier transform domain. The increasing phase shift, observed as a phase drift of the coherence between the signals, is used in a weighted least-squares framework to estimate the SRO. This method is referred to as least-squares coherence drift (LCD). Experimental results in different real-world recording and simulated scenarios show the effectiveness of LCD compared to different benchmark methods. The LCD is effective even for short signal segments. We finally demonstrate that the use of the LCD within a conventional compensation approach eliminates the performance loss due to SRO in a speech enhancement algorithm based on the multichannel Wiener filter.

Adaptive Quantization for Multichannel Wiener Filter-Based Speech Enhancement in Wireless Acoustic Sensor Networks

Speech enhancement in wireless acoustic sensor networks requires the exchange of audio signals. Since the wireless communication often dominates the nodes’ energy budget, techniques for data exchange reduction are crucial. Adaptive quantization aims to optimize the bit depth of each exchanged signal according to its contribution to the speech enhancement performance. This enables the network to scale its energy and communication bandwidth requirements according to the current operating environment. The impact metric was previously proposed to predict the effect of quantization in linear minimum mean squared error (MMSE) estimation. We provide new insights into greedy adaptive quantization based on this impact metric. We achieve this by expanding the mathematical framework to include a new metric based on the gradient of the MMSE as a function of the quantization noise power. Using these tools, we show how the MMSE gradient naturally leads to a greedy algorithm and how the impact metric is a generalization of the gradient metric and a previously proposed metric. Besides, we validate the impact metric for adaptive quantization both in a simulated and in a real wireless acoustic sensor network deployed in a home environment, showing the energy savings achievable through greedy adaptive quantization.

An analysis of environment, microphone and data simulation mismatches in robust speech recognition

Speech enhancement and automatic speech recognition (ASR) are most often evaluated in matched (or multi-condition) settings where the acoustic conditions of the training data match (or cover) those of the test data. Few studies have systematically assessed the impact of acoustic mismatches between training and test data, especially concerning recent speech enhancement and state-of-the-art ASR techniques. In this article, we study this issue in the context of the CHiME-3 dataset, which consists of sentences spoken by talkers situated in challenging noisy environments recorded using a 6-channel tablet based microphone array. We provide a critical analysis of the results published on this dataset for various signal enhancement, feature extraction, and ASR backend techniques and perform a number of new experiments in order to separately assess the impact of different noise environments, different numbers and positions of microphones, or simulated vs. real data on speech enhancement and ASR performance. We show that, with the exception of minimum variance distortionless response (MVDR) beamforming, most algorithms perform consistently on real and simulated data and can benefit from training on simulated data. We also find that training on different noise environments and different microphones barely affects the ASR performance, especially when several environments are present in the training data: only the number of microphones has a significant impact. Based on these results, we introduce the CHiME-4 Speech Separation and Recognition Challenge, which revisits the CHiME-3 dataset and makes it more challenging by reducing the number of microphones available for testing.

Regularized non-negative matrix factorization with Gaussian mixtures and masking model for speech enhancement

We introduce single-channel supervised speech enhancement algorithms based on regularized non-negative matrix factorization (NMF). In the proposed framework, the log-likelihood functions (LLF) of the magnitude spectra for both the clean speech and noise, based on Gaussian mixture models (GMM), are included as regularization terms in the NMF cost function. By using this proposed regularization as a priori information in the enhancement stage, we can exploit the statistical properties of both the clean speech and noise signals. For further improvement of the enhanced speech quality, we also incorporate a masking model of the human auditory system in our approach. Specifically, we construct a weighted Wiener filter (WWF) where the power spectral densities (PSD) of the speech and noise are estimated from the above mentioned NMF algorithm with the proposed regularization. The weighting factor in the WWF is selected based on a masking threshold which is obtained from the estimated PSD of the enhanced speech. Experimental results of perceptual evaluation of speech quality (PESQ), source-to-distortion ratio (SDR) and segmental signal-to-noise ratio (SNR) show that the proposed speech enhancement algorithms (i.e., regularized NMF with and without masking model) provide better performance in speech enhancement than the benchmark algorithms.

2차원 이진 마스크를 이용한 적응형 음성향상 잡음 제거기

Speech enhancement algorithm plays an important role in numerous speech signal processing applications. Over the last few decades, many algorithms have been studied for speech enhancement. The algorithms are based on spectral subtraction, Wiener filter, and subspace method etc. They have good performance of speech enhancement, but the performance can be deteriorated in specific noises or low SNR environment. In this paper, a new speech enhancement algorithms are proposed based on adaptive noise canceller. And the proposed algorithm improved performance of adaptive noise cancelling using 2-D binary mask. From objective experimental index, it is confirmed that the proposed algorithm is useful and has better performance than recently proposed speech enhancement algorithms.

A new regularized forward blind source separation algorithm for automatic speech quality enhancement

Abstract This paper addresses the problem of speech enhancement and acoustic noise reduction by adaptive filtering algorithms in a moving car through blind source separation (BSS) structures. In this paper we propose a new regularized forward blind source separation (RFBSS) algorithm that does not need voice activity detection (VAD) systems, and allows getting efficient speech enhancement performances with low complexity. The proposed RFBSS algorithm is compared with recent and classical speech enhancement algorithms in different noisy conditions. This comparison is evaluated in terms of Cepstral distance (CD), the system mismatch (SM) and the Segmental signal-to-noise ratio (SegSNR) criteria. The obtained results show the efficiency of the proposed algorithm and its superiority in comparison with competitive algorithms in speech enhancement applications.

EFFECT OF VARIATIONS IN MICROPHONE COVER SHAPE AND WEARING POSITION ON THE PERFORMANCE OF A HEARING-SUPPORT DEVICE MOUNTED ON EAR — SIMULATION STUDY

Morphological and positional factors that can affect the actual performance of the hearing-support (HS) devices are utilized to support the damaged hearing ability of the sensorineural hearing-impaired persons. However, there have been few studies that demonstrated the effects of variations in such design factors on the frequency response of the device experimentally. In this study, the effect of design variations in the shape of the microphone cover on the housing and the wearing position of the device mounted on the ear on the input frequency response of the device and on the performance of an embedded beamforming algorithm were investigated using a human upper body model, a hearing aid housing model, and an acoustic environment model using computer simulation. Experimental results showed that the implemented simulator could simulate the actual acoustic situations (differences less than 5 dB in audible frequency range) and that both of the response patterns of the device and beamforming algorithm were varied in accordance with the variations in the shape of the microphone cover and the mounting position of the device on the ear. These results demonstrate the necessity of additional design and algorithm fine-tuning of each (HS) device to improve its actual speech enhancement performance.

Single-channel speech enhancement method using reconstructive NMF with spectrotemporal speech presence probabilities

In this paper, a novel single microphone channel-based speech enhancement technique is presented. While most of the conventional nonnegative matrix factorization-based approaches focus on generating a basis matrix of speech and noise for enhancement, the proposed algorithm performs an additional process to reconstruct speech from noisy speech when these two elements are highly overlapped in selected spectral bands. This process involves a log-spectral amplitude based estimator, which provides the spectrotemporal speech presence probability to obtain a more accurate reconstruction. Moreover, the proposed algorithm applies an unsupervised learning method to the input noise, so it is adaptable to any type of environmental noise without a pre-trained dictionary. The experimental results demonstrate that the proposed algorithm obtains improved speech enhancement performance compared with conventional single channel-based approaches.

Discriminative Training of NMF Model Based on Class Probabilities for Speech Enhancement

In this letter, we introduce a discriminative training algorithm of the basis vectors in the nonnegative matrix factorization (NMF) model for single-channel speech enhancement. The basis vectors for the clean speech and noises are estimated simultaneously during the training stage by incorporating the concept of classification from machine learning. Specifically, we consider the probabilistic generative model (PGM) of classification, which is specified by class-conditional densities, along with the NMF model. The update rules of the NMF are jointly obtained with the parameters of the class-conditional densities using the expectation–maximization (EM) algorithm, which guarantees convergence. Experimental results show that the proposed algorithm provides better performance in speech enhancement than the benchmark algorithms.

Nonlinear speech enhancement by virtual increase of channels and maximum SNR beamformer

In this paper, we propose a new microphone array signal processing technique, which increases the number of microphones virtually by generating extra signal channels from real microphone signals. Microphone array signal processing methods such as speech enhancement are effective for improving the quality of various speech applications such as speech recognition and voice communication systems. However, the performance of speech enhancement and other signal processing methods depends on the number of microphones. Thus, special equipment such as a multichannel A/D converter or a microphone array is needed to achieve high processing performance. Therefore, our aim was to establish a technique for improving the performance of array signal processing with a small number of microphones and, in particular, to increase the number of channels virtually by synthesizing virtual microphone signals, or extra signal channels, from two channels of microphone signals. Each virtual microphone signal is generated by interpolating a short-time Fourier transform (STFT) representation of the microphone signals. The phase and amplitude of the signal are interpolated individually. The phase is linearly interpolated on the basis of a sound propagation model, and the amplitude is nonlinearly interpolated on the basis of β divergence. We also performed speech enhancement experiments using a maximum signal-to-noise ratio (SNR) beamformer equipped with virtual microphones and evaluated the improvement in performance upon introducing virtual microphones.

Coupled Dictionaries for Exemplar-Based Speech Enhancement and Automatic Speech Recognition

Exemplar-based speech enhancement systems work by decomposing the noisy speech as a weighted sum of speech and noise exemplars stored in a dictionary and use the resulting speech and noise estimates to obtain a time-varying filter in the full-resolution frequency domain to enhance the noisy speech. To obtain the decomposition, exemplars sampled in lower dimensional spaces are preferred over the full-resolution frequency domain for their reduced computational complexity and the ability to better generalize to unseen cases. But the resulting filter may be sub-optimal as the mapping of the obtained speech and noise estimates to the full-resolution frequency domain yields a low-rank approximation. This paper proposes an efficient way to directly compute the full-resolution frequency estimates of speech and noise using coupled dictionaries: an input dictionary containing atoms from the desired exemplar space to obtain the decomposition and a coupled output dictionary containing exemplars from the full-resolution frequency domain. We also introduce modulation spectrogram features for the exemplar-based tasks using this approach. The proposed system was evaluated for various choices of input exemplars and yielded improved speech enhancement performances on the AURORA-2 and AURORA-4 databases. We further show that the proposed approach also results in improved word error rates (WERs) for the speech recognition tasks using HMM-GMM and deep-neural network (DNN) based systems.

Speech Enhancement Based on Analysis–Synthesis Framework with Improved Parameter Domain Enhancement

This paper presents a speech enhancement approach based on analysis---synthesis framework. An improved multi-band summary correlogram (MBSC) algorithm is proposed for pitch estimation and voiced/unvoiced (V/UV) detection. The proposed pitch detection algorithm achieves a lower pitch detection error compared with the reference algorithm. The denoising autoencoder (DAE) is applied to enhance the line spectrum frequencies (LSFs). The reconstruction loss could be decreased compare with the swallow model. The proposed approach is evaluated using the perceptual evaluation of speech quality (PESQ) and the experimental results show that the proposed approach improves the performance of speech enhancement compared with the conventional speech enhancement approach. In addition, it could be applied to parametric speech coding even at low bit rate and low signal-noise ratio (SNR) environments.

잡음 데이터를 활용한 음성 기저 행렬과 NMF 기반 음성 향상 기법

This paper presents a speech enhancement method using non-negative matrix factorization (NMF). In the training phase, each basis matrix of source signal is obtained from a proper database, and these basis matrices are utilized for the source separation. In this case, the performance of speech enhancement relies heavily on the basis matrix. The proposed method for which speech basis matrix is made a high reconstruction error for noise signal shows a better performance than the standard NMF which basis matrix is trained independently. For comparison, we propose another method, and evaluate one of previous method. In the experiment result, the performance is evaluated by perceptual evaluation speech quality and signal to distortion ratio, and the proposed method outperformed the other methods. 논문 15-40-04-02 The Journal of Korean Institute of Communications and Information Sciences '15-04 Vol.40 No.04 http://dx.doi.org/10.7840/kics.2015.40.4.619 619 ※ 이 논문은 2012년도 정부(교육과학기술부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No. 2012R1A2A2A01045874). ※ 이 논문은 2014년도 대검찰청 지원을 받아 수행된 연구임. First Author : Department of Electrical and Computer Engineering and the Institute of New Media and Communications, Seoul National University, kskwon@hi.snu.ac.kr, 학생회원 ° Corresponding Author : Department of Electrical and Computer Engineering and the Institute of New Media and Communications, Seoul National University, nkim@snu.ac.kr, 종신회원 * 서울대학교 전기.정보공학부 및 뉴미디어통신공동연구소, hykim@hi.snu.ac.kr 논문번호:KICS2014-12-478, Received December 4, 2014; Revised March 13, 2015; Accepted April 15, 2015 The Journal of Korean Institute of Communications and Information Sciences '15-04 Vol.40 No.04

A speech enhancement approach based on noise classification

For speech enhancement, most existing approaches do not consider the differences, between various types of noise, which significantly affect the performance of speech enhancement. In this paper, we propose a novel speech enhancement approach by taking into account the different characteristic statistical properties of various noise on the basis of noise classification. To classify noise, an effective noise classification method is firstly developed by exploiting the features of noise energy distribution in the Bark domain. Then, based on the noise types, the speech enhancement approach is obtained by forming the optimal parameter combinations for the optimally modified log-spectral amplitude (OM-LSA) speech estimator with the improved minima controlled recursive averaging (IMCRA) noise estimator, where the parameter combinations consisting of the smoothing parameters for smoothing the noisy power spectrum and the recursive averaging in the noise spectrum estimation as well as the weighting factor for the a priori SNR estimation, are built through the enhancement of noisy speech samples. Finally, extensive experiments are carried out in terms of objective evaluation under various noise conditions, and the experimental results show that the proposed approach yields better performance compared with the conventional OM-LSA with IMCRA in speech enhancement.

A Method of Wide Dynamic Range Compression Based on Curve Compression Applying on Hearing Aids

Recently, there is a growing trend to investigate wide dynamic range compression technique with the development of hearing aids. However, existing methods used to enhance expected speech as well as the background noise and classical linear compression can’t achieve the requirements of hearing impaired individuals’ reaction to the loudness compensation flexibly. To cope with the problems, this paper proposes a new approach to suppress the enhanced background noise and to dismiss abnormal sound caused by severe jittering from high frequency signal. What’s more, a scheme of curve compression is introduced to improve the loudness amplifying flexibility instead of conventional linear compression. Both the theoretical simulation and testing on cellphone hearing aid APP based on android system prove the proposed method can improve the performance of speech enhancement and provide more choices to strength sound with the curve compression ratio.

Optimization of Acoustic Echo and Noise Reduction in Non Stationary Environment

Optimized speech enhancement method combines acoustic echo reduction and noise reduction in a unified framework for non stationary environment. Simultaneous optimization of noise and echo reduction is already done in stationary environment. In most of the times in transmission, signal properties change over time. We need to remove the artifacts of sound in those conditions. Recursive least square method proposed for noise and echo reduction. It gives little amount of mean square error and better results. Normally, partial optimization of acoustic echo reduction and noise reduction does not lead to total optimization. A cascade method of multiple functions causes mutual interference between these functions and degrades eventual speech enhancement performance. Unlike cascade methods, the proposed method combines all functions to optimize eventual speech enhancement performance based on a unified framework, which is also robust against the mutual interference problem. With the proposed method, in addition to time-invariant linear filters, time-varying filters are used to reduce residual acoustic echo signal, and background noise signal which cannot be reduced using time-invariant filters. These time-invariant filters and time-varying filters are also optimized based on a unified likelihood function to avoid the mutual interference problem. Under this, all the parameters are optimized simultaneously based on the expectation-maximization algorithm and calculates a minimum mean squared error estimate of a desired signal. The experimental results show that the proposed method is superior to the cascade methods.

Simultaneous Optimization of Acoustic Echo Reduction, Speech Dereverberation, and Noise Reduction against Mutual Interference

We propose an optimized speech enhancement method that combines acoustic echo reduction, speech dereverberation, and noise reduction in a unified framework. Normally, partial optimization of acoustic echo reduction, speech dereverberation, and noise reduction does not lead to total optimization. A cascade method of multiple functions causes mutual interference between these functions and degrades eventual speech enhancement performance. Unlike cascade methods, the proposed method combines all functions to optimize eventual speech enhancement performance based on a unified framework, which is also robust against the mutual interference problem. With the proposed method, in addition to time-invariant linear filters, time-varying filters are used to reduce residual reverberation, residual acoustic echo signal, and background noise signal which cannot be reduced using time-invariant filters. These time-invariant filters and time-varying filters are also optimized based on a unified likelihood function to avoid the mutual interference problem. By combining the time-invariant linear filters and the time-varying filters, the proposed method uses a local Gaussian model with a full-rank covariance matrix and a non-zero average vector as a probabilistic model of the microphone input signal. In the local Gaussian model, non-stationary characteristics of speech sources are considered to effectively enhance speech sources. Under this probabilistic model, all the parameters are optimized simultaneously based on the expectation-maximization algorithm and calculates a minimum mean squared error estimate of a desired signal. The experimental results show that the proposed method is superior to the cascade methods.

Spectro-Temporal Filtering for Multichannel Speech Enhancement in Short-Time Fourier Transform Domain

In this letter, we propose a spectro-temporal filtering algorithm for multichannel speech enhancement in the short-time Fourier transform (STFT) domain. Compared with the traditional multiplicative filtering technique, the proposed method takes account of interdependencies between components in adjacent frames and frequency bins. For spectro-temporal filtering, speech and noise power spectral density (PSD) matrices are estimated based on an extended formulation utilizing temporal and spectral correlations, and the parametric noise reduction filter based on these PSD matrices is applied to the input microphone array signal. Moreover, multichannel speech presence probabilities are also estimated within a unified framework. A number of experimental results show that the proposed spectro-temporal filtering method improves the performance of multichannel speech enhancement.

Speech enhancement using empirical mode decomposition and the Teager–Kaiser energy operator

In this paper a speech denoising strategy based on time adaptive thresholding of intrinsic modes functions (IMFs) of the signal, extracted by empirical mode decomposition (EMD), is introduced. The denoised signal is reconstructed by the superposition of its adaptive thresholded IMFs. Adaptive thresholds are estimated using the Teager-Kaiser energy operator (TKEO) of signal IMFs. More precisely, TKEO identifies the type of frame by expanding differences between speech and non-speech frames in each IMF. Based on the EMD, the proposed speech denoising scheme is a fully data-driven approach. The method is tested on speech signals with different noise levels and the results are compared to EMD-shrinkage and wavelet transform (WT) coupled with TKEO. Speech enhancement performance is evaluated using output signal to noise ratio (SNR) and perceptual evaluation of speech quality (PESQ) measure. Based on the analyzed speech signals, the proposed enhancement scheme performs better than WT-TKEO and EMD-shrinkage approaches in terms of output SNR and PESQ. The noise is greatly reduced using time-adaptive thresholding than universal thresholding. The study is limited to signals corrupted by additive white Gaussian noise.