Single-microphone Noise Reduction Research Articles

Deep learning restores speech intelligibility in multi-talker interference for cochlear implant users

Cochlear implants (CIs) do not offer the same level of effectiveness in noisy environments as in quiet settings. Current single-microphone noise reduction algorithms in hearing aids and CIs only remove predictable, stationary noise, and are ineffective against realistic, non-stationary noise such as multi-talker interference. Recent developments in deep neural network (DNN) algorithms have achieved noteworthy performance in speech enhancement and separation, especially in removing speech noise. However, more work is needed to investigate the potential of DNN algorithms in removing speech noise when tested with listeners fitted with CIs. Here, we implemented two DNN algorithms that are well suited for applications in speech audio processing: (1) recurrent neural network (RNN) and (2) SepFormer. The algorithms were trained with a customized dataset (∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 30 h), and then tested with thirteen CI listeners. Both RNN and SepFormer algorithms significantly improved CI listener’s speech intelligibility in noise without compromising the perceived quality of speech overall. These algorithms not only increased the intelligibility in stationary non-speech noise, but also introduced a substantial improvement in non-stationary noise, where conventional signal processing strategies fall short with little benefits. These results show the promise of using DNN algorithms as a solution for listening challenges in multi-talker noise interference.

Beamforming and Single-Microphone Noise Reduction: Effects on Signal-to-Noise Ratio and Speech Recognition of Bimodal Cochlear Implant Users.

We have investigated the effectiveness of three noise-reduction algorithms, namely an adaptive monaural beamformer (MB), a fixed binaural beamformer (BB), and a single-microphone stationary-noise reduction algorithm (SNRA) by assessing the speech reception threshold (SRT) in a group of 15 bimodal cochlear implant users. Speech was presented frontally towards the listener and background noise was established as a homogeneous field of long-term speech-spectrum-shaped (LTSS) noise or 8-talker babble. We pursued four research questions, namely: whether the benefits of beamforming on the SRT differ between LTSS noise and 8-talker babble; whether BB is more effective than MB; whether SNRA improves the SRT in LTSS noise; and whether the SRT benefits of MB and BB are comparable to their improvement of the signal-to-noise ratio (SNR). The results showed that MB and BB significantly improved SRTs by an average of 2.6 dB and 2.9 dB, respectively. These benefits did not statistically differ between noise types or between the two beamformers. By contrast, physical SNR improvements obtained with a manikin revealed substantially greater benefits of BB (6.6 dB) than MB (3.3 dB). SNRA did not significantly affect SRTs per se in omnidirectional microphone settings, nor in combination with MB and BB. We conclude that in the group of bimodal listeners tested, BB had no additional benefits on speech recognition over MB in homogeneous noise, despite the finding that BB had a substantial larger benefit on the SNR than MB. SNRA did not improve speech recognition.

Measuring the Influence of Noise Reduction on Listening Effort in Hearing-Impaired Listeners Using Response Times to an Arithmetic Task in Noise.

Single microphone noise reduction (NR) in hearing aids can provide a subjective benefit even when there is no objective improvement in speech intelligibility. A possible explanation lies in a reduction of listening effort. Previously, we showed that response times (a proxy for listening effort) to an auditory-only dual-task were reduced by NR in normal-hearing (NH) listeners. In this study, we investigate if the results from NH listeners extend to the hearing-impaired (HI), the target group for hearing aids. In addition, we assess the relevance of the outcome measure for studying and understanding listening effort. Twelve HI subjects were asked to sum two digits of a digit triplet in noise. We measured response times to this task, as well as subjective listening effort and speech intelligibility. Stimuli were presented at three signal-to-noise ratios (SNR; –5, 0, +5 dB) and in quiet. Stimuli were processed with ideal or nonideal NR, or unprocessed. The effect of NR on response times in HI listeners was significant only in conditions where speech intelligibility was also affected (–5 dB SNR). This is in contrast to the previous results with NH listeners. There was a significant effect of SNR on response times for HI listeners. The response time measure was reasonably correlated (R142 = 0.54) to subjective listening effort and showed a sufficient test–retest reliability. This study thus presents an objective, valid, and reliable measure for evaluating an aspect of listening effort of HI listeners.

Musical-noise-free noise reduction by using biased harmonic regeneration and considering relationship between a priori SNR and sound quality

This paper focuses on two representative single-microphone noise reduction problems: speech distortion and musical noise. Many noise reduction methods have been proposed for each problem. Harmonic regeneration noise reduction (HRNR) was introduced for the improvements of speech distortion and the a priori signal-to-noise ratio (SNR) estimator. HRNR involves using a unique signal to regenerate harmonics, which had been eliminated. Musical-noise-free noise reduction based on the minimum-mean square error short-time spectral amplitude estimator (musical-noise-free MMSE-STSA estimator) has also been proposed. This method can suppress a noisy signal without generating musical noise by introducing a bias into the classical a priori SNR estimator. We propose a noise reduction method for addressing these problems simultaneously by improving the classical a priori SNR estimator. We investigated the behavior of the internal parameters for the proposed and conventional methods with regard to speech quality and show the effectiveness of the proposed method in terms of speech distortion and musical noise. We consider and discuss the relationship between the estimation accuracy of an a priori SNR and speech quality. Specifically, we consider the factors to improve speech quality in terms of biasing.

Subjective evaluation of single microphone noise reduction with different time constants

Objective: Previous studies on single microphone noise reduction (NR) in hearing aids (HAs) have shown that some NR algorithms provide beneficial effects in terms of listener preference. To improve HA user satisfaction, we are interested in characteristics that determine preferences for NR, and in the inter-individual variability. The aim of this study was to test if dynamic properties of NR influence listener preference.Design: The gain reduction at speech offsets of a NR algorithm was slowed down by applying temporal exponential smoothing. At speech onsets the gain recovery was left unchanged. Test signals consisted of speech in continuous and modulated speech-shaped background noise, processed with three time constants: 0, 100, and 200 ms.Study sample: 16 Normal hearing (NH) and 16 hearing impaired (HI) subjects participated in a paired-comparison listening test.Results: NH subjects as a group had a significant preference for NR with time constants of 100, or 200 ms (slower acting NR). HI listeners as a group preferred NR over no NR, but had no clear preference for fast or slow NR. Patterns of preference differed between individual listeners.Conclusions: NR dynamics had an impact on individual listener preference and should be considered when optimising HAs.

Optimising the effect of noise reduction algorithm ClearVoice in cochlear implant users by increasing the maximum comfort levels

Objective: ClearVoice is a single-microphone noise reduction algorithm in Advanced Bionics cochlear implant(CI) systems with the aim to improve performance in background noise. The present study investigated a hypothesised increased effect of ClearVoice if combined with a structural increase of maximum comfort stimulation levels (M-levels) in the CI fitting. Design: We tested performance with ClearVoice (Medium) in four conditions, defined by combined settings of ClearVoice off/on and with/without 5% increase of M-levels. The main outcome measures were the Acceptable Noise Level (ANL) and the speech reception threshold in continuous background noise (SRTn). Study sample: Participants were 16 experienced cochlear implant recipients with Advanced Bionics implants and a Naida Q70 processor. Results: The ANL significantly improved by using either ClearVoice or an increase of M-levels. Combining both settings gave the largest improvement in ANL. For the SRTn, we found a small, but significant interaction between ClearVoice and an increase of M-levels, implying that ClearVoice improved speech understanding slightly, but only if combined with a 5% increase of M-levels. Conclusions: Optimal profit from ClearVoice is obtained if combined with a structural 5% increase of M-levels.

The Influence of Noise Reduction on Speech Intelligibility, Response Times to Speech, and Perceived Listening Effort in Normal-Hearing Listeners.

Single-microphone noise reduction leads to subjective benefit, but not to objective improvements in speech intelligibility. We investigated whether response times (RTs) provide an objective measure of the benefit of noise reduction and whether the effect of noise reduction is reflected in rated listening effort. Twelve normal-hearing participants listened to digit triplets that were either unprocessed or processed with one of two noise-reduction algorithms: an ideal binary mask (IBM) and a more realistic minimum mean square error estimator (MMSE). For each of these three processing conditions, we measured (a) speech intelligibility, (b) RTs on two different tasks (identification of the last digit and arithmetic summation of the first and last digit), and (c) subjective listening effort ratings. All measurements were performed at four signal-to-noise ratios (SNRs): −5, 0, +5, and +∞ dB. Speech intelligibility was high (>97% correct) for all conditions. A significant decrease in response time, relative to the unprocessed condition, was found for both IBM and MMSE for the arithmetic but not the identification task. Listening effort ratings were significantly lower for IBM than for MMSE and unprocessed speech in noise. We conclude that RT for an arithmetic task can provide an objective measure of the benefit of noise reduction. For young normal-hearing listeners, both ideal and realistic noise reduction can reduce RTs at SNRs where speech intelligibility is close to 100%. Ideal noise reduction can also reduce perceived listening effort.

Comparison of single-microphone noise reduction schemes: can hearing impaired listeners tell the difference?

Objective: The perceived qualities of nine different single-microphone noise reduction (SMNR) algorithms were to be evaluated and compared in subjective listening tests with normal hearing and hearing impaired (HI) listeners. Design: Speech samples added with traffic noise or with party noise were processed by the SMNR algorithms. Subjects rated the amount of speech distortions, intrusiveness of background noise, listening effort and overall quality, using a simplified MUSHRA (ITU-R, 2003) assessment method. Study sample: 18 normal hearing and 18 moderately HI subjects participated in the study. Results: Significant differences between the rating behaviours of the two subject groups were observed: While normal hearing subjects clearly differentiated between different SMNR algorithms, HI subjects rated all processed signals very similarly. Moreover, HI subjects rated speech distortions of the unprocessed, noisier signals as being more severe than the distortions of the processed signals, in contrast to normal hearing subjects. Conclusions: It seems harder for HI listeners to distinguish between additive noise and speech distortions or/and they might have a different understanding of the term “speech distortion” than normal hearing listeners have. The findings confirm that the evaluation of SMNR schemes for hearing aids should always involve HI listeners.

Are Experienced Hearing Aid Users Faster at Grasping the Meaning of a Sentence Than Inexperienced Users? An Eye-Tracking Study

This study assessed the effects of hearing aid (HA) experience on how quickly a participant can grasp the meaning of an acoustic sentence-in-noise stimulus presented together with two similar pictures that either correctly (target) or incorrectly (competitor) depict the meaning conveyed by the sentence. Using an eye tracker, the time taken by the participant to start fixating the target (the processing time) was measured for two levels of linguistic complexity (low vs. high) and three HA conditions: clinical linear amplification (National Acoustic Laboratories-Revised), single-microphone noise reduction with National Acoustic Laboratories-Revised, and linear amplification ensuring a sensation level of ≥ 15 dB up to at least 4 kHz for the speech material used here. Timed button presses to the target stimuli after the end of the sentences (offline reaction times) were also collected. Groups of experienced (eHA) and inexperienced (iHA) HA users matched in terms of age, hearing loss, and working memory capacity took part (N = 15 each). For the offline reaction times, no effects were found. In contrast, processing times increased with linguistic complexity. Furthermore, for all HA conditions, processing times were longer (poorer) for the iHA group than for the eHA group, despite comparable speech recognition performance. Taken together, these results indicate that processing times are more sensitive to speech processing-related factors than offline reaction times. Furthermore, they support the idea that HA experience positively impacts the ability to process noisy speech quickly, irrespective of the precise gain characteristics.

A single microphone noise reduction algorithm based on the detection and reconstruction of spectro-temporal features

Animals throughout the animal kingdom excel at extracting individual sounds from competing background sounds, yet current state-of-the-art signal processing algorithms struggle to process speech in the presence of even modest background noise. Recent psychophysical experiments in humans and electrophysiological recordings in animal models suggest that the brain is adapted to process sounds within the restricted domain of spectro-temporal modulations found in natural sounds. Here, we describe a novel single microphone noise reduction algorithm called spectro-temporal detection–reconstruction (STDR) that relies on an artificial neural network trained to detect, extract and reconstruct the spectro-temporal features found in speech. STDR can significantly reduce the level of the background noise while preserving the foreground speech quality and improving estimates of speech intelligibility. In addition, by leveraging the strong temporal correlations present in speech, the STDR algorithm can also operate on predictions of upcoming speech features, retaining similar performance levels while minimizing inherent throughput delays. STDR performs better than a competing state-of-the-art algorithm for a wide range of signal-to-noise ratios and has the potential for real-time applications such as hearing aids and automatic speech recognition.

Erratum to: Comparative Evaluation of Voice Activity Detectors in Single Microphone Noise Reduction Algorithms

Erratum to: Comparative Evaluation of Voice Activity Detectors in Single Microphone Noise Reduction Algorithms

DFT-Domain Based Single-Microphone Noise Reduction for Speech Enhancement: A Survey of the State of the Art

As speech processing devices like mobile phones, voice controlled devices, and hearing aids have increased in popularity, people expect them to work anywhere and at any time without user intervention. However, the presence of acoustical disturbances limits the use of these applications, degrades their performance, or causes the user difficulties in understanding the conversation or appreciating the device. A common way to reduce the effects of such disturbances is through the use of single-microphone noise reduction algorithms for speech enhancement.

Comparative evaluation of voice activity detectors in single microphone noise reduction algorithms

Purpose Voice activity detection (VAD) is significantly associated with the performance of single microphone noise reduction algorithms. In this study, the cause and extent of the VAD effect on noise reduction performance were analyzed by comparing conventional VAD algorithms with ideal VAD.

Speech enhancement using a generic noise codebook

Although single-microphone noise reduction methods perform well in stationary noise environments, their performance in non-stationary conditions remains unsatisfactory. Use of prior knowledge about speech and noise power spectral densities in the form of trained codebooks has been previously shown to address this limitation. While it is possible to use trained speech codebooks in a practical system, the variety of noise types encountered in practice makes the use of trained noise codebooks less practical. This letter presents a method that uses a generic noise codebook for speech enhancement that can be generated on-the-fly and provides good performance.

Single and Multiple Microphone Noise Reduction Strategies in Cochlear Implants

To restore hearing sensation, cochlear implants deliver electrical pulses to the auditory nerve by relying on sophisticated signal processing algorithms that convert acoustic inputs to electrical stimuli. Although individuals fitted with cochlear implants perform well in quiet, in the presence of background noise, the speech intelligibility of cochlear implant listeners is more susceptible to background noise than that of normal hearing listeners. Traditionally, to increase performance in noise, single-microphone noise reduction strategies have been used. More recently, a number of approaches have suggested that speech intelligibility in noise can be improved further by making use of two or more microphones, instead. Processing strategies based on multiple microphones can better exploit the spatial diversity of speech and noise because such strategies rely mostly on spatial information about the relative position of competing sound sources. In this article, we identify and elucidate the most significant theoretical aspects that underpin single- and multi-microphone noise reduction strategies for cochlear implants. More analytically, we focus on strategies of both types that have been shown to be promising for use in current-generation implant devices. We present data from past and more recent studies, and furthermore we outline the direction that future research in the area of noise reduction for cochlear implants could follow.

Measurement and prediction of the acceptable noise level for single-microphone noise reduction algorithms

Objective: To measure the acceptable noise level (ANL) with and without noise reduction algorithms (NRAs), and to predict ΔANL, i.e. the difference in acceptable noise level with and without NRAs. Design: The ANL test was applied to three NRAs. Furthermore, the measured ΔANL was predicted using several methods based on either the calculation of the signal-to-noise ratio or correlation methods of the processed signals with an unprocessed reference signal. Study sample: Ten normal-hearing and eleven hearing-impaired subjects accomplished the ANL test. Results: In general, the ANL test could determine an increased acceptance of noise with some NRAs. However, great inter-individual differences also resulted that were attributed to audible distortions when an NRA was used. Prediction of the mean measured DANL was possible, but individual prediction of DANL failed due to inter-individual differences. Mean DANL was predicted more accurately for hearing-impaired subjects when individual hearing loss was taken into account. Conclusions: The ANL test is a suitable tool for measuring the advantage of one NRA. A prediction of the measured individual ΔANL failed. However, mean DANL could be predicted with some methods. Furthermore, the individual hearing loss should be taken into account for a more accurate prediction for hearing-impaired subjects.

TWO MICROPHONE NOISE REDUCTION SYSTEM

A two microphone noise reduction system is described. In an embodiment, input signals from each of the microphones are divided into subbands and each subband is then filtered independently to separate noise and desired signals and to suppress non-stationary and stationary noise. Filtering methods used include adaptive decorrelation filtering. A post-processing module using adaptive noise cancellation like filtering algorithms may be used to further suppress stationary and non-stationary noise in the output signals from the adaptive decorrelation filtering and a single microphone noise reduction algorithm may be used to further provide optimal stationary noise reduction performance of the system.

A comparative intelligibility study of single-microphone noise reduction algorithms

The evaluation of intelligibility of noise reduction algorithms is reported. IEEE sentences and consonants were corrupted by four types of noise including babble, car, street and train at two signal-to-noise ratio levels (0 and 5 dB), and then processed by eight speech enhancement methods encompassing four classes of algorithms: spectral subtractive, sub-space, statistical model based and Wiener-type algorithms. The enhanced speech was presented to normal-hearing listeners for identification. With the exception of a single noise condition, no algorithm produced significant improvements in speech intelligibility. Information transmission analysis of the consonant confusion matrices indicated that no algorithm improved significantly the place feature score, significantly, which is critically important for speech recognition. The algorithms which were found in previous studies to perform the best in terms of overall quality, were not the same algorithms that performed the best in terms of speech intelligibility. The subspace algorithm, for instance, was previously found to perform the worst in terms of overall quality, but performed well in the present study in terms of preserving speech intelligibility. Overall, the analysis of consonant confusion matrices suggests that in order for noise reduction algorithms to improve speech intelligibility, they need to improve the place and manner feature scores.

Speech pause detection for noise spectrum estimation by tracking power envelope dynamics

A speech pause detection algorithm is an important and sensitive part of most single-microphone noise reduction schemes for enhancement of speech signals corrupted by additive noise as an estimate of the background noise is usually determined when speech is absent. An algorithm is proposed which detects speech pauses by adaptively tracking minima in a noisy signal's power envelope both for the broadband signal and for the high-pass and low-pass filtered signal. In poor signal-to-noise ratios (SNRs), the proposed algorithm maintains a low false-alarm rate in the detection of speech pauses while the standardized algorithm of ITU G.729 shows an increasing false-alarm rate in unfavorable situations. These characteristics are found with different types of noise and indicate that the proposed algorithm is better suited to be used for noise estimation in noise reduction algorithms, as speech deterioration may thus be kept at a low level. It is shown that in connection with the Ephraim-Malah (1984) noise reduction scheme, the speech pause detection performance can even be further increased by using the noise-reduced signal instead of the noisy signal as input for the speech pause decision unit.