Cochlear Implant Simulations Research Articles

Perceptual learning of speech processed by a Cochlear Implant simulator — An fMRI investigation

Perceptual learning of speech processed by a Cochlear Implant simulator — An fMRI investigation

Speech recognition with varying numbers and types of competing talkers by normal-hearing, cochlear-implant, and implant simulation subjects

Cochlear-implant users perform far below normal-hearing subjects in background noise. Speech recognition with varying numbers of competing female, male, and child talkers was evaluated in normal-hearing subjects, cochlear-implant users, and normal-hearing subjects utilizing an eight-channel sine-carrier cochlear-implant simulation. Target sentences were spoken by a male. Normal-hearing subjects obtained considerably better speech reception thresholds than cochlear-implant subjects; the largest discrepancy was 24 dB with a female masker. Evaluation of one implant subject with normal hearing in the contralateral ear suggested that this difference is not caused by age-related disparities between the subject groups. Normal-hearing subjects showed a significant advantage with fewer competing talkers, obtaining release from masking with up to three talker maskers. Cochlear-implant and simulation subjects showed little such effect, although there was a substantial difference between the implant and simulation results with talker maskers. All three groups benefited from a voice pitch difference between target and masker, with the female talker providing significantly less masking than the male. Child talkers produced more masking than expected, given their fundamental frequency, syllabic rate, and temporal modulation characteristics. Neither a simulation nor testing in steady-state noise predicts the difficulties cochlear-implant users experience in real-life noisy situations.

Evaluation of companding-based spectral enhancement using simulated cochlear-implant processing

This study tested a time-domain spectral enhancement algorithm that was recently proposed by Turicchia and Sarpeshkar [IEEE Trans. Speech Audio Proc. 13, 243-253 (2005)]. The algorithm uses a filter bank, with each filter channel comprising broadly tuned amplitude compression, followed by more narrowly tuned expansion (companding). Normal-hearing listeners were tested in their ability to recognize sentences processed through a noise-excited envelope vocoder that simulates aspects of cochlear-implant processing. The sentences were presented in a steady background noise at signal-to-noise ratios of 0, 3, and 6 dB and were either passed directly through an envelope vocoder, or were first processed by the companding algorithm. Using an eight-channel envelope vocoder, companding produced small but significant improvements in speech reception. Parametric variations of the companding algorithm showed that the improvement in intelligibility was robust to changes in filter tuning, whereas decreases in the time constants resulted in a decrease in intelligibility. Companding continued to provide a benefit when the number of vocoder frequency channels was increased to sixteen. When integrated within a sixteen-channel cochlear-implant simulator, companding also led to significant improvements in sentence recognition. Thus, companding may represent a readily implementable way to provide some speech recognition benefits to current cochlear-implant users.

Perceptual adaptation by normally hearing listeners to a simulated “hole” in hearing

Simulations of cochlear implants have demonstrated that the deleterious effects of a frequency misalignment between analysis bands and characteristic frequencies at basally shifted simulated electrode locations are significantly reduced with training. However, a distortion of frequency-to-place mapping may also arise due to a region of dysfunctional neurons that creates a "hole" in the tonotopic representation. This study simulated a 10 mm hole in the mid-frequency region. Noise-band processors were created with six output bands (three apical and three basal to the hole). The spectral information that would have been represented in the hole was either dropped or reassigned to bands on either side. Such reassignment preserves information but warps the place code, which may in itself impair performance. Normally hearing subjects received three hours of training in two reassignment conditions. Speech recognition improved considerably with training. Scores were much lower in a baseline (untrained) condition where information from the hole region was dropped. A second group of subjects trained in this dropped condition did show some improvement; however, scores after training were significantly lower than in the reassignment conditions. These results are consistent with the view that speech processors should present the most informative frequency range irrespective of frequency misalignment.

Unintelligible Low-Frequency Sound Enhances Simulated Cochlear-Implant Speech Recognition in Noise

Speech can be recognized by multiple acoustic cues in both frequency and time domains. These acoustic cues are often thought to be redundant. One example is the low-frequency sound component below 300 Hz, which is not even transmitted by the majority of communication devices including telephones. Here, we showed that this low-frequency sound component, although unintelligible when presented alone, could improve the functional signal-to-noise ratio (SNR) by 10-15 dB for speech recognition in noise when presented in combination with a cochlear-implant simulation. A similar low-frequency enhancement effect could be obtained by presenting the low-frequency sound component to one ear and the cochlear-implant simulation to the other ear. However, a high-frequency sound could not produce a similar speech enhancement in noise. We argue that this low-frequency enhancement effect cannot be due to linear addition of intelligibility between low- and high-frequency components or an increase in the physical SNR. We suggest a brain-based mechanism that uses the voice pitch cue in the low-frequency sound to first segregate the target voice from the competing voice and then to group appropriate temporal envelope cues in the target voice for robust speech recognition under realistic listening situations.

Contribution of low-frequency acoustic information to Chinese speech recognition in cochlear implant simulations

Chinese sentence recognition strongly relates to the reception of tonal information. For cochlear implant (CI) users with residual acoustic hearing, tonal information may be enhanced by restoring low-frequency acoustic cues in the nonimplanted ear. The present study investigated the contribution of low-frequency acoustic information to Chinese speech recognition in Mandarin-speaking normal-hearing subjects listening to acoustic simulations of bilaterally combined electric and acoustic hearing. Subjects listened to a 6-channel CI simulation in one ear and low-pass filtered speech in the other ear. Chinese tone, phoneme, and sentence recognition were measured in steady-state, speech-shaped noise, as a function of the cutoff frequency for low-pass filtered speech. Results showed that low-frequency acoustic information below 500 Hz contributed most strongly to tone recognition, while low-frequency acoustic information above 500 Hz contributed most strongly to phoneme recognition. For Chinese sentences, speech reception thresholds (SRTs) improved with increasing amounts of low-frequency acoustic information, and significantly improved when low-frequency acoustic information above 500 Hz was preserved. SRTs were not significantly affected by the degree of spectral overlap between the CI simulation and low-pass filtered speech. These results suggest that, for CI patients with residual acoustic hearing, preserving low-frequency acoustic information can improve Chinese speech recognition in noise.

Effects of talker variability on vowel recognition by cochlear implant users and normal-hearing subjects listening to cochlear implant simulations

Different talker characteristics can significantly affect speech recognition, especially under conditions of reduced spectral resolution. In the current study, single- and multi-talker vowel recognition was tested with both cochlear implant (CI) patients and normal-hearing (NH) subjects listening to CI simulations. CI users were tested with their everyday processors. NH subjects were tested using four-channel acoustic processors in which the envelope cutoff frequency (20, 160 Hz) and the carrier band (noise-band, sine-wave) conditions were varied. Results showed that CI users’ single-talker vowel recognition was significantly better than multi-talker recognition. Among the simulations, both single- and multi-talker vowel recognition was best with the 160 Hz/sine-wave processor. Also, single-talker performance was significantly better than multi-talker performance only with the 160 Hz/sine-wave processor; there was no difference between single- and multi-talker recognition with the 20 Hz/sine-wave or noise-band processor. While the effects of talker variability were less for the 20 Hz/sine-wave and noise-band processors, overall vowel recognition was also significantly lower. Listeners’ sensitivity to talker variability may indicate their ability to register the relative vowel space for individual talkers. Without sensitivity to talker variability, multi-talker vowel space may collapse, causing cross-talker differences to be confused with phonetic differences. [Research is supported by NIDCD.]

Effects on performance of partial misalignments of spectral information in acoustic simulations of cochlear implants

Objective: To assess the effects on speech understanding of frequency misalignments in channel outputs for an acoustic model of a cochlear implant. Design: Consonants, vowels, and sentences were processed through a four-channel sine-wave simulation of a cochlear implant. In the five experimental conditions, channels 1 and 3 were always output at the correct frequency while channels 2 and 4 were output at frequencies varying from the correct frequency to frequencies 25%, 50%, and 75% lower than appropriate. In a fifth condition (2-of-4 condition), channels 2 and 4 were turned off. Results: Consonant recognition was reduced significantly with a 50% shift in channels 2 and 4. Vowel and sentence recognition were reduced significantly with a 75% shift in channels 2 and 4. For all material performance in the 75% shift condition was better than in the 2-of-4 condition. Conclusions: The perceiving system that underlies speech recognition is relatively tolerant of misplaced frequency information if other information is presented to the correct frequency location. This flexibility may account for some of the success of cochlear implants. [Research supported by a grant from NIDCD (No. RO1 00654-12) to the second author.]

The effect of spatial separation on informational and energetic masking of speech.

The effect of spatial separation of sources on the masking of a speech signal was investigated for three types of maskers, ranging from energetic to informational. Normal-hearing listeners performed a closed-set speech identification task in the presence of a masker at various signal-to-noise ratios. Stimuli were presented in a quiet sound field. The signal was played from 0 degrees azimuth and a masker was played either from the same location or from 90 degrees to the right. Signals and maskers were derived from sentences that were preprocessed by a modified cochlear-implant simulation program that filtered each sentence into 15 frequency bands, extracted the envelopes from each band, and used these envelopes to modulate pure tones at the center frequencies of the bands. In each trial, the signal was generated by summing together eight randomly selected frequency bands from the preprocessed signal sentence. Three maskers were derived from the preprocessed masker sentences: (1) different-band sentence, which was generated by summing together six randomly selected frequency bands out of the seven bands not present in the signal (resulting in primarily informational masking); (2) different-band noise, which was generated by convolving the different-band sentence with Gaussian noise; and (3) same-band noise, which was generated by summing the same eight bands from the preprocessed masker sentence that were used in the signal sentence and convolving the result with Gaussian noise (resulting in primarily energetic masking). Results revealed that in the different-band sentence masker, the effect of spatial separation averaged 18 dB (at 51% correct), while in the different-band and same-band noise maskers the effect was less than 10 dB. These results suggest that, in these conditions, the advantage due to spatial separation of sources is greater for informational masking than for energetic masking.

Identification of musical instruments by normal-hearing subjects listening through a cochlear-implant simulation

Signal processing in a cochlear implant (CI) is primarily designed to convey speech and environmental sounds, and can cause distortion of musical timbre. Systematic investigation of musical instrument identification through a CI has not yet revealed how timbre is affected by the implant’s processing. In this experiment, the bandpass filtering, rectification, and low-pass filtering of an implant are simulated in matlab. Synthesized signals representing 12 common instruments, each performing a major scale, are processed by simulations using up to 8 analysis channels. The unprocessed recordings, together with the 8 simulation conditions for 12 instruments, are presented in random order to each of the subjects. The subject’s task is to identify the instrument represented by each item. The subjects also subjectively score each item based on similarity and pleasantness. We anticipate performance using the simulation will be worse than the unprocessed condition because of the limited information delivered by the envelopes of the analysis channels. These results will be analyzed as a confusion matrix and provide a basis for contrasting the information used by subjects listening to the unprocessed and processed materials. Understanding these differences should aid in the development of new processing strategies to better represent music for cochlear implant users.

Cochlear-implant simulations of speech in complex acoustic backgrounds

Cochlear-implant simulations have shown that good speech intelligibility in quiet can be obtained with very poor spectral resolution and no temporal fine-structure information. It is not known how robust speech is to such processing in the presence of complex interfering sounds. This study examined speech reception for sentences in the presence of four types of interference (broadband noise, broadband noise modulated with a speech envelope, a single male talker, and a single female talker) and three processing schemes (24- and 8-channel cochlear-implant simulations and unprocessed). The spectral resolution in the 24-channel condition was designed to be comparable to that of normal hearing, but the original temporal fine structure was replaced by noise in both the 24- and 8-channel processing conditions. Performance with 24-channel processing was substantially poorer than in unprocessed conditions, despite the relatively good frequency resolution. Eight-channel processing produced the poorest performance. Stimulus processing was generally more detrimental to speech reception in the fluctuating interferers than in the unmodulated noise. The detrimental effects of eliminating fine-structure information may be due to the reduction in pitch cues, which listeners may use to help segregate the sources. [Work supported by NIDCD Grant R01DC05216.]

Hot topics in speech communication

Speech communication is a robust and flexible process. For listeners with normal hearing, speech remains intelligible when a substantial proportion of the spectrum is eliminated by filtering, when portions of the wave form are replaced with silence, and in the presence of interfering sounds, including sounds made by competing talkers. The resilience of speech depends on several interacting systems which exploit the redundancy of speech in different ways. New results are reported on the perceptual processing of talker variability, monaural and binaural mechanisms of sound separation, audio-visual integration in lip reading, and perceptual compensation for channel distortion. For listeners with hearing impairment, new research in the field of cochlear implants has focused on the development of new electrode arrays, the efficacy of bilateral implants and the development of sophisticated signal processing techniques to combat noise. The new electrode arrays are designed to limit electrode interaction and thus increase the number of perceptual channels of information. The benefit of bilateral implants is investigated in terms of sound localization and speech perception in noise. New results are reported on bilateral cochlear implants, speech-processing strategies for noisy environments and acoustic simulations of cochlear implants which aim to quantify the factors contributing to speech intelligibility.

Effects of the number of channels and speech-to-noise ratio on rate of connected discourse tracking through a simulated cochlear implant speech processor.

To investigate the effects of number of channels and speech-to-noise ratio on connected discourse tracking (CDT) through simulations of cochlear implant speech processing. Previous studies have used citation-form vowel and consonant materials or simple sentences. CDT rates were expected to be less likely to be limited by ceiling effects and more representative of everyday speech communication. Four normal-hearing subjects were presented with speech processed through a real-time sine-excited vocoder having three, four, eight, or 12 channels. Amplitude envelopes extracted from each band modulated sinusoidal carrier signals placed at each band center frequency. Speech-spectrum shaped noise was added to speech before vocoder processing at three signal to noise ratios based on real-time measurements of speech level (+7, +12, +17 dB). CDT rates increased significantly with number of channels up to eight in both quiet and noise, and decreased significantly with each increase in noise level from quiet. The effects on CDT rates of the number of channels and speech-to-noise ratio are highly correlated with intelligibility measures for Hearing in Noise Test (HINT) sentences, consonants and vowels. However, HINT sentence scores even in noise show ceiling effects that obscure the advantages of processors with eight or more channels. Moderate levels of noise that have only slight effects on other measures significantly affected CDT rate. CDT rates with three or four bands of spectral information were much lower than asymptotic rates, especially in the presence of noise.