Unprocessed Speech Research Articles

Noise estimation based on optimal smoothing and minimum controlled through recursive averaging for speech enhancement

One of the most significant challenges in the real time speech processing applications is the elimination of noise in the corrupted speech data. This noise can significantly impact the efficacy/performance of the applications of speech processing. However, developing a robust noise reduction algorithm is crucial for improving the accuracy of automatic speech recognition (ASR) and other speech processing systems under uncontrolled conditions. In this paper, we propose an algorithm to reduce the background noise in the degraded speech data under highly non-stationary conditions. The proposed optimal smoothing and minima controlled (OSMC) technique uses recursive averaging to enhance degraded speech data. Initially, a smoothed periodogram and local minima of the degraded speech data are computed and determined the time-frequency dependent threshold factor. The ratio of smoothed periodogram to local minima is used to find the active regions of speech in the degraded speech data by adapting the Bayesian minimum cost decision rule. To calculate the estimated noise spectrum for each frequency bin, a time-frequency smoothing factors are used. The perceptual evaluation of speech quality (PESQ) and normalized covariance metric (NCM) are used to evaluate the speech quality and intelligibility of the proposed technique over competing algorithms after speech enhancement. The experimental results demonstrated that the proposed algorithm has given a significant improvement in terms of average values of PESQ by 15.03% and 16.71% and NCM by 3.45% and 5.73% for NOIZEUS and Kannada speech databases at 5 dB and 10 dB respectively, over unprocessed speech under highly non-stationary noisy environments.

Phonological discrimination and contrast detection in pupillometry.

The perception of phonemes is guided by both low-level acoustic cues and high-level linguistic context. However, differentiating between these two types of processing can be challenging. In this study, we explore the utility of pupillometry as a tool to investigate both low- and high-level processing of phonological stimuli, with a particular focus on its ability to capture novelty detection and cognitive processing during speech perception. Pupillometric traces were recorded from a sample of 22 Danish-speaking adults, with self-reported normal hearing, while performing two phonological-contrast perception tasks: a nonword discrimination task, which included minimal-pair combinations specific to the Danish language, and a nonword detection task involving the detection of phonologically modified words within sentences. The study explored the perception of contrasts in both unprocessed speech and degraded speech input, processed with a vocoder. No difference in peak pupil dilation was observed when the contrast occurred between two isolated nonwords in the nonword discrimination task. For unprocessed speech, higher peak pupil dilations were measured when phonologically modified words were detected within a sentence compared to sentences without the nonwords. For vocoded speech, higher peak pupil dilation was observed for sentence stimuli, but not for the isolated nonwords, although performance decreased similarly for both tasks. Our findings demonstrate the complexity of pupil dynamics in the presence of acoustic and phonological manipulation. Pupil responses seemed to reflect higher-level cognitive and lexical processing related to phonological perception rather than low-level perception of acoustic cues. However, the incorporation of multiple talkers in the stimuli, coupled with the relatively low task complexity, may have affected the pupil dilation.

Accent rating of noise- and tone-vocoded foreign-accented speech

There are abundant acoustic-phonetic cues in speech signals for listeners to encode talker identity. However, speech signals in the real world are always less optimal due to various adverse listening sources. Vocoded speech is one type of simplified signal that has less spectral and/or temporal information in comparison to normal speech. The purpose of this study is to examine whether and how listeners’ judgment of talker accent is affected by noise and tone vocoding. Twelve Mandarin-accented English speakers with varying degree of accentedness and two native English speakers were recorded reading “The Rainbow Passage.” The recorded speech samples from each talker were segmented into small sections that were randomly selected for noise- or tone-excited vocoder processing into 1, 2, 4, 8, and 16 channels. The vocoded and unprocessed speech samples were randomly presented to a group of normal-hearing, monolingual English listeners for accent rating. The listeners judged the degree of talker accent on a 9-point Likert scale with “1” representing no accent and “9” representing extremely strong accent. The data are still in the process of being collected and analyzed. Results and implications of the present study will be discussed.

Age-related rollover for spectrally degraded temporal speech contrasts

Some speech sounds, or phonemes, are primarily distinguished acoustically from other phonemes by temporal cues. A decline in temporal-processing abilities is a common consequence of aging that might cause a listener to have difficulty differentiating between sounds, especially when the speech is degraded. One type of signal degradation occurs in a cochlear-implant vocoder simulation, which eliminates most of the spectral information in speech while maintaining the temporal envelope. In addition, temporal processing abilities may be challenged at higher presentation levels causing a decrease in performance (i.e., rollover), particularly in older listeners. This study measures the phoneme classification of younger and older listeners of four spectro-temporal contrasts when presented in quiet with and without vocoding at a range of presentation levels. We hypothesized that, due to declines in temporal processing abilities, older listeners would be more susceptible to rollover, measured by a decrease in phoneme classification performance for degraded temporal contrasts as presentation levels increase. We further hypothesized that performance rollover would occur more often for vocoded speech than for unprocessed speech, especially in older listeners. Preliminary data show interactions between speech contrasts, presentation levels, and vocoding. Our results will help facilitate understanding speech perception difficulties in older cochlear-implant users.

Speech Enhancement Algorithm Based on a Convolutional Neural Network Reconstruction of the Temporal Envelope of Speech in Noisy Environments

Temporal modulation processing is a promising technique for improving the intelligibility and quality of speech in noise. We propose a speech enhancement algorithm that constructs the temporal envelope (TEV) in the time-frequency domain by means of an embedded convolutional neural network (CNN). To accomplish this, the input speech signals are divided into sixteen parallel frequency bands (subbands) with bandwidths approximating 1.5 times that of auditory filters. The corrupted TEVs in each subband are extracted and then fed to the 1-dimensional CNN (1-D CNN) model to restore the TEVs distorted by noise. The method is evaluated using 2,700 words from nine different talkers, which are mixed with speech-spectrum shaped random noise (SSN), and babble noise, at different signal-to-noise ratios. The Short-time Objective Intelligibility (STOI) and Perceptual Evaluation of Speech Quality (PESQ) metrics are used to evaluate the performance of the 1-D CNN algorithm. Results suggest that the 1-D CNN model improves STOI scores on average by 27% and 34% for SSN and babble noise, respectively, and PESQ scores on average by 19% and 18%, respectively, compared to unprocessed speech. The 1-D CNN model is also shown to outperform a conventional TEV-based speech enhancement algorithm.

Severity Prediction over Parkinson's Disease Prediction by Using the Deep Brooke Inception Net Classifier.

Parkinson's disease (PD) is a neurodegenerative illness that progresses and is long-lasting. It becomes more difficult to talk, write, walk, and do other basic functions when the brain's dopamine-generating neurons are injured or killed. There is a gradual rise in the intensity of these symptoms over time. Using Parkinson's Telemonitoring Voice Data Set from UCI and deep neural networks, we provide a strategy for predicting the severity of Parkinson's disease in this research. An unprocessed speech recording contains a slew of unintelligible data that makes correct diagnosis difficult. Therefore, the raw signal data must be preprocessed using the signal error drop standardization while the features can be grouped by using the wavelet cleft fuzzy algorithm. Then the abnormal features can be selected by using the firming bacteria foraging algorithm for feature size decomposition process. Then classification was made using the deep brooke inception net classifier. The performances of the classifier are compared where the simulation results show that the proposed strategy accuracy in detecting severity of the Parkinson's disease is better than other conventional methods. The proposed DBIN model achieved better accuracy compared to other existing techniques. It is also found that the classification based on extracted voice abnormality data achieves better accuracy (99.8%) over PD prediction; hence it can be concluded as a better metric for severity prediction.

Beyond Recognition: Visual Contributions to Verbal Working Memory.

It is well recognized that adding the visual to the acoustic speech signal improves recognition when the acoustic signal is degraded, but how that visual signal affects postrecognition processes is not so well understood. This study was designed to further elucidate the relationships among auditory and visual codes in working memory, a postrecognition process. In a main experiment, 80 young adults with normal hearing were tested using an immediate serial recall paradigm. Three types of signals were presented (unprocessed speech, vocoded speech, and environmental sounds) in three conditions (audio-only, audio-video with dynamic visual signals, and audio-picture with static visual signals). Three dependent measures were analyzed: (a) magnitude of the recency effect, (b) overall recall accuracy, and (c) response times, to assess cognitive effort. In a follow-up experiment, 30 young adults with normal hearing were tested largely using the same procedures, but with a slight change in order of stimulus presentation. The main experiment produced three major findings: (a) unprocessed speech evoked a recency effect of consistent magnitude across conditions; vocoded speech evoked a recency effect of similar magnitude to unprocessed speech only with dynamic visual (lipread) signals; environmental sounds never showed a recency effect. (b) Dynamic and static visual signals enhanced overall recall accuracy to a similar extent, and this enhancement was greater for vocoded speech and environmental sounds than for unprocessed speech. (c) All visual signals reduced cognitive load, except for dynamic visual signals with environmental sounds. The follow-up experiment revealed that dynamic visual (lipread) signals exerted their effect on the vocoded stimuli by enhancing phonological quality. Acoustic and visual signals can combine to enhance working memory operations, but the source of these effects differs for phonological and nonphonological signals. Nonetheless, visual information can support better postrecognition processes for patients with hearing loss.

Perception of indexical cues in speech by children and adults with and without dyslexia: Regional dialect and gender identification.

Auditory research in developmental dyslexia proposes that deficient auditory processing of speech underlies difficulties with reading and spelling. Focusing predominantly on phonological processing, studies have not yet addressed the role of the speaker-related (indexical) properties of speech that enable the formation of phonological representations. Here, we assess auditory processing of indexical characteristics cueing a speaker's regional dialect and gender to determine whether dyslexia constraints recognition of dialect features and voice gender. Adults and children aged 11-14 years with dyslexia and their age-matched controls responded to 360 unique sentences extracted from spontaneous conversations of 40 speakers. In addition to the original unprocessed speech, there were two focused filtered conditions (using lowpass filtering at 400 Hz and 8-channel noise vocoding) probing listeners' responses to segmental and prosodic cues. Compared with controls, both groups with dyslexia were significantly limited in their abilities to recognize dialect features from either set of cues. The results for gender suggest that their comparatively worse gender recognition in the noise-vocoded condition was possibly related to poor temporal resolution. We propose that the deficient processing of indexical cues by individuals with dyslexia originates in peripheral auditory processes, of which impaired processing of relevant temporal cues in amplitude envelope is a likely candidate.

Accuracy and cue use in word segmentation for cochlear-implant listeners and normal-hearing listeners presented vocoded speech.

Cochlear-implant (CI) listeners experience signal degradation, which leads to poorer speech perception than normal-hearing (NH) listeners. In the present study, difficulty with word segmentation, the process of perceptually parsing the speech stream into separate words, is considered as a possible contributor to this decrease in performance. CI listeners were compared to a group of NH listeners (presented with unprocessed speech and eight-channel noise-vocoded speech) in their ability to segment phrases with word segmentation ambiguities (e.g., "an iceman" vs "a nice man"). The results showed that CI listeners and NH listeners were worse at segmenting words when hearing processed speech than NH listeners were when presented with unprocessed speech. When viewed at a broad level, all of the groups used cues to word segmentation in similar ways. Detailed analyses, however, indicated that the two processed speech groups weighted top-down knowledge cues to word boundaries more and weighted acoustic cues to word boundaries less relative to NH listeners presented with unprocessed speech.

Speech intelligibility and talker gender classification with noise-vocoded and tone-vocoded speech.

Vocoded speech provides less spectral information than natural, unprocessed speech, negatively affecting listener performance on speech intelligibility and talker gender classification tasks. In this study, young normal-hearing participants listened to noise-vocoded and tone-vocoded (i.e., sinewave-vocoded) sentences containing 1, 2, 4, 8, 16, or 32 channels, as well as non-vocoded sentences, and reported the words heard as well as the gender of the talker. Overall, performance was significantly better with tone-vocoded than noise-vocoded speech for both tasks. Within the talker gender classification task, biases in performance were observed for lower numbers of channels, especially when using the noise carrier.

Age-Related Changes in Voice Emotion Recognition by Postlingually Deafened Listeners With Cochlear Implants.

Identification of emotional prosody in speech declines with age in normally hearing (NH) adults. Cochlear implant (CI) users have deficits in the perception of prosody, but the effects of age on vocal emotion recognition by adult postlingually deaf CI users are not known. The objective of the present study was to examine age-related changes in CI users' and NH listeners' emotion recognition. Participants included 18 CI users (29.6 to 74.5 years) and 43 NH adults (25.8 to 74.8 years). Participants listened to emotion-neutral sentences spoken by a male and female talker in five emotions (happy, sad, scared, angry, neutral). NH adults heard them in four conditions: unprocessed (full spectrum) speech, 16-channel, 8-channel, and 4-channel noise-band vocoded speech. The adult CI users only listened to unprocessed (full spectrum) speech. Sensitivity (d') to emotions and Reaction Times were obtained using a single-interval, five-alternative, forced-choice paradigm. For NH participants, results indicated age-related declines in Accuracy and d', and age-related increases in Reaction Time in all conditions. Results indicated an overall deficit, as well as age-related declines in overall d' for CI users, but Reaction Times were elevated compared with NH listeners and did not show age-related changes. Analysis of Accuracy scores (hit rates) were generally consistent with d' data. Both CI users and NH listeners showed age-related deficits in emotion identification. The CI users' overall deficit in emotion perception, and their slower response times, suggest impaired social communication which may in turn impact overall well-being, particularly so for older CI users, as lower vocal emotion recognition scores have been associated with poorer subjective quality of life in CI patients.

Emotion and Word Recognition for Unprocessed and Vocoded Speech Stimuli.

Individuals with cochlear implants (CIs) show reduced word and auditory emotion recognition abilities relative to their peers with normal hearing. Modern CI processing strategies are designed to preserve acoustic cues requisite for word recognition rather than those cues required for accessing other signal information (e.g., talker gender or emotional state). While word recognition is undoubtedly important for communication, the inaccessibility of this additional signal information in speech may lead to negative social experiences and outcomes for individuals with hearing loss. This study aimed to evaluate whether the emphasis on word recognition preservation in CI processing has unintended consequences on the perception of other talker information, such as emotional state. Twenty-four young adult listeners with normal hearing listened to sentences and either reported a target word in each sentence (word recognition task) or selected the emotion of the talker (emotion recognition task) from a list of options (Angry, Calm, Happy, and Sad). Sentences were blocked by task type (emotion recognition versus word recognition) and processing condition (unprocessed versus 8-channel noise vocoder) and presented randomly within the block at three signal-to-noise ratios (SNRs) in a background of speech-shaped noise. Confusion matrices showed the number of errors in emotion recognition by listeners. Listeners demonstrated better emotion recognition performance than word recognition performance at the same SNR. Unprocessed speech resulted in higher recognition rates than vocoded stimuli. Recognition performance (for both words and emotions) decreased with worsening SNR. Vocoding speech resulted in a greater negative impact on emotion recognition than it did for word recognition. These data confirm prior work that suggests that in background noise, emotional prosodic information in speech is easier to recognize than word information, even after simulated CI processing. However, emotion recognition may be more negatively impacted by background noise and CI processing than word recognition. Future work could explore CI processing strategies that better encode prosodic information and investigate this effect in individuals with CIs as opposed to vocoded simulation. This study emphasized the need for clinicians to consider not only word recognition but also other aspects of speech that are critical to successful social communication.

Perception of temporally enhanced and hearing aid processed speech in children with late-onset Auditory Neuropathy Spectrum Disorder

ObjectiveTo investigate phrase perception and subjective quality preference of temporal enhancement-based speech processing strategies: Deep-band modulation and Stretching, and hearing aid processed speech in adolescents with late-onset Auditory Neuropathy. Methods15 participants with Auditory Neuropathy Spectrum Disorder were involved. Speech perception was assessed using unprocessed, deep-band modulated, stretched, and hearing aid processed conditions. Subjective preference was also assessed using the pair-wise comparison technique. Gap detection test was done using broadband noise to investigate the possible correlation with benefit from processing strategies. ResultsAs a group, no significant benefits were found with processed speech. Clinically relevant trends emerged on subdividing the group into good and poor performers. Deep-band modulation processing was significantly better than unprocessed speech in those with poorer speech recognition abilities. There was a trend for those with poorer temporal processing abilities to benefit more with deep-band modulation processing. In those with relatively better speech recognition abilities, processing showed no benefit, and hearing aid processed speech was inferior to unprocessed speech. Quality-wise, the unprocessed speech was preferred by all the participants. Among the processed conditions, deep-band modulation was preferred by most, followed by stretching and hearing aid processed speech. ConclusionResults indicate that temporal enhancement strategies may be beneficial for a subset of individuals with ANSD with poorer temporal processing and speech perception abilities. However, processing strategies need to take into account the quality aspect and not just intelligibility.

Testing Speech Perception with Cochlear Implants Through Digital Audio Streaming in a Virtual Sound Booth: A Feasibility Study

For patients who have received cochlear implants (CIs), speech-perception testing requires specialized equipment. This limits locations where these services can be provided, which can introduce barriers for provision of care. Providing speech test stimuli directly to the CI via wireless digital audio streaming (DAS) or wired direct audio input (DAI) allows for testing without the need for a sound booth (SB). A few studies have investigated the use of DAI for testing speech perception in CIs, but none have evaluated DAS. The goal of this study was to compare speech perception testing in CI users via DAS versus a traditional SB to determine if differences exist between the two presentation modes. We also sought to determine whether pre-processing the DAS signal with room acoustics (reverberation and noise floor) to emulate the SB environment would affect performance differences between the SB and DAS. In Experiment 1, speech perception was measured for monosyllabic words in quiet and sentences in quiet and in noise. Scores were obtained in a SB and compared to those obtained via DAS with unprocessed speech (DAS-U) for 11 adult CI users (12 ears). In Experiment 2, speech perception was measured for sentences in noise, where both the speech and noise stimuli were pre-processed to emulate the SB environment. Scores were obtained for 11 adult CI users (12 ears) in the SB, via DAS-U, and via DAS with the processed speech (DAS-P). For Experiment 1, there was no significant difference between SB and DAS-U conditions for words or sentences in quiet. However, DAS-U scores were significantly better than SB scores for sentences in noise. For Experiment 2, there was no significant difference between the SB and DAS-P conditions. Similar to Experiment 1, DAS-U scores were significantly better than SB or DAS-P scores. By pre-processing the test materials to emulate the noise and reverberation characteristics of a traditional SB, we can account for differences in speech-perception scores between those obtained via DAS and in a SB.

Perceptual Learning of Vocoded Speech With and Without Contralateral Hearing: Implications for Cochlear Implant Rehabilitation.

Purpose An increasing number of individuals with residual or even normal contralateral hearing are being considered for cochlear implantation. It remains unknown whether the presence of contralateral hearing is beneficial or detrimental to their perceptual learning of cochlear implant (CI)-processed speech. The aim of this experiment was to provide a first insight into this question using acoustic simulations of CI processing. Method Sixty normal-hearing listeners took part in an auditory perceptual learning experiment. Each subject was randomly assigned to one of three groups of 20 referred to as NORMAL, LOWPASS, and NOTHING. The experiment consisted of two test phases separated by a training phase. In the test phases, all subjects were tested on recognition of monosyllabic words passed through a six-channel "PSHC" vocoder presented to a single ear. In the training phase, which consisted of listening to a 25-min audio book, all subjects were also presented with the same vocoded speech in one ear but the signal they received in their other ear differed across groups. The NORMAL group was presented with the unprocessed speech signal, the LOWPASS group with a low-pass filtered version of the speech signal, and the NOTHING group with no sound at all. Results The improvement in speech scores following training was significantly smaller for the NORMAL than for the LOWPASS and NOTHING groups. Conclusions This study suggests that the presentation of normal speech in the contralateral ear reduces or slows down perceptual learning of vocoded speech but that an unintelligible low-pass filtered contralateral signal does not have this effect. Potential implications for the rehabilitation of CI patients with partial or full contralateral hearing are discussed.

Acoustical and Perceptual Analysis of Noise Reduction Strategies in Individuals With Auditory Neuropathy Spectrum Disorders.

Purpose The conventional amplification devices render minimal or no benefit at abating the speech perception problems of individuals with auditory neuropathy spectrum disorder (ANSD). This study was undertaken to evaluate the effect of noise reduction strategies (multiband spectral subtraction, Wiener-as, Karhunen-Loeve transform [Subspace], and ideal binary mask [IdBM] algorithm) on speech using speech perception measures and acoustic measure among individuals with ANSD. Method Two groups of participants (age: 17-43 years) were recruited in the study. Group I comprised 12 individuals with a confirmed diagnosis of ANSD and not exceeding moderate degree of hearing loss and Group II of 10 individuals with normal hearing in both ears. The signal-to-noise required for 50% speech recognition (SNR-50) was measured for the participants in five conditions, that is, unprocessed speech and speech processed with four noise reduction strategies. Additionally, an acoustic objective measure Extended Short-Time Objective Intelligibility algorithm was employed to estimate the intelligibility index across the conditions. Results Significant difference was found across conditions in both the groups. Pairwise comparison revealed significantly better speech perception on SNR-50 measure with IdBM strategy, for both the groups. No significant difference in SNR-50 was observed with other noise reduction strategies. IdBM condition also gave the highest intelligibility index (d) values using Extended Short-Time Objective Intelligibility algorithm. This finding needs to be verified on a larger group of individuals with ANSD. Conclusions IdBM noise reduction strategy rendered significantly lower SNR-50 compared to other noise reduction strategies for individuals with ANSD in this study. This provides clinical evidence for the same and also recommends trying on a larger group of participants before its implementation in hearing devices. Apart from this, the current strategies used in hearing aids provide no improvement in speech identification in noise for this population. Hence, though the present hearing aids may show benefit in quiet condition, chances of its rejection are high in noisy backgrounds.

Effects of Spectral Resolution and Frequency Mismatch on Speech Understanding and Spatial Release From Masking in Simulated Bilateral Cochlear Implants.

Due to interaural frequency mismatch, bilateral cochlear-implant (CI) users may be less able to take advantage of binaural cues that normal-hearing (NH) listeners use for spatial hearing, such as interaural time differences and interaural level differences. As such, bilateral CI users have difficulty segregating competing speech even when the target and competing talkers are spatially separated. The goal of this study was to evaluate the effects of spectral resolution, tonotopic mismatch (the frequency mismatch between the acoustic center frequency assigned to CI electrode within an implanted ear relative to the expected spiral ganglion characteristic frequency), and interaural mismatch (differences in the degree of tonotopic mismatch in each ear) on speech understanding and spatial release from masking (SRM) in the presence of competing talkers in NH subjects listening to bilateral vocoder simulations. During testing, both target and masker speech were presented in five-word sentences that had the same syntax but were not necessarily meaningful. The sentences were composed of five categories in fixed order (Name, Verb, Number, Color, and Clothes), each of which had 10 items, such that multiple sentences could be generated by randomly selecting a word from each category. Speech reception thresholds (SRTs) for the target sentence presented in competing speech maskers were measured. The target speech was delivered to both ears and the two speech maskers were delivered to (1) both ears (diotic masker), or (2) different ears (dichotic masker: one delivered to the left ear and the other delivered to the right ear). Stimuli included the unprocessed speech and four 16-channel sine-vocoder simulations with different interaural mismatch (0, 1, and 2 mm). SRM was calculated as the difference between the diotic and dichotic listening conditions. With unprocessed speech, SRTs were 0.3 and -18.0 dB for the diotic and dichotic maskers, respectively. For the spectrally degraded speech with mild tonotopic mismatch and no interaural mismatch, SRTs were 5.6 and -2.0 dB for the diotic and dichotic maskers, respectively. When the tonotopic mismatch increased in both ears, SRTs worsened to 8.9 and 2.4 dB for the diotic and dichotic maskers, respectively. When the two ears had different tonotopic mismatch (e.g., there was interaural mismatch), the performance drop in SRTs was much larger for the dichotic than for the diotic masker. The largest SRM was observed with unprocessed speech (18.3 dB). With the CI simulations, SRM was significantly reduced to 7.6 dB even with mild tonotopic mismatch but no interaural mismatch; SRM was further reduced with increasing interaural mismatch. The results demonstrate that frequency resolution, tonotopic mismatch, and interaural mismatch have differential effects on speech understanding and SRM in simulation of bilateral CIs. Minimizing interaural mismatch may be critical to optimize binaural benefits and improve CI performance for competing speech, a typical listening environment. SRM (the difference in SRTs between diotic and dichotic maskers) may be a useful clinical tool to assess interaural frequency mismatch in bilateral CI users and to evaluate the benefits of optimization methods that minimize interaural mismatch.

Meta-Analysis on the Identification of Linguistic and Emotional Prosody in Cochlear Implant Users and Vocoder Simulations.

This study quantitatively assesses how cochlear implants (CIs) and vocoder simulations of CIs influence the identification of linguistic and emotional prosody in nontonal languages. By means of meta-analysis, it was explored how accurately CI users and normal-hearing (NH) listeners of vocoder simulations (henceforth: simulation listeners) identify prosody compared with NH listeners of unprocessed speech (henceforth: NH listeners), whether this effect of electric hearing differs between CI users and simulation listeners, and whether the effect of electric hearing is influenced by the type of prosody that listeners identify or by the availability of specific cues in the speech signal. Records were found by searching the PubMed Central, Web of Science, Scopus, Science Direct, and PsycINFO databases (January 2018) using the search terms "cochlear implant prosody" and "vocoder prosody." Records (published in English) were included that reported results of experimental studies comparing CI users' and/or simulation listeners' identification of linguistic and/or emotional prosody in nontonal languages to that of NH listeners (all ages included). Studies that met the inclusion criteria were subjected to a multilevel random-effects meta-analysis. Sixty-four studies reported in 28 records were included in the meta-analysis. The analysis indicated that CI users and simulation listeners were less accurate in correctly identifying linguistic and emotional prosody compared with NH listeners, that the identification of emotional prosody was more strongly compromised by the electric hearing speech signal than linguistic prosody was, and that the low quality of transmission of fundamental frequency (f0) through the electric hearing speech signal was the main cause of compromised prosody identification in CI users and simulation listeners. Moreover, results indicated that the accuracy with which CI users and simulation listeners identified linguistic and emotional prosody was comparable, suggesting that vocoder simulations with carefully selected parameters can provide a good estimate of how prosody may be identified by CI users. The meta-analysis revealed a robust negative effect of electric hearing, where CIs and vocoder simulations had a similar negative influence on the identification of linguistic and emotional prosody, which seemed mainly due to inadequate transmission of f0 cues through the degraded electric hearing speech signal of CIs and vocoder simulations.

Spectral Contrast Effects Reveal Different Acoustic Cues for Vowel Recognition in Cochlear-Implant Users.

The identity of a speech sound can be affected by the spectrum of a preceding stimulus in a contrastive manner. Although such aftereffects are often reduced in people with hearing loss and cochlear implants (CIs), one recent study demonstrated larger spectral contrast effects in CI users than in normal-hearing (NH) listeners. The present study aimed to shed light on this puzzling finding. We hypothesized that poorer spectral resolution leads CI users to rely on different acoustic cues not only to identify speech sounds but also to adapt to the context. Thirteen postlingually deafened adult CI users and 33 NH participants (listening to either vocoded or unprocessed speech) participated in this study. Psychometric functions were estimated in a vowel categorization task along the /I/ to /ε/ (as in "bit" and "bet") continuum following a context sentence, the long-term average spectrum of which was manipulated at the level of either fine-grained local spectral cues or coarser global spectral cues. In NH listeners with unprocessed speech, the aftereffect was determined solely by the fine-grained local spectral cues, resulting in a surprising insensitivity to the larger, global spectral cues utilized by CI users. Restricting the spectral resolution available to NH listeners via vocoding resulted in patterns of responses more similar to those found in CI users. However, the size of the contrast aftereffect remained smaller in NH listeners than in CI users. Only the spectral contrasts used by listeners contributed to the spectral contrast effects in vowel identification. These results explain why CI users can experience larger-than-normal context effects under specific conditions. The results also suggest that adaptation to new spectral cues can be very rapid for vowel discrimination, but may follow a longer time course to influence spectral contrast effects.

Assessing the cortical processing of spectrally rotated speech using functional near-infrared spectroscopy

Rotated speech (RS) is unintelligible to untrained listeners but contains similar temporal and spectral complexity as ordinary speech. It has been used in many studies to investigate cortical pathways for intelligible speech. Typically, the generation of RS involves the rotation of the spectrum of low-pass filtered speech around a center frequency (e.g., 2 kHz). Due to the inversion, voiced segments of RS contain harmonics that are typically not multiples of the fundamental frequency, and therefore the RS signal is not truly periodic. Here, it was assessed whether cortical responses to such aperiodic RS differ from quasi-periodic RS with preserved harmonic structure. Harmonic RS was constructed by rotating only the spectral envelope while preserving the characteristics of the source, whereas for inharmonic RS both components were rotated. A story listening paradigm was used, in which subjects were presented with 25-s blocks of unprocessed speech, inharmonic RS, and harmonic RS. Functional near-infrared spectroscopy (fNIRS) was used to measure cortical activation based on changes in the concentration of oxygenated and deoxygenated hemoglobin. The results show increased activity in the right posterior superior temporal sulcus (STS) for harmonic RS in comparison with inharmonic RS but not unprocessed speech. This demonstrates that the periodicity of RS should be considered in the interpretation of previous and the planning of future neuroimaging studies. [Wellcome Trust Multi-user Equipment Award and SNSF grant P400PG_180693.]Rotated speech (RS) is unintelligible to untrained listeners but contains similar temporal and spectral complexity as ordinary speech. It has been used in many studies to investigate cortical pathways for intelligible speech. Typically, the generation of RS involves the rotation of the spectrum of low-pass filtered speech around a center frequency (e.g., 2 kHz). Due to the inversion, voiced segments of RS contain harmonics that are typically not multiples of the fundamental frequency, and therefore the RS signal is not truly periodic. Here, it was assessed whether cortical responses to such aperiodic RS differ from quasi-periodic RS with preserved harmonic structure. Harmonic RS was constructed by rotating only the spectral envelope while preserving the characteristics of the source, whereas for inharmonic RS both components were rotated. A story listening paradigm was used, in which subjects were presented with 25-s blocks of unprocessed speech, inharmonic RS, and harmonic RS. Functional near-infrared sp...