Advanced Combination Encoder Strategy Research Articles

ElectrodeNet—A Deep-Learning-Based Sound Coding Strategy for Cochlear Implants

ElectrodeNet, a deep learning based sound coding strategy for the cochlear implant (CI), is proposed to emulate the advanced combination encoder (ACE) strategy by replacing the conventional envelope detection using various artificial neural networks. The extended ElectrodeNet-CS strategy further incorporates the channel selection (CS). Network models of deep neural network (DNN), convolutional neural network (CNN), and long short-term memory (LSTM) were trained using the Fast Fourier Transformed bins and channel envelopes obtained from the processing of clean speech by the ACE strategy. Objective speech understanding using short-time objective intelligibility (STOI) and normalized covariance metric (NCM) was estimated for ElectrodeNet using CI simulations. Sentence recognition tests for vocoded Mandarin speech were conducted with normal-hearing listeners. DNN, CNN, and LSTM based ElectrodeNets exhibited strong correlations to ACE in objective and subjective scores using mean squared error (MSE), linear correlation coefficient (LCC) and Spearman's rank correlation coefficient (SRCC). The ElectrodeNet-CS strategy was capable of producing N-of-M compatible electrode patterns using a modified DNN network to embed maxima selection, and to perform in similar or even slightly higher average in STOI and sentence recognition compared to ACE. The methods and findings demonstrated the feasibility and potential of using deep learning in CI coding strategy.

Temporal and spectral analysis of melody perception with different simulated cochlear implant coding strategies

This study examined the contribution of pitch and rhythm cues in melody recognition, as well as the quality from three well-known CI coding strategies, Advanced Combinational Encoder (ACE), Fundamental Frequency Modulation (F0mod), and Envelope Enhancement (EE). Firstly, we processed the 30 popular melodies of Taiwan children's songs using NCU-CI, a cochlear implant simulation software. In the pilot subjective listening test (n = 5), rhythm cues perceived better (p < 0.05) by having a higher percent correct and faster response time than pitch cues. Foremost, rhythm cues processed by ACE strategy achieved the best score, 86.80%. Then, objective tests were conducted to measure the spectral and temporal quality of processed melody using log spectral distance (LSD), intensity mismatch pattern, and envelope difference index (EDI). LSD between original and processed by the ACE, F0mod, and EE strategy were 2.10, 2.16, and 2.19, respectively. Then, average intensity mismatch patterns between original and processed by the ACE, F0mod, and EE strategy were 5.9, 6.4, and 6.0, respectively. The lower LSD and intensity mismatch pattern, the better the amplitude melody and spectral quality was preserved. Then, EDI between original and processed by the ACE, F0mod, and EE strategy were 0.11, 0.11, and 0.15, respectively. The higher EDI, the better the temporal envelope was preserved. It reveals that rhythm cues combined with the ACE strategy performed the best feature in CI melody perception.

Precompensating for spread of excitation in a cochlear implant coding strategy

Cochlear implant users’ limited ability to understand speech in noisy environments has been linked to the poor spatial resolution and the high degree of spectral smearing associated with the spread of neural excitation. A sound coding algorithm that aims to improve the spectro-temporal representation of the sound signal at the implanted ear by precompensating the electrical stimulation for the spread of excitation is presented in this study. The spread precompensation algorithm was integrated into the standard clinical advanced combination encoder (ACE) strategy and the resulting strategy was called SPACE. SPACE was evaluated acutely with a group of six implant users and was compared to their daily used ACE strategy in terms of preference rating and speech recognition in four-talker babble and stationary speech-shaped noise. While no significant differences in preference rating were observed, speech recognition in four-talker babble was improved by SPACE processing. Analysis of the group results revealed a significant improvement in mean speech reception threshold (SRT) over the ACE strategy of 1.4 dB in four-talker babble, whereas the difference of 0.9 dB in stationary noise did not reach statistical significance. Assessment of individual differences showed that four out of six listeners obtained significant SRT improvements with SPACE and that no subject scored significantly worse compared to ACE. The results suggest that the proposed sound coding strategy has the potential to improve speech perception for cochlear implant users in challenging listening situations.

A bio-inspired coding (BIC) strategy for cochlear implants

A bio-inspired coding (BIC) strategy was implemented in this study with the goal of better representation of spectral and temporal information. The auditory nerve fibers' (ANFs) characteristics such as refractory recovery, facilitation and spatial spread were obtained from ECAP data recorded in 11 CI recipients. These characteristics, together with a non-individualized model-derived adaptation effect, were integrated into the BIC strategy for a better selection of channels. Two variations of the BIC strategy were compared to the conventional advanced combination encoder (ACE) coding strategy: the BIC-I strategy based on the individual CI recipients' ECAP parameters, and the BIC-G strategy based on the median values of ECAP parameters from all CI recipients who participated in the study. The melodic contour identification (MCI) and Oldenburg sentence recognition in noise (OLSA) tests were used to assess and compare the three coding strategies. A significantly better performance in the transformed MCI test results with the rationalized arcsine transformation, was observed for both BIC strategy variations compared to the ACE strategy. There was no significant difference between the two variations of the BIC strategy and the ACE strategy in the OLSA test. No correlation was found between recovery time constants, absolute refractory periods, left and right width of SOE functions from three test electrodes and CI recipients’ performances in the two experiments. However, significant correlations were found between facilitation time constant and amplitude and the results of the MCI and OLSA tests for the two variations of the BIC strategy.

Evaluation of the Optimized Pitch and Language Strategy in Cochlear Implant Recipients.

The Optimized Pitch and Language (OPAL) strategy enhances pitch perception through coding of fundamental frequency (F0) amplitude modulation information in the stimulus envelope delivered to a cochlear implant. Previous research using a prototype of the strategy demonstrated significant benefits in musical pitch and lexical tone discrimination tasks with no degradation in speech recognition when compared with the clinical Advanced Combination Encoder (ACE) strategy in a small group of subjects. Based on those studies, a modified version of the strategy was implemented in the commercial Nucleus CP900 series processor. The aims of the present study were to establish whether the CP900 OPAL implementation continued to provide improved F0 pitch perception in a speech intonation task with no degradation to speech perception in quiet and noise, when compared with the clinical ACE strategy in a larger cohort of subjects. Further aims were to evaluate fitting procedures and subject acclimatization to the strategy after take-home experience. Twenty experienced adult cochlear implant recipients were enrolled in the study. Two subjects withdrew during the study leaving 18 sets of data for analysis. A repeated-measures single-subject design with take-home experience was used to test for improved speech intonation perception using OPAL compared with ACE and for comparable performance between strategies for open-set word recognition in quiet at two presentation levels, sentence recognition in adaptive 4-talker babble noise, and speech intelligibility ratings. The stimulation rate employed for OPAL was 1200 pulses per second/channel which was higher than the default clinical rate of 900 pulses per second/channel used for ACE by all subjects in the present study. Two variations of the OPAL "F0 restore gain" (the gain applied to restore the loudness of modulated channels) were investigated: "custom" measured per subject and "default" which was the average of all subject custom gains. A significant group mean benefit on the intonation test of 8.5% points was shown for OPAL compared with ACE. There was a significant period of adaptation to OPAL with significantly poorer sentence in noise scores acutely and after only 2 weeks of take-home experience. After 4 weeks of take-home experience, comparable word perception in quiet and sentence perception in noise for OPAL were obtained. Furthermore, there was good subject acceptability in the field with comparable speech intelligibility ratings between strategies. Results of the fitting procedure showed that OPAL did not require any additional steps compared with fitting of ACE. A default F0 restore gain provided comparable outcomes to a custom gain setting. The CP900 OPAL implementation provided a significant benefit to perception of speech intonation when compared with ACE. Comparable speech perception (in quiet and noise) and subjective ratings of speech intelligibility between strategies were also achieved after a period of acclimatization. These outcomes are consistent with results of earlier studies using prototype versions of the strategy and reaffirm its potential for improvement of F0 pitch perception in speech while preserving coding of segmental speech information. Furthermore, the OPAL strategy can be programmed into subject's processors using the same fitting procedures used for ACE thereby simplifying its adoption in clinical settings.

FMRI as a Preimplant Objective Tool to Predict Children’s Postimplant Auditory and Language Outcomes as Measured by Parental Observations

The trends in cochlear implantation candidacy and benefit have changed rapidly in the last two decades. It is now widely accepted that early implantation leads to better postimplant outcomes. Although some generalizations can be made about postimplant auditory and language performance, neural mechanisms need to be studied to predict individual prognosis. The aim of this study was to use functional magnetic resonance imaging (fMRI) to identify preimplant neuroimaging biomarkers that predict children's postimplant auditory and language outcomes as measured by parental observation/reports. This is a pre-post correlational measures study. Twelve possible cochlear implant candidates with bilateral severe to profound hearing loss were recruited via referrals for a clinical magnetic resonance imaging to ensure structural integrity of the auditory nerve for implantation. Participants underwent cochlear implantation at a mean age of 19.4 mo. All children used the advanced combination encoder strategy (ACE, Cochlear Corporation™, Nucleus® Freedom cochlear implants). Three participants received an implant in the right ear; one in the left ear whereas eight participants received bilateral implants. Participants' preimplant neuronal activation in response to two auditory stimuli was studied using an event-related fMRI method. Blood oxygen level dependent contrast maps were calculated for speech and noise stimuli. The general linear model was used to create z-maps. The Auditory Skills Checklist (ASC) and the SKI-HI Language Development Scale (SKI-HI LDS) were administered to the parents 2 yr after implantation. A nonparametric correlation analysis was implemented between preimplant fMRI activation and postimplant auditory and language outcomes based on ASC and SKI-HI LDS. Statistical Parametric Mapping software was used to create regression maps between fMRI activation and scores on the aforementioned tests. Regression maps were overlaid on the Imaging Research Center infant template and visualized in MRIcro. Regression maps revealed two clusters of brain activation for the speech versus silence contrast and five clusters for the noise versus silence contrast that were significantly correlated with the parental reports. These clusters included auditory and extra-auditory regions such as the middle temporal gyrus, supramarginal gyrus, precuneus, cingulate gyrus, middle frontal gyrus, subgyral, and middle occipital gyrus. Both positive and negative correlations were observed. Correlation values for the different clusters ranged from -0.90 to 0.95 and were significant at a corrected p value of <0.05. Correlations suggest that postimplant performance may be predicted by activation in specific brain regions. The results of the present study suggest that (1) fMRI can be used to identify neuroimaging biomarkers of auditory and language performance before implantation and (2) activation in certain brain regions may be predictive of postimplant auditory and language performance as measured by parental observation/reports.

Sensitivity to interaural timing differences using the advanced combinational encoder strategy

When listening through clinical processors, sound localization in bilateral cochlear implant (BiCI) users is highly variable, and some patients perform rather poorly. While this is often ascribed to lack of access to temporal fine-structure interaural timing differences (ITDs), the potential role of envelope ITDs is often disregarded. Furthermore, sensitivity to ITDs alone has not been closely examined with the commonly used Advanced Combinational Encoder (ACE) strategy. In theory, envelope ITDs could be encoded with ACE and some BiCI users may be sensitive to this cue to locate sounds, which may explain why some BiCI users have better localization performance. ITD just noticeable differences (JNDs) were measured in 9 participants, using a novel setup of presenting whole-waveform-delayed words through the auxillary port of clinical processors. Listening with ACE, ITD JNDs ranged from 105-680 µs (unmeasurable in one listener). Localization root-mean-squared error ranged from 19-68 degrees. However, ITD sensitivity and sound localization were not significantly correlated (ρ = 0.5, p = 0.17). The unexplained range of sound localization performance in BiCI users is likely due to distorted and inconsistent binaural cues that arise from uncoordinated device programming and signal processing across the two ears. [Work supported by NIH-NIDCD (R03DC015321 to AK and R01DC003083 to RYL) and NIH-NICHD (P30HD03352 to Waisman Center).]

Investigating the use of a Gammatone filterbank for a cochlear implant coding strategy

BackgroundContemporary speech processing strategies in cochlear implants (CIs) such as the Advanced Combination Encoder (ACE) use a standard Fast Fourier Transform (FFT) filterbank to extract envelopes. The assignment of the FFT bins to approximate the frequency resolution of the basilar membrane is only partly based on physiology, especially since the bins are distributed linearly below 1000Hz and logarithmically above 1000Hz. New methodA Gammatone filterbank which provides a closer approximation to the bandwidths of filters in the human auditory system could replace the standard FFT filterbank in the ACE strategy. An infinite impulse response (IIR) all-pole design of the Gammatone filterbank was compared to the FFT filterbank with 128, 256 and 512 points resolutions and the effect of the frequency boundaries of the filters was also investigated. ResultsMelodic contour identification (MCI) and just noticeable difference (JND) experiments, both involving synthetic clarinet notes in octaves 3 and 4, were conducted with 6 normal hearing (NH) participants using noise vocoded stimuli; and 10 CI recipients just performed the MCI experiment. The MCI results for both NH and CI subjects, showed a significant effect of the filterbank on the percentage correct responses of the participants. Comparison with existing methodsThe Gammatone filterbank can better resolve the harmonics of tested synthetic clarinet notes which led to better performances in the MCI experiment. ConclusionsThe total delay of the Gammatone filterbank can be made smaller than the delay of the FFT filterbank with the same frequency resolution at low frequencies.

Comparison of Mandarin tone and speech perception between advanced combination encoder and continuous interleaved sampling speech-processing strategies in children

ObjectiveThis study was performed to compare cochlear implant (CI) users' performance in Mandarin speech and tone perception between 2 types of speech-processing strategies—advanced combination encoder (ACE) and continuous interleaved sampling (CIS)—under quiet and noisy conditions. MethodsThis study involved 10 congenitally deaf children (age range, 5.7–15.3 years; mean, 9.2 years) who received the Nucleus 24-channel CI system cochlear device (CI24R; Cochlear Ltd, Lane Cove NSW, Australia). The subjects used ACE since switching on their CI devices. Speech and tone perception tests were administered under quiet and noisy (+5 dB signal-to-noise ratio) conditions with ACE and CIS strategies 20 minutes and 2 weeks apart. ResultsRegardless of the strategy used, subjects showed significantly higher scores in speech perception than in tone recognition. Under noisy conditions, subjects had significantly higher tone identification scores with the CIS than the ACE strategy (P = .038). There was no significant difference in speech identification score between the strategies. Subjects showed significant higher tone identification and speech perception scores under quiet than noisy (+5 dB signal-to-noise ratio) conditions. Subjectively, 6 subjects preferred the ACE strategy, and the remaining 4 preferred the CIS strategy. The strategy preference of the subjects was related to speech perception performance rather than tone identification. A significant correlation was observed between tone identification and speech recognition, regardless of whether speech was evaluated by consonants (r = 0.669, P < .001), vowels (r = 0.426, P = .001), or sentences (r = 0.294, P = .023). ConclusionThere are only 4 patterns of tone in Mandarin, which is far fewer than the number of speech sounds. However, tone identification is poorer than speech perception. The CIS speech-processing strategy may improve tone identification under noisy conditions. Before improved speech strategies to code acoustic characteristics of tone can be developed, it would be worthwhile to try both CIS and ACE for CI users and to select the most suitable speech-processing strategy according to the subjective preference and objective performance.

Across-Frequency Delays Based on the Cochlear Traveling Wave: Enhanced Speech Presentation for Cochlear Implants

Cochlear implants stimulate the auditory nerve with the outputs of a bank of narrow-band filters. We propose that cochlear implant users are better able to perceive speech when these frequency bands are desynchronized, as occurs in the normal cochlea. The first part of this study was a computational investigation of across-frequency delays on the stimulation patterns generated by the advanced combination encoder (ACE) sound-processing strategy. By offsetting frequency bands from each other, fewer stimuli were discarded from voiced speech by maxima selection. Background noise, however, was not affected in this way. The second part of this study was an assessment of speech perception with across-frequency delays in cochlear implant users with the ACE strategy. In the perception of sentences in noise, three subjects improved with delays, four showed no change and one was worse. For words in quiet, four subjects had improved word recognition and four showed no change. A significant group improvement ( P < 0.05 ) was seen for speech in quiet. These results are encouraging for cochlear implant sound processing because across-frequency delays can be incorporated easily and efficiently into existing sound-processing strategies.

A new sound coding strategy for suppressing noise in cochlear implants

In the n-of-m strategy, the signal is processed through m bandpass filters from which only the n maximum envelope amplitudes are selected for stimulation. While this maximum selection criterion, adopted in the advanced combination encoder strategy, works well in quiet, it can be problematic in noise as it is sensitive to the spectral composition of the input signal and does not account for situations in which the masker completely dominates the target. A new selection criterion is proposed based on the signal-to-noise ratio (SNR) of individual channels. The new criterion selects target-dominated (SNR > or = 0 dB) channels and discards masker-dominated (SNR<0 dB) channels. Experiment 1 assessed cochlear implant users' performance with the proposed strategy assuming that the channel SNRs are known. Results indicated that the proposed strategy can restore speech intelligibility to the level attained in quiet independent of the type of masker (babble or continuous noise) and SNR level (0-10 dB) used. Results from experiment 2 showed that a 25% error rate can be tolerated in channel selection without compromising speech intelligibility. Overall, the findings from the present study suggest that the SNR criterion is an effective selection criterion for n-of-m strategies with the potential of restoring speech intelligibility.

Conversion from the SPEAK to the ACE strategy in children using the nucleus 24 cochlear implant system: speech perception and speech production outcomes.

The main objective of this study was to assess whether speech perception and speech production in children using the Nucleus 24 cochlear implant system improved with a change in speech processing strategy from the SPEAK to the Advanced Combination Encoder (ACE) strategy. The major difference between the two strategies is that ACE uses a higher stimulation rate (in this study the stimulation rate was 900 Hz per channel) compared with the SPEAK strategy, where the stimulation rate is 250 Hz per channel. Information also was obtained regarding the adjustment period after conversion to the ACE strategy. An ABA experimental design was used where scores were initially obtained using the SPEAK strategy' (in the initial A time interval), and subsequently performance was assessed using the ACE strategy (B time interval) and then again with the SPEAK strategy (second A time interval). The duration of the B interval was 10 wk, and the duration for the second A interval was 4 wk. Seven children aged between 9 and 16 yr who had at least 6 mo experience with the SPEAK strategy participated. Open-set monosyllabic CNC word perception in quiet and Speech Intelligibility Test sentence perception in noise was evaluated at the end of each of the time intervals. Word perception was also monitored at fortnightly intervals during the B time interval. Speech production was assessed at the end of the initial A time interval and at the end of the B time interval. Mean word and phoneme scores for open-set words in quiet for the group of seven children were significantly higher with the ACE strategy as compared with the SPEAK strategy scores obtained in both of the A time intervals. For sentences in noise, mean scores using the ACE strategy as well as the SPEAK strategy at the second A evaluation point were significantly higher than the scores using the SPEAK strategy measured at the first A time interval. This suggests that learning effects may have influenced outcomes. For some subjects, an initial decrease in scores was found during the initial 2-wk period after fitting the ACE strategy; however, scores subsequently were found to be similar to or higher than those when using the initial SPEAK strategy. Analysis of speech production assessments showed an improvement in the medial consonant scores after using the ACE strategy. This study demonstrated that some children were able to benefit from the additional information provided by the ACE strategy as compared with the SPEAK strategy. However, the differences in overall performance between the two strategies appear to be relatively small.