Bimodal Speech Research Articles

Frequency importance functions in simulated bimodal cochlear-implant users with spectral holes.

Frequency importance functions (FIFs) for simulated bimodal hearing were derived using sentence perception scores measured in quiet and noise. Acoustic hearing was simulated using low-pass filtering. Electric hearing was simulated using a six-channel vocoder with three input frequency ranges, resulting in overlap, meet, and gap maps, relative to the acoustic cutoff frequency. Spectral holes present in the speech spectra were created within electric stimulation by setting amplitude(s) of channels to zero. FIFs were significantly different between frequency maps. In quiet, the three FIFs were similar with gradually increasing weights with channels 5 and 6 compared to the first three channels. However, the most and least weighted channels slightly varied depending on the maps. In noise, the patterns of the three FIFs were similar to those in quiet, with steeper increasing weights with channels 5 and 6 compared to the first four channels. Thus, channels 5 and 6 contributed to speech perception the most, while channels 1 and 2 contributed the least, regardless of frequency maps. Results suggest that the contribution of cochlear implant frequency bands for bimodal speech perception depends on the degree of frequency overlap between acoustic and electric stimulation and if noise is absent or present.

Objective discrimination of bimodal speech using frequency following responses

Bimodal hearing, in which a contralateral hearing aid is combined with a cochlear implant (CI), provides greater speech recognition benefits than using a CI alone. Factors predicting individual bimodal patient success are not fully understood. Previous studies have shown that bimodal benefits may be driven by a patient's ability to extract fundamental frequency (f0) and/or temporal fine structure cues (e.g., F1). Both of these features may be represented in frequency following responses (FFR) to bimodal speech. Thus, the goals of this study were to: 1) parametrically examine neural encoding of f0 and F1 in simulated bimodal speech conditions; 2) examine objective discrimination of FFRs to bimodal speech conditions using machine learning; 3) explore whether FFRs are predictive of perceptual bimodal benefit. Three vowels (/ε/, /i/, and /ʊ/) with identical f0 were manipulated by a vocoder (right ear) and low-pass filters (left ear) to create five bimodal simulations for evoking FFRs: Vocoder-only, Vocoder +125 Hz, Vocoder +250 Hz, Vocoder +500 Hz, and Vocoder +750 Hz. Perceptual performance on the BKB-SIN test was also measured using the same five configurations. Results suggested that neural representation of f0 and F1 FFR components were enhanced with increasing acoustic bandwidth in the simulated “non-implanted” ear. As spectral differences between vowels emerged in the FFRs with increased acoustic bandwidth, FFRs were more accurately classified and discriminated using a machine learning algorithm. Enhancement of f0 and F1 neural encoding with increasing bandwidth were collectively predictive of perceptual bimodal benefit on a speech-in-noise task. Given these results, FFR may be a useful tool to objectively assess individual variability in bimodal hearing.

Visual Speech Recognition Using Lips Movement

The visual speech recognition technology look for increase noise-sound in mobile context by extracting lip movement from input face pictures. Bimodal speech recognition algorithms depended on the frontal face (lip) photos, but because users must be able to speak holding a device by using this device in front of the user face, these techniques are difficult for users to utilize. Our suggested method records lip movement using a tiny camera built into a smartphone, making it more practical, simple, and natural. Additionally, this technique effectively prevents a decline in the input signal-to-noise ratio (SNR). For CNN-based recognition, visual properties are extracted via optical-flow analysis and merged with audio data. Although in the previous model, it doesn't give the output in the audio format. Hence in our proposed model, we can provide the output in the audio format. Our proposed aim to record a user speaking into the camera or user will upload the video. The system will initially detect only the lip area from the video. The system will divide this lip video into multiple frames. After sequencing the lip frames, feature extraction will be done from the lip frames. The model will be trained to extract these features further these extracted features from the trained model will be used to find out the sequence of phoneme distributions. The final output will be the word or phrase spoken by the user displayed on the camera.

Deep Learning Approaches for Bimodal Speech Emotion Recognition: Advancements, Challenges, and a Multi-Learning Model

Deep Learning Approaches for Bimodal Speech Emotion Recognition: Advancements, Challenges, and a Multi-Learning Model

Underspecification in time

AbstractSubstance-free phonology or SFP (Reiss 2017) has renewed interest in the question of abstraction in phonology. Perhaps the most common form of abstraction through the absence of substance is underspecification, where some aspects of speech lack representation in memorized representations, within the phonology or in the phonetic implementation (Archangeli 1988, Keating 1988, Lahiri and Reetz 2010 among many others). The fundamental basis for phonology is argued to be a mental model of speech events in time, following Raimy (2000) and Papillon (2020). Each event can have properties (one-place predicates that are true of the event), which include the usual phonological features, and also structural entities for extended events like moras and syllables. Features can be bound together in an event, yielding segment-like properties. Pairs of events can be ordered in time by the temporal logic precedence relation represented by ‘<’. Events, features and precedence form a directed multigraph structure with edges in the graph interpreted as “maybe next”. Some infant bimodal speech perception results are examined using this framework, arguing for underspecification in time in the developing phonological representations.

Objective discrimination of bimodal speech using the frequency following response: A machine learning approach

Bimodal hearing, which combines a cochlear implant (CI) with a contralateral hearing aid, provides significant speech recognition benefits relative to a monaural CI. Factors predicting bimodal benefit remain poorly understood but may involve extracting fundamental frequency and/or formant information from the non-implanted ear. This study investigated whether neural responses (frequency following responses, FFRs) to simulated bimodal signals can be (1) accurately classified using machine learning and (2) used to predict perceptual bimodal benefit. We hypothesized that FFR classification accuracy would improve with increasing acoustic bandwidth due to greater fundamental and formant frequency access. Three vowels (/e/, /i/, and /ʊ/) with identical fundamental frequencies were manipulated to create five bimodal simulations (vocoder in right ear, lowpass filtered in left ear): Vocoder-only, Vocoder +125 Hz, Vocoder +250 Hz, Vocoder +500 Hz, and Vocoder +750 Hz. Perceptual performance on the BKB-SIN test was also measured using the same five configurations. FFR classification accuracy improved with increasing bimodal acoustic bandwidth. Furthermore, FFR bimodal benefit predicted behavioral bimodal benefit. These results indicate that the FFR may be useful in objectively verifying and tuning bimodal configurations.

Visual Speech Recognition

The audio-visual speech recognition approach attempts to boost noise-robustness in mobile situations by extracting lip movement from side-face images. Although earlier bimodal speech recognition algorithms used frontal face (lip) images, these approaches are difficult for consumers to utilize because they need them to talk while holding a device with a camera in front of their face. Our proposed solution, which uses a small camera put in a handset to capture lip movement, is more natural, simple, and convenient. This approach also effectively avoids a reduction in the input speech's signal-to-noise ratio (SNR). Optical-flow analysis extracts visual features, which are then coupled with audio features in the context of CNN-based recognition.

Bimodal cochlear implantation in elderly patients

Objective Bimodal stimulation is a standard option for asymmetric hearing loss in adults. Questions have been raised whether receiving two stimulations may conflict in elderly listeners where the central integration of an acoustic/electrical signal may be very important to obtain benefit in terms of speech perception. Design Clinical retrospective study. Study sample The outcomes from 17 bimodal cochlear implant (CI) users were analysed. The test material consisted of speech audiometry in quiet and in noise (STARR and Matrix). Results Bimodal PTA and speech perception both in quiet and in noise were significantly better than CI or HA alone. Age showed a significant effect on bimodal STARR outcomes. Similarly, bimodal STARR scores improved significantly in comparison to Better Ear. Conclusion Both Matrix and STARR tests were very difficult for many elderly CI listeners from the present study group, especially in unilateral listening condition. The performance improved significantly, emphasising a good integration of acoustic and electric hearing in this group of elderly bimodal listeners. Overall results highlighted how a specific study, based on speech perception in noise in the elderly listeners, might shed light on the effect of speech test modality on bimodal outcomes.

Evaluation of a novel bimodal fitting formula in Advanced Bionics cochlear implant recipients

Purpose: The study’s objectives were to (1) evaluate benefit from a novel bimodal fitting formula (Adaptive Phonak Digital Bimodal Fitting Formula [APDB]), and (2) compare outcomes with APDB and a traditional fitting formula (NAL-NL2). Methods: This prospective study evaluated outcomes in ten adults with unilateral Advanced Bionics (AB) cochlear implants (CI). Participants were tested bimodally with NAL-NL2 and APDB programed on Naída Link UP HAs. Measures of speech perception, sound quality, and preference were obtained with two bimodal configurations (CI + HANAL-NL2 and CI + HAAPDB). Participants used the CI + HAAPDB configuration for an acclimation period, after which measures were repeated. Results: Significant bimodal benefit was measured from both HA fitting formulae for speech perception in noise compared to the CI-only condition. Improved individual outcomes with the APDB were observed, but group differences were not statistically significant. Participants reported subjective benefit from APDB on blind comparisons of preference and sound quality. Conclusions: Significant benefit was found with both bimodal conditions compared to the CI-only condition; however, bimodal speech perception results were not significantly different. Users reported benefit from the APDB formula over NAL-NL2 formula. Due to individual improved speech perception and overall subjective preference for APDB, clinicians should consider APDB with AB CI recipients.

Multi-Modal Multi-Channel Target Speech Separation

Target speech separation refers to extracting a target speaker's voice from an overlapped audio of simultaneous talkers. Previously the use of visual modality for target speech separation has demonstrated great potentials. This work proposes a general multi-modal framework for target speech separation by utilizing all the available information of the target speaker, including his/her spatial location, voice characteristics and lip movements. Also, under this framework, we investigate on the fusion methods for multi-modal joint modeling. A factorized attention-based fusion method is proposed to aggregate the high-level semantic information of multi-modalities at embedding level. This method firstly factorizes the mixture audio into a set of acoustic subspaces, then leverages the target's information from other modalities to enhance these subspace acoustic embeddings with a learnable attention scheme. To validate the robustness of proposed multi-modal separation model in practical scenarios, the system was evaluated under the condition that one of the modalities is temporarily missing, invalid or corrupted. Experiments are conducted on a large-scale audio-visual dataset collected from YouTube (to be released) that spatialized by simulated room impulse responses (RIRs). Experiment results illustrate that our proposed multi-modal framework significantly outperforms single-modal and bi-modal speech separation approaches, while can still support real-time processing.

The influence of the bimodal stimulation on the auditory ability of temporal ordering

ABSTRACT Objective: to analyze temporal ordering auditory ability in unilateral cochlear implant users with bimodal stimulation. Methods: the study included 15 unilateral cochlear implant users, six with bimodal stimulation and nine with exclusive cochlear implant and a control group consisting of 15 hearing individuals. Both groups underwent the standard duration and the standard frequency tests for temporal ordering auditory ability analysis. In the inferential analysis of the data, the nonparametric statistical tests Mann-Whitney and Kruskal-Wallis were used (p<0,05). Results: there was a significant difference in the performance of the standard frequency test, and subjects with bimodal stimulation presented a better performance than those with exclusive cochlear implants. The standard frequency test was associated with education and speech therapy. On the other hand, in the standard duration test, there was a relation with the implanted ear. The most frequent errors were discrimination for both temporal tests. Conclusion: bimodal stimulation, education and speech therapy influenced the performance of the frequency pattern test, while the implanted side influenced the performance of the duration pattern test.

The Effect of Hearing Aid Bandwidth and Configuration of Hearing Loss on Bimodal Speech Recognition in Cochlear Implant Users.

(1) To determine the effect of hearing aid (HA) bandwidth on bimodal speech perception in a group of unilateral cochlear implant (CI) patients with diverse degrees and configurations of hearing loss in the nonimplanted ear, (2) to determine whether there are demographic and audiometric characteristics that would help to determine the appropriate HA bandwidth for a bimodal patient. Participants were 33 experienced bimodal device users with postlingual hearing loss. Twenty three of them had better speech perception with the CI than the HA (CI>HA group) and 10 had better speech perception with the HA than the CI (HA>CI group). Word recognition in sentences (AzBio sentences at +10 dB signal to noise ratio presented at 0° azimuth) and in isolation [CNC (consonant-nucleus-consonant) words] was measured in unimodal conditions [CI alone or HAWB, which indicates HA alone in the wideband (WB) condition] and in bimodal conditions (BMWB, BM2k, BM1k, and BM500) as the bandwidth of an actual HA was reduced from WB to 2 kHz, 1 kHz, and 500 Hz. Linear mixed-effect modeling was used to quantify the relationship between speech recognition and listening condition and to assess how audiometric or demographic covariates might influence this relationship in each group. For the CI>HA group, AzBio scores were significantly higher (on average) in all bimodal conditions than in the best unimodal condition (CI alone) and were highest at the BMWB condition. For CNC scores, on the other hand, there was no significant improvement over the CI-alone condition in any of the bimodal conditions. The opposite pattern was observed in the HA>CI group. CNC word scores were significantly higher in the BM2k and BMWB conditions than in the best unimodal condition (HAWB), but none of the bimodal conditions were significantly better than the best unimodal condition for AzBio sentences (and some of the restricted bandwidth conditions were actually worse). Demographic covariates did not interact significantly with bimodal outcomes, but some of the audiometric variables did. For CI>HA participants with a flatter audiometric configuration and better mid-frequency hearing, bimodal AzBio scores were significantly higher than the CI-alone score with the WB setting (BMWB) but not with other bandwidths. In contrast, CI>HA participants with more steeply sloping hearing loss and poorer mid-frequency thresholds (≥82.5 dB) had significantly higher bimodal AzBio scores in all bimodal conditions, and the BMWB did not differ significantly from the restricted bandwidth conditions. HA>CI participants with mild low-frequency hearing loss showed the highest levels of bimodal improvement over the best unimodal condition on CNC words. They were also less affected by HA bandwidth reduction compared with HA>CI participants with poorer low-frequency thresholds. The pattern of bimodal performance as a function of the HA bandwidth was found to be consistent with the degree and configuration of hearing loss for both patients with CI>HA performance and for those with HA>CI performance. Our results support fitting the HA for all bimodal patients with the widest bandwidth consistent with effective audibility.

Measuring the effect of high-speed video data on the audio-visual speech recognition accuracy

Introduction: The effectiveness of modern automatic speech recognition systems in quiet acoustic conditions is quite high and reaches 90-95%. However, in noisy uncontrolled environment, acoustic signals are often distorted, which greatly reduces the resulting recognition accuracy. In adverse conditions, it seems appropriate to use the visual information about the speech, as it is not affected by the acoustic noise. At the moment, there are no studies which objectively reflect the dependence of visual speech recognition accuracy on the video frame rate, and there are no relevant audio-visual databases for model training. Purpose : Improving the reliability and accuracy of the automatic audio-visual Russian speech recognition system; collecting representative audio-visual database and developing an experimental setup. Methods : For audio-visual speech recognition, we used coupled hidden Markov model architectures. For parametric representation of audio and visual features, we used mel-frequency cepstral coefficients and principal component analysis-based pixel features. Results: In the experiments, we studied 5 different rates of video data: 25, 50, 100, 150, and 200 fps. Experiments have shown a positive effect from the use of a high-speed video camera: we achieved an absolute increase in accuracy of 1.48% for a bimodal system and 3.10% for a unimodal one, as compared to the standard recording speed of 25 fps. During the experiments, test data for all speakers were added with two types of noise: wide-band white noise and “babble noise”. Analysis shows that bimodal speech recognition exceeds unimodal in accuracy, especially for low SNR values <15 dB. At very low SNR values <5 dB, the acoustic information becomes non-informative, and the best results are achieved by a unimodal visual speech recognition system. Practical relevance : The use of a high-speed camera can improve the accuracy and robustness of a continuous audio-visual Russian speech recognition system.

Cochlear implantation for single-sided deafness in children and adolescents

ObjectiveTo evaluate outcomes in pediatric and adolescent patients with single-sided deafness (SSD) undergoing cochlear implantation. MethodsA retrospective cohort design at two tertiary level academic cochlear implant centers. The subjects included nine children ages 1.5 to 15 years-old with single-sided deafness (SSD) who had undergone cochlear implantation in the affected ear. Objective outcome measures included were speech reception testing in quiet and noise, bimodal speech reception threshold testing in noise, tinnitus suppression, and device usage. ResultsNine pediatric and adolescent patients with SSD were implanted between 2011 and 2017. The median age at implantation was 8.9 years (range, 1.5–15.1) and the children had a median duration of deafness 2.9 years (range, 0.8–9.5). There was variability in testing measures due to patient age. Median pre-operative aided word recognition scores on the affected side were <30% regardless of the testing paradigm used. Six patients had pre-operative word testing (4 CNC, median score 25%; 2 MLNT, 8% and 17%). Four patients had pre-operative sentence testing (3 AzBio, median score 44%; 1 HINT-C, 57%). Median post-implantation follow-up interval was 12.3 months (range, 3–27.6 months). Six subjects had post-operative word recognition testing (CNC median, 70%; MLNT 50%, 92%) with a median improvement of 45.5% points. Five subjects had post-operative sentence testing (AzBio, median 82%; HINT, median 76%), with a median improvement of 40.5% points. Eight patients are full time users of their device. Tinnitus and bimodal speech reception thresholds in noise were improved. ConclusionPediatric subjects with SSD benefit substantially from cochlear implantation. Objective speech outcome measures are improved in both quiet and noise, and bimodal speech reception thresholds in noise are greatly improved. There is a low rate of device non-use.

Results in Adult Cochlear Implant Recipients With Varied Asymmetric Hearing: A Prospective Longitudinal Study of Speech Recognition, Localization, and Participant Report.

Asymmetric hearing with severe to profound hearing loss (SPHL) in one ear and better hearing in the other requires increased listening effort and is detrimental for understanding speech in noise and sound localization. Although a cochlear implant (CI) is the only treatment that can restore hearing to an ear with SPHL, current candidacy criteria often disallows this option for patients with asymmetric hearing. The present study aimed to evaluate longitudinal performance outcomes in a relatively large group of adults with asymmetric hearing who received a CI in the poor ear. Forty-seven adults with postlingual hearing loss participated. Test materials included objective and subjective measures meant to elucidate communication challenges encountered by those with asymmetric hearing. Test intervals included preimplant and 6 and 12 months postimplant. Preimplant testing was completed in participants' everyday listening condition: bilateral hearing aids (HAs) n = 9, better ear HA n = 29, and no HA n = 9; postimplant, each ear was tested separately and in the bimodal condition. Group mean longitudinal results in the bimodal condition postimplant compared with the preimplant everyday listening condition indicated significantly improved sentence scores at soft levels and in noise, improved localization, and higher ratings of communication function by 6 months postimplant. Group mean, 6-month postimplant results were significantly better in the bimodal condition compared with either ear alone. Audibility and speech recognition for the poor ear alone improved significantly with a CI compared with preimplant. Most participants had clinically meaningful benefit on most measures. Contributory factors reported for traditional CI candidates also impacted results for this population. In general, older participants had poorer bimodal speech recognition in noise and localization abilities than younger participants. Participants with early SPHL onset had better bimodal localization than those with later SPHL onset, and participants with longer SPHL duration had poorer CI alone speech understanding in noise but not in quiet. Better ear pure-tone average (PTA) correlated with all speech recognition measures in the bimodal condition. To understand the impact of better ear hearing on bimodal performance, participants were grouped by better ear PTA: group 1 PTA ≤40 dB HL (n = 19), group 2 PTA = 41 to 55 dB HL (n = 14), and group 3 PTA = 56 to 70 dB HL (n = 14). All groups showed bimodal benefit on speech recognition measures in quiet and in noise; however, only group 3 obtained benefit when noise was toward the CI ear. All groups showed improved localization and ratings of perceived communication. Receiving a CI for the poor ear was an effective treatment for this population. Improved audibility and speech recognition were evident by 6 months postimplant. Improvements in sound localization and self-reports of communication benefit were significant and not related to better ear hearing. Participants with more hearing in the better ear (group 1) showed less bimodal benefit but greater bimodal performance for speech recognition than groups 2 and 3. Test batteries for this population should include quality of life measures, sound localization, and adaptive speech recognition measures with spatially separated noise to capture the hearing loss deficits and treatment benefits reported by this patient population.

Children with dyslexia show a reduced processing benefit from bimodal speech information compared to their typically developing peers

During information processing, individuals benefit from bimodally presented input, as has been demonstrated for speech perception (i.e., printed letters and speech sounds) or the perception of emotional expressions (i.e., facial expression and voice tuning). While typically developing individuals show this bimodal benefit, school children with dyslexia do not. Currently, it is unknown whether the bimodal processing deficit in dyslexia also occurs for visual-auditory speech processing that is independent of reading and spelling acquisition (i.e., no letter-sound knowledge is required). Here, we tested school children with and without spelling problems on their bimodal perception of video-recorded mouth movements pronouncing syllables. We analyzed the event-related potential Mismatch Response (MMR) to visual-auditory speech information and compared this response to the MMR to monomodal speech information (i.e., auditory-only, visual-only). We found a reduced MMR with later onset to visual-auditory speech information in children with spelling problems compared to children without spelling problems. Moreover, when comparing bimodal and monomodal speech perception, we found that children without spelling problems showed significantly larger responses in the visual-auditory experiment compared to the visual-only response, whereas children with spelling problems did not. Our results suggest that children with dyslexia exhibit general difficulties in bimodal speech perception independently of letter-speech sound knowledge, as apparent in altered bimodal speech perception and lacking benefit from bimodal information. This general deficit in children with dyslexia may underlie the previously reported reduced bimodal benefit for letter-speech sound combinations and similar findings in emotion perception.

Comparison of Cepstral Normalization Techniques in Whispered Speech Recognition

This article presents an analysis of different cepstral normalization techniques in automatic recognition of whispered and bimodal speech (speech+whisper). In these experiments, conven ...

Monaural Beamforming in Bimodal Cochlear Implant Users: Effect of (A)symmetric Directivity and Noise Type.

ObjectiveTo evaluate monaural beamforming in bimodally aided cochlear implant (CI) users.DesignThe study enrolled twelve adult bimodal listeners with at least six months of CI-experience and using a contralateral hearing aid (HA) most of the daytime. Participants were uniformly fitted with the same CI speech processor and HA, giving access to an identical monaural beamformer in both ears. A within-subject repeated measures design evaluated three directional configurations [omnidirectional, asymmetric directivity (in CI alone) and symmetric directivity (in both CI and HA)] in two noise types [stationary and fluctuating]. Bimodal speech reception thresholds (SRT) as well as listening effort ratings were assessed in a diffuse noise field.ResultsSymmetric monaural beamforming provided a significant SRT improvement of 2.6 dB SNR, compared to 1.6 dB SNR for asymmetric monaural beamforming. Directional benefits were similarly observed in stationary and fluctuating noise. Directivity did not contribute to less listening effort in addition to improvement in speech intelligibility. Bimodal performance was about 7 dB SNR worse in fluctuating than in stationary noise.ConclusionsMonaural beamforming provided substantial benefit for speech intelligibility in noise for bimodal listeners. The greatest benefit occurred when monaural beamforming was activated symmetrically in both CI and HA. Monaural beamforming does not bridge the gap between bimodal and normal hearing performance, especially in fluctuating noise. Results advocate further bimodal co-operation.Trial RegistrationThis trial was registered in www.trialregister.nl under number NTR4901.

Bimodal Dynamics of Faces and Voices Influence Older Infants' Perception of Emotion

Given the importance of infants' perception of bimodal speech for emerging language and emotion development, this study used eye‐tracking technology to examine infants' attention to face+voice displays differing by emotion (fear, sad, happy) and visual stimulus (dynamic versus static). Peripheral distracters were presented to measure attention disengagement. It was predicted that infants would look longer at and disengage more slowly from dynamic bimodal emotion displays, especially when viewing dynamic fear. However, the results from twenty‐two 10‐month‐olds found significantly greater attention on dynamic versus static trials, independent of emotion. Interestingly, infants looked equally to mouth and eye regions of speakers' faces except when viewing/hearing dynamic fear; in this case, they fixated more on the speakers' mouth region. Average latencies to distracters were longer on dynamic compared to static bimodal stimuli, but not differentiated by emotion. Thus, infants' attention was enhanced (in terms of both elicitation and maintenance) by dynamic, bimodal emotion displays. Results are compared to conflicting findings using static emotion displays, with suggestions for future research using more ecologically relevant dynamic, multimodal displays to gain a richer understanding of infants' processing of emotion.

Contributions of electric and acoustic hearing to bimodal speech and music perception.

Cochlear implant (CI) users have difficulty understanding speech in noisy listening conditions and perceiving music. Aided residual acoustic hearing in the contralateral ear can mitigate these limitations. The present study examined contributions of electric and acoustic hearing to speech understanding in noise and melodic pitch perception. Data was collected with the CI only, the hearing aid (HA) only, and both devices together (CI+HA). Speech reception thresholds (SRTs) were adaptively measured for simple sentences in speech babble. Melodic contour identification (MCI) was measured with and without a masker instrument; the fundamental frequency of the masker was varied to be overlapping or non-overlapping with the target contour. Results showed that the CI contributes primarily to bimodal speech perception and that the HA contributes primarily to bimodal melodic pitch perception. In general, CI+HA performance was slightly improved relative to the better ear alone (CI-only) for SRTs but not for MCI, with some subjects experiencing a decrease in bimodal MCI performance relative to the better ear alone (HA-only). Individual performance was highly variable, and the contribution of either device to bimodal perception was both subject- and task-dependent. The results suggest that individualized mapping of CIs and HAs may further improve bimodal speech and music perception.