Auditory-visual Speech Research Articles

Speech perception: Auditory and visual cue integration in children with and without phonological disorder in voiceless fricatives

ABSTRACT The literature reports contradictory results regarding the influence of visual cues on speech perception tasks in children with phonological disorder (PD). This study aimed to compare the performance of children with (n = 15) and without PD (n = 15) in audiovisual perception task in voiceless fricatives. Assuming that PD could be associated with an inability to integrate phonological information from two sensory sources, we presumed that children with PD would present difficulties in integrating auditory and visual cues compared to typical children. A syllable identification task was conducted. The stimuli were presented according to four conditions: auditory-only (AO); visual-only (VO); audiovisual congruent (AV+); and audiovisual incongruent (AV-). The percentages of correct answers and the respective reaction times in the AO, VO, and AV+ conditions were considered for the analysis. The correct percentage of auditory stimuli was considered for the AV- condition, as well as the percentage of perceptual preference: auditory, visual, and/or illusion (McGurk effect), with the respective reaction time. In comparing the four conditions, children with PD presented a lower number of correct answers and longer reaction time than children with typical development, mainly for the VO. Both groups showed a preference for auditory stimuli for the AV- condition. However, children with PD showed higher percentages for visual perceptual preference and the McGurk effect than typical children. The superiority of typical children over PD children in auditory-visual speech perception depends on type of stimuli and condition of presentation.

Uso de máscara na comunicação oral para usuários de dispositivos auxiliares à audição

ABSTRACT Purpose: to assess the influence of wearing a mask on auditory-visual speech recognition, in a favorable listening situation, in hearing devices users. Methods: a cross-sectional observational study comprising 52 hearing aid users, whose speech recognition was assessed with six video-recorded lists of sentences with and without masks. The mean test results in the various situations were compared using the Friedman test with Bonferroni post hoc, the significance level being set at 5%. Results: speech recognition assessment results differed between the situations with and without masks and between mask types, with a small effect size. The post hoc, with p-values adjusted with the Bonferroni method, showed a difference between transparent masks and others. The transparent one had a higher mean (77.8%) of auditory-visual sentence recognition between the various situations. There were statistically significant differences, as the transparent mask provided a better performance than the others. Conclusion: the auditory-visual recognition of the hearing-impaired people was better with the transparent mask.

Seeing a Talking Face Matters: Gaze Behavior and the Auditory-Visual Speech Benefit in Adults' Cortical Tracking of Infant-directed Speech.

In face-to-face conversations, listeners gather visual speech information from a speaker's talking face that enhances their perception of the incoming auditory speech signal. This auditory-visual (AV) speech benefit is evident even in quiet environments but is stronger in situations that require greater listening effort such as when the speech signal itself deviates from listeners' expectations. One example is infant-directed speech (IDS) presented to adults. IDS has exaggerated acoustic properties that are easily discriminable from adult-directed speech (ADS). Although IDS is a speech register that adults typically use with infants, no previous neurophysiological study has directly examined whether adult listeners process IDS differently from ADS. To address this, the current study simultaneously recorded EEG and eye-tracking data from adult participants as they were presented with auditory-only (AO), visual-only, and AV recordings of IDS and ADS. Eye-tracking data were recorded because looking behavior to the speaker's eyes and mouth modulates the extent of AV speech benefit experienced. Analyses of cortical tracking accuracy revealed that cortical tracking of the speech envelope was significant in AO and AV modalities for IDS and ADS. However, the AV speech benefit [i.e., AV > (A + V)] was only present for IDS trials. Gaze behavior analyses indicated differences in looking behavior during IDS and ADS trials. Surprisingly, looking behavior to the speaker's eyes and mouth was not correlated with cortical tracking accuracy. Additional exploratory analyses indicated that attention to the whole display was negatively correlated with cortical tracking accuracy of AO and visual-only trials in IDS. Our results underscore the nuances involved in the relationship between neurophysiological AV speech benefit and looking behavior.

How do human newborns come to understand the multimodal environment?

For a long time, newborns were considered as human beings devoid of perceptual abilities who had to learn with effort everything about their physical and social environment. Extensive empirical evidence gathered in the last decades has systematically invalidated this notion. Despite the relatively immature state of their sensory modalities, newborns have perceptions that are acquired, and are triggered by, their contact with the environment. More recently, the study of the fetal origins of the sensory modes has revealed that in utero all the senses prepare to operate, except for the vision mode, which is only functional starting from the first minutes after birth. This discrepancy between the maturation of the different senses leads to the question of how human newborns come to understand our multimodal and complex environment. More precisely, how the visual mode interacts with the tactile and auditory modes from birth. After having defined the tools that newborns use to interact with other sensory modalities, we review studies across different fields of research such as the intermodal transfer between touch and vision, auditory-visual speech perception, and the existence of links between the dimensions of space, time, and number. Overall, evidence from these studies supports the idea that human newborns are spontaneously driven, and cognitively equipped, to link information collected by the different sensory modes in order to create a representation of a stable world.

Consonant Perception in Connected Syllables Spoken at a Conversational Syllabic Rate.

Closed-set consonant identification, measured using nonsense syllables, has been commonly used to investigate the encoding of speech cues in the human auditory system. Such tasks also evaluate the robustness of speech cues to masking from background noise and their impact on auditory-visual speech integration. However, extending the results of these studies to everyday speech communication has been a major challenge due to acoustic, phonological, lexical, contextual, and visual speech cue differences between consonants in isolated syllables and in conversational speech. In an attempt to isolate and address some of these differences, recognition of consonants spoken in multisyllabic nonsense phrases (e.g., aBaSHaGa spoken as /ɑbɑʃɑɡɑ/) produced at an approximately conversational syllabic rate was measured and compared with consonant recognition using Vowel-Consonant-Vowel bisyllables spoken in isolation. After accounting for differences in stimulus audibility using the Speech Intelligibility Index, consonants spoken in sequence at a conversational syllabic rate were found to be more difficult to recognize than those produced in isolated bisyllables. Specifically, place- and manner-of-articulation information was transmitted better in isolated nonsense syllables than for multisyllabic phrases. The contribution of visual speech cues to place-of-articulation information was also lower for consonants spoken in sequence at a conversational syllabic rate. These data imply that auditory-visual benefit based on models of feature complementarity from isolated syllable productions may over-estimate real-world benefit of integrating auditory and visual speech cues.

Benefits of Text Supplementation on Sentence Recognition and Subjective Ratings With and Without Facial Cues for Listeners With Normal Hearing.

Recognizing speech through telecommunication can be challenging in unfavorable listening conditions. Text supplementation or provision of facial cues can facilitate speech recognition under some circumstances. However, our understanding of the combined benefit of text and facial cues in telecommunication is limited. The purpose of this study was to investigate the potential benefit of text supplementation for sentence recognition scores and subjective ratings of spoken speech with and without facial cues available. Twenty adult females (M = 24 years, range 21 to 29 years) with normal hearing performed a sentence recognition task and also completed a subjective rating questionnaire in 24 conditions. The conditions varied by integrity of the available facial cues (clear facial cues, slight distortion facial cues, great distortion facial cues, no facial cues), signal-to-noise ratio (quiet, +1 dB, -3 dB), and text availability (with text, without text). When present, the text was an 86 to 88% accurate transcription of the auditory signal presented at a 500 ms delay relative to the auditory signal. The benefits of text supplementation were largest when facial cues were not available and when the signal-to-noise ratio was unfavorable. Although no recognition score benefit was present in quiet, recognition benefit was significant in all levels of background noise for all levels of facial cue integrity. Moreover, participant subjective ratings of text benefit were robust and present even in the absence of recognition benefit. Consistent with previous literature, facial cues were beneficial for sentence recognition scores in the most unfavorable signal-to-noise ratio, even when greatly distorted. It is interesting that, although all levels of facial cues were beneficial for recognition scores, participants rated a significant benefit only with clear facial cues. The benefit of text for auditory-only and auditory-visual speech recognition is evident in recognition scores and subjective ratings; the benefit is larger and more robust for subjective ratings than for scores. Therefore, text supplementation might provide benefit that extends beyond speech recognition scores. Combined, these findings support the use of text supplementation in telecommunication, even when facial cues are concurrently present, such as during teleconferencing or watching television.

Multisensory benefits for speech recognition in noisy environments.

A series of our previous studies explored the use of an abstract visual representation of the amplitude envelope cues from target sentences to benefit speech perception in complex listening environments. The purpose of this study was to expand this auditory-visual speech perception to the tactile domain. Twenty adults participated in speech recognition measurements in four different sensory modalities (AO, auditory-only; AV, auditory-visual; AT, auditory-tactile; AVT, auditory-visual-tactile). The target sentences were fixed at 65 dB sound pressure level and embedded within a simultaneous speech-shaped noise masker of varying degrees of signal-to-noise ratios (-7, -5, -3, -1, and 1 dB SNR). The amplitudes of both abstract visual and vibrotactile stimuli were temporally synchronized with the target speech envelope for comparison. Average results showed that adding temporally-synchronized multimodal cues to the auditory signal did provide significant improvements in word recognition performance across all three multimodal stimulus conditions (AV, AT, and AVT), especially at the lower SNR levels of -7, -5, and -3 dB for both male (8-20% improvement) and female (5-25% improvement) talkers. The greatest improvement in word recognition performance (15-19% improvement for males and 14-25% improvement for females) was observed when both visual and tactile cues were integrated (AVT). Another interesting finding in this study is that temporally synchronized abstract visual and vibrotactile stimuli additively stack in their influence on speech recognition performance. Our findings suggest that a multisensory integration process in speech perception requires salient temporal cues to enhance speech recognition ability in noisy environments.

Seeing a talking face matters: Infants' segmentation of continuous auditory-visual speech.

Visual speech cues from a speaker's talking face aid speech segmentation in adults, but despite the importance of speech segmentation in language acquisition, little is known about the possible influence of visual speech on infants' speech segmentation. Here, to investigate whether there is facilitation of speech segmentation by visual information, two groups of English-learning 7-month-old infants were presented with continuous speech passages, one group with auditory-only (AO) speech and the other with auditory-visual (AV) speech. Additionally, the possible relation between infants' relative attention to the speaker's mouth versus eye regions and their segmentation performance was examined. Both the AO and the AV groups of infants successfully segmented words from the continuous speech stream, but segmentation performance persisted for longer for infants in the AV group. Interestingly, while AV group infants showed no significant relation between the relative amount of time spent fixating the speaker's mouth versus eyes and word segmentation, their attention to the mouth was greater than that of AO group infants, especially early in test trials. The results are discussed in relation to the possible pathways through which visual speech cues aid speech perception.

Seeing a talking face matters: The relationship between cortical tracking of continuous auditory‐visual speech and gaze behaviour in infants, children and adults

An auditory-visual speech benefit, the benefit that visual speech cues bring to auditory speech perception, is experienced from early on in infancy and continues to be experienced to an increasing degree with age. While there is both behavioural and neurophysiological evidence for children and adults, only behavioural evidence exists for infants – as no neurophysiological study has provided a comprehensive examination of the auditory-visual speech benefit in infants. It is also surprising that most studies on auditory-visual speech benefit do not concurrently report looking behaviour especially since the auditory-visual speech benefit rests on the assumption that listeners attend to a speaker's talking face and that there are meaningful individual differences in looking behaviour. To address these gaps, we simultaneously recorded electroencephalographic (EEG) and eye-tracking data of 5-month-olds, 4-year-olds and adults as they were presented with a speaker in auditory-only (AO), visual-only (VO), and auditory-visual (AV) modes. Cortical tracking analyses that involved forward encoding models of the speech envelope revealed that there was an auditory-visual speech benefit [i.e., AV > (A + V)], evident in 5-month-olds and adults but not 4-year-olds. Examination of cortical tracking accuracy in relation to looking behaviour, showed that infants’ relative attention to the speaker's mouth (vs. eyes) was positively correlated with cortical tracking accuracy of VO speech, whereas adults’ attention to the display overall was negatively correlated with cortical tracking accuracy of VO speech. This study provides the first neurophysiological evidence of auditory-visual speech benefit in infants and our results suggest ways in which current models of speech processing can be fine-tuned.

Detection and Recognition of Asynchronous Auditory/Visual Speech: Effects of Age, Hearing Loss, and Talker Accent.

This investigation examined age-related differences in auditory-visual (AV) integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous (referred to as the “AV simultaneity window”). The older participants were also expected to exhibit greater declines in speech recognition for asynchronous AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which they judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio (SNR) corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech.

Speechreading as a secondary diagnostic tool in bipolar disorder

Speech perception used to be seen as a solely an auditory phenomenon until the discovery of the “McGurk Effect” and in recent decades it has come to be understood as a multimodal, specifically, an auditory-visual process. Behavioural studies have shown that the magnitude of auditory-visual integration shows differences across ages and languages. A new question regarding the integration of auditory and visual speech information pertains to the case of mental disorders – a domain in which a significant paucity of data exits. The available studies (e.g., schizophrenic patients) generally point out to lessened levels of integration on both behavioural and cortical levels in respective clinical populations. A recent study with bipolar disordered individuals has shown that while those in manic and depressive episodes give similar behavioural responses to McGurk stimuli revealing an integration of auditory and visual speech information similar to the control group. However, in a visual-only condition (i.e., lipreading), compared to the control group and manic episode patients, those in the depressive episode show very impoverished levels of lipreading. We hypothesize that, during depressive episode of bipolar patients, abnormal lipreading is associated with impaired occipito-temporal connections in the dominant hemisphere, which are compensated by early activation of anterior cingulate gyrus and IFG, either through dominant or non-dominant hemisphere pathways. This can relay the signal in reverse fashion to the multimodal procession areas like posterior superior temporal sulcus, hence allowing audiovisual integration. This finding calls for cortical imaging studies to unearth the physiological processes involved in both lipreading and auditory-visual speech processing in the bipolar disordered individuals. A full understanding of these processes will provide practitioners with secondary diagnostic tools and knowledge regarding this disorder.

Multimodal Second-Language Communication: Research Findings and Pedagogical Implications

This article reviews research findings involving visual input in speech processing in the form of facial cues and co-speech gestures for second-language (L2) learners, and provides pedagogical implications for the teaching of listening and speaking. It traces the foundations of auditory–visual speech research and explores the role of a speaker’s facial cues in L2 perception training and of gestural cues in listening comprehension. There is a strong role for pedagogy to maximize the salience of multimodal cues for L2 learners. Visible articulatory gestures that precede the acoustic signal and the preparation phase of a hand gesture that precedes the acoustic onset of a word provide a priming effect on perceivers’ attention to signal upcoming information and facilitate processing, and visible gestures that co-occur with speech aid ongoing processing and comprehension. L2 learners benefit from an awareness of these visual cues and exposure to input.

Auditory and auditory-visual frequency-band importance functions for consonant recognition.

The relative importance of individual frequency regions for speech intelligibility has been firmly established for broadband auditory-only (AO) conditions. Yet, speech communication often takes place face-to-face. This study tested the hypothesis that under auditory-visual (AV) conditions, where visual information is redundant with high-frequency auditory cues, lower frequency regions will increase in relative importance compared to AO conditions. Frequency band-importance functions for consonants were measured for eight hearing-impaired and four normal-hearing listeners. Speech was filtered into four 1/3-octave bands each separated by an octave to minimize energetic masking. On each trial, the signal-to-noise ratio (SNR) in each band was selected randomly from a 10-dB range. AO and AV band-importance functions were estimated using three logistic-regression analyses: a primary model relating performance to the four independent SNRs; a control model that also included band-interaction terms; and a different set of four control models, each examining one band at a time. For both listener groups, the relative importance of the low-frequency bands increased under AV conditions, consistent with earlier studies using isolated speech bands. All three analyses showed similar results, indicating the absence of cross-band interactions. These results suggest that accurate prediction of AV speech intelligibility may require different frequency-importance functions than for AO conditions.

Age Differences in the Effects of Speaking Rate on Auditory, Visual, and Auditory-Visual Speech Perception.

This study was designed to examine how speaking rate affects auditory-only, visual-only, and auditory-visual speech perception across the adult lifespan. In addition, the study examined the extent to which unimodal (auditory-only and visual-only) performance predicts auditory-visual performance across a range of speaking rates. The authors hypothesized significant Age × Rate interactions in all three modalities and that unimodal performance would account for a majority of the variance in auditory-visual speech perception for speaking rates that are both slower and faster than normal. Participants (N = 145), ranging in age from 22 to 92, were tested in conditions with auditory-only, visual-only, and auditory-visual presentations using a closed-set speech perception test. Five different speaking rates were presented in each modality: an unmodified (normal rate), two rates that were slower than normal, and two rates that were faster than normal. Signal to noise ratios were set individually to produce approximately 30% correct identification in the auditory-only condition and this signal to noise ratio was used in the auditory-only and auditory-visual conditions. Age × Rate interactions were observed for the fastest speaking rates in both the visual-only and auditory-visual conditions. Unimodal performance accounted for at least 60% of the variance in auditory-visual performance for all five speaking rates. The findings demonstrate that the disproportionate difficulty that older adults have with rapid speech for auditory-only presentations can also be observed with visual-only and auditory-visual presentations. Taken together, the present analyses of age and individual differences indicate a generalized age-related decline in the ability to understand speech produced at fast speaking rates. The finding that auditory-visual speech performance was almost entirely predicted by unimodal performance across all five speaking rates has important clinical implications for auditory-visual speech perception and the ability of older adults to use visual speech information to compensate for age-related hearing loss.

Auditory–visual integration during nonconscious perception

Our study proposes a test of a key assumption of the most prominent model of consciousness – the global workspace (GWS) model (e.g., Baars, 2002, 2005, 2007; Dehaene & Naccache, 2001; Mudrik, Faivre, & Koch, 2014). This assumption is that multimodal integration requires consciousness; however, few studies have explicitly tested if integration can occur between nonconscious information from different modalities. The proposed study examined whether a classic indicator of multimodal integration – the McGurk effect – can be elicited with subliminal auditory–visual speech stimuli. We used a masked speech priming paradigm developed by Kouider and Dupoux (2005) in conjunction with continuous flash suppression (CFS; Tsuchiya & Koch, 2005), a binocular rivalry technique for presenting video stimuli subliminally. Applying these techniques together, we carried out two experiments in which participants categorised auditory syllable targets which were preceded by subliminal auditory–visual (AV) speech primes. Subliminal AV primes were either illusion-inducing (McGurk) or illusion-neutral (Incongruent) combinations of speech stimuli. In Experiment 1, the categorisation of the syllable target (“pa”) was facilitated by the same syllable prime when it was part of a McGurk combination (auditory “pa” and visual “ka”) but not when part of an Incongruent combination (auditory “pa” and visual “wa”). This dependency on specific AV combinations indicated a nonconscious AV interaction. Experiment 2 presented a different syllable target (“ta”) which matched the predicted illusory outcome of the McGurk combination – here, both the McGurk combination (auditory “pa” and visual “ka”) and the Incongruent combination (auditory “ta” and visual “ka”) failed to facilitate target categorisation. The combined results of both Experiments demonstrate a type of nonconscious multimodal interaction that is distinct from integration – it allows unimodal information that is compatible for integration (i.e., McGurk combinations) to persist and influence later processes, but does not actually combine and alter that information. As the GWS model does not account for non-integrative multimodal interactions, this places some pressure on such models of consciousness.

Auditory–Visual Speech Integration in Bipolar Disorder: A Preliminary Study

This study aimed to investigate how individuals with bipolar disorder integrate auditory and visual speech information compared to healthy individuals. Furthermore, we wanted to see whether there were any differences between manic and depressive episode bipolar disorder patients with respect to auditory and visual speech integration. It was hypothesized that the bipolar group’s auditory–visual speech integration would be weaker than that of the control group. Further, it was predicted that those in the manic phase of bipolar disorder would integrate visual speech information more robustly than their depressive phase counterparts. To examine these predictions, a McGurk effect paradigm with an identification task was used with typical auditory–visual (AV) speech stimuli. Additionally, auditory-only (AO) and visual-only (VO, lip-reading) speech perceptions were also tested. The dependent variable for the AV stimuli was the amount of visual speech influence. The dependent variables for AO and VO stimuli were accurate modality-based responses. Results showed that the disordered and control groups did not differ in AV speech integration and AO speech perception. However, there was a striking difference in favour of the healthy group with respect to the VO stimuli. The results suggest the need for further research whereby both behavioural and physiological data are collected simultaneously. This will help us understand the full dynamics of how auditory and visual speech information are integrated in people with bipolar disorder.

Speech intelligibility of virtual humans

The speech intelligibility benefit of visual speech cues during oral communication is well-established. Therefore, an ecologically valid approach of auditory assessment should include the processing of both auditory and visual speech cues. This study describes the development and evaluation of a virtual human speaker designed to present speech auditory-visually. A male and female virtual human speaker were created and evaluated in two experiments: a visual-only speech reading test of words and sentences and an auditory-visual speech intelligibility sentence test. A group of five hearing, skilled speech reading adults participated in the speech reading test whereas a group of young normal-hearing participants (N = 35) was recruited for the intelligibility test. Skilled speech readers correctly identified 57 to 67% of the words and sentences uttered by the virtual speakers. The presence of the virtual speaker improved the speech intelligibility of sentences in noise by 1.5 to 2 dB. These results demonstrate the potential applicability of virtual humans in future auditory-visual speech assessment paradigms.

Assessing perceived listening difficulty using behavioral gaze patterns for audiovisual speech

Listeners alternate their gaze between the talker’s eyes, nose, and mouth during conversation, spending more time gazing directly at the mouth in more difficult listening conditions. Perceived listening difficulty and speech perception performance vary independently, demonstrated with indirect measures of perceived listening difficulty/effort such as pupil dilation and dual task paradigms. This study took a more direct approach, evaluating whether perceived listening difficulty could be assessed by observing a listener’s natural gaze pattern in a challenging speech-perception task. The proportion of time spent gazing at the talker’s mouth was measured for auditory-visual speech perception in conditions that produced equal percentage-correct scores, but where one condition was designed to produce greater perceived difficulty. Experiment 1 compared gaze proportion for a speech-shaped noise versus two same-gender interfering talkers. Experiment 2 examined the effect of sentence context using low and high-probability sentences. The results are discussed in terms of the potential application of the eye-gaze metric to evaluate rehabilitative interventions in cases where subjectively reported benefits do not manifest in improved speech-understanding scores. [The views expressed in this abstract are those of the authors and do not reflect the official policy of the Department of Army/Navy/Air Force, Department of Defense, or U.S. Government.]

Language-General Auditory-Visual Speech Perception: Thai-English and Japanese-English McGurk Effects.

Cross-language McGurk Effects are used to investigate the locus of auditory-visual speech integration. Experiment 1 uses the fact that [], as in 'sing', is phonotactically legal in word-final position in English and Thai, but in word-initial position only in Thai. English and Thai language participants were tested for 'n' perception from auditory [m]/visual [] (A[m]V[]) in word-initial and -final positions. Despite English speakers' native language bias to label word-initial [] as 'n', the incidence of 'n' percepts to A[m]V[] was equivalent for English and Thai speakers in final and initial positions. Experiment 2 used the facts that (i) [ð] as in 'that' is not present in Japanese, and (ii) English speakers respond more often with 'tha' than 'da' to A[ba]V[ga], but more often with 'di' than 'thi' to A[bi]V[gi]. English and three groups of Japanese language participants (Beginner, Intermediate, Advanced English knowledge) were presented with A[ba]V[ga] and A[bi]V[gi] by an English (Experiment 2a) or a Japanese (Experiment 2b) speaker. Despite Japanese participants' native language bias to perceive 'd' more often than 'th', the four groups showed a similar phonetic level effect of [a]/[i] vowel context× 'th' vs. 'd' responses to A[b]V[g] presentations. In Experiment 2b this phonetic level interaction held, but was more one-sided as very few 'th' responses were evident, even in Australian English participants. Results are discussed in terms of a phonetic plus postcategorical model, in which incoming auditory and visual information is integrated at a phonetic level, after which there are post-categorical phonemic influences.

A case of auditory neuropathy revealed by OTOF gene mutation analysis in a junior high school girl

ObjectiveCongenital auditory neuropathy (AN) affects hearing and speech development. The degree of hearing difficulty in congenital AN varies as a function of pathology at the inner ear hair cell (IHC) synapses or the auditory nerve. We report a case of a Chinese girl with AN revealed by OTOF (otoferlin) gene mutation analysis who had only a mild hearing loss. PatientA 13-year-old Chinese girl was diagnosed as having congenital AN on the basis of OTOF gene mutation analysis. She manifest a mild sensorineural hearing loss with 50% maximum monosyllable speech discrimination rate, normal DPOAEs (distortion product otoacoustic emissions) beyond ambient noise levels, only SPs (summating potentials) evoked during ECoG (electrocochleography) and absent ABRs (auditory evoked brainstem responses) bilaterally to clicks presented at 100 dBnHL. She was able to effectively communicate with others by speech reading owing to her mild hearing loss. Moreover, bilateral hearing aids helped her to communicate. ConclusionsOur patient was demonstrated to have a mutation on the OTOF gene. Nevertheless, she was able to communicate using auditory visual speech reading in spite of a mild auditory threshold elevation probably due to partial pathology at the IHC synapses or in the auditory nerve.