Speech Perception Processing Research Articles

Lexical processing depends on sublexical processing: Evidence from the visual world paradigm and aphasia.

Some early studies of people with aphasia reported strikingly better performance on lexical than on sublexical speech perception tasks. These findings challenged the claim that lexical processing depends on sublexical processing and suggested that acoustic information could be mapped directly to lexical representations. However, Dial and Martin (Neuropsychologia 96: 192-212, 2017) argued that these studies failed to match the discriminability of targets and distractors for the sublexical and lexical stimuli and showed that when using closely matched tasks with natural speech tokens, no patient performed substantially better at the lexical than at the sublexical processing task. In the current study, we sought to provide converging evidence for the dependence of lexical on sublexical processing by examining the perception of synthetic speech stimuli varied on a voice-onset time continuum using eye-tracking methodology, which is sensitive to online speech perception processes. Eight individuals with aphasia and ten age-matched controls completed two visual world paradigm tasks: phoneme (sublexical) and word (lexical) identification. For both identification and eye-movement data, strong correlations were observed between the sublexical and lexical tasks. Critically, no patient within the control range on the lexical task was impaired on the sublexical task. Overall, the current study supports the claim that lexical processing depends on sublexical processing. Implications for inferring deficits in people with aphasia and the use of sublexical tasks to assess sublexical processing are also discussed.

The role of linguistic variability in the perception of voice cues

Talkers’ voices play an important role in speech perception. Through voices, we identify individual talkers and can facilitate speech communication in challenging conditions (e.g., cocktail party situations). While previous research has suggested that several linguistic factors broadly influence talker perception, how these factors influence perception of the individual voice cues remains unclear. The current study investigated the role of linguistic variability in voice cue perception, specifically fundamental frequency (F0) and vocal-tract length (VTL). Just-Noticeable-Differences (JNDs) were obtained using a 3AFC adaptive paradigm. Effects of word status (words, nonwords), word characteristics (lexical frequency, neighborhood density), and nonword characteristics (phonotactic probability, neighborhood density) were examined. Results demonstrated that voice cue perception was influenced by linguistic variability. While overall similar for words and nonwords, F0 and VTL JNDs were affected by phonological information in nonwords, i.e., phonotactic probability, but not on lexical information, i.e., lexical frequency and neighborhood density. However, VTL JNDs varied less across linguistic conditions than F0 JNDs, suggesting different processing mechanisms for these voice cues. These findings provide better insight into the interaction of voice and linguistic information, which may also improve our understanding of speech perception processes in populations with limitations in voice perception, such as cochlear-implant listeners.

Computer simulations of coupled idiosyncrasies in speech perception and speech production with COSMO, a perceptuo-motor Bayesian model of speech communication.

The existence of a functional relationship between speech perception and production systems is now widely accepted, but the exact nature and role of this relationship remains quite unclear. The existence of idiosyncrasies in production and in perception sheds interesting light on the nature of the link. Indeed, a number of studies explore inter-individual variability in auditory and motor prototypes within a given language, and provide evidence for a link between both sets. In this paper, we attempt to simulate one study on coupled idiosyncrasies in the perception and production of French oral vowels, within COSMO, a Bayesian computational model of speech communication. First, we show that if the learning process in COSMO includes a communicative mechanism between a Learning Agent and a Master Agent, vowel production does display idiosyncrasies. Second, we implement within COSMO three models for speech perception that are, respectively, auditory, motor and perceptuo-motor. We show that no idiosyncrasy in perception can be obtained in the auditory model, since it is optimally tuned to the learning environment, which does not include the motor variability of the Learning Agent. On the contrary, motor and perceptuo-motor models provide perception idiosyncrasies correlated with idiosyncrasies in production. We draw conclusions about the role and importance of motor processes in speech perception, and propose a perceptuo-motor model in which auditory processing would enable optimal processing of learned sounds and motor processing would be helpful in unlearned adverse conditions.

О природе речевых ошибок многоязычных школьников

Elaboration of well-considered classification of speech errors of multi-language schoolchildren is the necessary factor of successful providing Russian language programs in the situation of multicultural education. Such classification could allow differentiated approach towards the education and certification of multilingual pupils. The article is a study of different types of errors and the reasons of such innovations. The article describes common errors of Russian-speaking and multilingual children. Special errors of bilingual children (including those appeared as the results of interference) are also studied in the article. Errors caused by linguistic reasons and failures caused by particularities of processes of speech perception and generation are under consideration. While analyzing the linguistic nature of deviation, authors give examples of errors: connected with (1) the system violations (a person is not well acquainted with language rules); (2) violation of the norm (a person owns language rules but does not acquainted with constraints imposed by the norm); (3) violations of usus (a person is not acquainted with the conditions of realization of the language norm). Interlingual interference is supposed to be a cause of errors of the first type: authors list the examples of sintagmatic and paradigmatic interference; analyze the cases of improper segmentation of speech flow under the influence of native/first language and wrong differentiation of language elements on phonetic, grammar and lexical levels. The errors connected with violation of norm are studied as ones caused with the language system structure: they appear when there are absolute or relative lacunas in language. Facts that are controversial to modern language rules and cases when the means of units of high level does not come from the units of low level. The study allows to differentiate the errors caused by the input failure and genetic errors — the deviations caused by the processes of natural language learning that pass with the accumulation of pupils language experience.

Experience, attention, and context in the processing of systematic variation in spoken language

The acoustic speech signal is characterized by enormous variability. Characteristics of individual speakers and groups of speakers profoundly change the way in which linguistic structure is realized. A substantial body of research suggests that language users track, retain, and use systematic variation to restructure linguistic representation and processing in order to maximize intelligibility of spoken language. Less clear is whether sources of variation differ in relevance during speech processing and how relevance changes as a function of experience and context. Research will be presented examining talker-, task-, and listener-related factors that mediate memory and learning of systematic variation in spoken language. The findings suggest that although listeners dynamically adapt to systematic changes in linguistic structure as a function of experience, this adaptation depends on the characteristics and frequency of particular sources of variations, the modulation of attention driven by the structure of the learning environment, and expectations and subsequent sensitivity to socially relevant variation. The considerable behavioral and representational plasticity that is characteristic of speech perception and spoken language processing may depend in part on the social, linguistic, and contextual relevance of the variation associated with both individual talkers and classes of talkers.

Does Implicit Voice Learning Improve Spoken Language Processing? Implications for Clinical Practice.

In typical interactions with other speakers, including a clinical environment, listeners become familiar with voices through implicit learning. Previous studies have found evidence for a Familiar Talker Advantage (better speech perception and spoken language processing for familiar voices) following explicit voice learning. The current study examined whether a Familiar Talker Advantage would result from implicit voice learning. Thirty-three adults and 16 second graders were familiarized with 1 of 2 talkers' voices over 2 days through live interactions as 1 of 2 experimenters administered standardized tests and interacted with the listeners. To assess whether this implicit voice learning would generate a Familiar Talker Advantage, listeners completed a baseline sentence recognition task and a post-learning sentence recognition task with both the familiar talker and the unfamiliar talker. No significant effect of voice familiarity was found for either the children or the adults following implicit voice learning. Effect size estimates suggest that familiarity with the voice may benefit some listeners, despite the lack of an overall effect of familiarity. We discuss possible clinical implications of this finding and directions for future research.

Who's who? The incremental processing of presuppositions (and beyond)

Who's who? The incremental processing of presuppositions (and beyond)

The electrophysiological correlates of sublexical speech perception: A combined electrophysiological/cognitive neuropsychology approach

The electrophysiological correlates of sublexical speech perception: A combined electrophysiological/cognitive neuropsychology approach

Reanalyzing neurocognitive data on the role of the motor system in speech perception within COSMO, a Bayesian perceptuo-motor model of speech communication

While neurocognitive data provide clear evidence for the involvement of the motor system in speech perception, its precise role and the way motor information is involved in perceptual decision remain unclear. In this paper, we discuss some recent experimental results in light of COSMO, a Bayesian perceptuo-motor model of speech communication. COSMO enables us to model both speech perception and speech production with probability distributions relating phonological units with sensory and motor variables. Speech perception is conceived as a sensory-motor architecture combining an auditory and a motor decoder thanks to a Bayesian fusion process. We propose the sketch of a neuroanatomical architecture for COSMO, and we capitalize on properties of the auditory vs. motor decoders to address three neurocognitive studies of the literature. Altogether, this computational study reinforces functional arguments supporting the role of a motor decoding branch in the speech perception process.

Effects of Aging and Background Babble Noise on Speech Perception Processing: An fMRI Study

Speech perception processing in a noisy environment is subjected to age-related decline. We used functional magnetic resonance imaging (fMRI) to examine cortical activation associated with such processing across four groups of participants with age ranges of 23–29, 30–37, 41–47 and 50–65 years old. All participants performed a forward repeat task in quiet environment (SQ) and in the presence of multi-talker babble noise (SN; 5-dB signal-to-noise ratio, SNR). Behavioral test results demonstrated a decrease in the performance accuracy associated with increasing age for both SQ and SN. However, a significant difference in the performance accuracy between these conditions could only be seen among the elderly (60–65 years old) subjects. The fMRI results across the four age groups showed a nearly similar pattern of brain activation in the auditory, speech, and attention areas during SQ and SN. Comparisons between SQ and SN demonstrated significantly lower brain activation in the left precentral gyrus, left postcentral gyrus, left Heschly’s gyrus, and right middle temporal gyrus under the latter condition. Other activated brain areas showed no significant differences in brain activation between SQ and SN. The decreases in cortical activation in the activated regions positively correlated with the decrease in the behavioral performance across age groups. These findings are discussed based on a dedifferentiation hypothesis that states that increased brain activation among older participants, as compared to young participants, is due to the age-related deficits in neural communication.

Evidence for causal top-down frontal contributions to predictive processes in speech perception

Perception relies on the integration of sensory information and prior expectations. Here we show that selective neurodegeneration of human frontal speech regions results in delayed reconciliation of predictions in temporal cortex. These temporal regions were not atrophic, displayed normal evoked magnetic and electrical power, and preserved neural sensitivity to manipulations of sensory detail. Frontal neurodegeneration does not prevent the perceptual effects of contextual information; instead, prior expectations are applied inflexibly. The precision of predictions correlates with beta power, in line with theoretical models of the neural instantiation of predictive coding. Fronto-temporal interactions are enhanced while participants reconcile prior predictions with degraded sensory signals. Excessively precise predictions can explain several challenging phenomena in frontal aphasias, including agrammatism and subjective difficulties with speech perception. This work demonstrates that higher-level frontal mechanisms for cognitive and behavioural flexibility make a causal functional contribution to the hierarchical generative models underlying speech perception.

Brief Stimulus Exposure Fully Remediates Temporal Processing Deficits Induced by Early Hearing Loss

In childhood, partial hearing loss can produce prolonged deficits in speech perception and temporal processing. However, early therapeutic interventions targeting temporal processing may improve later speech-related outcomes. Gap detection is a measure of auditory temporal resolution that relies on the auditory cortex (ACx), and early auditory deprivation alters intrinsic and synaptic properties in the ACx. Thus, early deprivation should induce deficits in gap detection, which should be reflected in ACx gap sensitivity. We tested whether earplugging-induced, early transient auditory deprivation in male and female Mongolian gerbils caused correlated deficits in behavioral and cortical gap detection, and whether these could be rescued by a novel therapeutic approach: brief exposure to gaps in background noise. Two weeks after earplug removal, animals that had been earplugged from hearing onset throughout auditory critical periods displayed impaired behavioral gap detection thresholds (GDTs), but this deficit was fully reversed by three 1 h sessions of exposure to gaps in noise. In parallel, after earplugging, cortical GDTs increased because fewer cells were sensitive to short gaps, and gap exposure normalized this pattern. Furthermore, in deprived animals, both first-spike latency and first-spike latency jitter increased, while spontaneous and evoked firing rates decreased, suggesting that deprivation causes a wider range of perceptual problems than measured here. These cortical changes all returned to control levels after gap exposure. Thus, brief stimulus exposure, perhaps in a salient context such as the unfamiliar placement into a testing apparatus, rescued impaired gap detection and may have potential as a remediation tool for general auditory processing deficits.SIGNIFICANCE STATEMENT Hearing loss in early childhood leads to impairments in auditory perception and language processing that can last well beyond the restoration of hearing sensitivity. Perceptual deficits can be improved by training, or by acoustic enrichment in animal models, but both approaches involve extended time and effort. Here, we used a novel remediation technique, brief periods of auditory stimulus exposure, to fully remediate cortical and perceptual deficits in gap detection induced by early transient hearing loss. This technique also improved multiple cortical response properties. Rescue by this efficient exposure regime may have potential as a therapeutic tool to remediate general auditory processing deficits in children with perceptual challenges arising from early hearing loss.

Effectiveness of an intervention program combining direct vocabulary instruction and individualized phonetic training for Mandarin-speaking young children with specific language impairment

Children with specific language impairment (SLI) demonstrate deficits in vocabulary development and novel word learning process, which have been proposed to stem from their speech perception and phonological processing deficits. In this study, we designed an intervention program combining small-group direct vocabulary instruction and individualized phonetic training computer program. Twenty-five 5- to 7-year-old children with SLI participated in the study, and another 15 children with SLI served as control group. All participants in the experimental group attended 50-minutes direct vocabulary instruction class with interactive activities for ten weeks. The phonetic training computer games included AX discrimination and identification tasks of Mandarin consonant and lexical tones minimal pairs presented in single word and sentence levels. Children were asked to play the phonetic training games at least 30 minutes a week at home and encouraged to run more if they prefer. Results showed that children in the experimental group performed better than control group in the post test, including receptive and expressive vocabulary knowledge. In addition, there was a correlation between the gain in the post test and the time spent in the phonetic training game. The result suggests the vocabulary intervention programs incorporated with speech perception training promote children’s vocabulary abilities.

Evaluating the relationship between sublexical and lexical processing in speech perception: Evidence from aphasia

Several studies have reported that aphasic patients may perform substantially better on lexical than sublexical perception tasks (e.g., Miceli et al., 1980). These findings challenge claims made by models of speech perception which assume obligatory sublexical processing (e.g., McClelland and Elman, 1986; Norris, 1994). However, prior studies have not closely matched the phonological similarity of targets and distractors or task demands of the sublexical and lexical perception tasks. The current study addressed shortcomings of these prior studies, testing 13 aphasic patients on sublexical and lexical tasks matched in phonological similarity of stimuli and task demands. When the lexical and sublexical tasks were not matched (Experiment 1a), as in prior studies (e.g., Miceli et al., 1980), several patients with impaired sublexical perception were within the control range on tasks tapping lexical perception. In contrast, when the lexical and sublexical tasks (sublexical: syllable discrimination, auditory-written syllable matching (AWSM); lexical: word discrimination, lexical decision, and picture-word matching (PWM)) were matched on these factors (Experiments 1b and 2), in most instances, patients were impaired on both sublexical and lexical tasks relative to controls and performance on the lexical tasks was not significantly greater than that on the sublexical tasks. For two patients, performance on one lexical task was statistically better than that on one sublexical task, but the advantage was not replicated across other task comparisons. The current study is consistent with models of speech perception which assume obligatory sublexical processing and fails to support models that do not require successful sublexical perception in order to access lexical levels (e.g., Goldinger, 1998; Hickok and Poeppel, 2000).

Visual Cortical Entrainment to Motion and Categorical Speech Features during Silent Lipreading.

Speech is a multisensory percept, comprising an auditory and visual component. While the content and processing pathways of audio speech have been well characterized, the visual component is less well understood. In this work, we expand current methodologies using system identification to introduce a framework that facilitates the study of visual speech in its natural, continuous form. Specifically, we use models based on the unheard acoustic envelope (E), the motion signal (M) and categorical visual speech features (V) to predict EEG activity during silent lipreading. Our results show that each of these models performs similarly at predicting EEG in visual regions and that respective combinations of the individual models (EV, MV, EM and EMV) provide an improved prediction of the neural activity over their constituent models. In comparing these different combinations, we find that the model incorporating all three types of features (EMV) outperforms the individual models, as well as both the EV and MV models, while it performs similarly to the EM model. Importantly, EM does not outperform EV and MV, which, considering the higher dimensionality of the V model, suggests that more data is needed to clarify this finding. Nevertheless, the performance of EMV, and comparisons of the subject performances for the three individual models, provides further evidence to suggest that visual regions are involved in both low-level processing of stimulus dynamics and categorical speech perception. This framework may prove useful for investigating modality-specific processing of visual speech under naturalistic conditions.

Towards human-like and transhuman perception in AI 2.0: a review

Perception is the interaction interface between an intelligent system and the real world. Without sophisticated and flexible perceptual capabilities, it is impossible to create advanced artificial intelligence (AI) systems. For the next-generation AI, called ‘AI 2.0’, one of the most significant features will be that AI is empowered with intelligent perceptual capabilities, which can simulate human brain’s mechanisms and are likely to surpass human brain in terms of performance. In this paper, we briefly review the state-of-the-art advances across different areas of perception, including visual perception, auditory perception, speech perception, and perceptual information processing and learning engines. On this basis, we envision several RD (2) auditory perception and computation in an actual auditory setting; (3) speech perception and computation in a natural interaction setting; (4) autonomous learning of perceptual information; (5) large-scale perceptual information processing and learning platforms; and (6) urban omnidirectional intelligent perception and reasoning engines. We believe these research directions should be highlighted in the future plans for AI 2.0.

Psycholinguistic Approach to the Foreign Language Competence Formation of Future Seafarers

This article describes the study of psycholinguistic process of speech perception by future seafarers. The described study proves the role of psycholinguistic aspects in the process of forming foreign language competence of seafarers. Diagnostics of the level of perceptual skill development has been carried out and the way of its further correction in the process of English language teaching at higher educational institutions for maritime training areas have been worked out. Requirements for psycho-acoustic properties of the voices of the future seafarers are caused by the conditions of their life activity at sea and their future professional duties. Noise, vibration, bad weather require skills to wield the power of voice (psycho-acoustic dimensions). The need to speak in such conditions requires the ability to control the volume of voice when speaking. Based on the fact that the students of higher educational institutions of marine areas—the future officers, their speech should be clear and assertive. Psycho-acoustical aspect, in our opinion, precedes the development of speech perception skills and the ability to control the sound of speech. The results of diagnostics of the formed level of seafarers’ skills to assess the quality of their speech and their formed level of skills and abilities as a listener have prompted us to develop psycholinguistic exercises to improve speech perceptual skills of future seafarers that will enable them to improve professional communication and interpersonal communication at sea.

The Role of Categorical Speech Perception and Phonological Processing in Familial Risk Children With and Without Dyslexia.

This study assessed whether a categorical speech perception (CP) deficit is associated with dyslexia or familial risk for dyslexia, by exploring a possible cascading relation from speech perception to phonology to reading and by identifying whether speech perception distinguishes familial risk (FR) children with dyslexia (FRD) from those without dyslexia (FRND). Data were collected from 9-year-old FRD (n = 37) and FRND (n = 41) children and age-matched controls (n = 49) on CP identification and discrimination and on the phonological processing measures rapid automatized naming, phoneme awareness, and nonword repetition. The FRD group performed more poorly on CP than the FRND and control groups. Findings on phonological processing align with the literature in that (a) phonological processing related to reading and (b) the FRD group showed the lowest phonological processing outcomes. Furthermore, CP correlated weakly with reading, but this relationship was fully mediated by rapid automatized naming. Although CP phonological skills are related to dyslexia, there was no strong evidence for a cascade from CP to phonology to reading. Deficits in CP at the behavioral level are not directly associated with dyslexia.

Exploiting the Ganong effect to probe for phonetic uncertainty resulting from hearing loss

The “Ganong effect” is the tendency to perceive an ambiguous speech sound as a phoneme that would complete a real word, rather than completing a nonsense/fake word. For example, a sound that could be heard as either /g/ or /k/ is perceived as /g/ when followed by “ift” but perceived as /k/ when followed by “iss.” Because the target speech sound (/g/ or /k/) is the same across both environments, this effect demonstrates the influence of top-down processing rather than simply bottom-up processing in speech perception. We hypothesized that degradations in the auditory system (including simulations of hearing loss or cochlear implantation), speech stimuli will be rendered more ambiguous, and an increased Ganong effect should be observed. Participants heard three speech continua that varied by spectral cues of varying speeds, including fast (stop formant transitions), medium (fricative spectra), and slow (vowel formants). Stimuli were presented with clear spectral quality, or with varying amounts of spectral degradation using a noise vocoder. Responses were analyzed within stimulus pairs using binomial logistic regression, revealing increased Ganong effect with degraded speech. This test is proposed as a simple measure of how phonetic uncertainty can lead to an individual’s increased reliance on top-down processing.

Separation and integration of sound sources in auditory processing

Some perceptual experiments were conducted to elucidate sound source separation processing in speech perception. The phenomenon of phonemic restoration was tested if it occurs in dichotic listening. An original speech is Japanese /VCV/, the stimulus is made by manipulating the /VCV/ as was replaced the consonant part with noise. The duration of replacement was varied at six levels: 10, 20, 40, 60, 80, and 100 ms. There were two hearing modes: in one case, the monaural stimulus was presented to both the ear, and in the other case, the speech part of the stimulus was presented to an ear and the noise part to the opposite ear simultaneously. Thirty subjects in the age of twenties participate the listening test. The results show that phonemic restoration effect is clearly observed in both hearing modes. The rate of restoration is 92%, and 95% even in dichotic listening. There was no feature of speech in the noise has no relation with speech sound. Moreover, the direction of the noise sound was opposite to that of speech sound source, different in 180 degree. Even though the two sound signal are somehow integrated to hear restored speech.