Classes Of Speech Sounds Research Articles

Active Models in Speech Perception: A Critical Review in Phonetics

This critical review examines four prominent active models of speech perception: quantal theory, motor theory, hyper-hypo-articulation theory, and exemplar theory. Each of these models offers distinctive insights into how listeners perceive, interpret, and categorize speech sounds. Quantal theory explores the non-linear relationship between articulation and acoustic output, emphasizing stable and unstable regions in speech production. Motor theory posits that speech perception is grounded in articulatory gestures, suggesting a direct link between production and perception. The hyper-hypo-articulation theory focuses on the speaker’s adaptive articulation, which ranges from hyper- to hypo-articulation depending on the listener's needs and the communicative context. Exemplar theory, rooted in cognitive psychology, highlights the role of memory and experience, with individual speech sounds compared to stored exemplars in memory to aid perception. Despite their contributions, none of these models offers a comprehensive solution to all speech perception challenges. Each theory is suited to specific contexts, speech features, or tasks, making it crucial for researchers to understand their limitations. Furthermore, the review underscores the importance of selecting the appropriate model for particular research or practical applications, given the variability in their functionality. Finally, it calls for future research to delve deeper into passive models of speech perception and suggests that comparative studies between active and passive models could yield valuable insights.

Individual differences in the perception of phonetic category structure predict speech-in-noise performance.

Speech sounds exist in a complex acoustic-phonetic space, and listeners vary in the extent to which they are sensitive to variability within the speech sound category ("gradience") and the degree to which they show stable, consistent responses to phonetic stimuli. Here, we investigate the hypothesis that individual differences in the perception of the sound categories of one's language may aid speech-in-noise performance across the adult lifespan. Declines in speech-in-noise performance are well documented in healthy aging, and are, unsurprisingly, associated with differences in hearing ability. Nonetheless, hearing status and age are incomplete predictors of speech-in-noise performance, and long-standing research suggests that this ability draws on more complex cognitive and perceptual factors. In this study, a group of adults ranging in age from 18 to 67 years performed online assessments designed to measure phonetic category sensitivity, questionnaires querying recent noise exposure history and demographic factors, and crucially, a test of speech-in-noise perception. Results show that individual differences in the perception of two consonant contrasts significantly predict speech-in-noise performance, even after accounting for age and recent noise exposure history. This finding supports the hypothesis that individual differences in sensitivity to phonetic categories mediates speech perception in challenging listening situations.

When Jack isn't Jacques: Simultaneous opposite language-specific speech perceptual learning in French-English bilinguals.

Humans are remarkably good at understanding spoken language, despite the huge variability of the signal as a function of the talker, the situation, and the environment. This success relies on having access to stable representations based on years of speech input, coupled with the ability to adapt to short-term deviations from these norms, e.g. accented speech or speech altered by ambient noise. In the last two decades, there has been a robust research effort focused on a possible mechanism for adjusting to accented speech. In these studies, listeners typically hear 15 - 20 words in which a speech sound has been altered, creating a short-term deviation from its longer-term representation. After exposure to these items, listeners demonstrate "lexically driven phonetic recalibration"-they alter their categorization of speech sounds, expanding a speech category to take into account the recently heard deviations from their long-term representations. In the current study, we investigate such adjustments by bilingual listeners. French-English bilinguals were first exposed to nonstandard pronunciations of a sound (/s/ or /f/) in one language and tested for recalibration in both languages. Then, the exposure continued with both the original type of mispronunciation in the same language, plus mispronunciations in the other language, in the opposite direction. In a final test, we found simultaneous recalibration in opposite directions for the two languages-listeners shifted their French perception in one direction and their English in the other: Bilinguals can maintain separate adjustments, for the same sounds, when a talker's speech differs across two languages.

Speech Sound Categories Affect Lexical Competition: Implications for Analytic Auditory Training.

We provide a novel application of psycholinguistic theories and methods to the field of auditory training to provide preliminary data regarding which minimal pair contrasts are more difficult for listeners with typical hearing to distinguish in real-time. Using eye-tracking, participants heard a word and selected the corresponding image from a display of four: the target word, two unrelated words, and a word from one of four contrast categories (i.e., voiced-initial [e.g., peach-beach], voiced-final [e.g., back-bag], manner-initial [e.g., talk-sock], and manner-final [e.g., bat-bass]). Fixations were monitored to measure how strongly words compete for recognition depending on the contrast type (voicing, manner) and location (word-initial or final). Manner contrasts competed more for recognition than did voicing contrasts, and contrasts that occurred in word-final position were harder to distinguish than word-initial position. These results are an important initial step toward creating an evidence-based hierarchy for auditory training for individuals who use cochlear implants.

Age and category structure in phonetic category learning

When learning new speech sound categories in a second language, listeners must decide which sounds belong to the same category and which belong to different categories. This learning process is contingent on perceptual systems that change across the lifespan, which may, in turn, make certain categories easier to learn depending on which systems are employed by learners. In the present study, learners across a variety of ages categorize speech sounds taken from German that vary in their category structure (i.e., the complexity of the categories being learned within the stimulus space). Participants aged 7 to 70 are recruited and run at community sites such as museums and libraries. The preliminary dataset indicates that the simpler category structure saw strong age effects (with poorer performance on the younger and older ends of the age range and better performance at intermediate ages), while differences across ages were much smaller for the more complex category structure. These findings suggest that learning simpler category structures may rely on age-dependent learning systems, while learning more complex categories may rely on systems that are less age-dependent.

Listeners' convergence towards an artificial agent in a joint phoneme categorization task

This study focuses on inter-individual convergence effects in the perception and categorization of speech sounds. We ask to what extent two listeners can come to establish a shared set of categorization criteria in a phoneme identification task that they accomplish together. Several hypotheses are laid out in the framework of a Bayesian model of speech perception that we have developed to account for how two listeners may each infer the parameters that govern their partner’s responses. In our experimental paradigm, participants were asked to perform a joint phoneme identification task with a partner that, unbeknownst to them, was an artificial agent, whose responses we manipulated along two dimensions, the location of the categorical boundary and the slope of the identification function. Convergence was found to arise for bias but not for slope. Numerical simulations suggested that lack of convergence in slope may stem from the listeners’ prior level of confidence in the variance in VOT for the two phonemic categories. This study sheds new light on perceptual convergence between listeners in the categorization of speech sounds, a phenomenon that has received little attention so far in spite of its central importance for speech communication.

Linking lexically guided perceptual learning to statistical patterns in speech input

Listeners use lexical information to modify the mapping between speech acoustics and speech sound categories. Despite convention to consider lexically guided perceptual learning as a binary outcome, the magnitude of the learning effect varies in the extant literature. We hypothesize that graded learning outcomes can be linked, in part, to statistical characteristics of the to-be-learned input, consistent with the ideal adapter theory of speech adaptation. Following standard methods (i.e., waveform averaging to create ambiguous variants), a lexically guided perceptual learning stimulus set for the /ʃ/-/s/ contrast was created for each of 16 talkers, yielding variability in the statistical cues specifying this contrast across talkers. Experiment 1 will (a) measure lexically guided perceptual learning for each talker, (b) identify input characteristics that are associated with learning magnitude, and (c) examine whether a computational instantiation of the ideal adapter theory can model the input-learning link. Experiment 2 will provide a confirmatory test of the patterns observed in Experiment 1 by manipulating the to-be-learned input holding talker constant. The results will provide a critical test of the ideal adapter framework for speech adaptation, thus informing an understanding of the mechanisms that allow listeners to solve the lack of invariance problem for speech perception.

Nonverbal Intelligence Does Matter for the Perception of Second Language Sounds

Although there has been considerable research on the interplay between intelligence and second language (L2) learning, research focusing on the intelligence and L2 speech perception link is limited. The present study aims to fill this gap. The native language of the participants was Cypriot Greek and they spoke English as an L2. The participants completed a forced-choice psychoacoustic test in which they discriminated L2 sound contrasts and a nonverbal intelligence test which measured their nonverbal intelligence capacities. They were divided into two groups according to their performance in the intelligence test, namely, a low IQ and a high IQ group. The results showed that the high IQ group discriminated the majority of the L2 contrasts better than the low IQ group. In addition, the degree of perceived difficulty for most L2 contrasts differed between the two groups. It is concluded that nonverbal intelligence is associated with the discrimination of L2 sounds. This can be explained by the possibility that either intelligence triggers the more efficient functioning of other domains, such as information processing and attention, leading to increased speech perception skills, or that it directly affects the categorization of speech sounds resulting in the development of more robust L2 categories.

Flexibility and stability of speech sounds: The time course of lexically-driven recalibration

Perceptual stability is obviously advantageous, but being able to adjust to the prevailing environment is also adaptive. Previous research has identified ways in which the categorization of speech sounds shifts as a function of recently heard speech. Dozens of studies have examined “lexically driven recalibration”, an adjustment to categorization after listeners hear a number of words with a particular speech sound designed to be perceptually ambiguous. Despite the large number of these studies, little is known about how long the adjustment endures. Using two different stimulus sets, we assess the recovery time after lexically driven recalibration. In addition, we examine whether the size of the recalibration effect diminishes during the identification test used to measure it, and whether the recalibration effect is stronger for one side of a tested contrast or the other. The effect did in fact decline during its measurement, and one side of the contrast (/s/) produced stronger shifts than others (/ʃ/ or /θ/) under the conditions typically examined in recalibration studies. Recalibration was quite robust after 24 hours for both stimulus sets, and still measurable after one week for one of them. This time course is strikingly different than the recovery times reported in previous studies for two other adjustment processes – selective adaptation and audiovisually driven recalibration. The vastly different time courses pose a major challenge for models that ascribe these phenomena to the same adjustment function. Thus, such models will need to be substantially modified, or alternative models will need to be developed.

The nature of non-native speech sound representations.

Most current theories and models of second language speech perception are grounded in the notion that learners acquire speech sound categories in their target language. In this paper, this classic idea in speech perception is revisited, given that clear evidence for formation of such categories is lacking in previous research. To understand the debate on the nature of speech sound representations in a second language, an operational definition of "category" is presented, and the issues of categorical perception and current theories of second language learning are reviewed. Following this, behavioral and neuroimaging evidence for and against acquisition of categorical representations is described. Finally, recommendations for future work are discussed. The paper concludes with a recommendation for integration of behavioral and neuroimaging work and theory in this area.

Using frequency selectivity to examine category-informative dimension-selective attention

In everyday situations, listeners may selectively attend to acoustic dimensions (e.g., frequency) within complex sounds and ignore other simultaneous dimensions. Speech sound categories are defined over multiple dimensions that vary in informativeness; thus, speech perception may demand selective attention to diagnostic dimensions. Here, we examine how selective attention affects cortical representations of informative acoustic dimensions during category learning. Participants completed a five-day training regimen wherein they learned four novel non-speech categories to criterion (Obasih et al., in prep). Crucially, stimuli were designed such that for two categories, categorization required reliance on – and perhaps selective attention to – acoustic patterns within high-frequency bands; the other two categories were differentiated by acoustic patterns in low-frequency bands. After training, listeners completed an fMRI session using the same non-speech categorization task. Best frequency for each auditory cortex voxel was obtained using tonotopic mapping. We hypothesized that when categorization depended on distinctions in certain frequency ranges (e.g., high frequencies), listeners would selectively attend to those frequencies, resulting in greater recruitment of voxels preferring those frequencies and possibly attenuation of signal in voxels preferring other frequencies. We present preliminary data from our study, with the goal of clarifying the cortical mechanisms supporting dimension-based auditory selective attention.

Auditory perceptual learning in autistic adults.

The automatic retuning of phoneme categories to better adapt to the speech of a novel talker has been extensively documented across various (neurotypical) populations, including both adults and children. However, no studies have examined auditory perceptual learning effects in populations atypical in perceptual, social, and language processing for communication, such as populations with autism. Employing a classic lexically-guided perceptual learning paradigm, the present study investigated perceptual learning effects in Australian English autistic and non-autistic adults. The findings revealed that automatic attunement to existing phoneme categories was not activated in the autistic group in the same manner as for non-autistic control subjects. Specifically, autistic adults were able to both successfully discern lexical items and to categorize speech sounds; however, they did not show effects of perceptual retuning to talkers. These findings may have implications for the application of current sensory theories (e.g., Bayesian decision theory) to speech and language processing by autistic individuals. LAY SUMMARY: Lexically guided perceptual learning assists in the disambiguation of speech from a novel talker. The present study established that while Australian English autistic adult listeners were able to successfully discern lexical items and categorize speech sounds in their native language, perceptual flexibility in updating speaker-specific phonemic knowledge when exposed to a novel talker was not available. Implications for speech and language processing by autistic individuals as well as current sensory theories are discussed.

Speech sound categorization: The contribution of non-auditory and auditory cortical regions

Which processes in the human brain lead to the categorical perception of speech sounds? Investigation of this question is hampered by the fact that categorical speech perception is normally confounded by acoustic differences in the stimulus. By using ambiguous sounds, however, it is possible to dissociate acoustic from perceptual stimulus representations. Twenty-seven normally hearing individuals took part in an fMRI study in which they were presented with an ambiguous syllable (intermediate between /da/ and /ga/) in one ear and with disambiguating acoustic feature (third formant, F3) in the other ear. Multi-voxel pattern searchlight analysis was used to identify brain areas that consistently differentiated between response patterns associated with different syllable reports. By comparing responses to different stimuli with identical syllable reports and identical stimuli with different syllable reports, we disambiguated whether these regions primarily differentiated the acoustics of the stimuli or the syllable report. We found that BOLD activity patterns in left perisylvian regions (STG, SMG), left inferior frontal regions (vMC, IFG, AI), left supplementary motor cortex (SMA/pre-SMA), and right motor and somatosensory regions (M1/S1) represent listeners’ syllable report irrespective of stimulus acoustics. Most of these regions are outside of what is traditionally regarded as auditory or phonological processing areas. Our results indicate that the process of speech sound categorization implicates decision-making mechanisms and auditory-motor transformations.

A protracted developmental trajectory for English-learning children’s detection of consonant mispronunciations in newly learned words

ABSTRACT Children are adept at learning their language’s speech-sound categories, but just how these categories function in their developing lexicon has not been mapped out in detail. Here, we addressed whether, in a language-guided looking procedure, 2-year-olds would respond to a mispronunciation of the voicing of the initial consonant of a newly learned word. First, to provide a baseline of mature native-speaker performance, adults were taught a new word under training conditions of low prosodic variability. In a second experiment, 24- and 30-month-olds were taught a new word under training conditions of high or low prosodic variability. Children and adults showed evidence of learning the taught word. Adults’ target looking was reduced when the novel word was realized at test with a change in the voicing of the initial consonant, but children did not show any such decrement in target fixation. For both children and adults, most learners did not treat the phonologically distinct variant as a different word. Acoustic-phonetic variability during teaching did not have consistent effects. Thus, under conditions of intensive short-term training, 24- and 30-month-olds did not differentiate a newly learned word from a variant differing only in consonant voicing. High task complexity during training could explain why mispronunciation detection was weaker here than in some prior studies.

Sleep affects higher-level categorization of speech sounds, but not frequency encoding

Sleep can increase consolidation of new knowledge and skills. It is less clear whether sleep plays a role in other aspects of experience-dependent neuroplasticity, which underlie important human capabilities such as spoken language processing. Theories of sensory learning differ in their predictions; some imply rapid learning at early sensory levels, while other propose a slow, progressive timecourse such that higher-level categorical representations guide immediate, novice learning, while lower-level sensory changes do not emerge until later stages. In this study, we investigated the role of sleep across both behavioural and physiological indices of auditory neuroplasticity. Forty healthy young human adults (23 female) who did not speak a tonal language participated in the study. They learned to categorize non-native Mandarin lexical tones using a sound-to-category training paradigm, and were then randomly assigned to a Nap or Wake condition. Polysomnographic data were recorded to quantify sleep during a 3 h afternoon nap opportunity, or equivalent period of quiet wakeful activity. Measures of behavioural performance accuracy revealed a significant improvement in learning the sound-to-category training paradigm between Nap and Wake groups. Conversely, a neural index of fine sound encoding fidelity of speech sounds known as the frequency-following response (FFR) suggested no change due to sleep, and a null model was supported, using Bayesian statistics. Together, these results support theories that propose a slow, progressive and hierarchical timecourse for sensory learning. Sleep's effect may play the biggest role in the higher-level learning, although contributions to more protracted processes of plasticity that exceed the study duration cannot be ruled out.

Noise helps cochlear implant listeners to categorize vowels.

Theoretical studies demonstrate that controlled addition of noise can enhance the amount of information transmitted by a cochlear implant (CI). The present study is a proof-of-principle for whether stochastic facilitation can improve the ability of CI users to categorize speech sounds. Analogue vowels were presented to CI users through a single electrode with independent noise on multiple electrodes. Noise improved vowel categorization, particularly in terms of an increase in information conveyed by the first and second formant. Noise, however, did not significantly improve vowel recognition: the miscategorizations were just more consistent, giving the potential to improve with experience.

Later but Not Weaker: Neural Categorization of Native Vowels of Children at Familial Risk of Dyslexia

Although allophonic speech processing has been hypothesized to be a contributing factor in developmental dyslexia, experimental evidence is limited and inconsistent. The current study compared the categorization of native similar sounding vowels of typically developing (TD) children and children at familial risk (FR) of dyslexia. EEG response was collected in a non-attentive passive oddball paradigm from 35 TD and 35 FR Dutch 20-month-old infants who were matched on vocabulary. The children were presented with two nonwords “giep” [ɣip] and “gip” [ɣIp] that contrasted solely with respect to the vowel. In the multiple-speaker condition, both nonwords were produced by twelve different speakers while in the single-speaker condition, single tokens of each word were used as stimuli. For both conditions and for both groups, infant positive mismatch response (p-MMR) was elicited, and the p-MMR amplitude was comparable between the two groups, although the FR children had a later p-MMR peak than the TD children in the multiple-speaker condition. These findings indicate that FR children are able to categorize speech sounds, but that they may do so in a more effortful way than TDs.

Arousal States as a Key Source of Variability in Speech Perception and Learning

The human brain exhibits the remarkable ability to categorize speech sounds into distinct, meaningful percepts, even in challenging tasks like learning non-native speech categories in adulthood and hearing speech in noisy listening conditions. In these scenarios, there is substantial variability in perception and behavior, both across individual listeners and individual trials. While there has been extensive work characterizing stimulus-related and contextual factors that contribute to variability, recent advances in neuroscience are beginning to shed light on another potential source of variability that has not been explored in speech processing. Specifically, there are task-independent, moment-to-moment variations in neural activity in broadly-distributed cortical and subcortical networks that affect how a stimulus is perceived on a trial-by-trial basis. In this review, we discuss factors that affect speech sound learning and moment-to-moment variability in perception, particularly arousal states—neurotransmitter-dependent modulations of cortical activity. We propose that a more complete model of speech perception and learning should incorporate subcortically-mediated arousal states that alter behavior in ways that are distinct from, yet complementary to, top-down cognitive modulations. Finally, we discuss a novel neuromodulation technique, transcutaneous auricular vagus nerve stimulation (taVNS), which is particularly well-suited to investigating causal relationships between arousal mechanisms and performance in a variety of perceptual tasks. Together, these approaches provide novel testable hypotheses for explaining variability in classically challenging tasks, including non-native speech sound learning.

Overnight changes to dual-memory processes reflected in speech-perceptual performance.

Adults' ability to attain and retain nonnative speech sound categories vary substantially among individuals. While we know that speech-perceptual skills play a role, we know less about how consolidation-related changes in acoustic-phonetic memory contribute to perceptual tasks. The goal of this investigation was to examine contributions of memory and perceptual skills to the perceptual performance on a trained nonnative speech contrast over two days. Twenty-one adult participants (ages 18-24) completed four different experiments. Three of these assessed learning and memory: visual statistical learning (implicit), visual object recognition (explicit), and nonnative (Hindi dental-retroflex) speech-sound training. Participants completed the learning tasks around 8 p.m., and performance was measured shortly after learning and again 12 hours later. On a separate day, participants completed a categorical perception task on a native (/a/-/e/) vowel continuum. Nonnative speech perception was associated with implicit learning performance when both were assessed shortly after learning, and associated with the retention of explicit memory when both were assessed after an overnight delay. Native speech-sounds were at least marginally associated with nonnative speech perception performance on both days, but with a stronger association observed with performance assessed on Day 2. These findings provide preliminary support for the interpretation that speech-sounds are encoded by at least two memory systems in parallel, but that perceptual performance may reflect acoustic-phonetic knowledge learned by different memory systems over time since exposure. Moreover, performance on speech perception tasks in both native and nonnative speech-sounds may rely on similar retrieval mechanisms for long-term storage of speech-sound information.

Analysis and classification of speech sounds of children with autism spectrum disorder using acoustic features

Children with autism spectrum disorder (ASD) produce speech sounds different from that of Normal or non-ASD children. Hence, analyzing acoustic features can help characterizing the ASD speech signals. In this study, the distinguishing characteristics of speech production are examined for ASD affected children, with comparison to Normal children’s speech. Acoustic features are analyzed first and then classification of ASD vs Normal speech is attempted using different machine learning techniques. Two speech sound databases are recorded for this study: the speech database of children with ASD and the speech database of Normal children. English speech utterances are recorded for children of Indian regional (Tamil and Telugu) nativity. The changes due to autism effect are examined in context of 5 English vowel sounds (/a/, /e/, /i/, /o/, and /u/). Changes in the speech production characteristics of children are explored using three sets of features. Firstly, changes in the excitation source features are examined using strength of excitation (SoE) and instantaneous fundamental frequency (F0). Secondly, changes in the vocal tract (VT) filter features are examined using dominant frequencies (FD1, FD2) and formant frequencies (F1 to F5). Thirdly, changes in the source-filter combined features are examined using signal energy (E), zero-crossing rate (ZCR), linear prediction cepstrum coefficients (LPCC), and Mel-frequency cepstral coefficients (MFCC). Then, various combinations of the acoustic features are classified utilizing machine learning methods such as probabilistic neural network (PNN), multilayer perceptron (MLP), support vector machine (SVM), and K-nearest neighbors (KNN). Analyses of acoustic features shows significant differences between the speech of children with ASD and the Normal children. Results up to 98.17% accuracy are obtained for classification between acoustic features of the speech sounds of children with ASD and the Normal children. The observations and this study results may be useful as acoustic biomarkers to identify autism and its progression/cure among children. This study may also be valuable towards developing a system for ASD diagnosis from children’s speech sounds, in the future.