Phoneme Onset Research Articles

Phonetic categorization in phonological lexical neighborhoods: Facilitatory and inhibitory effects

Phonetic processing, whereby the bottom-up speech signal is translated into higher-level phonological representations such as phonemes, has been demonstrated to be influenced by phonological lexical neighborhoods. Previous studies show facilitatory effects of lexicality and phonological neighborhood density on phonetic categorization. However, given the evidence for lexical competition in spoken word recognition, we hypothesize that there are concurrent facilitatory and inhibitory effects of phonological lexical neighborhoods on phonetic processing. In Experiments 1 and 2, participants categorized the onset phoneme in word-nonword and nonword-word acoustic continua. The results show that the target word of the continuum exhibits facilitatory lexical influences whereas rhyme neighbors inhibit phonetic categorization. The results support the hypothesis that sublexical phonetic processing is affected by multiple facilitatory and inhibitory lexical forces in the processing stream.

Objective speech intelligibility prediction using a deep learning model with continuous speech-evoked cortical auditory responses.

Auditory prostheses provide an opportunity for rehabilitation of hearing-impaired patients. Speech intelligibility can be used to estimate the extent to which the auditory prosthesis improves the user’s speech comprehension. Although behavior-based speech intelligibility is the gold standard, precise evaluation is limited due to its subjectiveness. Here, we used a convolutional neural network to predict speech intelligibility from electroencephalography (EEG). Sixty-four–channel EEGs were recorded from 87 adult participants with normal hearing. Sentences spectrally degraded by a 2-, 3-, 4-, 5-, and 8-channel vocoder were used to set relatively low speech intelligibility conditions. A Korean sentence recognition test was used. The speech intelligibility scores were divided into 41 discrete levels ranging from 0 to 100%, with a step of 2.5%. Three scores, namely 30.0, 37.5, and 40.0%, were not collected. The speech features, i.e., the speech temporal envelope (ENV) and phoneme (PH) onset, were used to extract continuous-speech EEGs for speech intelligibility prediction. The deep learning model was trained by a dataset of event-related potentials (ERP), correlation coefficients between the ERPs and ENVs, between the ERPs and PH onset, or between ERPs and the product of the multiplication of PH and ENV (PHENV). The speech intelligibility prediction accuracies were 97.33% (ERP), 99.42% (ENV), 99.55% (PH), and 99.91% (PHENV). The models were interpreted using the occlusion sensitivity approach. While the ENV models’ informative electrodes were located in the occipital area, the informative electrodes of the phoneme models, i.e., PH and PHENV, were based on the occlusion sensitivity map located in the language processing area. Of the models tested, the PHENV model obtained the best speech intelligibility prediction accuracy. This model may promote clinical prediction of speech intelligibility with a comfort speech intelligibility test.

Early Auditory Event-Related Potentials Are Modulated by Alphabetic Literacy Skills in Logographic Chinese Readers.

The acquisition of an alphabetic orthography transforms speech processing in the human brain. Behavioral evidence shows that phonological awareness as assessed by meta-phonological tasks like phoneme judgment, is enhanced by alphabetic literacy acquisition. The current study investigates the time-course of the neuro-cognitive operations underlying this enhancement as revealed by event-related potentials (ERPs). Chinese readers with and without proficiency in Jyutping, a Romanization system of Cantonese, were recruited for an auditory onset phoneme judgment task; their behavioral responses and the elicited ERPs were examined. Proficient readers of Jyutping achieved higher response accuracy and exhibited more negative-going ERPs in three early ERP time-windows corresponding to the P1, N1, and P2 components. The phonological mismatch negativity component exhibited sensitivity to both onset and rhyme mismatch in the speech stimuli, but it was not modulated by alphabetic literacy skills. The sustained negativity in the P1-N1-P2 time-windows is interpreted as reflecting enhanced phonetic/phonological processing or attentional/awareness modulation associated with alphabetic literacy and phonological awareness skills.

Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus

Objective. To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the ‘hand knob’ area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. Approach. Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode’s binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the ‘Brain-to-Speech’ pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes’ onset times. Main results. A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio. Significance. The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

Differences in phonological awareness performance

Abstract Children who have knowledge of two languages may show better phonological awareness than their monolingual peers (e.g. Bruck & Genesee, 1995). It remains unclear how much bilingual experience is needed for such advantages to appear, and whether differences in language or cognitive skills alter the relation between bilingualism and phonological awareness. These questions were investigated in this cross-sectional study. Participants (n = 294; 4–7 year-olds, in the first three grades of primary school) were Dutch-speaking pupils attending mainstream monolingual Dutch primary schools or early-English schools providing English lessons from grade 1, and simultaneous Dutch-English bilinguals. We investigated phonological awareness (rhyming, phoneme blending, onset phoneme identification, and phoneme deletion) and its relation to age, Dutch vocabulary, English vocabulary, working memory and short-term memory, and the balance between Dutch and English vocabulary. Small significant (α < .05) effects of bilingualism were found on onset phoneme identification and phoneme deletion, but post-hoc comparisons revealed no robust pairwise differences between the groups. Furthermore, effects of bilingualism sometimes disappeared when differences in language or memory skills were taken into account. Learning two languages simultaneously is not beneficial to – and importantly, also not detrimental to – phonological awareness.

Task-free auditory EEG paradigm for probing multiple levels of speech processing in the brain.

While previous studies on language processing highlighted several ERP components in relation to specific stages of sound and speech processing, no study has yet combined them to obtain a comprehensive picture of language abilities in a single session. Here, we propose a novel task-free paradigm aimed at assessing multiple levels of speech processing by combining various speech and nonspeech sounds in an adaptation of a multifeature passive oddball design. We recorded EEG in healthy adult participants, who were presented with these sounds in the absence of sound-directed attention while being engaged in a primary visual task. This produced a range of responses indexing various levels of sound processing and language comprehension: (a) P1-N1 complex, indexing obligatory auditory processing; (b) P3-like dynamics associated with involuntary attention allocation for unusual sounds; (c) enhanced responses for native speech (as opposed to nonnative phonemes) from ∼50 ms from phoneme onset, indicating phonological processing; (d) amplitude advantage for familiar real words as opposed to meaningless pseudowords, indexing automatic lexical access; (e) topographic distribution differences in the cortical activation of action verbs versus concrete nouns, likely linked with the processing of lexical semantics. These multiple indices of speech-sound processing were acquired in a single attention-free setup that does not require any task or subject cooperation; subject to future research, the present protocol may potentially be developed into a useful tool for assessing the status of auditory and linguistic functions in uncooperative or unresponsive participants, including a range of clinical or developmental populations.

In Spoken Word Recognition, the Future Predicts the Past.

Speech is an inherently noisy and ambiguous signal. To fluently derive meaning, a listener must integrate contextual information to guide interpretations of the sensory input. Although many studies have demonstrated the influence of prior context on speech perception, the neural mechanisms supporting the integration of subsequent context remain unknown. Using MEG to record from human auditory cortex, we analyzed responses to spoken words with a varyingly ambiguous onset phoneme, the identity of which is later disambiguated at the lexical uniqueness point. Fifty participants (both male and female) were recruited across two MEG experiments. Our findings suggest that primary auditory cortex is sensitive to phonological ambiguity very early during processing at just 50 ms after onset. Subphonemic detail is preserved in auditory cortex over long timescales and re-evoked at subsequent phoneme positions. Commitments to phonological categories occur in parallel, resolving on the shorter timescale of ∼450 ms. These findings provide evidence that future input determines the perception of earlier speech sounds by maintaining sensory features until they can be integrated with top-down lexical information.SIGNIFICANCE STATEMENT The perception of a speech sound is determined by its surrounding context in the form of words, sentences, and other speech sounds. Often, such contextual information becomes available later than the sensory input. The present study is the first to unveil how the brain uses this subsequent information to aid speech comprehension. Concretely, we found that the auditory system actively maintains the acoustic signal in auditory cortex while concurrently making guesses about the identity of the words being said. Such a processing strategy allows the content of the message to be accessed quickly while also permitting reanalysis of the acoustic signal to minimize parsing mistakes.

Spoken-word production in Korean: A non-word masked priming and phonological Stroop task investigation.

Speech production studies have shown that phonological unit initially used to fill the metrical frame during phonological encoding is language specific, that is, a phoneme for English and Dutch, an atonal syllable for Mandarin Chinese, and a mora for Japanese. However, only a few studies chronometrically investigated speech production in Korean, and they obtained mixed results. Korean is particularly interesting as there might be both phonemic and syllabic influences during phonological encoding. The purpose of this study is to further examine the initial phonological preparation unit in Korean, employing a masked priming task (Experiment 1) and a phonological Stroop task (Experiment 2). The results showed that significant onset (and onset-plus, that is, consonant-vowel [CV]) effects were found in both experiments, but there was no compelling evidence for a prominent role for the syllable. When the prime words were presented in three different forms related to the targets, namely, without any change, with re-syllabified codas, and with nasalised codas, there were no significant differences in facilitation among the three forms. Alternatively, it is also possible that participants may not have had sufficient time to process the primes up to the point that re-syllabification or nasalisation could have been carried out. In addition, the results of a Stroop task demonstrated that the onset phoneme effect was not driven by any orthographic influence. These findings suggest that the onset segment and not the syllable is the initial (or proximate) phonological unit used in the segment-to-frame encoding process during speech planning in Korean.

The influence of orthographic experience on the development of phonological preparation in spoken word production.

Three sets of experiments using the picture naming tasks with the form preparation paradigm investigated the influence of orthographic experience on the development of phonological preparation unit in spoken word production in native Mandarin-speaking children. Participants included kindergarten children who have not received formal literacy instruction, Grade 1 children who are comparatively more exposed to the alphabetic pinyin system and have very limited Chinese character knowledge, Grades 2 and 4 children who have better character knowledge and more exposure to characters, and skilled adult readers who have the most advanced character knowledge and most exposure to characters. Only Grade 1 children showed the form preparation effect in the same initial consonant condition (i.e., when a list of target words shared the initial consonant). Both Grade 4 children and adults showed the preparation effect when the initial syllable (but not tone) among target words was shared. Kindergartners and Grade 2 children only showed the preparation effect when the initial syllable including tonal information was shared. These developmental changes in phonological preparation could be interpreted as a joint function of the modification of phonological representation and attentional shift. Extensive pinyin experience encourages speakers to attend to and select onset phoneme in phonological preparation, whereas extensive character experience encourages speakers to prepare spoken words in syllables.

Dynamic Encoding of Acoustic Features in Neural Responses to Continuous Speech.

Humans are unique in their ability to communicate using spoken language. However, it remains unclear how the speech signal is transformed and represented in the brain at different stages of the auditory pathway. In this study, we characterized electroencephalography responses to continuous speech by obtaining the time-locked responses to phoneme instances (phoneme-related potential). We showed that responses to different phoneme categories are organized by phonetic features. We found that each instance of a phoneme in continuous speech produces multiple distinguishable neural responses occurring as early as 50 ms and as late as 400 ms after the phoneme onset. Comparing the patterns of phoneme similarity in the neural responses and the acoustic signals confirms a repetitive appearance of acoustic distinctions of phonemes in the neural data. Analysis of the phonetic and speaker information in neural activations revealed that different time intervals jointly encode the acoustic similarity of both phonetic and speaker categories. These findings provide evidence for a dynamic neural transformation of low-level speech features as they propagate along the auditory pathway, and form an empirical framework to study the representational changes in learning, attention, and speech disorders.SIGNIFICANCE STATEMENT We characterized the properties of evoked neural responses to phoneme instances in continuous speech. We show that each instance of a phoneme in continuous speech produces several observable neural responses at different times occurring as early as 50 ms and as late as 400 ms after the phoneme onset. Each temporal event explicitly encodes the acoustic similarity of phonemes, and linguistic and nonlinguistic information are best represented at different time intervals. Finally, we show a joint encoding of phonetic and speaker information, where the neural representation of speakers is dependent on phoneme category. These findings provide compelling new evidence for dynamic processing of speech sounds in the auditory pathway.

Facilitation and interference of phoneme repetition and phoneme similarity in speech production

ABSTRACTSpeakers know that it is hard to produce tongue-twister-like utterances involving repeated and similar phonemes (“She sells socks in a small shoe shop”). While utterances involving phonologically similar phonemes consistently give rise to speech errors, repeated phonemes have been shown to interfere as well as to facilitate production. Here we try to shed light on this paradox by investigating brain processes underlying planning utterances involving similar and repeated phonemes (tongue-twister-like) and utterances with only repeated phonemes elicited with picture naming during electroencephalographic/event-related potentials (EEG/ERP) recording. Tongue-twister-like utterances increased error rates while repeated onset phonemes decreased reaction times. Both types of utterances modulated ERPs after 370 ms following picture onset by shortening a period of stable electrophysiological pattern. Another microstate was lengthened for tongue-twister-like utterances only, wiping out the timing difference of the previous microstate. These results indicate priming of repeated phonemes and cost of encoding similar phonemes, both phenomena co-occurring in tongue-twisters.

Dynamic representation of acoustic features in neural responses to continuous speech

Humans are unique in their ability to communicate using spoken language. However, it remains unclear how distinct acoustic features of speech sounds are represented in the auditory pathway over time. In this study, we applied a novel analysis technique to electroencephalography (EEG) signals as subjects listened to continuous speech and characterize the neural representation of acoustic features and the progression of responses over time. We averaged the time-aligned neural responses to phoneme instances and calculated a phoneme-related potential (PRP). We show that phonemes in continuous speech evoke multiple observable responses which are clearly separated in time. These recurrent responses have different scalp distributions, and occur as early as 50 ms, and as late as 400 ms after the phoneme onset. We show that the responses explicitly represent the acoustic distinctions of phonemes, and that linguistic and non-linguistic information appear at different time intervals. Finally, we show a joint encoding of phonetic and speaker information, where the neural representation of speakers is dependent on phoneme category. This study provides evidence for a dynamic neural transformation of low-level speech features and form an empirical framework to study the representational changes in learning, attention, and speech disorders.

The proximate unit in the production of Korean words

An important issue in recent psycholinguistic research is whether the first selectable phonological units in spoken-word production vary across languages. Data from many Indo-European languages demonstrate that word-onset phonemes are first selected in production planning, whereas whole syllables and CV morae are selected in native Chinese and Japanese, respectively. To empirically test the languagespecific view of proximate units, we investigate the roles of syllables and onset phonemes as proximate units in Korean, using a picture-naming task (Li et al. 2015). This is based on the fact that both syllables and phonemes are salient units in Korean phonological and orthographic structures. A significant priming effect was found for syllables, and a marginally significant effect for onset segments. These results indicate that in Korean, syllables function as proximate units in spoken-word production planning, but onset phonemes also show facilitation effects, albeit less robust ones than syllables. Our findings suggest that the functional phonological unit in spoken-word production is language dependent.

Effects of concurrent task demands on language planning in fluent children and adults.

The aim of the present study was to investigate how children and adults allocate cognitive resources to performing segmental encoding and monitoring in a dual task paradigm and the response patterns of the primary and secondary tasks in the dual task. Participants were 20 children divided equally into two age groups-7 to 11 years, 12 to 15 years, and 10 adults. The primary task required participants to monitor phonemic segments in a picture - written word interference paradigm while silently naming the pictures. The picture and distractor word were the same (replica), related (phoneme onset overlap), or unrelated. The secondary task required participants to make pitch judgments on tones presented at short (330 ms) or long (1130 ms) stimulus onset asynchrony (SOA) from picture onset. Developmental differences were observed in both response times and percent errors in the primary and secondary tasks. Slower responses to the primary task were evident at the long SOA, related condition. Slower response times to the tone decision task were evident at the short than the long SOA. The findings support the capacity sharing account of dual task performance and suggest that dual task costs during language planning are higher in children than adults.

Top-down influences in perception with spectrally degraded resynthesized natural speech

Lexical processes can influence perception of ambiguous phonemes (e.g., Ganong, 1980). To date, research has focused on these influences in quiet conditions with stimuli that have not been degraded. This project examines the interplay between lexical and acoustic information in speech perception, with stimuli in non-degraded and spectrally degraded conditions. Continua of /t, k/ onsets were developed by wavelet resynthesis of natural speech using TandemSTRAIGHT software, concatenated to the vowel × coda portion of words/nonwords, and distorted using AngelSim (TigerCIS) vocoding software. Normal hearing adult participants identified the onset phonemes in non-degraded and spectrally degraded conditions. Identification functions and the effects of lexical status in phoneme perception with spectrally degraded stimuli will be discussed.

Phoneme-free prosodic representations are involved in pre-lexical and lexical neurobiological mechanisms underlying spoken word processing

Recently we reported that spoken stressed and unstressed primes differently modulate Event Related Potentials (ERPs) of spoken initially stressed targets. ERP stress priming was independent of prime–target phoneme overlap. Here we test whether phoneme-free ERP stress priming involves the lexicon. We used German target words with the same onset phonemes but different onset stress, such as MANdel (“almond”) and manDAT (“mandate”; capital letters indicate stress). First syllables of those words served as primes. We orthogonally varied prime–target overlap in stress and phonemes. ERP stress priming did neither interact with phoneme priming nor with the stress pattern of the targets. However, polarity of ERP stress priming was reversed to that previously obtained. The present results are evidence for phoneme-free prosodic processing at the lexical level. Together with the previous results they reveal that phoneme-free prosodic representations at the pre-lexical and lexical level are recruited by neurobiological spoken word recognition.

The locus of the masked onset priming effect

This study investigates the masked onset priming effect (MOPE) in Korean. The results revealed facilitated naming for nonwords written in the alphabetic syllabary hangul when primes and targets shared an initial consonant-vowel (CV) syllable as well as when they shared only an initial onset (C) phoneme. However, there was greater priming at the syllable level than at the phoneme level. Taken together with previous research on Korean (Kim & Davis, 2002), these findings indicate that the MOPE reflects facilitated grapheme-to-phoneme conversion processes when the pronunciation of the target is computed through the nonlexical route, and are interpreted to support the Forster and Davis (1991) model of this effect.

Changing only the probability that spoken words will be distorted changes how they are recognized

An eye-tracking experiment examined contextual flexibility in speech processing in response to distortions in spoken input. Dutch participants heard Dutch sentences containing critical words and saw four-picture displays. The name of one picture either had the same onset phonemes as the critical word or had a different first phoneme and rhymed. Participants fixated on onset-overlap more than rhyme-overlap pictures, but this tendency varied with speech quality. Relative to a baseline with noise-free sentences, participants looked less at onset-overlap and more at rhyme-overlap pictures when phonemes in the sentences (but not in the critical words) were replaced by noises like those heard on a badly tuned AM radio. The position of the noises (word-initial or word-medial) had no effect. Noises elsewhere in the sentences apparently made evidence about the critical word less reliable: Listeners became less confident of having heard the onset-overlap name but also less sure of having not heard the rhyme-overlap name. The same acoustic information has different effects on spoken-word recognition as the probability of distortion changes.

Electroacoustic Comparison of Hearing Aid Output of Phonemes in Running Speech versus Isolation: Implications for Aided Cortical Auditory Evoked Potentials Testing

Background. Functioning of nonlinear hearing aids varies with characteristics of input stimuli. In the past decade, aided speech evoked cortical auditory evoked potentials (CAEPs) have been proposed for validation of hearing aid fittings. However, unlike in running speech, phonemes presented as stimuli during CAEP testing are preceded by silent intervals of over one second. Hence, the present study aimed to compare if hearing aids process phonemes similarly in running speech and in CAEP testing contexts. Method. A sample of ten hearing aids was used. Overall phoneme level and phoneme onset level of eight phonemes in both contexts were compared at three input levels representing conversational speech levels. Results. Differences of over 3 dB between the two contexts were noted in one-fourth of the observations measuring overall phoneme levels and in one-third of the observations measuring phoneme onset level. In a majority of these differences, output levels of phonemes were higher in the running speech context. These differences varied across hearing aids. Conclusion. Lower output levels in the isolation context may have implications for calibration and estimation of audibility based on CAEPs. The variability across hearing aids observed could make it challenging to predict differences on an individual basis.

Pseudohomophone priming in lexical decision is not fragile in a sparse lexical neighborhood.

In lexical decision, to date few studies in English have found a reliable pseudohomophone priming advantage with orthographically similar primes (the klip-plip effect; Frost, Ahissar, Gotesman, & Tayeb, 2003; see Rastle & Brysbaert, 2006, for a review). On the basis of the Bayseian reader model of lexical decision (Norris, 2006, 2009), we hypothesized that this was because in previous studies, lexical decisions could have been made without finding a match between the input and a unique lexical representation. Consistent with our hypothesis, we found that words from dense neighborhoods showed neither an orthographic form priming effect nor a pseudohomophone priming advantage; in contrast, with words from a sparse lexical neighborhood, a sizeable orthographic form priming effect was found, and a robust pseudohomophone priming advantage, which was not limited to the overlap of onset phoneme, was also observed. Identity primes produced greater facilitation than pseudohomophone primes. We consider the implication of these findings for the role of assembled phonology in lexical access.