Acoustic-phonetic Information Research Articles

Impaired Cortical Tracking of Speech in Children with Developmental Language Disorder.

In developmental language disorder (DLD), learning to comprehend and express oneself with spoken language is impaired, but the reason for this remains unknown. Using millisecond-scale magnetoencephalography recordings combined with machine learning models, we investigated whether the possible neural basis of this disruption lies in poor cortical tracking of speech. The stimuli were common spoken Finnish words (e.g., dog, car, hammer) and sounds with corresponding meanings (e.g., dog bark, car engine, hammering). In both children with DLD (10 boys and 7 girls) and typically developing (TD) control children (14 boys and 3 girls), aged 10-15 years, the cortical activation to spoken words was best modeled as time-locked to the unfolding speech input at ∼100 ms latency between sound and cortical activation. Amplitude envelope (amplitude changes) and spectrogram (detailed time-varying spectral content) of the spoken words, but not other sounds, were very successfully decoded based on time-locked brain responses in bilateral temporal areas; based on the cortical responses, the models could tell at ∼75-85% accuracy which of the two sounds had been presented to the participant. However, the cortical representation of the amplitude envelope information was poorer in children with DLD compared with TD children at longer latencies (at ∼200-300 ms lag). We interpret this effect as reflecting poorer retention of acoustic-phonetic information in short-term memory. This impaired tracking could potentially affect the processing and learning of words as well as continuous speech. The present results offer an explanation for the problems in language comprehension and acquisition in DLD.

Do individual differences in lexical reliance reflect states or traits?

Research suggests that individuals differ in the degree to which they rely on lexical information to support speech perception. However, the locus of these differences is not yet known; nor is it known whether these individual differences reflect a context-dependent “state” or a stable listener “trait.” Here we test the hypothesis that individual differences in lexical reliance are a stable trait that is linked to individuals' relative weighting of lexical and acoustic-phonetic information for speech perception. At each of two sessions, listeners (n = 73) completed a Ganong task, a phonemic restoration task, and a locally time-reversed speech task – three tasks that have been used to demonstrate a lexical influence on speech perception. Robust lexical effects on speech perception were observed for each task in the aggregate. Individual differences in lexical reliance were stable across sessions; however, relationships among the three tasks in each session were weak. For the Ganong and locally time-reversed speech tasks, increased reliance on lexical information was associated with weaker reliance on acoustic-phonetic information. Collectively, these results (1) provide some evidence to suggest that individual differences in lexical reliance for a given task are a stable reflection of the relative weighting of acoustic-phonetic and lexical cues for speech perception in that task, and (2) highlight the need for a better understanding of the psychometric characteristics of tasks used in the psycholinguistic domain to build theories that can accommodate individual differences in mapping speech to meaning.

Effect of Age on Informational Masking: Differential Effects of Phonetic and Semantic Information in the Masker.

Speech recognition in noise is a ubiquitous problem in older listeners. Speech, the most commonly encountered noise in the real world, causes greater masking than noise maskers, a phenomenon called informational masking (IM). This is due to the lexical-semantic and/or acoustic-phonetic information present in speech maskers. In this study, we aimed to observe the age-related differences in speech recognition and the magnitudes of IM when the maskers varied in the type of linguistic information. In 30 young and 30 older individuals, we measured the signal-to-noise ratio required to obtain 50% correct identification under four-talker babble (lexical-semantic and acoustic-phonetic information), four-talker reverse babble (predominantly acoustic-phonetic information), and speech-shaped noise (SSN; energetic). In both groups, the four-talker babble caused the greatest masking effect (worst performances), whereas the SSN resulted in the least masking effect (best performances). The effectiveness of IM due to the lexical-semantic information was comparable between the two groups. However, the effectiveness of IM due to the acoustic-phonetic information was significantly higher in the older listeners, causing worse performances. The greater effectiveness of IM due to the acoustic-phonetic information (worse performance) could be due to the minimal-to-mild high-frequency hearing loss and the consequent temporal processing deficits observed in the older listeners. However, it is possible that the older listeners can employ compensatory mechanisms (such as life experiences, contextual cues, employing higher listening efforts, among many possible other mechanisms) to overcome some of these deficits. https://doi.org/10.23641/asha.20405730.

Do individual differences in lexical reliance reflect states or traits?

Individual differences in listeners’ reliance on lexical information for speech perception have been established in the literature. However, it is not yet clear whether these individual differences reflect a context-dependent “state” or whether they reflect a stable listener “trait.” To examine this question, listeners completed three measures of lexical processing and one measure of acoustic-phonetic processing at two time points (n = 80 at session 1; n = 41 at session 2). Results to date suggest limited within-session convergence among the three measures of lexical processing, inconsistent with past results. However, strong between-session relationships for two of the three measures of lexical processing and the measure of acoustic-phonetic processing and were found, suggesting that performance in these tasks may reflect stable individual traits. A strong trading relationship between reliance on lexical versus acoustic-phonetic information was only found when both sources of information were available in a given task, suggesting that individual differences in cue use may emerge most strongly when cues conflict in speech input. Collectively, the current results suggest that individuals are stable in their acoustic-phonetic and lexical cue use strategies for a given task, but that an individual’s cue use strategy may be task-dependent.

Sonority as a Phonological Cue in Early Perception of Written Syllables in French.

Many studies focused on the letter and sound co-occurrences to account for the well-documented syllable-based effects in French in visual (pseudo)word processing. Although these language-specific statistical properties are crucial, recent data suggest that studies that go all-in on phonological and orthographic regularities may be misguided in interpreting how—and why—readers locate syllable boundaries and segment clusters. Indeed, syllable-based effects could depend on more abstract, universal phonological constraints that rule and govern how letter and sound occur and co-occur, and readers could be sensitive to sonority—a universal phonological element—for processing (pseudo)words. Here, we investigate whether French adult skilled readers rely on universal phonological sonority-related markedness continuum across the syllable boundaries for segmentation (e.g., from marked, illegal intervocalic clusters /zl/ to unmarked, legal intervocalic clusters /lz/). To address this question, we ran two tasks with 128 French adult skilled readers using two versions of the illusory conjunction paradigm (Task 1 without white noise; Task 2 with white noise). Our results show that syllable location and segmentation in reading is early and automatically modulated by phonological sonority-related markedness in the absence or quasi-absence of statistical information and does not require acoustic-phonetic information. We discuss our results toward the overlooked role of phonological universals and the over-trusted role of statistical information during reading processes.

Non-native consonant acquisition in noise: Effects of exposure/test similarity.

When faced with speech in noise, do listeners rely on robust cues or can they make use of joint speech-plus-noise patterns based on prior experience? Recent studies have suggested that listeners are better able to identify words in noise if they experienced the same word-in-noise tokens in an earlier exposure phase. The current study examines the role of token similarity in exposure and test conditions. In three experiments, Spanish learners of English were exposed to intervocalic consonants during an extensive training phase, bracketed by pre- and post-tests. Distinct cohorts experienced tokens that were either matched or mismatched across test and training phases in one or both of two factors: signal-to-noise ratio (SNR) and talker. Cohorts with fully matching test-training exposure were no better at identifying consonants at the post-test phase than those trained in partially or fully mismatched conditions. Indeed, at more adverse test SNRs, training at more favourable SNRs was beneficial. These findings argue against the use of joint speech-plus-noise representations at the segmental level and instead suggest that listeners are able to extract useful acoustic-phonetic information across a range of exposure conditions.

Effect of informational masking on auditory working memory: role of linguistic information in the maskers

Purpose: The current study aimed at observing how informational maskers with varying degrees of linguistic information affect working memory (WM) performance. The linguistic information in the maskers was varied by increasing the number of speakers in the babbles as well as by time-reversal of these babbles.Method: Backward digit (BD) span was measured on 24 normal-hearing native speakers of Kannada language. BD spans were measured under six background conditions – quiet, 2- and 8-speaker babbles (2SB and 8SB), time-reversed 2- and 8- speaker babbles (2RB and 8RB), and speech spectrum noise. The digits were presented at 75 dB SPL and the SNR was maintained at +5 dB to ensure complete audibility and intelligibility of the target digits. Using an adaptive procedure, BD span was calculated as the average of the last four reversals.Results: One-way repeated measures ANOVA showed that the 2SB caused the greatest deterioration in the BD spans. The 8SB and the two RBs resulted in significantly better scores than the 2SB, but were statistically no different from each other. The speech spectrum noise caused negligible deteriorations on the BD spans and was statistically similar to the quiet background condition.Conclusions: Results showed that the lexical-semantic information in the masker (2SB) causes the greatest deterioration in the WM scores, followed by acoustic–phonetic information (8SB and RBs). Energetic masking resulted in minimal deterioration on the BD spans. These results are further discussed based interpretations from different theories and models of working memory and speech perception.

Dichotic integration of acoustic-phonetic information: Competition from extraneous formants increases the effect of second-formant attenuation on intelligibility.

Differences in ear of presentation and level do not prevent effective integration of concurrent speech cues such as formant frequencies. For example, presenting the higher formants of a consonant-vowel syllable in the opposite ear to the first formant protects them from upward spread of masking, allowing them to remain effective speech cues even after substantial attenuation. This study used three-formant (F1+F2+F3) analogues of natural sentences and extended the approach to include competitive conditions. Target formants were presented dichotically (F1+F3; F2), either alone or accompanied by an extraneous competitor for F2 (i.e., F1±F2C+F3; F2) that listeners must reject to optimize recognition. F2C was created by inverting the F2 frequency contour and using the F2 amplitude contour without attenuation. In experiment 1, F2C was always absent and intelligibility was unaffected until F2 attenuation exceeded 30 dB; F2 still provided useful information at 48-dB attenuation. In experiment 2, attenuating F2 by 24 dB caused considerable loss of intelligibility when F2C was present, but had no effect in its absence. Factors likely to contribute to this interaction include informational masking from F2C acting to swamp the acoustic-phonetic information carried by F2, and interaural inhibition from F2C acting to reduce the effective level of F2.

Lexical frequency effects in English and Spanish word misperceptions.

When listeners misperceive words in noise, do they report words that are more common? Lexical frequency differences between misperceived and target words in English and Spanish were examined for five masker types. Misperceptions had a higher lexical frequency in the presence of pure energetic maskers, but frequency effects were reduced or absent for informational maskers. The tendency to report more common words increased with the degree of energetic masking, suggesting that uncertainty about segment identity provides a role for lexical frequency. However, acoustic-phonetic information from an informational masker may additionally constrain lexical choice.

Investigating Text-Independent Speaker Verification Systems Under Varied Data Conditions

This work makes an investigation into speaker verification (SV) from the view of practical systems. Limited data SV is preferred in order to have user comfort and effective decision delivery for regular usage. However, reduction in speech data affects the SV performance that becomes a concern for field deployment. In this work, varied data conditions for SV are explored, and sufficient train with limited test data is presented as a preferable anatomy for practical systems. Different explorations are made from the perspective of improving performance in varied data conditions. These explorations include vocal tract constriction feature to include speaker-specific acoustic–phonetic information, different attributes of voice source features that carry alternative/complementary information from that carried by conventional mel-frequency cepstral coefficient features. Further, kernel discriminant analysis is performed at the back end of i-vector-based speaker modeling for channel/session compensation that is found to work well for varied data conditions. Finally, a framework is proposed in combination with the stated explorations to have a better speaker characterization, which is more effective in case of sufficient train and limited test data scenario. The proposed framework achieves significant improvement in performance [equal error rate (EER): 11.20%, detection cost function (DCF): 0.1990], compared to the baseline (EER: 22.31%, DCF: 0.4128) for sufficient train with 2-s test segment case, showing scope toward application-oriented systems.

Perceptual Discrimination of Speaking Style Under Cochlear Implant Simulation.

Objectives:Real-life, adverse listening conditions involve a great deal of speech variability, including variability in speaking style. Depending on the speaking context, talkers may use a more casual, reduced speaking style or a more formal, careful speaking style. Attending to fine-grained acoustic-phonetic details characterizing different speaking styles facilitates the perception of the speaking style used by the talker. These acoustic-phonetic cues are poorly encoded in cochlear implants (CIs), potentially rendering the discrimination of speaking style difficult. As a first step to characterizing CI perception of real-life speech forms, the present study investigated the perception of different speaking styles in normal-hearing (NH) listeners with and without CI simulation.Design:The discrimination of three speaking styles (conversational reduced speech, speech from retold stories, and carefully read speech) was assessed using a speaking style discrimination task in two experiments. NH listeners classified sentence-length utterances, produced in one of the three styles, as either formal (careful) or informal (conversational). Utterances were presented with unmodified speaking rates in experiment 1 (31 NH, young adult Dutch speakers) and with modified speaking rates set to the average rate across all utterances in experiment 2 (28 NH, young adult Dutch speakers). In both experiments, acoustic noise-vocoder simulations of CIs were used to produce 12-channel (CI-12) and 4-channel (CI-4) vocoder simulation conditions, in addition to a no-simulation condition without CI simulation.Results:In both experiments 1 and 2, NH listeners were able to reliably discriminate the speaking styles without CI simulation. However, this ability was reduced under CI simulation. In experiment 1, participants showed poor discrimination of speaking styles under CI simulation. Listeners used speaking rate as a cue to make their judgements, even though it was not a reliable cue to speaking style in the study materials. In experiment 2, without differences in speaking rate among speaking styles, listeners showed better discrimination of speaking styles under CI simulation, using additional cues to complete the task.Conclusions:The findings from the present study demonstrate that perceiving differences in three speaking styles under CI simulation is a difficult task because some important cues to speaking style are not fully available in these conditions. While some cues like speaking rate are available, this information alone may not always be a reliable indicator of a particular speaking style. Some other reliable speaking styles cues, such as degraded acoustic-phonetic information and variability in speaking rate within an utterance, may be available but less salient. However, as in experiment 2, listeners’ perception of speaking styles may be modified if they are constrained or trained to use these additional cues, which were more reliable in the context of the present study. Taken together, these results suggest that dealing with speech variability in real-life listening conditions may be a challenge for CI users.

Neural substrates of subphonemic variation and lexical competition in spoken word recognition

ABSTRACTIn spoken word recognition, subphonemic variation influences lexical activation, with sounds near a category boundary increasing phonetic competition as well as lexical competition. The current study investigated the interplay of these factors using a visual world task in which participants were instructed to look at a picture of an auditory target (e.g. peacock). Eyetracking data indicated that participants were slowed when a voiced onset competitor (e.g. beaker) was also displayed, and this effect was amplified when acoustic-phonetic competition was increased. Simultaneously-collected fMRI data showed that several brain regions were sensitive to the presence of the onset competitor, including the supramarginal, middle temporal, and inferior frontal gyri, and functional connectivity analyses revealed that the coordinated activity of left frontal regions depends on both acoustic-phonetic and lexical factors. Taken together, results suggest a role for frontal brain structures in resolving lexical competition, particularly as atypical acoustic-phonetic information maps on to the lexicon.

Assessing the importance of several acoustic properties to the perception of spontaneous speech.

Spoken language manifests itself as change over time in various acoustic dimensions. While it seems clear that acoustic-phonetic information in the speech signal is key to language processing, little is currently known about which specific types of acoustic information are relatively more informative to listeners. This problem is likely compounded when considering reduced speech: Which specific acoustic information do listeners rely on when encountering spoken forms that are highly variable, and often include altered or elided segments? This work explores contributions of spectral shape, f0 contour, target duration, and time varying intensity in the perception of reduced speech. This work extends previous laboratory-speech based perception studies into the realm of casual speech, and also provides support for use of an algorithm that quantifies phonetic reduction. Data suggest the role of spectral shape is extensive, and that its removal degrades signals in a way that hinders recognition severely. Information reflecting f0 contour and target duration both appear to aid the listener somewhat, though their influence seems small compared to that of short term spectral shape. Finally, information about time varying intensity aids the listener more than noise filled gaps, and both aid the listener beyond presentation of acoustic context with duration-matched silence.

Across-ear integration of acoustic-phonetic information under competitive conditions: Effects of attenuation increase in the presence of an extraneous formant

Previous research using consonant-vowel syllables where one ear receives the first formant (F1) and the other receives the second and third has shown that dichotic release from masking allows F2 + F3 to remain effective speech cues even after substantial attenuation. This study used three-formant analogues of natural sentences and extended the approach to include competitive conditions. Target formants were presented dichotically (F1 + F3; F2), either alone or accompanied by an extraneous competitor for F2 (i.e., F1±F2C + F3; F2) that listeners must reject to optimize recognition. In experiment 1, F2C was absent and F2 was attenuated in 6-dB steps (range: -6-48 dB). Intelligibility was unaffected until attenuation >30 dB; F2 still provided useful information at 48-dB attenuation. In experiment 2, F2 was attenuated in 12-dB steps (range: 0-36 dB). F2C was created by inverting the F2 frequency contour and using the F2 amplitude contour without attenuation. When F2C was present, 12-dB attenuation was sufficient to cause some loss of intelligibility; the effect was large when attenuation ≥24 dB. This interaction suggests that some mandatory across-ear spectral integration occurs, such that informational masking arising from F2C rapidly swamps the acoustic-phonetic information carried by F2 as its relative level is attenuated. [Work supported by ESRC.]

Ganong shifts for noise- and sine-vocoded speech continua in normal-hearing listeners

Speech perception involves interpreting incoming sensory information in the context of stored linguistic knowledge. Lexical bias is the tendency to perceive an ambiguous speech sound as a phoneme that completes a word rather than a non-word; the more ambiguous the auditory signal, the greater the reliance on lexical knowledge. For example, a speech sound ambiguous between [g] and [k] is more likely to be perceived as [g] when preceding [Ift] and as [k] when preceding [Is]. The magnitude of this between-context difference—the Ganong shift—also increases when the listener experiences high cognitive load, limiting the resources available to process acoustic-phonetic information. In this study, an eight-step series between monotonized natural tokens of [gI] and [kI] was created using morphing software. Two continua—“gift”-“kift” and “giss”-“kiss”—were derived by adding [ft] or [s] to each step. High-resolution noise- (16-band) and sine-vocoded (32-band) versions were derived from these continua. Ganong shifts were substantially larger for sine- than for noise-vocoded stimuli, despite the greater number of channels for the former and similar slopes for the identification functions, perhaps reflecting the higher cognitive load associated with the relative spectral sparsity and less natural timbre of the sine-vocoded stimuli. [Work supported by ESRC.]

Adults with Specific Language Impairment fail to consolidate speech sounds during sleep

Specific Language Impairment (SLI) is a common learning disability that is associated with poor speech sound representations. These differences in representational quality are thought to impose a burden on spoken language processing. The underlying mechanism to account for impoverished speech sound representations remains in debate. Previous findings that implicate sleep as important for building speech representations, combined with reports of atypical sleep in SLI, motivate the current investigation into a potential consolidation mechanism as a source of impoverished representations in SLI. In the current study, we trained individuals with SLI on a new (nonnative) set of speech sounds, and tracked their perceptual accuracy and neural responses to these sounds over two days. Adults with SLI achieved comparable performance to typical controls during training, however demonstrated a distinct lack of overnight gains on the next day. We propose that those with SLI may be impaired in the consolidation of acoustic-phonetic information.

Individual differences in cue weights are correlated across contrasts

When listeners make judgments about phonological contrasts, they integrate different acoustic dimensions putting more weight on some than others. Individuals differ in how they weight different cues. Schultz, Francis & Llanos (2012) found that weights for VOT and f0 as cues to initial stop voicing in English are weakly positively correlated across individuals. We ask whether this is specific to VOT and f0 or a more general property of cue weighting across individuals. Secondly, across contrasts do the same listeners have stronger cue weights? 43 listeners performed a 2AFC task for four sets of minimal pairs that each varied orthogonally in two dimensions. All heard bet-bat and Luce-lose (vowel spectral quality vs. duration) and bog-dog (burst spectrum vs. formant transitions). 24 participants also heard sock-shock (sibilant spectrum vs. formant transitions) and the other 19 heard dear-tear (VOT vs. f0). Cue weights were fit with random slopes in a logistic regression for each minimal pair. Weights were positively correlated across individuals both within and across contrasts for bet-bat, Luce-lose, and sock-shock. Bog-dog and dear-tear had less consistent results. Overall this indicates that some individuals are better able to extract and use acoustic-phonetic information across different acoustic dimensions.

Speaking Style Influences the Brain's Electrophysiological Response to Grammatical Errors in Speech Comprehension.

This electrophysiological study asked whether the brain processes grammatical gender violations in casual speech differently than in careful speech. Native speakers of Dutch were presented with utterances that contained adjective-noun pairs in which the adjective was either correctly inflected with a word-final schwa (e.g., een spannende roman, "a suspenseful novel") or incorrectly uninflected without that schwa ( een spannend roman). Consistent with previous findings, the uninflected adjectives elicited an electrical brain response sensitive to syntactic violations when the talker was speaking in a careful manner. When the talker was speaking in a casual manner, this response was absent. A control condition showed electrophysiological responses for carefully as well as casually produced utterances with semantic anomalies, showing that listeners were able to understand the content of both types of utterance. The results suggest that listeners take information about the speaking style of a talker into account when processing the acoustic-phonetic information provided by the speech signal. Absent schwas in casual speech are effectively not grammatical gender violations. These changes in syntactic processing are evidence of contextually driven neural flexibility.

Effect of Context and Hearing Loss on Time-Gated Word Recognition in Children.

The purpose of this study was to examine word recognition in children who are hard of hearing (CHH) and children with normal hearing (CNH) in response to time-gated words presented in high- versus low-predictability sentences (HP, LP), where semantic cues were manipulated. Findings inform our understanding of how CHH combine cognitive-linguistic and acoustic-phonetic cues to support spoken word recognition. It was hypothesized that both groups of children would be able to make use of linguistic cues provided by HP sentences to support word recognition. CHH were expected to require greater acoustic information (more gates) than CNH to correctly identify words in the LP condition. In addition, it was hypothesized that error patterns would differ across groups. Sixteen CHH with mild to moderate hearing loss and 16 age-matched CNH participated (5 to 12 years). Test stimuli included 15 LP and 15 HP age-appropriate sentences. The final word of each sentence was divided into segments and recombined with the sentence frame to create series of sentences in which the final word was progressively longer by the gated increments. Stimuli were presented monaurally through headphones and children were asked to identify the target word at each successive gate. They also were asked to rate their confidence in their word choice using a five- or three-point scale. For CHH, the signals were processed through a hearing aid simulator. Standardized language measures were used to assess the contribution of linguistic skills. Analysis of language measures revealed that the CNH and CHH performed within the average range on language abilities. Both groups correctly recognized a significantly higher percentage of words in the HP condition than in the LP condition. Although CHH performed comparably with CNH in terms of successfully recognizing the majority of words, differences were observed in the amount of acoustic-phonetic information needed to achieve accurate word recognition. CHH needed more gates than CNH to identify words in the LP condition. CNH were significantly lower in rating their confidence in the LP condition than in the HP condition. CHH, however, were not significantly different in confidence between the conditions. Error patterns for incorrect word responses across gates and predictability varied depending on hearing status. The results of this study suggest that CHH with age-appropriate language abilities took advantage of context cues in the HP sentences to guide word recognition in a manner similar to CNH. However, in the LP condition, they required more acoustic information (more gates) than CNH for word recognition. Differences in the structure of incorrect word responses and their nomination patterns across gates for CHH compared with their peers with NH suggest variations in how these groups use limited acoustic information to select word candidates.

Syntactic predictability in the recognition of carefully and casually produced speech.

The present study investigated whether the recognition of spoken words is influenced by how predictable they are given their syntactic context and whether listeners assign more weight to syntactic predictability when acoustic-phonetic information is less reliable. Syntactic predictability was manipulated by varying the word order of past participles and auxiliary verbs in Dutch subordinate clauses. Acoustic-phonetic reliability was manipulated by presenting sentences either in a careful or a casual speaking style. In 3 eye-tracking experiments, participants recognized past participles more quickly when they occurred after their associated auxiliary verbs than when they preceded them. Response measures tapping into later stages of processing suggested that this effect was stronger for casually than for carefully produced sentences. These findings provide further evidence that syntactic predictability can influence word recognition and that this type of information is particularly useful for coping with acoustic-phonetic reductions in conversational speech. We conclude that listeners dynamically adapt to the different sources of linguistic information available to them.