Acoustic-phonetic Processing Research Articles

Neural evidence suggests phonological acceptability judgments reflect similarity, not constraint evaluation

Acceptability judgments are a primary source of evidence in formal linguistic research. Within the generative linguistic tradition, these judgments are attributed to evaluation of novel forms based on implicit knowledge of rules or constraints governing well-formedness. In the domain of phonological acceptability judgments, other factors including ease of articulation and similarity to known forms have been hypothesized to influence evaluation. We used data-driven neural techniques to identify the relative contributions of these factors. Granger causality analysis of magnetic resonance imaging (MRI)-constrained magnetoencephalography (MEG) and electroencephalography (EEG) data revealed patterns of interaction between brain regions that support explicit judgments of the phonological acceptability of spoken nonwords. Comparisons of data obtained with nonwords that varied in terms of onset consonant cluster attestation and acceptability revealed different cortical regions and effective connectivity patterns associated with phonological acceptability judgments. Attested forms produced stronger influences of brain regions implicated in lexical representation and sensorimotor simulation on acoustic-phonetic regions, whereas unattested forms produced stronger influence of phonological control mechanisms on acoustic-phonetic processing. Unacceptable forms produced widespread patterns of interaction consistent with attempted search or repair. Together, these results suggest that speakers' phonological acceptability judgments reflect lexical and sensorimotor factors.

Do individual differences in lexical reliance reflect states or traits?

Individual differences in listeners’ reliance on lexical information for speech perception have been established in the literature. However, it is not yet clear whether these individual differences reflect a context-dependent “state” or whether they reflect a stable listener “trait.” To examine this question, listeners completed three measures of lexical processing and one measure of acoustic-phonetic processing at two time points (n = 80 at session 1; n = 41 at session 2). Results to date suggest limited within-session convergence among the three measures of lexical processing, inconsistent with past results. However, strong between-session relationships for two of the three measures of lexical processing and the measure of acoustic-phonetic processing and were found, suggesting that performance in these tasks may reflect stable individual traits. A strong trading relationship between reliance on lexical versus acoustic-phonetic information was only found when both sources of information were available in a given task, suggesting that individual differences in cue use may emerge most strongly when cues conflict in speech input. Collectively, the current results suggest that individuals are stable in their acoustic-phonetic and lexical cue use strategies for a given task, but that an individual’s cue use strategy may be task-dependent.

Informational masking of speech by acoustically similar intelligible and unintelligible interferers.

Masking experienced when target speech is accompanied by a single interfering voice is often primarily informational masking (IM). IM is generally greater when the interferer is intelligible than when it is not (e.g., speech from an unfamiliar language), but the relative contributions of acoustic-phonetic and linguistic interference are often difficult to assess owing to acoustic differences between interferers (e.g., different talkers). Three-formant analogues (F1+F2+F3) of natural sentences were used as targets and interferers. Targets were presented monaurally either alone or accompanied contralaterally by interferers from another sentence (F0 = 4 semitones higher); a target-to-masker ratio (TMR) between ears of 0, 6, or 12 dB was used. Interferers were either intelligible or rendered unintelligible by delaying F2 and advancing F3 by 150 ms relative to F1, a manipulation designed to minimize spectro-temporal differences between corresponding interferers. Target-sentence intelligibility (keywords correct) was 67% when presented alone, but fell considerably when an unintelligible interferer was present (49%) and significantly further when the interferer was intelligible (41%). Changes in TMR produced neither a significant main effect nor an interaction with interferer type. Interference with acoustic-phonetic processing of the target can explain much of the impact on intelligibility, but linguistic factors-particularly interferer intrusions-also make an important contribution to IM.

Informational masking of speech analogues by intelligible and non-intelligible but acoustically similar interferers

Informational masking of target speech is generally greater when the interfering speech is intelligible than when it is not (e.g., speech from an unfamiliar language), but the relative contributions of acoustic-phonetic and linguistic interference are often difficult to assess owing to acoustic differences between interferers (e.g., different talkers). This study used three-formant analogues (F1 + F2 + F3) of natural sentences as targets and interferers. Target formants were presented monaurally (F0 = 120.3 Hz) either alone or accompanied with the contralateral ear by interfering formants from another sentence (F0 = 151.5 Hz); a target-to-masker ratio (TMR) between ears of 0, 6, or 12 dB was used. Interferers were either intelligible or rendered non-intelligible by delaying F2 and advancing F3 by 150 ms relative to F1, a manipulation designed to minimize spectro-temporal differences between corresponding interferers. Target-sentence intelligibility (keywords correct) was 67% when presented alone but fell considerably when a non-intelligible interferer was present (49%) and significantly further when the interferer was intelligible (41%). The changes in TMR produced neither a significant main effect nor an interaction with interferer type. The results suggest that although linguistic factors contribute to informational masking, interference with acoustic-phonetic processing of the target can explain much of the impact on intelligibility. [Work supported by ESRC.]

Sentential influences on acoustic-phonetic processing: a Granger causality analysis of multimodal imaging data

ABSTRACTSentential context influences the way that listeners identify phonetically ambiguous or perceptual degraded speech sounds. Unfortunately, inherent inferential limitations on the interpretation of behavioural or BOLD imaging results make it unclear whether context influences perceptual processing directly, or acts at a post-perceptual decision stage. In this paper, we use Kalman-filter enabled Granger causation analysis of MRI-constrained MEG/EEG data to distinguish between these possibilities. Using a retrospective probe verification task, we found that sentential context strongly affected the interpretation of words with ambiguous initial voicing (e.g. DUSK-TUSK). This behavioural context effect coincided with increased influence by brain regions associated with lexical representation on regions associated with acoustic-phonetic processing. These results support an interactive view of sentence context effects on speech perception.

Lexical mediation of phonotactic frequency effects on spoken word recognition: A Granger causality analysis of MRI-constrained MEG/EEG data.

Phonotactic frequency effects play a crucial role in a number of debates over language processing and representation. It is unclear however, whether these effects reflect prelexical sensitivity to phonotactic frequency, or lexical "gang effects" in speech perception. In this paper, we use Granger causality analysis of MR-constrained MEG/EEG data to understand how phonotactic frequency influences neural processing dynamics during auditory lexical decision. Effective connectivity analysis showed weaker feedforward influence from brain regions involved in acoustic-phonetic processing (superior temporal gyrus) to lexical areas (supramarginal gyrus) for high phonotactic frequency words, but stronger top-down lexical influence for the same items. Low entropy nonwords (nonwords judged to closely resemble real words) showed a similar pattern of interactions between brain regions involved in lexical and acoustic-phonetic processing. These results contradict the predictions of a feedforward model of phonotactic frequency facilitation, but support the predictions of a lexically mediated account.

Does listening to non-native speech impair speech perception?

Previous research with highly fluent, very early, Catalan–Spanish bilinguals has shown that L1 Spanish bilinguals accept certain mispronounced Catalan words at extremely high rates, and even L1 Catalan bilinguals do so at surprisingly high rates. Using similarly highly fluent, very early Basque–Spanish bilinguals, we investigate why this occurs. We test three possibilities: (1) There could be a failure to distinguish two similar sounds at the phonetic level, (2) Listeners might store exemplars they have heard in their lexical representations, with one accurate and one mispronounced version, or (3) Listeners might map two different phonetic codes to a lexical representation, either during perception or in a developed representation. A discrimination test showed that the problem is not due to a failure in acoustic-phonetic processing. To test the dual-exemplar representation versus dual mapping alternatives, we taught our subjects new Basque words under conditions that only allowed for a single lexical representation. Listeners showed the same tolerance for erroneous pronunciations in these uniquely learned items, ruling out the dual lexical representation option and supporting dual mapping. In regions in which bilinguals routinely hear accented versions of their language, it is a useful adaptation to the resulting variation to treat certain sounds as allophonic for purposes of lexical access. Looked at this way, accepting a “wrong” pronunciation is really not an error at all – it is an efficient adaptation to the prevailing linguistic environment.

A Review on Acoustic Phonetic Approach for Marathi Speech Recognition

paper discusses the phoneme used in Marathi language as a possible basic unit of speech recognition, for which there is some empirical psychoacoustic support in the case of human and some engineering justification in the case of machines striving to imitate human abilities. For the purpose of the research described in this paper, a basic unit of speech recognition is the intermediate form of speech information around which much of the recognition processing is organized for human beings or for machines. The general opinion of phonetician and psycholinguists is that there is indeed such a unit with relatively few distinct types 1 . For this research a basic unit is ideally an output of acoustic-phonetic processing and an input to the lexical processing stages.

Independence of Early Speech Processing from Word Meaning

We combined magnetoencephalography (MEG) with magnetic resonance imaging and electrocorticography to separate in anatomy and latency 2 fundamental stages underlying speech comprehension. The first acoustic-phonetic stage is selective for words relative to control stimuli individually matched on acoustic properties. It begins ∼60 ms after stimulus onset and is localized to middle superior temporal cortex. It was replicated in another experiment, but is strongly dissociated from the response to tones in the same subjects. Within the same task, semantic priming of the same words by a related picture modulates cortical processing in a broader network, but this does not begin until ∼217 ms. The earlier onset of acoustic-phonetic processing compared with lexico-semantic modulation was significant in each individual subject. The MEG source estimates were confirmed with intracranial local field potential and high gamma power responses acquired in 2 additional subjects performing the same task. These recordings further identified sites within superior temporal cortex that responded only to the acoustic-phonetic contrast at short latencies, or the lexico-semantic at long. The independence of the early acoustic-phonetic response from semantic context suggests a limited role for lexical feedback in early speech perception.

Long-term phonological learning begins at the level of word form

Incidental learning of phonological structures through repeated exposure is an important component of native and foreign-language vocabulary acquisition that is not well understood at the neurophysiological level. It is also not settled when this type of learning occurs at the level of word forms as opposed to phoneme sequences. Here, participants listened to and repeated back foreign phonological forms (Korean words) and new native-language word forms (Finnish pseudowords) on two days. Recognition performance was improved, repetition latency became shorter and repetition accuracy increased when phonological forms were encountered multiple times. Cortical magnetoencephalography responses occurred bilaterally but the experimental effects only in the left hemisphere. Superior temporal activity at 300-600 ms, probably reflecting acoustic-phonetic processing, lasted longer for foreign phonology than for native phonology. Formation of longer-term auditory-motor representations was evidenced by a decrease of a spatiotemporally separate left temporal response and correlated increase of left frontal activity at 600-1200 ms on both days. The results point to item-level learning of novel whole-word representations.

Speech Perception: Motoric Contributions versus the Motor Theory

Recent studies indicate that the motor cortex is involved not only in the production of speech, but also in its perception. These studies have sparked a renewed interest in gesture-based theories of speech perception.

Cross-modal perceptual learning of spectrally degraded speech: Behavioral and neuroimaging studies

We investigated normal-hearing listeners' ability to adapt to a speech signal that simulates aspects of the stimulation received from a cochlear implant. The training materials were spoken sentences which were spectrally degraded by noiseband-vocoding and shifted upwards in frequency. A control condition consisted of spectrally inverted, unintelligible versions of these stimuli. Participants listened passively to these sentences, and on each trial received visual feedback which could be either a written version of the sentence or a video of the talker who originally produced it. Learning under both feedback conditions was relatively fast: Subjects improved on average by 25% on keyword recognition scores after 100 trials. No learning effects were observed in the control condition. We further used functional magnetic resonance imaging to investigate which cortical areas may be recruited during learning. A comparison of degraded and learnable sentences with the unlearnable control stimuli showed activity in the left superior temporal sulcus both during passive listening and receiving feedback. In contrast, the left inferior frontal gyrus was activated only when subjects were receiving feedback in the learnable condition. These results suggest that the inferior frontal gyrus plays an important role in integrating acoustic-phonetic processing with externally provided feedback.

Recognition of gated verbs by children with Grammatical-Specific Language Impairment: Effects of inflection and frequency

A common feature of language disorders, particularly in English, is an impairment in inflectional morphology. One view claims that this deficit is caused by impaired speech processing and resulting impoverished phonological representations. We investigated the accuracy of spoken word recognition in Specific Language Impairment (SLI) using a successive forward gating paradigm, with target verbs manipulated for frequency and past tense inflection. Children with Grammatical-SLI were compared to age and language controls. We scored responses according to (1) proportion of gates to the first correct response, (2) proportion of gates to the first of three consistently correct responses. G-SLI children generally performed at the same level as age and vocabulary controls, although worse than age controls on uninflected verbs with respect to the second criterion, indicating that they activated the correct word at the same point, but took longer to reach a consistent response. Low frequency and inflection of the target word did not disadvantage G-SLI children to a greater extent than any of their controls. These results do not support the hypothesis that G-SLI children's morphological impairment is caused by poor acoustic-phonetic processing.

Acoustic-phonetic processing in semantic dementia

University of Pennsylvania School of Medicine, Department of Neurology, Philadelphia, Pennsylvania, PA, USAAccepted 6 July 2006IntroductionSemantic dementia (SD) results in a progressive loss of conceptualknowledge in the context of preserved syntactic, auditory, and visual spa-tial abilities (Grossman & Ash, 2004; Hodges, 2003; Paterson, Knott, H moderate: 66). The mean MMSEwas 18.75 (mild: 24; moderate: 13.5). Semantic memory impairment asmeasured by the Pyramids and Palm Trees test (Howard & Patterson,1992) averaged, for pictures and words, 43.75/52 (mild: 47.25; moderate:40.25). Average confrontation naming on the Boston Naming Test (Kap-lan, Goodglass, & Weintraub, 1976) averaged 5.75 of 15 (mild: 9; moder-ate: 2.5). All patients passed a pure tone hearing screening at 1, 2, and4 kHz. Patients were instructed through training sessions before each task.All auditory stimuli were digitized and administered by headphones with afrequency response range of 8–25 kHz.Structural MRI scans of milder SD revealed atrophy in the left ventral,anterolateral cortex (see Fig. 1a for VBM analysis of significant corticalatrophy in a mild SD patient). Patients with moderate SD showed atrophyspread to the posterior temporal cortex (see Fig. 1b for a VBM analysis ofsignificant cortical atrophy in a moderate SD patient).Experiment 1: Pure tone acoustic discriminationPure tones varying in frequency from 300 to 5000 Hz were presented inpairs, with a 1000 ms interstimulus interval (ISI). Half the tone pairs wereidentical; the remainder differed by increments of 25–4000 Hz. ParticipantsindicatedwhethereachtonepairwastheSAMEorDIFFERENTwithakeypress.ResultsOverall accuracy of pure tone discrimination was 82.5% (p < .001,binomial test differing from chance). The within-group contrast of mildSD versus moderate SD demonstrated greater discrimination difficultyin moderate SD (mild accuracy: 90%; moderate accuracy: 75%;v

Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory–auditory/orosensory internal models

This experiment investigates neural processes underlying perceptual identification of the same phonemes for native- and second-language speakers. A model is proposed implicating the use of articulatory–auditory and articulatory–orosensory mappings to facilitate perceptual identification under conditions in which the phonetic contrast is ambiguous, as in the case of second-language speakers. In contrast, native-language speakers are predicted to use auditory-based phonetic representations to a greater extent for perceptual identification than second-language speakers. The English /r–l/ phonetic contrast, although easy for native English speakers, is extremely difficult for native Japanese speakers who learned English as a second language after childhood. Twenty-two native English and twenty-two native Japanese speakers participated in this study. While undergoing event-related fMRI, subjects were aurally presented with syllables starting with a /r/, /l/, or a vowel and were required to rapidly identify the phoneme perceived by pushing one of three buttons with the left thumb. Consistent with the proposed model, the results show greater activity for second- over native-language speakers during perceptual identification of /r/ and /l/ relative to vowels in brain regions implicated with instantiating forward and inverse articulatory–auditory articulatory–orosensory models [Broca's area, anterior insula, anterior superior temporal sulcus/gyrus (STS/G), planum temporale (PT), superior temporal parietal area (Stp), SMG, and cerebellum]. The results further show that activity in brain regions implicated with instantiating these internal models is correlated with better /r/ and /l/ identification performance for second-language speakers. Greater activity found for native-language speakers especially in the anterior STG/S for /r/ and /l/ perceptual identification is consistent with the hypothesis that native-language speakers use auditory phonetic representations more extensively than second-language speakers.

Learning-induced neural plasticity associated with improved identification performance after training of a difficult second-language phonetic contrast

Adult native Japanese speakers have difficulty perceiving the English /r–l/ phonetic contrast even after years of exposure. However, after extensive perceptual identification training, long-lasting improvement in identification performance can be attained. This fMRI study investigates localized changes in brain activity associated with 1 month of extensive feedback-based perceptual identification training by native Japanese speakers learning the English /r–l/ phonetic contrast. Before and after training, separate functional brain imaging sessions were conducted for identification of the English /r–l/ contrast (difficult for Japanese speakers), /b–g/ contrast (easy), and /b–v/ contrast (difficult), in which signal-correlated noise served as the reference control condition. Neural plasticity, denoted by exclusive enhancement in brain activity for the /r–l/ contrast, does not involve only reorganization in brain regions concerned with acoustic-phonetic processing (superior and medial temporal areas) but also the recruitment of additional bilateral cortical (supramarginal gyrus, planum temporale, Broca’s area, premotor cortex, supplementary motor area) and subcortical regions (cerebellum, basal ganglia, substantia nigra) involved with auditory–articulatory (perceptual–motor) mappings related to verbal speech processing and learning. Contrary to what one may expect, brain activity for perception of a difficult contrast does not come to resemble that of an easy contrast as learning proceeds. Rather, the results support the hypothesis that improved identification performance may be due to the acquisition of auditory–articulatory mappings allowing for perception to be made in reference to potential action.

Learning-induced neural plasticity associated with acquisition of a difficult second-language phonetic contrast

Adult native Japanese speakers have difficulty perceiving the English /rl/ phonetic contrast even after years of exposure. However, after extensive perceptual identification training long lasting improvement in identification performance that generalizes to novel stimuli can be attained. In this fMRI study we investigate localized changes in brain activity associated with one-month of extensive feedback-based perceptual identification training by native Japanese speakers learning the English /rl/ phonetic contrast. Before and after training separate functional brain imaging sessions were conducted for identification of the English /rl/ contrast (difficult for Japanese speakers), /bg/ contrast (easy), and /bv/ contrast (difficult). Neural plasticity, denoted by exclusive enhancement in brain activity for the /rl/ contrast (not present for the /bv/ and /bg/ conditions), does not only occur in brain regions involved with acoustic–phonetic processing (superior temporal areas, supramarginal gyrus, planum temporale) but also in additional bilateral cortical (Broca’s area, premotor cortex, orosensory cortex) and subcortical regions (cerebellum, basal ganglia, substantia nigra) involved with speech production as well as with formation of perceptual–motor mappings. The results support the hypothesis that learning of an auditory–articulatory mapping improves identification performance by allowing a perception to be made in reference to potential action. [Work supported by TAO, Japan.]

Analysis of lesions by MRI in stroke patients with acoustic‐phonetic processing deficits

We tested 10 aphasic stroke patients for the ability to discriminate and identify English phonemes. All patients underwent MRI and had their scans analyzed morphometrically. Patients with impairments in acoustic-phonetic processing tended to have lesions involving the left posterior supramarginal gyrus and the bordering parietal operculum, an observation further supported by regression and correlation analyses. These results are interpreted as evidence that the region including the left posterior supramarginal gyrus and parietal operculum plays a significant role in acoustic-phonetic processing.

Speech perception during spoken-language processing in French: How comprehension processes interact with the perception of on-going speech

A study group of 60 common untrained French listeners (30 female and 30 male equally distributed in three age groups and two language groups) was used to investigate the rote of word-recognition processes and sentential context on acoustic-phonetic input processing. The monitoring latencies to a nonsense CV syllable occurring in three different contexts (acoustic-phonetic analysis alone; word-recognition processes; or contextual constraints) was examined. The results show that, in spite of slow RTs with age and more laborious acoustic-phonetic analysis with bilingualism, the word-recognition processes and the contextual constaints have a similar effect. These effects are discussed within the theoretical framework of interactive models of spoken-language comprehension.