Speech Motor Cortex Research Articles

A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages.

Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish-English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.

Lesions to Dorsal Stream Modulate Responses to Language Repetition in Speech-motor Cortex (S18.006)

Lesions to Dorsal Stream Modulate Responses to Language Repetition in Speech-motor Cortex (S18.006)

Decoding kinematic information from beta-band motor rhythms of speech motor cortex: a methodological/analytic approach using concurrent speech movement tracking and magnetoencephalography.

Articulography and functional neuroimaging are two major tools for studying the neurobiology of speech production. Until now, however, it has generally not been feasible to use both in the same experimental setup because of technical incompatibilities between the two methodologies. Here we describe results from a novel articulography system dubbed Magneto-articulography for the Assessment of Speech Kinematics (MASK), which is technically compatible with magnetoencephalography (MEG) brain scanning systems. In the present paper we describe our methodological and analytic approach for extracting brain motor activities related to key kinematic and coordination event parameters derived from time-registered MASK tracking measurements. Data were collected from 10 healthy adults with tracking coils on the tongue, lips, and jaw. Analyses targeted the gestural landmarks of reiterated utterances/ipa/ and /api/, produced at normal and faster rates. The results show that (1) Speech sensorimotor cortex can be reliably located in peri-rolandic regions of the left hemisphere; (2) mu (8-12 Hz) and beta band (13-30 Hz) neuromotor oscillations are present in the speech signals and contain information structures that are independent of those present in higher-frequency bands; and (3) hypotheses concerning the information content of speech motor rhythms can be systematically evaluated with multivariate pattern analytic techniques. These results show that MASK provides the capability, for deriving subject-specific articulatory parameters, based on well-established and robust motor control parameters, in the same experimental setup as the MEG brain recordings and in temporal and spatial co-register with the brain data. The analytic approach described here provides new capabilities for testing hypotheses concerning the types of kinematic information that are encoded and processed within specific components of the speech neuromotor system.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. “alpha” for “a”). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.

Speech motor facilitation is not affected by ageing but is modulated by task demands during speech perception

Motor areas for speech production activate during speech perception. Such activation may assist speech perception in challenging listening conditions. It is not known how ageing affects the recruitment of articulatory motor cortex during active speech perception. This study aimed to determine the effect of ageing on recruitment of speech motor cortex during speech perception.Single-pulse Transcranial Magnetic Stimulation (TMS) was applied to the lip area of left primary motor cortex (M1) to elicit lip Motor Evoked Potentials (MEPs). The M1 hand area was tested as a control site. TMS was applied whilst participants perceived syllables presented with noise (−10, 0, +10 dB SNRs) and without noise (clear). Participants detected and counted syllables throughout MEP recording. Twenty younger adult subjects (aged 18–25) and twenty older adult subjects (aged 65–78) participated in this study.Results indicated a significant interaction between age and noise condition in the syllable task. Specifically, older adults significantly misidentified syllables in the 0 dB SNR condition, and missed the syllables in the −10 dB SNR condition, relative to the clear condition. There were no differences between conditions for younger adults. There was a significant main effect of noise level on lip MEPs. Lip MEPs were unexpectedly inhibited in the 0 dB SNR condition relative to clear condition. There was no interaction between age group and noise condition. There was no main effect of noise or age group on control hand MEPs. These data suggest that speech-induced facilitation in articulatory motor cortex is abolished when performing a challenging secondary task, irrespective of age.

Asymmetry of Auditory-Motor Speech Processing is Determined by Language Experience.

Speech processing relies on interactions between auditory and motor systems and is asymmetrically organized in the human brain. The left auditory system is specialized for processing of phonemes, whereas the right is specialized for processing of pitch changes in speech affecting prosody. In speakers of tonal languages, however, processing of pitch (i.e., tone) changes that alter word meaning is left-lateralized indicating that linguistic function and language experience shape speech processing asymmetries. Here, we investigated the asymmetry of motor contributions to auditory speech processing in male and female speakers of tonal and non-tonal languages. We temporarily disrupted the right or left speech motor cortex using transcranial magnetic stimulation (TMS) and measured the impact of these disruptions on auditory discrimination (mismatch negativity; MMN) responses to phoneme and tone changes in sequences of syllables using electroencephalography (EEG). We found that the effect of motor disruptions on processing of tone changes differed between language groups: disruption of the right speech motor cortex suppressed responses to tone changes in non-tonal language speakers, whereas disruption of the left speech motor cortex suppressed responses to tone changes in tonal language speakers. In non-tonal language speakers, the effects of disruption of left speech motor cortex on responses to tone changes were inconclusive. For phoneme changes, disruption of left but not right speech motor cortex suppressed responses in both language groups. We conclude that the contributions of the right and left speech motor cortex to auditory speech processing are determined by the functional roles of acoustic cues in the listener's native language.SIGNIFICANCE STATEMENT The principles underlying hemispheric asymmetries of auditory speech processing remain debated. The asymmetry of processing of speech sounds is affected by low-level acoustic cues, but also by their linguistic function. By combining transcranial magnetic stimulation (TMS) and electroencephalography (EEG), we investigated the asymmetry of motor contributions to auditory speech processing in tonal and non-tonal language speakers. We provide causal evidence that the functional role of the acoustic cues in the listener's native language affects the asymmetry of motor influences on auditory speech discrimination ability [indexed by mismatch negativity (MMN) responses]. Lateralized top-down motor influences can affect asymmetry of speech processing in the auditory system.

Noninvasive neurostimulation of left ventral motor cortex enhances sensorimotor adaptation in speech production

Sensorimotor adaptation—enduring changes to motor commands due to sensory feedback—allows speakers to match their articulations to intended speech acoustics. How the brain integrates auditory feedback to modify speech motor commands and what limits the degree of these modifications remain unknown. Here, we investigated the role of speech motor cortex in modifying stored speech motor plans. In a within-subjects design, participants underwent separate sessions of sham and anodal transcranial direct current stimulation (tDCS) over speech motor cortex while speaking and receiving altered auditory feedback of the first formant. Anodal tDCS increased the rate of sensorimotor adaptation for feedback perturbation. Computational modeling of our results using the Directions Into Velocities of Articulators (DIVA) framework of speech production suggested that tDCS primarily affected behavior by increasing the feedforward learning rate. This study demonstrates how focal noninvasive neurostimulation can enhance the integration of auditory feedback into speech motor plans.

Failure of tDCS to modulate motor excitability and speech motor learning

Transcranial direct current stimulation (tDCS) modulates cortical excitability in a polarity-specific way and, when used in combination with a behavioural task, it can alter performance. TDCS has the potential, therefore, for use as an adjunct to therapies designed to treat disorders affecting speech, including, but not limited to acquired aphasias and developmental stuttering. For this reason, it is important to conduct studies evaluating its effectiveness and the parameters optimal for stimulation. Here, we aimed to evaluate the effects of bi-hemispheric tDCS over speech motor cortex on performance of a complex speech motor learning task, namely the repetition of tongue twisters. A previous study in older participants showed that tDCS could modulate performance on a similar task. To further understand the effects of tDCS, we also measured the excitability of the speech motor cortex before and after stimulation. Three groups of 20 healthy young controls received: (i) anodal tDCS to the left IFG/LipM1 and cathodal tDCS to the right hemisphere homologue; or (ii) cathodal tDCS over the left and anodal over the right; or (iii) sham stimulation. Participants heard and repeated novel tongue twisters and matched simple sentences before, during and 10 min after the stimulation. One mA tDCS was delivered concurrent with task performance for 13 min. Motor excitability was measured using transcranial magnetic stimulation to elicit motor-evoked potentials in the lip before and immediately after tDCS. The study was double-blind, randomized, and sham-controlled; the design and analysis were pre-registered. Performance on the task improved from baseline to after stimulation but was not significantly modulated by tDCS. Similarly, a small decrease in motor excitability was seen in all three stimulation groups but did not differ among them and was unrelated to task performance. Bayesian analyses provide substantial evidence in support of the null hypotheses in both cases, namely that tongue twister performance and motor excitability were not affected by tDCS. We discuss our findings in the context of the previous positive results for a similar task. We conclude that tDCS may be most effective when brain function is sub-optimal due to age-related declines or pathology. Further study is required to determine why tDCS failed to modulate excitability in the speech motor cortex in the expected ways.

Speech rhythms and their neural foundations.

The recognition of spoken language has typically been studied by focusing on either words or their constituent elements (for example, low-level features or phonemes). More recently, the 'temporal mesoscale' of speech has been explored, specifically regularities in the envelope of the acoustic signal that correlate with syllabic information and that play a central role in production and perception processes. The temporal structure of speech at this scale is remarkably stable across languages, with a preferred range of rhythmicity of 2- 8 Hz. Importantly, this rhythmicity is required by the processes underlying the construction of intelligible speech. A lot of current work focuses on audio-motor interactions in speech, highlighting behavioural and neural evidence that demonstrates how properties of perceptual and motor systems, and their relation, can underlie the mesoscale speech rhythms. The data invite the hypothesis that the speech motor cortex is best modelled as a neural oscillator, a conjecture that aligns well with current proposals highlighting the fundamental role of neural oscillations in perception and cognition. The findings also show motor theories (of speech) in a different light, placing new mechanistic constraints on accounts of the action-perception interface.

Generating Natural, Intelligible Speech From Brain Activity in Motor, Premotor, and Inferior Frontal Cortices.

Neural interfaces that directly produce intelligible speech from brain activity would allow people with severe impairment from neurological disorders to communicate more naturally. Here, we record neural population activity in motor, premotor and inferior frontal cortices during speech production using electrocorticography (ECoG) and show that ECoG signals alone can be used to generate intelligible speech output that can preserve conversational cues. To produce speech directly from neural data, we adapted a method from the field of speech synthesis called unit selection, in which units of speech are concatenated to form audible output. In our approach, which we call Brain-To-Speech, we chose subsequent units of speech based on the measured ECoG activity to generate audio waveforms directly from the neural recordings. Brain-To-Speech employed the user's own voice to generate speech that sounded very natural and included features such as prosody and accentuation. By investigating the brain areas involved in speech production separately, we found that speech motor cortex provided more information for the reconstruction process than the other cortical areas.

Dopamine drives left-hemispheric lateralization of neural networks during human speech.

Although the concept of left-hemispheric lateralization of neural processes during speech production has been known since the times of Broca, its physiological underpinnings still remain elusive. We sought to assess the modulatory influences of a major neurotransmitter, dopamine, on hemispheric lateralization during real-life speaking using a multimodal analysis of functional MRI, intracranial EEG recordings, and large-scale neural population simulations based on diffusion-weighted MRI. We demonstrate that speech-induced phasic dopamine release into the dorsal striatum and speech motor cortex exerts direct modulation of neuronal activity in these regions and drives left-hemispheric lateralization of speech production network. Dopamine-induced lateralization of functional activity and networks during speaking is not dependent on lateralization of structural nigro-striatal and nigro-motocortical pathways. Our findings provide the first mechanistic explanation for left-hemispheric lateralization of human speech that is due to left-lateralized dopaminergic modulation of brain activity and functional networks.

Functional Segregation of Cortical Regions Underlying Speech Timing and Articulation

Spoken language is a central part of our everyday lives, but the precise roles that individual cortical regions play in the production of speech are often poorly understood. To address this issue, we focally lowered the temperature of distinct cortical regions in awake neurosurgical patients, and we relate this perturbation to changes in produced speech sequences. Using this method, we confirm that speech is highly lateralized, with the vast majority of behavioral effects seen on the left hemisphere. We then use this approach to demonstrate a clear functional dissociation between nearby cortical speech sites. Focal cooling of pars triangularis/pars opercularis (Broca's region) and the ventral portion of the precentral gyrus (speech motor cortex) resulted in the manipulation of speech timing and articulation, respectively. Our results support a class of models that have proposed distinct processing centers underlying motor sequencing and execution for speech.

Speech dynamics are coded in the left motor cortex in fluent speakers but not in adults who stutter.

The precise excitability regulation of neuronal circuits in the primary motor cortex is central to the successful and fluent production of speech. Our question was whether the involuntary execution of undesirable movements, e.g. stuttering, is linked to an insufficient excitability tuning of neural populations in the orofacial region of the primary motor cortex. We determined the speech-related time course of excitability modulation in the left and right primary motor tongue representation. Thirteen fluent speakers (four females, nine males; aged 23-44) and 13 adults who stutter (four females, nine males, aged 21-55) were asked to build verbs with the verbal prefix 'auf'. Single-pulse transcranial magnetic stimulation was applied over the primary motor cortex during the transition phase between a fixed labiodental articulatory configuration and immediately following articulatory configurations, at different latencies after transition onset. Bilateral electromyography was recorded from self-adhesive electrodes placed on the surface of the tongue. Off-line, we extracted the motor evoked potential amplitudes and normalized these amplitudes to the individual baseline excitability during the fixed configuration. Fluent speakers demonstrated a prominent left hemisphere increase of motor cortex excitability in the transition phase (P = 0.009). In contrast, the excitability of the right primary motor tongue representation was unchanged. Interestingly, adults afflicted with stuttering revealed a lack of left-hemisphere facilitation. Moreover, the magnitude of facilitation was negatively correlated with stuttering frequency. Although orofacial midline muscles are bilaterally innervated from corticobulbar projections of both hemispheres, our results indicate that speech motor plans are controlled primarily in the left primary speech motor cortex. This speech motor planning-related asymmetry towards the left orofacial motor cortex is missing in stuttering. Moreover, a negative correlation between the amount of facilitation and stuttering severity suggests that we discovered a main physiological principle of fluent speech production and its role in stuttering.

Direct classification of all American English phonemes using signals from functional speech motor cortex

Objective. Although brain–computer interfaces (BCIs) can be used in several different ways to restore communication, communicative BCI has not approached the rate or efficiency of natural human speech. Electrocorticography (ECoG) has precise spatiotemporal resolution that enables recording of brain activity distributed over a wide area of cortex, such as during speech production. In this study, we sought to decode elements of speech production using ECoG. Approach. We investigated words that contain the entire set of phonemes in the general American accent using ECoG with four subjects. Using a linear classifier, we evaluated the degree to which individual phonemes within each word could be correctly identified from cortical signal. Main results. We classified phonemes with up to 36% accuracy when classifying all phonemes and up to 63% accuracy for a single phoneme. Further, misclassified phonemes follow articulation organization described in phonology literature, aiding classification of whole words. Precise temporal alignment to phoneme onset was crucial for classification success. Significance. We identified specific spatiotemporal features that aid classification, which could guide future applications. Word identification was equivalent to information transfer rates as high as 3.0 bits s−1 (33.6 words min−1), supporting pursuit of speech articulation for BCI control.

Visemic processing in audiovisual discrimination of natural speech: A simultaneous fMRI–EEG study

In a noisy environment, visual perception of articulatory movements improves natural speech intelligibility. Parallel to phonemic processing based on auditory signal, visemic processing constitutes a counterpart based on “visemes”, the distinctive visual units of speech. Aiming at investigating the neural substrates of visemic processing in a disturbed environment, we carried out a simultaneous fMRI–EEG experiment based on discriminating syllabic minimal pairs involving three phonological contrasts, each bearing on a single phonetic feature characterised by different degrees of visual distinctiveness. The contrasts involved either labialisation of the vowels, or place of articulation or voicing of the consonants. Audiovisual consonant–vowel syllable pairs were presented either with a static facial configuration or with a dynamic display of articulatory movements related to speech production.In the sound-disturbed MRI environment, the significant improvement of syllabic discrimination achieved in the dynamic audiovisual modality, compared to the static audiovisual modality was associated with activation of the occipito-temporal cortex (MT+V5) bilaterally, and of the left premotor cortex. While the former was activated in response to facial movements independently of their relation to speech, the latter was specifically activated by phonological discrimination. During fMRI, significant evoked potential responses to syllabic discrimination were recorded around 150 and 250ms following the onset of the second stimulus of the pairs, whose amplitude was greater in the dynamic compared to the static audiovisual modality. Our results provide arguments for the involvement of the speech motor cortex in phonological discrimination, and suggest a multimodal representation of speech units.

Response bias modulates the speech motor system during syllable discrimination

Recent evidence suggests that the speech motor system may play a significant role in speech perception. Repetitive transcranial magnetic stimulation (TMS) applied to a speech region of premotor cortex impaired syllable identification, while stimulation of motor areas for different articulators selectively facilitated identification of phonemes relying on those articulators. However, in these experiments performance was not corrected for response bias. It is not currently known how response bias modulates activity in these networks. The present functional magnetic resonance imaging experiment was designed to produce specific, measureable changes in response bias in a speech perception task. Minimal consonant-vowel stimulus pairs were presented between volume acquisitions for same-different discrimination. Speech stimuli were embedded in Gaussian noise at the psychophysically determined threshold level. We manipulated bias by changing the ratio of same-to-different trials: 1:3, 1:2, 1:1, 2:1, 3:1. Ratios were blocked by run and subjects were cued to the upcoming ratio at the beginning of each run. The stimuli were physically identical across runs. Response bias (criterion, C) was measured in individual subjects for each ratio condition. Group mean bias varied in the expected direction. We predicted that activation in frontal but not temporal brain regions would co-vary with bias. Group-level regression of bias scores on percent signal change revealed a fronto-parietal network of motor and sensory-motor brain regions that were sensitive to changes in response bias. We identified several pre- and post-central clusters in the left hemisphere that overlap well with TMS targets from the aforementioned studies. Importantly, activity in these regions covaried with response bias even while the perceptual targets remained constant. Thus, previous results suggesting that speech motor cortex participates directly in the perceptual analysis of speech should be called into question.

Changes in emotional state modulate neuronal firing rates of human speech motor cortex: A case study in long-term recording

In many brain areas, modulations in neuronal firing rates are thought to code information. However, in electrophysiological recording experiments, especially recordings in human patients, the type of information that is coded by a neuron's discharge patterns is often not known, or difficult to determine. From our long experience with chronic recordings in humans, we have come to suspect that such unexplained modulations in firing rates are often due to state changes in the subject. We here present two case studies, with extensive data in one subject to illustrate the point that a change in the subject's emotions, such as sudden fear, surprise, or happiness, may trigger substantial changes in firing rates.

Cortical Spatio-temporal Dynamics Underlying Phonological Target Detection in Humans

Selective processing of task-relevant stimuli is critical for goal-directed behavior. We used electrocorticography to assess the spatio-temporal dynamics of cortical activation during a simple phonological target detection task, in which subjects press a button when a prespecified target syllable sound is heard. Simultaneous surface potential recordings during this task revealed a highly ordered temporal progression of high gamma (HG, 70-200 Hz) activity across the lateral hemisphere in less than 1 sec. The sequence demonstrated concurrent regional sensory processing of speech syllables in the posterior superior temporal gyrus (STG) and speech motor cortex, and then transitioned to sequential task-dependent processing from prefrontal cortex (PFC), to the final motor response in the hand sensorimotor cortex. STG activation was modestly enhanced for target over nontarget sounds, supporting a selective gain mechanism in early sensory processing, whereas PFC was entirely selective to targets, supporting its role in guiding response behavior. These results reveal that target detection is not a single cognitive event, but rather a process of progressive target selectivity that involves large-scale rapid parallel and serial processing in sensory, cognitive, and motor structures to support goal-directed human behavior.

Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex.

We conducted a neurophysiological study of attempted speech production in a paralyzed human volunteer using chronic microelectrode recordings. The volunteer suffers from locked-in syndrome leaving him in a state of near-total paralysis, though he maintains good cognition and sensation. In this study, we investigated the feasibility of supervised classification techniques for prediction of intended phoneme production in the absence of any overt movements including speech. Such classification or decoding ability has the potential to greatly improve the quality-of-life of many people who are otherwise unable to speak by providing a direct communicative link to the general community. We examined the performance of three classifiers on a multi-class discrimination problem in which the items were 38 American English phonemes including monophthong and diphthong vowels and consonants. The three classifiers differed in performance, but averaged between 16 and 21% overall accuracy (chance-level is 1/38 or 2.6%). Further, the distribution of phonemes classified statistically above chance was non-uniform though 20 of 38 phonemes were classified with statistical significance for all three classifiers. These preliminary results suggest supervised classification techniques are capable of performing large scale multi-class discrimination for attempted speech production and may provide the basis for future communication prostheses.

Chapter 1 - Making the lifetime connection between brain and machine for restoring and enhancing function

A reliable neural interface that lasts a lifetime will lead to the development of neural prosthetic devices as well as the possibility that brain function can be enhanced. Our data demonstrate that a reliable neural interface is best achieved when the surrounding neuropil grows into the electrode tip where it is held securely, allowing myelinated axons to be recorded using implanted amplifiers. Stable single and multiunits were recorded from three implanted subjects and classified according to amplitudes and firing rates. In one paralyzed and mute subject implanted for over 5 years with a double electrode in the speech motor cortex, the single units allowed recognition of over half the 39 English language phonemes detected using a variety of decoding methods. These single units were used by the subject in a speech task where vowel phonemes were recognized and fed back to the subject using audio output. Weeks of training resulted in an 80% success rate in producing four vowels in an adaptation of the classic center-out task used in motor control studies. The importance of using single units was shown in a different task using pure tones that the same subject heard and then sung or hummed in his head. Feedback was associated with smoothly coordinated unit firings. The plasticity of the unit firings was demonstrated over several sessions first without, and then with, feedback. These data suggest that units can be reliably recorded over years, that there is an inverse relationship between single unit firing rate and amplitude, that pattern recognition decoding paradigms can allow phoneme recognition, that single units appear more important than multiunits when precision is important, and that units are plastic in their functional relationships. These characteristics of a reliable neural interface are essential for the development of neural prostheses and also for the future enhancement of human brain function.