Articulatory Movements Research Articles

Driving Digitalization: Action Research to Innovate Pronunciation Pedagogies by Building a Virtual Voice Laboratory

This paper explores the potential of digitalization in supporting English as a Second Language (ESL) students' development of pronunciation skills through a parallel action research. The author created a Virtual Voice Laboratory (VVL) platform to innovate pronunciation pedagogies in higher education; and simultaneously conducted an ongoing action research study to examine three research questions. The questions examined the efficacy of the VVL in facilitating blended learning, self-directed practice, and self-evaluation within a pronunciation course, and its impact on student engagement, participation, and pronunciation skill development. This VVL as a tailormade digital platform was implemented with undergraduate ESL students in Hong Kong's higher education context. It enabled blended learning, wherein students engaged with the mechanics and articulatory movements essential for improving English pronunciation through self-directed practice using the VVL. Adopting an action research methodology, the study investigated the process of implementing digitalization and blended learning strategies. This was achieved through the analysis of student recordings, sound chart comparisons, and qualitative-quantitative feedback collected during a pilot study. The findings reveal nuances of student experiences, their learning trajectories, and the results from the pilot study, illuminating the impacts and outcomes facilitated by the e-learning approach. The research holds valuable implications for integrating digitalization and blended learning in language education, contributing to the ongoing scholarly dialogue on the transformative potential of technology-mediated approaches in fostering dynamic and impactful learning experiences for diverse student populations.

Temporal Coordination of Articulatory Gestures in Nonnative Onset Clusters: Evidence From American English Speakers Using Electromagnetic Articulography.

Research in cross-language speech production indicates that, although the production of nonnative consonant clusters is often difficult, speakers of American English can produce some nonnative clusters (e.g., /fn/) with high accuracy. This ease of production for select nonnative clusters may occur due to similarity of phonetic structure with native clusters (e.g., nonnative /fn/ and native /sm/ are both fricative-nasal sequences). The current study tested this hypothesis by examining the extent of transfer of articulatory coordination from phonetically similar native onset clusters (i.e., /fl/, /sm/) to nonnative /fn/ clusters. Using electromagnetic articulography, lip, tongue, and jaw movements were recorded in nine participants during the production of 22 nonwords (eight tokens per nonword) containing the native and nonnative clusters in different carrier phrases. We examined the temporal lags between each consonantal gesture in a cluster and the flanking vowel gesture, which were compared to the matched singleton conditions. Analyses revealed that, as in native speech, when the syllable onset became more complex (i.e., CV ➔ CCV [C as consonant, V as vowel]), there was an increase in lag (less temporal overlap) between the leftmost consonantal gesture and the vocalic gesture, whereas there was a decrease in lag (more temporal overlap) between the rightmost consonant and the vocalic gesture (i.e., C-center timing). However, the segmental makeup of the cluster and type of carrier phrase used were also found to influence this change in temporal organization, raising new questions for future research. By and large, the findings are in agreement with the idea that the temporal coordination of articulator movements may be transferred from native clusters to phonetically similar nonnative clusters. However, kinematic measures of a broader range of nonnative clusters in different contexts are needed to fully explore this position.

Targeted deep brain stimulation of the motor thalamus improves speech and swallowing motor functions after cerebral lesions.

Speech and swallowing are complex motor acts that depend upon the integrity of input neural signals from motor cortical areas to control muscles of the head and neck. Lesions damaging these neural pathways result in weakness of key muscles causing dysarthria and dysphagia, leading to profound social isolation and risk of aspiration and suffocation. Here we show that Deep Brain Stimulation (DBS) of the motor thalamus improved speech and swallowing functions in two participants with dysarthria and dysphagia. First, we proved that DBS increased excitation of the face motor cortex, augmenting motor evoked potentials, and range and speed of motion of orofacial articulators in n = 10 volunteers with intact neural pathways. Then, we demonstrated that this potentiation led to immediate improvement in swallowing functions in a patient with moderate dysphagia and profound dysarthria as a consequence of a traumatic brain lesion. In this subject and in another with mild dysarthria, we showed that DBS immediately ameliorated impairments of respiratory, phonatory, resonatory, and articulatory control thus resulting in a clinically significant improvement in speech intelligibility. Our data provide first-in-human evidence that DBS can be used to treat dysphagia and dysarthria in people with cerebral lesions.

EXPRESS: Prior multisensory learning can facilitate auditory-only voice-identity and speech recognition in noise.

Seeing the visual articulatory movements of a speaker, while hearing their voice, helps with understanding what is said. This multisensory enhancement is particularly evident in noisy listening conditions. Multisensory enhancement also occurs even in auditory-only conditions: auditory-only speech and voice-identity recognition is superior for speakers previously learned with their face, compared to control learning; an effect termed the "face-benefit". Whether the face-benefit can assist in maintaining robust perception in increasingly noisy listening conditions, similar to concurrent multisensory input, is unknown. Here, in two behavioural experiments, we examined this hypothesis. In each experiment, participants learned a series of speakers' voices together with their dynamic face, or control image. Following learning, participants listened to auditory-only sentences spoken by the same speakers and recognised the content of the sentences (speech recognition, Experiment 1) or the voice-identity of the speaker (Experiment 2) in increasing levels of auditory noise. For speech recognition, we observed that 14/30 participants (47%) showed a face-benefit. While 19/25 participants (76%) showed a face-benefit for voice-identity recognition. For those participants who demonstrated a face-benefit, the face-benefit increased with auditory noise levels. Taken together, the results support an audio-visual model of auditory communication and suggest that the brain can develop a flexible system in which learned facial characteristics are used to deal with varying auditory uncertainty.

The encoding of prominence relations in supra-laryngeal articulation across speaking styles

Speakers modify their supra-laryngeal articulation on the one hand as a function of prosodic prominence and on the other hand as a function of speaking style. Articulatory movements are spatio-temporally expanded in words that are highlighted due to prosodic strengthening, but also in utterances that are produced in a loud speaking style. What remains unclear, to date, is how these two phenomena interact, or, in other words, how prominence relations are encoded in loud speech. In this paper, we use electromagnetic articulography to investigate lip and tongue kinematics and analyze selected acoustic parameters across several focus conditions in habitual and loud speech. Prosodic prominence relations are studied as the simultaneous strengthening of highlighted words and the attenuation of words in the background. The results show that prominence relations can be encoded multidimensionally in loud speech. Some modifications are even stronger in loud speech than in habitual speech, despite the general articulatory modifications associated with a loud speaking style. In both speaking styles, the attenuation of words in the background tends to be more robust than the strengthening of highlighted words. The paper underlines the flexibility of the multidimensional phonetic space in the encoding of prosodic prominence across varying communicative demands.

Immediate auditory feedback regulates inter-articulator speech coordination in service to phonetic structure.

Research has shown that talkers reliably coordinate the timing of articulator movements across variation in production rate and syllable stress, and that this precision of inter-articulator timing instantiates phonetic structure in the resulting acoustic signal. We here tested the hypothesis that immediate auditory feedback helps regulate that consistent articulatory timing control. Talkers with normal hearing recorded 480 /tV#Cat/ utterances using electromagnetic articulography, with alternative V (/ɑ/-/ɛ/) and C (/t/-/d/), across variation in production rate (fast-normal) and stress (first syllable stressed-unstressed). Utterances were split between two listening conditions: unmasked and masked. To quantify the effect of immediate auditory feedback on the coordination between the jaw and tongue-tip, the timing of tongue-tip raising onset for C, relative to the jaw opening-closing cycle for V, was obtained in each listening condition. Across both listening conditions, any manipulation that shortened the jaw opening-closing cycle reduced the latency of tongue-tip movement onset, relative to the onset of jaw opening. Moreover, tongue-tip latencies were strongly affiliated with utterance type. During auditory masking, however, tongue-tip latencies were less strongly affiliated with utterance type, demonstrating that talkers use afferent auditory signals in real-time to regulate the precision of inter-articulator timing in service to phonetic structure.

The impact of Playdough games on the development of oral motor skills in preschool children with childhood apraxia of speech

The relevant challenges are the use of innovative approaches that conserve both the speech therapist’s and the child’s resources while addressing multiple goals in speech therapy. The authors of this article have adopted such an approach to develop oral motor skills, which are crucial for speech development in preschoolers, particularly those with childhood apraxia of speech. This study aimed to evaluate the effectiveness of an author-developed therapy using Playdough games for preschool children with childhood apraxia of speech. A controlled design with repeated measures was employed. The primary diagnosis of CAS was established based on clinical assessment. A comparative evaluation of two groups (control and experimental) was conducted using a developed methodology for assessing the level of manual and oral motor skills in preschool children before and after the author-developed therapy using Playdough games. Researchers discovered that the systematic use of their author-developed Playdough games positively impacted the motor skills of children with CAS. Children in the experimental group, who participated in Playdough therapy, demonstrated significant improvements in both manual and oral motor skills, increased precision and focus in movement execution, and enhanced accuracy and speed of articulatory movements. There was a reduction in delays and incorrect movement sequences during finger movement tasks. Children required less additional time and significantly less assistance from the speech therapist, indicating a positive impact of using Playdough in therapy. The following skills improved: transitions between articulatory positions, which helped children avoid coarticulation difficulties; planning of movement sequences; and smooth transitions between sounds and syllables. The practical significance of this study lies in the effectiveness of using Playdough games for significantly improving oral motor skills, movement precision, and concentration in children with childhood apraxia of speech, thereby reducing the need for additional speech therapy support

Audio–visual deepfake detection using articulatory representation learning

Advancements in generative artificial intelligence have made it easier to manipulate auditory and visual elements, highlighting the critical need for robust audio–visual deepfake detection methods. In this paper, we propose an articulatory representation-based audio–visual deepfake detection approach, ART-AVDF. First, we devise an audio encoder to extract articulatory features that capture the physical significance of articulation movement, integrating with a lip encoder to explore audio–visual articulatory correspondences in a self-supervised learning manner. Then, we design a multimodal joint fusion module to further explore inherent audio–visual consistency using the articulatory embeddings. Extensive experiments on the DFDC, FakeAVCeleb, and DefakeAVMiT datasets demonstrate that ART-AVDF obtains a significant performance improvement compared to many deepfake detection models.

Prosody matters: Preserved prominence marking strategies in people with Parkinson's disease independent of motor status.

While many studies focus on segmental variation in Parkinsonian speech, little is known about prosodic modulations reflecting the ability to adapt to communicative demands in people with Parkinson's disease (PwPD). This type of prosodic modulation is important for social interaction, and it involves modifications in speech melody (intonational level) and articulation of consonants and vowels (segmental level). The present study investigates phonetic cues of prosodic modulations with respect to different focus structures in mild dysarthric PwPD as a function of levodopa. Acoustic and kinematic speech parameters of 25 PwPD were assessed in two motor conditions. Speech production data from PwPD were collected before (medication-OFF) and after levodopa intake (medication-ON) by means of 3-D electromagnetic articulography. On the acoustic level, intensity, pitch, and syllable durations were analyzed. On the kinematic level, movement duration and amplitude were investigated. Spatio-temporal modulations of speech parameters were examined and compared across three different prosodic focus structures (out-of-focus, broad focus, contrastive focus) to display varying speech demands. Overall, levodopa had beneficial effects on motor performance, speech loudness, and pitch modulation. Acoustic syllable durations and kinematic movement durations did not change, revealing no systematic effects of motor status on the temporal domain. In contrast, there were spatial modulations of the oral articulators: tongue tip movements were smaller and lower lip movements were larger in amplitude under levodopa, reflecting a more agile and efficient articulatory movement under levodopa. Thus, respiratory-phonatory functions and consonant production improved, while syllable duration and tongue body kinematics did not change. Interestingly, prominence marking strategies were comparable between the medication conditions under investigation, and in fact, appear to be preserved in mild dysarthric PwPD.

Is the Articulatory Trajectory of Changing Syllables Important for Achieving Higher Syllable Rates Compared to Repeated Syllables?

The production of speech-like sequences composed of varying syllables has been reported to achieve higher syllable rates than the production of repeated syllables (commonly designed as sequential motion rate [SMR] and alternating motion rate [AMR] tasks, respectively). The faster rate for SMR relative to AMR sequences is explained by different interpretative hypotheses, which remain empirically unexplored. In the present study, we aimed to investigate whether the high syllable rates of SMR sequences are due to the specific sequences used in most studies that involve front-to-back articulatory movements. Syllable rates of SMR sequences composed of different articulatory trajectories (i.e., labial-alveolar-velar [/pateko/], alveolar-velar-labial [/tekopa/], and velar-labial-alveolar [/kopate/]) were compared with those of the AMR sequences /papapa/, /tetete/, and /kokoko/ in 20 participants. The results show higher syllable rates for each of the three SMR sequences as compared to AMR, suggesting that the trajectory of the articulatory movements in the sequential sequences is not the key to achieving higher syllable rates. The syllable rate advantage for SMR over AMR sequences is not explained by the articulatory trajectories included in the former task, indicating that the front-to-back movements generally included in SMR sequences (i.e., /pataka/) are not decisive in achieving a higher syllable rate.

Speaker-independent speech inversion for recovery of velopharyngeal port constriction degreea).

For most of his illustrious career, Ken Stevens focused on examining and documenting the rich detail about vocal tract changes available to listeners underlying the acoustic signal of speech. Current approaches to speech inversion take advantage of this rich detail to recover information about articulatory movement. Our previous speech inversion work focused on movements of the tongue and lips, for which "ground truth" is readily available. In this study, we describe acquisition and validation of ground-truth articulatory data about velopharyngeal port constriction, using both the well-established measure of nasometry plus a novel technique-high-speed nasopharyngoscopy. Nasometry measures the acoustic output of the nasal and oral cavities to derive the measure nasalance. High-speed nasopharyngoscopy captures images of the nasopharyngeal region and can resolve velar motion during speech. By comparing simultaneously collected data from both acquisition modalities, we show that nasalance is a sufficiently sensitive measure to use as ground truth for our speech inversion system. Further, a speech inversion system trained on nasalance can recover known patterns of velopharyngeal port constriction shown by American English speakers. Our findings match well with Stevens' own studies of the acoustics of nasal consonants.

Imagined speech event detection from electrocorticography and its transfer between speech modes and subjects

Speech brain-computer interfaces aim to support communication-impaired patients by translating neural signals into speech. While impressive progress was achieved in decoding performed, perceived and attempted speech, imagined speech remains elusive, mainly due to the absence of behavioral output. Nevertheless, imagined speech is advantageous since it does not depend on any articulator movements that might become impaired or even lost throughout the stages of a neurodegenerative disease. In this study, we analyzed electrocortigraphy data recorded from 16 participants in response to 3 speech modes: performed, perceived (listening), and imagined speech. We used a linear model to detect speech events and examined the contributions of each frequency band, from delta to high gamma, given the speech mode and electrode location. For imagined speech detection, we observed a strong contribution of gamma bands in the motor cortex, whereas lower frequencies were more prominent in the temporal lobe, in particular of the left hemisphere. Based on the similarities in frequency patterns, we were able to transfer models between speech modes and participants with similar electrode locations.

Memória, discurso e universo acadêmico

This article analyzes the situation, aiming to inaugurate reflections on the following questions: under what vital conditions is knowledge produced in the space-time of neoliberal universities; whether there is gender inequality in these conditions; how it is felt and experienced by bodies that accumulate multiple exclusions; what socio-epistemological consequences this entails; and how to resist and act to reverse this situation. The method used is to review bibliographic and information sources. As a result, the article makes the first movement of articulation and systematization of statements. They are precariousness, acceleration, suffering, gender blindness, substantive (in)equality, epistemic violence, feminist epistemology, social memory and French discourse analysis. In a detailed and sustained manner, the conclusion presents the guidelines for carrying out future empirical studies.

Perceptual Adaptation to Noise-Vocoded Speech by Lip-Read Information: No Difference between Dyslexic and Typical Readers.

Auditory speech can be difficult to understand but seeing the articulatory movements of a speaker can drastically improve spoken-word recognition and, on the longer-term, it helps listeners to adapt to acoustically distorted speech. Given that individuals with developmental dyslexia (DD) have sometimes been reported to rely less on lip-read speech than typical readers, we examined lip-read-driven adaptation to distorted speech in a group of adults with DD ( N = 29) and a comparison group of typical readers ( N = 29). Participants were presented with acoustically distorted Dutch words (six-channel noise-vocoded speech, NVS) in audiovisual training blocks (where the speaker could be seen) interspersed with audio-only test blocks. Results showed that words were more accurately recognized if the speaker could be seen (a lip-read advantage), and that performance steadily improved across subsequent auditory-only test blocks (adaptation). There were no group differences, suggesting that perceptual adaptation to disrupted spoken words is comparable for dyslexic and typical readers. These data open up a research avenue to investigate the degree to which lip-read-driven speech adaptation generalizes across different types of auditory degradation, and across dyslexic readers with decoding versus comprehension difficulties.

Representation of internal speech by single neurons in human supramarginal gyrus

Speech brain–machine interfaces (BMIs) translate brain signals into words or audio outputs, enabling communication for people having lost their speech abilities due to diseases or injury. While important advances in vocalized, attempted and mimed speech decoding have been achieved, results for internal speech decoding are sparse and have yet to achieve high functionality. Notably, it is still unclear from which brain areas internal speech can be decoded. Here two participants with tetraplegia with implanted microelectrode arrays located in the supramarginal gyrus (SMG) and primary somatosensory cortex (S1) performed internal and vocalized speech of six words and two pseudowords. In both participants, we found significant neural representation of internal and vocalized speech, at the single neuron and population level in the SMG. From recorded population activity in the SMG, the internally spoken and vocalized words were significantly decodable. In an offline analysis, we achieved average decoding accuracies of 55% and 24% for each participant, respectively (chance level 12.5%), and during an online internal speech BMI task, we averaged 79% and 23% accuracy, respectively. Evidence of shared neural representations between internal speech, word reading and vocalized speech processes was found in participant 1. SMG represented words as well as pseudowords, providing evidence for phonetic encoding. Furthermore, our decoder achieved high classification with multiple internal speech strategies (auditory imagination/visual imagination). Activity in S1 was modulated by vocalized but not internal speech in both participants, suggesting no articulator movements of the vocal tract occurred during internal speech production. This work represents a proof-of-concept for a high-performance internal speech BMI.

Optimization-based planning of speech articulation using general Tau Theory

This paper presents a model of speech articulation planning and generation based on General Tau Theory and Optimal Control Theory. Because General Tau Theory assumes that articulatory targets are always reached, the model accounts for speech variation via context-dependent articulatory targets. Targets are chosen via the optimization of a composite objective function. This function models three different task requirements: maximal intelligibility, minimal articulatory effort and minimal utterance duration. The paper shows that systematic phonetic variability can be reproduced by adjusting the weights assigned to each task requirement. Weights can be adjusted globally to simulate different speech styles, and can be adjusted locally to simulate different levels of prosodic prominence. The solution of the optimization procedure contains Tau equation parameter values for each articulatory movement, namely position of the articulator at the movement offset, movement duration, and a parameter which relates to the shape of the movement’s velocity profile. The paper presents simulations which illustrate the ability of the model to predict or reproduce several well-known characteristics of speech. These phenomena include close-to-symmetric velocity profiles for articulatory movement, variation related to speech rate, centralization of unstressed vowels, lengthening of stressed vowels, lenition of unstressed lingual stop consonants, and coarticulation of stop consonants.

Impact of Bilingualism on Speech Sound Disorder/ Articulatory and Phonological Disorders

Impact of Bilingualism on Speech Sound Disorder/ Articulatory and Phonological Disorders

Acoustic Effects of Speaker Sex, Speech Sample, and Mandarin Tone on Vowel Production of Poststroke Spastic Dysarthria

Plain Language SummaryMandarin speakers with poststroke spastic dysarthria often have vowel production deficits that affect articulation and speech intelligibility. Vowel production is influenced by many factors. This study explored the acoustic effects of speaker sex, tones, and speech samples and their interactions on vowel production in Mandarin speakers with poststroke spastic dysarthria and confirmed the existence of these effects. We found that regardless of the speech samples or tones, there were the reduced range of articulatory movements, centralized vowel production, and prolonged vowel duration in dysarthria compared to healthy people. The most sensitive and useful analysis measures were the vowel articulation index and vowel duration. Specific influencing factors (speaker sex, speech sample, and tone) of vowel production need to be taken seriously by speech and language pathologists and considered in the assessment and rehabilitation.

Segmental and Syllabic Articulations: A Descriptive Approach.

This article reports on jaw articulation, in connection with the segmental articulations of tongue tip and lips. The jaw is considered a syllable articulator as it opens and closes for each syllable, and the amount of jaw opening is related to syllable strength, given the same phonemic vowel. Here, we investigate the syllabic and the segmental articulations relationships to the acoustic segments. We analyze electromagnetic articulography data of 18 Southern Swedish speakers by testing hypotheses based on the Descriptive Approach to Segmental Articulations. In this approach, articulatory movements are divided into fast intervals and steady states, and the crucial ("primary") articulators' deceleration and acceleration peaks align with the acoustic segment boundaries. Previous findings on the lips and the tongue tip are confirmed. New results are the jaw as noncrucial ("nonprimary") does not align with the acoustic segment boundaries: Its steady states are shorter than consonant and vowel segments. Moreover, place of articulation of the coda affects the jaw cycle, while syllable type only affects jaw closing. No such effects are found on the edges of the jaw cycle, instead only gender affects jaw deceleration at syllable onset. The jaw is not crucial for the timing of the acoustic segment boundaries; also, syllable borders seem to not be affected by place of segmental articulation. The results strengthen the view of the jaw as syllable articulator providing a frame and the other articulators (e.g., lips and tongue tip) as the segmental articulators providing the content.

Prospectively accelerated dynamic speech magnetic resonance imaging at 3 T using a self-navigated spiral-based manifold regularized scheme.

This work develops and evaluates a self-navigated variable density spiral (VDS)-based manifold regularization scheme to prospectively improve dynamic speech magnetic resonance imaging (MRI) at 3 T. Short readout duration spirals (1.3-ms long) were used to minimize sensitivity to off-resonance. A custom 16-channel speech coil was used for improved parallel imaging of vocal tract structures. The manifold model leveraged similarities between frames sharing similar vocal tract postures without explicit motion binning. The self-navigating capability of VDS was leveraged to learn the Laplacian structure of the manifold. Reconstruction was posed as a sensitivity-encoding-based nonlocal soft-weighted temporal regularization scheme. Our approach was compared with view-sharing, low-rank, temporal finite difference, extra dimension-based sparsity reconstruction constraints. Undersampling experiments were conducted on five volunteers performing repetitive and arbitrary speaking tasks at different speaking rates. Quantitative evaluation in terms of mean square error over moving edges was performed in a retrospective undersampling experiment on one volunteer. For prospective undersampling, blinded image quality evaluation in the categories of alias artifacts, spatial blurring, and temporal blurring was performed by three experts in voice research. Region of interest analysis at articulator boundaries was performed in both experiments to assess articulatory motion. Improved performance with manifold reconstruction constraints was observed over existing constraints. With prospective undersampling, a spatial resolution of 2.4 × 2.4 mm2/pixel and a temporal resolution of 17.4 ms/frame for single-slice imaging, and 52.2 ms/frame for concurrent three-slice imaging, were achieved. We demonstrated implicit motion binning by analyzing the mechanics of the Laplacian matrix. Manifold regularization demonstrated superior image quality scores in reducing spatial and temporal blurring compared with all other reconstruction constraints. While it exhibited faint (nonsignificant) alias artifacts that were similar to temporal finite difference, it provided statistically significant improvements compared with the other constraints. In conclusion, the self-navigated manifold regularized scheme enabled robust high spatiotemporal resolution dynamic speech MRI at 3 T.