Speech Movements Research Articles

Predicting Cognitive Load and Operational Performance in a Simulated Marksmanship Task.

Modern operational environments can place significant demands on a service member's cognitive resources, increasing the risk of errors or mishaps due to overburden. The ability to monitor cognitive burden and associated performance within operational environments is critical to improving mission readiness. As a key step toward a field-ready system, we developed a simulated marksmanship scenario with an embedded working memory task in an immersive virtual reality environment. As participants performed the marksmanship task, they were instructed to remember numbered targets and recall the sequence of those targets at the end of the trial. Low and high cognitive load conditions were defined as the recall of three- and six-digit strings, respectively. Physiological and behavioral signals recorded included speech, heart rate, breathing rate, and body movement. These features were input into a random forest classifier that significantly discriminated between the low- and high-cognitive load conditions (AUC = 0.94). Behavioral features of gait were the most informative, followed by features of speech. We also showed the capability to predict performance on the digit recall (AUC = 0.71) and marksmanship (AUC = 0.58) tasks. The experimental framework can be leveraged in future studies to quantify the interaction of other types of stressors and their impact on operational cognitive and physical performance.

Multimodal Coordination of Sound and Movement in Music and Speech

ABSTRACT Speech and music emerge from a spectrum of nested motor and perceptual coordination patterns across timescales of brief movements to actions. Intuitively, this nested clustering in movements should be reflected in sound. We examined similarities and differences in multimodal, multiscale coordination of speech and music using two complementary measures: We computed spectra for envelopes of acoustic amplitudes and motion amplitudes and correlated spectral powers across modalities as a function of frequency. We also correlated smoothed envelopes and examined peaks in their cross-correlation functions. YouTube videos of five different modes of speaking and five different types of music were analyzed. Speech performances yielded stronger, more reliable relationships between sound and movement compared with music. Interestingly, a cappella singing patterned more with music, and improvisational jazz piano patterned more with speech. Results suggest that nested temporal structures in sound and movement are coordinated as a function of communicative aspects of performance.

Performing virtual bodies

ABSTRACT Following the technical advancement of cross-reality platforms – including virtual, mixed and augmented reality, the term ‘virtual bodies’ has become widespread. Despite its increased use, however, there has not been much investigation of ‘virtual bodies’ taking into account a cross-cultural approach that links traditional methods and digital outcomes. This article will demonstrate the methods of experiencing virtual bodies, in the context of Korean healing ritual, along with discussions of my avatar life-review projects, in the phase of wishing, facing, creating and performing virtual bodies. Since 2016, I have been working on avatar life-review platforms that have a mirrored body movement and lip-synchronization of live speech. This platform was implemented in two projects: in the Voicing Elder (2016), the avatar life-review for older adults at Gayton Terrace Senior Living, Richmond, VA; The Voicing Han (2019), the avatar life-review for cancer patients, at VCU palliative care, Massey cancer center, Richmond, VA. These avatar life-review sessions will be discussed as examples of wishing, facing, creating and performing virtual bodies, to proactively experiment, design and facilitate emotional and therapeutic experiences. It will enrich the potential of technically mediated bodies in digital space, supported by the constant development of virtual, interactive and simulation technologies.

Apkinson: the smartphone application for telemonitoring Parkinson's patients through speech, gait and hands movement.

Aim: This paper introduces Apkinson, a mobile application for motor evaluation and monitoring of Parkinson's disease patients. Materials & methods: The App is based on previously reported methods, for instance, the evaluation of articulation and pronunciation in speech, regularity and freezing of gait in walking, and tapping accuracy in hand movement. Results: Preliminary experiments indicate that most of the measurements are suitable to discriminate patients and controls. Significance is evaluated through statistical tests. Conclusion: Although the reported results correspond to preliminary experiments, we think that Apkinson is a very useful App that can help patients, caregivers and clinicians, in performing a more accurate monitoring of the disease progression. Additionally, the mobile App can be a personal health assistant.

Integration or Separation? Malcolm X’s College Debates, Free Speech, and the Challenge to Racial Liberalism on Campus

Abstract Malcolm X participated in over thirty speaking engagements at prominent colleges and universities between 1960 and 1963. His popularity on campuses coincided with a new epoch of civil rights struggle as students became involved in Freedom Rides and sit-in campaigns to desegregate lunch counters and interstate travel. Most invitations were debates on the topic “Integration or Separation?” which pitted Malcolm against an integrationist opponent. The insertion of racial separatism into a discourse that took integration as an unquestioned aim of the movement pushed students to question and defend their own understandings of racial liberalism. Nearly a dozen invitations were extended by NAACP student chapters that had been revitalized amid the new flurry of student involvement. Years before the founding of the first Black Student Union (BSU) at San Francisco State, these chapters were far more ideologically diverse and active than their forbearers, and often invited Malcolm X to speak out of a commitment to students’ rights to free speech and academic freedom. When administrations blocked and cancelled his visits, students became politicized around issues of academic liberties, thereby situating the Nation of Islam and Malcolm X at the nexus of early debates within the student free speech movement. These became part of the early challenge to university paternalism. While these debates and lectures have often been discussed individually, this essay looks at their cumulative effect by situating them during the emergence of student radicalism on campus and the growth of youth participation in the civil rights movement.

Changes of orofacial somatosensory attenuation during speech production

Modulation of auditory activity occurs before and during voluntary speech movement. However, it is unknown whether orofacial somatosensory input is modulated in the same manner. The current study examined whether or not the somatosensory event-related potentials (ERPs) in response to facial skin stretch are changed during speech and nonspeech production tasks. Specifically, we compared ERP changes to somatosensory stimulation for different orofacial postures and speech utterances. Participants produced three different vowel sounds (voicing) or non-speech oral tasks in which participants maintained a similar posture without voicing. ERP’s were recorded from 64 scalp sites in response to the somatosensory stimulation under six task conditions (three vowels × voicing/posture) and compared to a resting baseline condition. The first negative peak for the vowel /u/ was reliably reduced from the baseline in both the voicing and posturing tasks, but the other conditions did not differ. The second positive peak was reduced for all voicing tasks compared to the posturing tasks. The results suggest that the sensitivity of somatosensory ERP to facial skin deformation is modulated by the task and that somatosensory processing during speaking may be modulated differently relative to phonetic identity.

SAT-LB50 Fahr’s Syndrome: A Rare Neurological Disorder Unmasked by a Psychiatric Illness

Fahr’s syndrome is a rare familial disorder characterized by abnormal accumulation of calcium deposits bilaterally at basal ganglia. It commonly affects middle-aged adults and presents with a range of neuropsychiatric symptoms. The exact prevalence of Fahr’s syndrome is uncertain; however, intracranial calcifications suggestive of this disorder are detected incidentally in approximately 0.3 % to 1.2 % of CT imaging of the brain with a prevalence of 1/1,000,000. It may be idiopathic or secondary to numerous causes dominated by phosphorous and calcium disorders, with the most common etiology being hypoparathyroidism. We report the case of a 27 years old female patient with a medical history of insulin-dependent Diabetes Mellitus type 1, Bipolar disorder, Autoimmune Polyglandular Syndrome Type 1, Thalassemia major, Primary Hypoparathyroidism and Bronchial Asthma who was admitted to the hospital after presenting an episode of dizziness, slurred speech and involuntary movements associated to hypoglycemia. The patient had a medical history of recurrent episodes of conscious self-induced hypoglycemia with double doses of insulin therapy and noncompliance with home medications. Upon evaluation, patient presents aggressive and defiant behavior. Physical and neurological examination was difficult to assess since she refused to be examined. Laboratories were remarkable for serum calcium of 6.2mg/dl, albumin of 3.5g/dl, with corrected calcium levels of 6.5mg/dl, suggestive of severe hypercalcemia. Head CT scan showed bilateral subcortical, basal ganglia clouded, thalamic, and cerebellar calcifications with preserved gray and white matter differentiation. Treatment was tailored to symptoms control and correction of underlying abnormalities. These case present the most critical features of the diagnostic criteria of Fahr’s syndrome. Pathologically, calcifications occur in the vascular walls and in the perivascular spaces of arterioles, capillaries, and veins. Clinical findings of Fahr’s syndrome vary from neurological disorder to those mimicking Bipolar disorder. In this case, there were no neurological symptoms, and this patient only presented with psychiatric manifestations suggestive of bipolar disorder. For any psychiatric condition, it is essential to rule out organic brain disorders before labeling a patient, especially one who is young and has multiple endocrinopathies which could be associated with this rare condition.

Mild encephalitis/encephalopathy with a reversible splenial lesion (MERS) following influenza virus infection

A 50-year-old man presented to the emergency department with fever, altered mental status, incoherent speech and involuntary movements. He had a high-grade fever with upper respiratory symptoms 9 days prior,...

Databases, Features and Classification Techniques for Speech Emotion Recognition

Emotion recognition is a rapidly growing research field. Emotions can be effectively expressed through speech and can provide insight about speaker’s intentions. Although, humans can easily interpret emotions through speech, physical gestures, and eye movement but to train a machine to do the same with similar preciseness is quite a challenging task. SER systems can improve human-machine interaction when used with automatic speech recognition, as emotions have the tendency to change the semantics of a sentence. Many researchers have contributed their extremely impressive work in this research area, leading to development of numerous classification, feature selection, feature extraction and emotional speech databases. This paper reviews recent accomplishments in the area of speech emotion recognition. It also present a detailed review of various types of emotional speech databases, and different classification techniques which can be used individually or in combination and a brief description of various speech features for emotion recognition.

Speech-Induced Suppression for Delayed Auditory Feedback in Adults Who Do and Do Not Stutter

Speech-induced suppression is the normal, relative amplitude reduction of the auditory evoked potential for self-, compared to externally-generated, auditory stimulation. It remains controversial as to whether adults who stutter exhibit expected auditory modulation during speech; some studies have reported a significant difference between stuttering and fluent groups in speech-induced suppression during speech movement planning, while others have not. We compared auditory evoked potentials (N1 component) for auditory feedback arising from one’s own voice (Speaking condition) with passive listening to a recording of one’s own voice (Listening condition) in 24 normally-fluent speakers and 16 adults who stutter under various delayed auditory feedback (DAF) time conditions (100 ms, 200 ms, 500 ms, and 1,000 ms). We presented the participant’s own voice with a delay, immediately after presenting it without a delay. Our working hypothesis was that the shorter the delay time, the more likely the delayed sound is perceived as self-generated. Therefore, shorter delay time conditions are proposed to result in relatively enhanced suppression of the auditory system. Results showed that in fluent speakers, the shorter the delay time, the more the auditory evoked potential in the Speaking condition tended to be suppressed. In the Listening condition, there was a larger evoked potential with shorter delay times. As a result, speech-induced suppression was only significant at the short delay time conditions of 100 and 200 ms. Adults who stutter did not show the opposing changes in the Speaking and Listening conditions seen in the fluent group. Although the evoked potential in the Listening condition tended to decrease as the delay time increased, that in the Speaking condition did not show a distinct trend, and there was a significant suppression only at 200 ms delay. For the 200 ms delay condition, speakers with more severe stuttering showed significantly greater speech-induced suppression than those with less severe stuttering. This preliminary study suggests our methods for investigating evoked potentials by presenting own voice with a delay may provide a clue as to the nature of auditory modulation in stuttering.

Robust algorithm for remote photoplethysmography in realistic conditions

Over the last decade remote photoplethysmography (rPPG) algorithms have been developed extensively. As a result, pulse rate can now be accurately estimated by video data for still subjects. However, in realistic conditions both the accuracy of these algorithms and benchmark datasets are far from perfect. In this paper we propose a new rPPG method which enables heart rate detection by video from a standard webcam. The algorithm is robust with respect to such factors as illumination changes or the subject's movements, and can track fast pulse rate changes. To do that, the algorithm determines the approximate value of the pulse rate and then specifies it with high time resolution. In order to comprehensively study the proposed method, we collected a new dataset consisting of videos recorded in various challenging conditions of several categories as well as reference photoplethysmograms recorded synchronously with a contact pulse oximeter. The proposed method showed high performance under all conditions including blinking illumination, speech and large-amplitude movements. We tested two simplified versions of the algorithm, which provided competitive scores as well. However, with human movement videos the full method showed better results than its simplified versions (p<0.001). The proposed algorithm was tested on the existing UBFC-RPPG database and compared with previous methods. Our method showed high results (2.10 MAE, 3.43 RMSE).

Development of a Psychometric Measure of the Propensity to Consciously Control and Monitor Speech Production.

Purpose A speech-specific reinvestment scale (SSRS) is a psychometric measure of the propensity to consciously control and monitor speech production. This study develops and validates an SSRS as well as examines its relationship with speech performance with the moderating effects of trait social anxieties (i.e., social interaction anxiety, public speaking anxiety, and social phobia). Method Scale development involves the following stages: (a) initial item generation based on relevant literature, (b) item evaluation through cognitive interviews with 24 healthy respondents, (c) scale reliability and validity tests using cross-sectional survey data from 498 healthy respondents, and (d) test-retest reliability assessment using longitudinal survey data from 185 healthy respondents. Respondents' speech performance is quantified using speech examination scores. Hierarchical moderated regression analyses are conducted to examine the moderating effects of trait social anxieties. Results The validated SSRS comprises 35 items, which can be categorized into four subdimensions, namely, speech movement self-consciousness, public consciousness of speech content, speech manner, and speech movement. Results show that respondents with low trait social anxieties indicate a generally positive relationship between public consciousness of speech movement and speech performance, whereas respondents with high trait social anxieties exhibit a nonsignificant relationship. Conclusions SSRS offers a reliable and valid method for assessing the predisposition for conscious speech control and monitoring, which plays a role in speech performance and is moderated by an individual's level of trait social anxiety. SSRS is a potential assessment tool for speech-language pathologists to evaluate the impacts of conscious speech control and monitoring on individuals with speech impairment.

Harsh adjustments: speech hygiene for disabled competitors in individual events

Disabled competitors face unique challenges in competitive forensics individual events (IE). However, formal and informal institutional forces often erase their stories or worsen their situations. Although the lived experiences of disabled speakers have been studied and celebrated in disability performance art and other non-forensics performance spaces, IE has not caught up largely due to the historical shadow of the speech hygiene movement. Through a confrontation with our activity’s past and a community autoethnography of disabled competitors from the last decade, we argue that this history must be uncovered and interrogated in order to mitigate harms and promote positive change in the IE community.

The perils of learning to move while speaking: One-sided interference between speech and visuomotor adaptation.

Our understanding of the adaptive processes that shape sensorimotor behavior is largely derived from studying isolated movements. Studies of visuomotor adaptation, in which participants adapt cursor movements to rotations of the cursor's screen position, have led to prominent theories of motor control. In response to changes in visual feedback of movements, explicit (cognitive) and implicit (automatic) learning processes adapt movements to counter errors. However, movements rarely occur in isolation. The extent to which explicit and implicit processes drive sensorimotor adaptation when multiple movements occur simultaneously, as in the real world, remains unclear. Here we address this problem in the context of speech and hand movements. Participants spoke in-time with rapid, hand-driven cursor movements. Using real-time alterations of vowel sound feedback, and visual rotations of the cursor's screen position, we induced sensorimotor adaptation in one or both movements simultaneously. Across three experiments (n = 60, n = 48 and n = 76, respectively), we demonstrate that visuomotor adaptation is markedly impaired by simultaneous speech adaptation, and the impairment is specific to the explicit learning process in visuomotor adaptation. In contrast, visuomotor adaptation had no impact on speech adaptation. The results demonstrate that the explicit learning process in visuomotor adaptation is sensitive to movements in other motor domains. They suggest that some forms of speech adaptation may lack an explicit learning process altogether.

Simplified Model of PKCγ Signaling Dysregulation and Cytosol-to-Membrane Translocation Kinetics During Neurodegenerative Spinocerebellar Ataxia Type 14 (SCA14).

Spinocerebellar ataxia type 14 (SCA14) is an autosomal neurodegenerative disease clinically characterized by progressive ataxia in the patient’s gait, accompanied by slurred speech and abnormal eye movements. These symptoms are linked to the loss of Purkinje cells (PCs), which leads to cerebellar neurodegeneration. PC observations link the mutations in PRKCG gene encoding protein kinase C γ (PKCγ) to SCA14. Observations also show that the link between PKCγ and SCA14 relies on a gain-of-function mechanism, and, in fact, both positive and negative regulation of PKCγ expression and activity may result in changes in cellular number, size, and complexity of the dendritic arbors in PCs. Here, through a systems biology approach, we investigate a key question relating to this system: why is PKCγ membrane residence time reduced in SCA14 mutant PCs compared to wild-type (WT) PCs? In this study, we investigate this question through two contrasting PKCγ signaling models in PCs. The first model proposed in this study describes the mechanism through which PKCγ signaling activity may be regulated in WT PCs. In contrast, the second model explores how mutations in PKCγ signaling affect the state of SCA14 in PCs. Numerical simulations of both models show that, in response to extracellular stimuli-induced depolarization of the membrane compartment, PKCγ and diacylglycerol kinase γ (DGKγ) translocate to the membrane. Results from our computational approach indicate that, for the same set of parameters, PKCγ membrane residence time is shorter in the SCA14 mutant model compared to the WT model. These results show how PKCγ membrane residence time is regulated by diacylglycerol (DAG), causing translocated PKCγ to return to the cytosol as DAG levels drop. This study shows that, when the strength of the extracellular signal is held constant, the membrane lifetime of mutant PKCγ is reduced. This reduction is due to the presence of constitutively active mutant PKCγ in the cytosol. Cytosolic PKCγ, in turn, leads to phosphorylation and activation of DGKγ while it is still residing in the cytosol. This effect occurs even during the resting conditions. Thus, the SCA14 mutant model explains that, when both DAG effector molecules are active in the cytosol, their interactions in the membrane compartment are reduced, critically influencing PKCγ membrane residence time.

Computational Psycholinguistics

Computational Psycholinguistics

Autonomy-supportive treatment for acquired apraxia of speech: feasibility and therapeutic effect

ABSTRACT Background: Most treatments for acquired apraxia of speech (AOS) rely on clinician-controlled practice conditions and repeated exposure to unimpaired speaker models. Recent motor learning research indicates that autonomy-support, expectation of competence, and external attentional focus may be more beneficial for motivation and skill learning. Aims: We evaluated the feasibility and basic therapeutic effect for the initial phase of a new speech production treatment program, ActionSC, that uses self-modeling and clinician coaching to help learners with AOS build their own practice program. Methods and Procedures: The single participant was a woman with moderate AOS and nonfluent aphasia. She met with project staff twice per week to review practice strategies, develop and adjust self-modeled video cues, work on her speech, and monitor progress. The program was structured around a custom app installed on a tablet computer. Most practice was directed by the participant based on options provided by the treating clinician. We used a multiple baseline across behavior design to evaluate the relationship between this treatment and oral reading probe performance for 30 conversational phrases the participant wanted to learn to say. Outcomes and Results: Experimental control was demonstrated, with target phrases remaining at baseline levels, then improving at the time treatment was introduced sequentially across three conversation topics. Effect sizes were moderate to large after 9–12 therapy sessions plus independent home practice. The participant assumed an active role in evaluating her own performance, administering and adjusting cues, and organizing her home practice. Conclusions: An autonomy-supportive and confidence-building format for speech practice can be feasible and effective for people with AOS. Fixed cueing hierarchies, augmented feedback, and attentional focus on speech movements may be less important in AOS treatment than previously thought. In addition to replicating our preliminary results with other participants and circumstances, there is a need to extend treatment development to later learning phases in order to promote positive change in real-life settings.

Synthesizing 3D Trump: Predicting and Visualizing the Relationship Between Text, Speech, and Articulatory Movements

The movements of articulators, such as lips, tongue and teeth, play an important role in increasing the language expression capability by unmasking the information hid in text or speech. Hence, it is necessary to deeply mine and visualize the relationship between text, speech and articulatory movements for understanding language in multi-modality and multi-level. As a case study, given text and audio of President Donald John Trump, this paper synthesizes a high quality 3D animation of him speaking with accurate synchronicity between speech and articulators. First, visual co-articulation is modeled by predicting the mapping from text/speech to articulatory movements. Then, based on a reconstructed 3D head model, physiological characteristics and statistical learning are combined to visualize each phoneme. Finally, the visualization results of consecutive phonemes are fused by visual co-articulation model to generate synchronized articulatory animations. Experiments show that the system can not only produce photo-realistic results in front but also distinguish the visual differences among phonemes from unconstrained views.

Dental Consideration and Diagnosis of TMJ Arthritis― A Curren Review

TMJ is a unique joint in which translatory as well as rotational movements are possible and where both the ends of bone articulate, in the same plane, with that of other bone. It is a hinge that connect the jaw to the temporal bones of the skull, which are in front of each ear. Temperomandibular joint (TMJ) diseases and disorders are refered to a complex and less understood set of conditions, and disorders of TMJ are manifested by pain in the area of the jaw and associated muscles, and limitations in the ability to make the normal movements of speech, facial expression, eating, chewing, and swallowing. Among the various disorders of TMJ this article is focused on a brief review on TMJ Arthritis.

Hand Motor Cortex Excitability During Speaking in Persistent Developmental Stuttering.

Persistent developmental stuttering (PDS) is a speech fluency disorder characterized by intermittent involuntary breakdowns of speech motor control, possibly related to motor cortex excitability. Whether motor cortex dysfunction extends into hand representations is unclear. We here studied task-dependent modulations of hand motor cortex excitability in 10 right-handed adults who stutter (AWS) and 13 age- and sex-matched fluent speaking control participants (ANS), covering a wide range of tasks in an exploratory study. Before, during and after a null speech/rest task, spontaneous speech, solo reading, chorus reading, singing, and non-verbal orofacial movements, transcranial magnetic stimulation was applied over the primary motor cortex and motor evoked potentials (MEPs) were recorded from the abductor digiti minimi muscle of either hand. In both groups, motor threshold was lower in the left than in the right motor cortex. During task performance, MEP amplitudes increased in both groups. A post hoc comparison of spontaneous speech and non-verbal orofacial movements yielded an interaction of group by task with AWS showing larger than ANS MEP amplitude increase in spontaneous speech, but a smaller than ANS MEP amplitude increase in non-verbal orofacial movements. We conclude that hemispheric specialization of hand motor representation is similar for both groups. Spontaneous speech as well as non-verbal orofacial movements are the orofacial tasks that merit further study. The excessive motor cortex facilitation could be reflecting a stronger activation of non-speech muscles during AWS’s speech.