Speech Production Process Research Articles

마비말장애 환자를 위한 청지각적 평가 시스템(DAS) 개발

Background and Objectives: Dysarthria is a significant motor speech disorder that arises from damage to the central or peripheral nervous system. This damage can result in paralysis and weakness of the muscles responsible for articulation, which negatively affects the speech production process. The objective of this study is to develop a Dysarthria Assessment System (DAS) for the auditory-perceptual evaluation of patients with dysarthria. Method: We conducted an analysis of assessment areas and tasks based on literature published after 2000. This analysis guided the development of assessment content, which was subsequently organized for integration into a web platform. Results: The DAS consists of three main components: Speech Mechanism, Speech, and Speech Motor. The Speech Mechanism component includes questions aimed at assessing the symmetry, range of motion, and strength of the articulatory organs involved in speech production, with items presented separately for rest and activity. The Speech component evaluates respiration, phonation, resonance, articulation, and prosody through tasks such as maximum phonation time, paragraph reading, and picture description. In the Speech Motor component, the system measures the frequency of alternating motion rates and sequential motion rates. Conclusions: The DAS developed in this study is expected to be highly beneficial in clinical practice due to its ability to visualize and automatically calculate results, enabling assessments to be conducted via PC or tablet. Future research will focus on standardizing the DAS through large-scale data collection.

The Impact of Online Task Planning Conditions on L2 Utterance Fluency and Self-Perceived Fluency: A Study of Chinese EFL Learners

This study examines the effects of online task planning conditions on L2 learners' utterance fluency and self-perceived fluency among Chinese EFL learners. Ninety participants were randomly assigned to Pressured Online Planning (POP), Unpressured Online Planning (UOP), and Hybrid Online Planning (HOP) groups. Participants completed a narrative task based on a Mr. Bean video clip under their respective planning conditions. Utterance fluency was measured using temporal and linguistic indicators, while self-perceived fluency was assessed through CEFR self-assessment and an Analytic Fluency Perception Scale. Results indicate that the hybrid condition yielded significantly higher speech and articulation rates, with fewer disfluencies compared to other conditions (p < .001). Notably, self-perceived fluency did not consistently align with objective measures or rater evaluations, particularly in the pressured condition. The study reveals complex relationships between task planning conditions, objective fluency measures, and learners' self-perceptions, contributing to our understanding of L2 speech production processes and informing task-based language teaching practices.

Visual Upward/Downward Motion Elicits Fast and Fluent High-/Low-Pitched Speech Production

Abstract Participants tend to produce a higher or lower vocal pitch in response to upward or downward visual motion, suggesting a pitch–motion correspondence between the visual and speech production processes. However, previous studies were contaminated by factors such as the meaning of vocalized words and the intrinsic pitch or tongue movements associated with the vowels. To address these issues, we examined the pitch–motion correspondence between simple visual motion and pitched speech production. Participants were required to produce a high- or low-pitched meaningless single vowel [a] in response to the upward or downward direction of a visual motion stimulus. Using a single vowel, we eliminated the artifacts related to the meaning, intrinsic pitch, and tongue movements of multiple vocalized vowels. The results revealed that vocal responses were faster when the pitch corresponded to the visual motion (consistent condition) than when it did not (inconsistent condition). This result indicates that the pitch–motion correspondence in speech production does not depend on the stimulus meaning, intrinsic pitch, or tongue movement of the vocalized words. In other words, the present study suggests that the pitch–motion correspondence can be explained more parsimoniously as an association between simple sensory (visual motion) and motoric (vocal pitch) features. Additionally, acoustic analysis revealed that speech production aligned with visual motion exhibited lower stress, greater confidence, and higher vocal fluency.

Tongue Twisters in English: Examining the Dynamics of Language Production and Working Memory in Pakistani English Learners

Analyzing speech errors can deepen our understanding of speech production processes, as these errors exhibit non-random patterns. (Traxler, 2012). By analyzing errors in the recall of previously listed tongue twisters, the study interprets errors occurring at different levels, providing valuable insights into individuals' performance. Focusing on undergraduate students enrolled in a Functional English course at a Pakistani university, the research investigates the relationship between mental processes and the production of tongue twisters, both in formal and informal contexts. The findings suggest that a higher occurrence of phonological errors in the formal tongue twister articulation indicates the influence of language production awareness. This observation supports the idea that the acquisition of English as a second language may involve an unconscious process, emphasizing mimicry over comprehension.

Task-generated processes in second language speech production: Exploring the neural correlates of task complexity during silent pauses

Abstract The last three decades have seen significant development in understanding and describing the effects of task complexity on learner internal processes. However, researchers have primarily employed behavioral methods to investigate task-generated cognitive load. Being the first to adopt neuroimaging to study second language (L2) task effects, we aimed to provide novel insights into the neural correlates of task-related variation in L2 oral production. To advance research methodology, we also tested the utility of a neuroimaging technique, functional magnetic resonance imaging (fMRI), in examining the impact of task-related variables on L2 speech production when combined with cognitive–behavioral tools (speech analysis, expert and learner judgments). Our research focus was the effects of task complexity on silent pausing. Twenty-four Japanese learners of English completed eight simple and complex versions of decision-making tasks, half in their first language and half in their L2. The dataset for the present study included the L2 speech and fMRI data, expert judgments, and participants’ difficulty ratings of the L1 and L2 tasks they completed. Based on our findings, we concluded that brain imaging and L1 task difficulty ratings were more sensitive to detecting task complexity effects than L2 self-ratings and pausing measures. These results point to the benefits of triangulating cognitive and neural data to study task-based neurocognitive processes.

Disfluencies as a window to macrostructure performance in the narrative of bilingual children

Disfluencies mirror processes of speech production and when examined in both languages of bilingual children, can provide valuable information regarding their linguistic skills. The aim of this study is to characterize the rates and the types of disfluencies across episodes of a narrative as well as across different story grammar (SG) elements in participants’ home language (Russian) and societal language (Hebrew) as a reflection of narrative macrostructure skills. Twenty bilingual Russian-Hebrew children aged 60–78 months produced narratives in Russian and Hebrew using the wordless picture book Frog, Where Are You? (Mayer, 1969). Narratives were transcribed, divided into five episodes, and coded for main SG elements (Character + Setting, Initiating event, Internal response, Plan, Attempt and Consequence). Four types of disfluencies (silent pauses, filled pauses, repetitions, and self-corrections) were analyzed in relation to each SG element per episode in both languages. Overall children produced more SG elements in Russian with a finer analysis showing that Initiating Event, Character and Setting differed across languages while Attempt, Plan and Internal response were similar. Disfluencies significantly impacted macrostructure differently across languages such that in Russian, a high percentage of disfluencies was associated with higher production of Character + Setting and Plan, whereas in Hebrew it was associated with lower production of Attempt and Consequence. The present research suggests that disfluency has impact on macrostructure, and consequently on overall narrative quality differently across the languages. The theoretical contribution may shed light on the understanding of planning and formulation processes among bilingual children. The application of the results may relate to these processes impacting diagnosis and/or intervention protocols in bilingual children.

Self-focused versus dialogic features of gesturing during simultaneous interpreting

The present study considers an implicit debate in the field of gesture studies as to whether gestures are produced primarily for the speaker or for the addressee. It considers the unique monologic setting of simultaneous interpreters working in a booth in which there is no visible audience present and where they only hear and do not see the speaker whose words they are interpreting. The hypotheses (H) are that the interpreters might produce more representational gestures, to aid in their own idea formulation (H1), and self-adapter movements, to maintain their self-focus (H2), rather than pragmatic gestures, which are known to serve interactive functions. Forty-nine interpreters were videorecorded as they interpreted two portions of popular science lectures, one from either English or German (their L2) into Russian (their L1) and one from Russian into their respective L2. The results showed that a vast majority of the gestures produced were either pragmatic in function or self adapters. H2 was thus supported, but H1 was not. The frequent use of pragmatic gestures is interpreted in terms of the internalized dialogic nature of talk and gesturing itself. Both beat gestures expressing emphasis and reduced forms of presentation gestures can facilitate the interpreters’ speaking by prompting the presentation and emphasis of ideas. Though focused on their own process of speech production, simultaneous interpreters may embody elements of the lecturer of the source text engaging with the audience, blended with their own dialogic speaking behaviors, aspects of which we may see in their gesturing.

Associations between social anxiety, physiological reactivity, and speech disfluencies in autistic young adults and controls

IntroductionThe aim of this study was to examine possible associations of social anxiety (SA) and speaking-related physiological reactivity with the frequencies of a) total disfluencies, b) typical disfluencies, and c) stuttering-like disfluencies, as well as d) stuttering-severity in autistic young adults and controls. MethodsThirty-two autistic young adults and 35 controls participated in this study. Participants were presented with video clips (viewing condition) and were then asked to talk about the videos (narrating condition). SA was measured by the self-report Social Phobia and Anxiety Inventory (SPAI). Speaking-related physiological reactivity was measured by the electrodermal activity (EDA), an index of emotional arousal. The speech samples from the narrating condition were analyzed for type and frequency of speech disfluencies and used for determining the stuttering severity. SA and speaking-related physiological reactivity were compared between the groups. Correlation between SA, physiological reactivity, disfluency frequencies, and stuttering severity were tested separately for both groups. ResultsNo between-group differences were found in the overall SA, yet differences were found in SPAI subscales of social interaction, group interaction, and avoidance, as well as in agoraphobia. Both groups had higher physiological arousal in narrating condition in comparison to the video viewing condition, yet there was no between-group difference in the reactivity. No associations were found between SPAI measures, physiological reactivity, disfluency frequencies, and stuttering severity in the autistic group. In the control group, a negative association was found between physiological reactivity and total and typical disfluency frequencies. ConclusionsSA or speaking-related physiological reactivity were not associated with disfluency frequencies or stuttering severity in autistic persons. Negative association between physiological reactivity and disfluency frequencies found in the control group may indicate that the physiological arousal may impact the speech production process by reducing the overt disfluencies.

The Impact of Phonological Variation on Speech Perception and

Purpose: The general aim of the study was to examine the impact of phonological variation on speech perception and production.
 Methodology: The study adopted a desktop research methodology. Desk research refers to secondary data or that which can be collected without fieldwork. Desk research is basically involved in collecting data from existing resources hence it is often considered a low cost technique as compared to field research, as the main cost is involved in executive’s time, telephone charges and directories. Thus, the study relied on already published studies, reports and statistics. This secondary data was easily accessed through the online journals and library.
 Findings: The findings reveal that there exists a contextual and methodological gap relating to phonological variation on speech perception and production. Preliminary empirical review revealed that that phonological variation significantly influences both speech perception and production processes. It highlighted the complex interplay between phonological variation, social factors, cultural context, and cognitive mechanisms. Moreover, the study emphasized the importance of addressing phonological variation in language education, clinical practice, and intercultural communication to promote inclusive communication practices in diverse settings. Overall, the findings contribute to a deeper understanding of language diversity and its implications for communication.
 Unique Contribution to Theory, Practice and Policy: Labov's theory of Variationist Sociolinguistics, Optimality theory and Speech Accommodation theory may be used to anchor future studies on phonological variation on speech perception and production. The study provided valuable recommendations that contributed to theoretical understanding, practical communication outcomes, and language policy decisions. It integrated insights from phonological variation research into linguistic theories, recommended the integration of phonological training into language instruction programs, and emphasized the importance of inclusive language policies. Additionally, the study suggested interventions to improve communication effectiveness, bridge communication gaps, address language disorders, and promote linguistic diversity. By incorporating these recommendations, stakeholders worked towards enhancing communication and fostering linguistic inclusivity across diverse linguistic contexts.

Timing and location of speech errors induced by direct cortical stimulation.

Cortical regions supporting speech production are commonly established using neuroimaging techniques in both research and clinical settings. However, for neurosurgical purposes, structural function is routinely mapped peri-operatively using direct electrocortical stimulation. While this method is the gold standard for identification of eloquent cortical regions to preserve in neurosurgical patients, there is lack of specificity of the actual underlying cognitive processes being interrupted. To address this, we propose mapping the temporal dynamics of speech arrest across peri-sylvian cortices by quantifying the latency between stimulation and speech deficits. In doing so, we are able to substantiate hypotheses about distinct region-specific functional roles (e.g. planning versus motor execution). In this retrospective observational study, we analysed 20 patients (12 female; age range 14-43) with refractory epilepsy who underwent continuous extra-operative intracranial EEG monitoring of an automatic speech task during clinical bedside language mapping. Latency to speech arrest was calculated as time from stimulation onset to speech arrest onset, controlling for individual speech rate. Most instances of motor-based arrest (87.5% of 96 instances) were in sensorimotor cortex with mid-range latencies to speech arrest with a distributional peak at 0.47 s. Speech arrest occurred in numerous regions, with relatively short latencies in supramarginal gyrus (0.46 s), superior temporal gyrus (0.51 s) and middle temporal gyrus (0.54 s), followed by relatively long latencies in sensorimotor cortex (0.72 s) and especially long latencies in inferior frontal gyrus (0.95 s). Non-parametric testing for speech arrest revealed that region predicted latency; latencies in supramarginal gyrus and in superior temporal gyrus were shorter than in sensorimotor cortex and in inferior frontal gyrus. Sensorimotor cortex is primarily responsible for motor-based arrest. Latencies to speech arrest in supramarginal gyrus and superior temporal gyrus (and to a lesser extent middle temporal gyrus) align with latencies to motor-based arrest in sensorimotor cortex. This pattern of relatively quick cessation of speech suggests that stimulating these regions interferes with the outgoing motor execution. In contrast, the latencies to speech arrest in inferior frontal gyrus and in ventral regions of sensorimotor cortex were significantly longer than those in temporoparietal regions. Longer latencies in the more frontal areas (including inferior frontal gyrus and ventral areas of precentral gyrus and postcentral gyrus) suggest that stimulating these areas interrupts a higher-level speech production process involved in planning. These results implicate the ventral specialization of sensorimotor cortex (including both precentral and postcentral gyri) for speech planning above and beyond motor execution.

Development of Theoretical Approaches to Modern Document Studies in Linguistic School of Professor Sofiya P. Lopushanskaya

The article presents fundamental positions of the linguistic school by Sofiya P. Lopushanskaya, Professor of Volgograd State University, and describes their theoretical advancement as being applied in the discourse studies on "Document text: history and modern state". It explains that the study of business and administrative communication should be based on a clarified definition of the term document as its main constituent. The major goal of documental linguistics is to process a diverse linguistic analysis, discover objective knowledge and reason linguistic comprehension of the documental nature of communication. Within the framework of this review, some general principles of the document text analysis are presented. Firstly, they are statements on: the unity of the speech-making process approach; the interaction and coordination between language phenomena and speech production processes; extralinguistic determination of the modern document. Secondly, these statements were particularized in the linguistic school of Volgograd State University with the aim of introducing basic approaches to the study of the document text. They involve identifying the research object and its characteristics, analyzing a diverse range of texts to assess the genre and style productivity of the document text equally, and defining linguistic categories that are manifested in textual and lingual specificity. The article presents renewed facts on discursive and linguistic features of modern documents united by such text properties as consistency, structure, locality, peculiarities of the language representation of the author and addressee. It is noted that the text of the document can contain information of various qualitative types (factual, conceptual, subtextual), and it is the main constituent of the documental system and the business and administrative communication act. As the text reflects an objective and social reality, its compositional structure and semantic realization greatly depend on extralinguistic constrains. The profiles of the author (addresser) and addressee are presented as being reconstructed with the detailed observation of multi-level lingual and discursive means. The variability of the addressee profiles (the document text addressability) is presented in models of lexical, style, and text adaptation. The principles and methods of detailed description of the documet text specified in the article open up new opportunities and directions for studying document typology and distinguishing relevant features of the document communication.

Destructive Changes in the Structure of Speech Action

This article considers the destructive changes in speech action, which are diagnosed by a comparative analysis of associative fields over time, and addresses two central problems: refinement of the long-standing concept of “speech action” and optimization of the established procedure for analyzing numerous and diverse associative data. The dynamics of the ways of predicating speech actions based on the material of associative fields was investigated. The principles of interpreting the cue, the target combination as a speech action, were further elaborated. The analysis relies on the results of free associative experiments and speech action modeling. The findings reveal a drastic change in the universal process of speech production over the past 27–30 years, with a significant decrease in the number of speech actions within the full syntactic cycle and a growing use of topic–comment structures and syntactic primitives as the types of speech action. The study is of great importance for psycholinguistics and associative lexicography, as it introduces a method for analyzing the associates indicative of the dynamics of language ability. In addition, the theoretical problem of classification of thematic or agrammatic responses as speech actions with a skipped operation of surface syntaxing was solved.

The NEF-SPA Approach as a Framework for Developing a Neurobiologically Inspired Spiking Neural Network Model for Speech Production.

The computer-based simulation of the whole processing route for speech production and speech perception in a neurobiologically inspired way remains a challenge. Only a few neural based models of speech production exist, and these models either concentrate on the cognitive-linguistic component or the lower-level sensorimotor component of speech production and speech perception. Moreover, these existing models are second-generation neural network models using rate-based neuron approaches. The aim of this paper is to describe recent work developing a third-generation spiking-neuron neural network capable of modeling the whole process of speech production, including cognitive and sensorimotor components. Our neural model of speech production was developed within the Neural Engineering Framework (NEF), incorporating the concept of Semantic Pointer Architecture (SPA), which allows the construction of large-scale neural models of the functioning brain based on only a few essential and neurobiologically well-grounded modeling or construction elements (i.e., single spiking neuron elements, neural connections, neuron ensembles, state buffers, associative memories, modules for binding and unbinding of states, modules for time scale generation (oscillators) and ramp signal generation (integrators), modules for input signal processing, modules for action selection, etc.). We demonstrated that this modeling approach is capable of constructing a fully functional model of speech production based on these modeling elements (i.e., biologically motivated spiking neuron micro-circuits or micro-networks). The model is capable of (i) modeling the whole processing chain of speech production and, in part, for speech perception based on leaky-integrate-and-fire spiking neurons and (ii) simulating (macroscopic) speaking behavior in a realistic way, by using neurobiologically plausible (microscopic) neural construction elements. The model presented here is a promising approach for describing speech processing in a bottom-up manner based on a set of micro-circuit neural network elements for generating a large-scale neural network. In addition, the model conforms to a top-down design, as it is available in a condensed form in box-and-arrow models based on functional imaging and electrophysiological data recruited from speech processing tasks.

Nonword reading by adults who stutter in a transparent orthography

PurposeUsing word- and nonword-reading passages in Kannada, which has a transparent orthography, we attempted to determine (a) whether orthographic differences between English and Kannada may explain the observed differences in stutter rates on nonwords, and (b) whether longer nonwords, like words, incur higher rates of stutters. MethodsStutters are defined as sound or syllable repetitions, sound prolongations, broken words or nonwords (a pause within a word or nonword), abnormal pauses, and intrusive vowel-like sounds. Twenty-six persons, who stutter, read the word and nonword passages. The nonwords were created by changing the first syllable of each word; otherwise words and nonwords were equivalent in length and syllable structure. Stutters were counted from audio-recordings and statistically analyzed. ResultsPWS stuttered on words in varying amounts and in significantly larger amounts on nonwords. Stutter frequency increased roughly in proportion to the increase in the length of phonological words (previously known) and nonwords (reported for the first time here). ConclusionThe results cannot be attributed to the difficulty of pronouncing nonwords because Kannada orthography has a one-to-one relationship between the written and spoken forms of words. Speech production is a multi-stage process consisting of ideation, lemma selection, phonological word creation, and the articulatory planning and execution. Because nonwords lack meaning and clearly identifiable part of speech, it appears that stutters arise late in the speech production process at the phonological word formation and articulatory planning stages. Meaning, lexicality, and morphosyntax may not contribute significantly to the occurrence of stutters.

Pausing and fluency in speech of patients with relapsing-remitting multiple sclerosis

ABSTRACT Multiple Sclerosis (MS) causes a variety of symptoms in speech production, such as more frequent pauses and an increase in the duration of pauses in the speech. However, there is almost no data on whether the disease affects speech fluency in other ways, such as changes in the frequency of disfluencies in speech. The main question of this study is the following: if we examine speech fluency in speech tasks requiring different cognitive load, will there be a difference between patients and controls? Twenty people with relapsing-remitting MS (3 men and 17 women) and 20 age- and education-matched control speakers (4 men and 16 women) participated in the study. Speech samples were recorded with each participant in three speech tasks: 1) spontaneous narratives about their own lives, 2) narratives about their previous day, and 3) narrative recalls based on a heard text. In the speech samples, pauses and disfluencies were annotated and the duration of pauses was measured. Then, the frequency of pauses and disfluencies were calculated and the types of disfluencies were examined. The results show that there are differences in the frequency and duration of pauses between people with MS and controls. However, there were no significant differences in the frequency of disfluencies between the groups. The same types of disfluencies occurred in the same frequency in both groups. The results help to better understand the speech production processes in MS.

Deficiencies in the Scope of Developmental Stuttering Speech Plans

This paper briefly introduces the process of speech production, EXPLAN model and the scope of speech planning, reviews recent studies on speech planning scope for developmental stuttering, and expounds the defects of the scope of speech planning for developmental stuttering. The aim is to provide more direction for the research of stuttering theory and some reference for the solution of stuttering problem.

The domain-separation language network dynamics in resting state support its flexible functional segregation and integration during language and speech processing

Modern linguistic theories and network science propose that language and speech processing are organized into hierarchical, segregated large-scale subnetworks, with a core of dorsal (phonological) stream and ventral (semantic) stream. The two streams are asymmetrically recruited in receptive and expressive language or speech tasks, which showed flexible functional segregation and integration. We hypothesized that the functional segregation of the two streams was supported by the underlying network segregation. A dynamic conditional correlation approach was employed to construct framewise time-varying language networks and k-means clustering was employed to investigate the temporal-reoccurring patterns. We found that the framewise language network dynamics in resting state were robustly clustered into four states, which dynamically reconfigured following a domain-separation manner. Spatially, the hub distributions of the first three states highly resembled the neurobiology of speech perception and lexical-phonological processing, speech production, and semantic processing, respectively. The fourth state was characterized by the weakest functional connectivity and was regarded as a baseline state. Temporally, the first three states appeared exclusively in limited time bins (∼15%), and most of the time (> 55%), state 4 was dominant. Machine learning-based dFC-linguistics prediction analyses showed that dFCs of the four states significantly predicted individual linguistic performance. These findings suggest a domain-separation manner of language network dynamics in resting state, which forms a dynamic “meta-network” framework to support flexible functional segregation and integration during language and speech processing.

Behind the Complex Interplay of Phonation: Investigating Elasticity of Vocal Folds With Pipette Aspiration Technique During Ex Vivo Phonation Experiments

Objectives: The vibration of the vocal folds produces the primary sound for the human speech. The vibration depends mainly on the pressure, airflow of the lungs, and the material properties of the vocal folds. In order to change them, muscles in the larynx stretch the vocal folds. This interplay is rarely investigated, but can give insight in the complex process of speech production. Most material properties studies are damaging the tissue; therefore, a nondestructive one is desired. Methods: An ex vivo phonation experiment combined with the dynamic Pipette Aspiration Technique is used to investigate 10 porcine larynges, under manipulations of different adduction and elongation levels. For each manipulation, the near surface material properties of the vocal folds are measured as well as different phonation parameters like the subglottal pressure, glottal resistance, frequency, and stiffness. Thereby, a high-speed camera was used to record the vocal fold movement. Results: On most of the measured parameters, the manipulations do show an effect. Both manipulations lead to a higher phonation frequency and an increase of the stiffness of the tissue. Comparing both manipulations, the elongation results in higher elasticity values than the adduction. Different measurement parameters have been compared with each other and correlations could be found. Where the strongest correlation are found among the elasticity values of different frequencies. But it can also be seen that the elasticity values correlate with phonation parameters. Conclusion: It was possible to produce a data set of 560 measurements in total. To our knowledge, this is the first time Pipette Aspiration Technique was combined with ex vivo phonation measurements for combined measurements. The amount of measurement data made it possible to carry out statistic investigations. The effect of the manipulations on material properties as well as on phonation parameters could be measured and different correlations could be found. The results lead to the hypothesis that the stretch does not have a huge effect on the material properties of the lamina propria, but more on the underlying muscle.

Speech Audio Synthesis from Tagged MRI and Non-Negative Matrix Factorization via Plastic Transformer.

The tongue's intricate 3D structure, comprising localized functional units, plays a crucial role in the production of speech. When measured using tagged MRI, these functional units exhibit cohesive displacements and derived quantities that facilitate the complex process of speech production. Non-negative matrix factorization-based approaches have been shown to estimate the functional units through motion features, yielding a set of building blocks and a corresponding weighting map. Investigating the link between weighting maps and speech acoustics can offer significant insights into the intricate process of speech production. To this end, in this work, we utilize two-dimensional spectrograms as a proxy representation, and develop an end-to-end deep learning framework for translating weighting maps to their corresponding audio waveforms. Our proposed plastic light transformer (PLT) framework is based on directional product relative position bias and single-level spatial pyramid pooling, thus enabling flexible processing of weighting maps with variable size to fixed-size spectrograms, without input information loss or dimension expansion. Additionally, our PLT framework efficiently models the global correlation of wide matrix input. To improve the realism of our generated spectrograms with relatively limited training samples, we apply pair-wise utterance consistency with Maximum Mean Discrepancy constraint and adversarial training. Experimental results on a dataset of 29 subjects speaking two utterances demonstrated that our framework is able to synthesize speech audio waveforms from weighting maps, outperforming conventional convolution and transformer models.

Stressed Speech Emotion Recognition Using Teager Energy and Spectral Feature Fusion with Feature Optimization.

The objective of speech emotion recognition (SER) is to enhance man-machine interface. It can also be used to cover the physiological state of a person in critical situations. In recent time, speech emotion recognition also finds its operations in medicine and forensics. A new feature extraction technique using Teager energy operator (TEO) is proposed for the detection of stressed emotions as Teager energy-autocorrelation envelope (TEO-Auto-Env). TEO is basically designed for increasing the energies of the stressed speech signals whose energies are reduced during the speech production process and hence used in this analysis. A stressed speech emotion recognition (SSER) system is developed using TEO-Auto-Env and spectral feature combination for detecting the emotions. The spectral features considered are Mel-frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), and relative spectra-perceptual linear prediction (RASTA-PLP). EMO-DB (German), EMOVO (Italian), IITKGP (Telugu), and EMA (English) databases are used in this analysis. The classification of the emotions is carried out using the k-nearest neighborhood (k-NN) classifier for gender-dependent (GD) and speaker-independent (SI) cases. The proposed SSER system provides improved accuracy compared to the existing ones. Average recall is used for performance evaluation. The highest classification accuracy is achieved using the feature combination of TEO-Auto-Env, MFCC, and LPCC features with 91.4% (SI), 91.4% (GD-male), and 93.1%(GD-female) for EMO-DB; 68.5% (SI), 68.5% (GD-male), and 74.6% (GD-female) for EMOVO; 90.6%(SI), 91% (GD-male), and 92.3% (GD-female) for EMA; and 95.1% (GD-female) for IITKGP female database.