Articulatory Features Research Articles

What R Mandarin Chinese /ɹ/s? - acoustic and articulatory features of Mandarin Chinese rhotics.

Rhotic sounds are well known for their considerable phonetic variation within and across languages and their complexity in speech production. Although rhotics in many languages have been examined and documented, the phonetic features of Mandarin rhotics remain unclear, and debates about the prevocalic rhotic (the syllable-onset rhotic) persist. This paper extends the investigation of rhotic sounds by examining the articulatory and acoustic features of Mandarin Chinese rhotics in prevocalic, syllabic (the rhotacized vowel [ɚ]), and postvocalic (r-suffix) positions. Eighteen speakers from Northern China were recorded using ultrasound imaging. Results showed that Mandarin syllabic and postvocalic rhotics can be articulated with various tongue shapes, including tongue-tip-up retroflex and tongue-tip-down bunched shapes. Different tongue shapes have no significant acoustic differences in the first three formants, demonstrating a many-to-one articulation-acoustics relationship. The prevocalic rhotics in our data were found to be articulated only with bunched tongue shapes, and were sometimes produced with frication noise at the start. In general, rhotics in all syllable positions are characterized by a close F2 and F3, though the prevocalic rhotic has a higher F2 and F3 than the syllabic and postvocalic rhotics. The effects of syllable position and vowel context are also discussed.

Ease of Articulation in Selecting First Names

Investigating the ease of articulation in selecting names contributes to emphasizing the effect of articulatory features on speech perception and production. It explains why some names are distinctive among multi-semantic references and sociocultural dominance and are more likely appropriate for one sex than the other. This study explores how distinctive features of phonemes subconsciously derive our conception of first names, regardless of rich meanings or morphological endings. It argues that all selected names demonstrate articulatory-easy phonemes. Sex distinction is assumed to attribute phonemes of easier configurations to characterize girl names more than boy names. By inspecting a sample that exhibited no feminine suffixes or compounding structures, the study analyzed the phonemic components based on the articulatory-ease module. The findings justified equivalent preference of boy names and girl names to the ease of bilabial, coronal, voiceless features, and disyllabic structures. However, interest in names with easier anterior coronal sonorants, posterior distributed strident and glottal dominated the selected girl names in line with sound symbolism. Moreover, prominent everlasting names indicated a mechanism of articulatory ease that identified one essential unit to develop a unique smooth flow of phonemes. The distinctive characteristic features of this unit marked sex distinction and determined popularity. KEYWORDS articulatory mechanism, articulatory module, conserving effort, distinctive features, prominent names, sound symbolism

Audio–visual deepfake detection using articulatory representation learning

Advancements in generative artificial intelligence have made it easier to manipulate auditory and visual elements, highlighting the critical need for robust audio–visual deepfake detection methods. In this paper, we propose an articulatory representation-based audio–visual deepfake detection approach, ART-AVDF. First, we devise an audio encoder to extract articulatory features that capture the physical significance of articulation movement, integrating with a lip encoder to explore audio–visual articulatory correspondences in a self-supervised learning manner. Then, we design a multimodal joint fusion module to further explore inherent audio–visual consistency using the articulatory embeddings. Extensive experiments on the DFDC, FakeAVCeleb, and DefakeAVMiT datasets demonstrate that ART-AVDF obtains a significant performance improvement compared to many deepfake detection models.

Speech's syllabic rhythm and articulatory features produced under different auditory feedback conditions identify Parkinsonism

Diagnostic tests for Parkinsonism based on speech samples have shown promising results. Although abnormal auditory feedback integration during speech production and impaired rhythmic organization of speech are known in Parkinsonism, these aspects have not been incorporated into diagnostic tests. This study aimed to identify Parkinsonism using a novel speech behavioral test that involved rhythmically repeating syllables under different auditory feedback conditions. The study included 30 individuals with Parkinson's disease (PD) and 30 healthy subjects. Participants were asked to rhythmically repeat the PA-TA-KA syllable sequence, both whispering and speaking aloud under various listening conditions. The results showed that individuals with PD had difficulties in whispering and articulating under altered auditory feedback conditions, exhibited delayed speech onset, and demonstrated inconsistent rhythmic structure across trials compared to controls. These parameters were then fed into a supervised machine-learning algorithm to differentiate between the two groups. The algorithm achieved an accuracy of 85.4%, a sensitivity of 86.5%, and a specificity of 84.3%. This pilot study highlights the potential of the proposed behavioral paradigm as an objective and accessible (both in cost and time) test for identifying individuals with Parkinson's disease.

La referencia lingüística y la decolonización de los nombres propios en las lenguas de señas

The subject of the proper name is the subject of reference. Proper names are paradigmatic examples because they refer to unique entities in the world (uniqueness). In sign languages, there are gestural expressions that identify people in deaf communities and are distinguished from the spelling of their name given in Latin alphabet. Studies on this topic have described the signed proper name according to hand configurations and its other articulatory characteristics, according to some general principles about the significance of having the proper name in the community and how it functions. However, almost no mention is made of the basic concepts involved in the theoretical formulation of names and uniqueness. In this paper, we argue that descriptions of this topic are deliberately simplistic and reductionist, not because there is a desire to marginalize sign languages, but because we have been historically conditioned to act this way. From a decolonial critique and an anti-representationalist perspective of language, we show the controversies that shape the issue of proper names and propose an integrated view that advances in the deconstruction of polarizations and false dichotomies in the linguistic and identity discourse.

Speech emotion recognition based on bi-directional acoustic–articulatory conversion

Acoustic and articulatory signals are naturally coupled and complementary. The challenge of acquiring articulatory data and the nonlinear ill-posedness of acoustic–articulatory conversions have resulted in previous studies on speech emotion recognition (SER) primarily relying on unidirectional acoustic–articulatory conversions. However, these studies have ignored the potential benefits of bi-directional acoustic–articulatory conversion. Addressing the problem of nonlinear ill-posedness and effectively extracting and utilizing these two modal features in SER remain open research questions. To bridge this gap, this study proposes a Bi-A2CEmo framework that simultaneously addresses the bi-directional acoustic–articulatory conversion for SER. This framework comprises three components: a Bi-MGAN that addresses the nonlinear ill-posedness problem, KCLNet that enhances the emotional attributes of the mapped features, and ResTCN-FDA that fully exploits the emotional attributes of the features. Another challenge is the absence of a parallel acoustic–articulatory emotion database. To overcome this issue, this study utilizes electromagnetic articulography (EMA) to create a multi-modal acoustic–articulatory emotion database for Mandarin Chinese called STEM-E2VA. A comparative analysis is then conducted between the proposed method and state-of-the-art models to evaluate the effectiveness of the framework. Bi-A2CEmo achieves an accuracy of 89.04% in SER, which is an improvement of 5.27% compared with the actual acoustic and articulatory features recorded by the EMA. The results for the STEM-E2VA dataset show that Bi-MGAN achieves a higher accuracy in mapping and inversion than conventional conversion networks. Visualization of the mapped features before and after enhancement reveals that KCLNet reduces the intra-class spacing while increasing the inter-class spacing of the features. ResTCN-FDA demonstrates high recognition accuracy on three publicly available datasets. The experimental results show that the proposed bi-directional acoustic–articulatory conversion framework can significantly improve the SER performance.

Articulatory Characteristics of Secondary Palatalization in Romanian Fricatives

Articulatory Characteristics of Secondary Palatalization in Romanian Fricatives

Visual scanning patterns of a talking face when evaluating phonetic information in a native and non-native language.

When comprehending speech, listeners can use information encoded in visual cues from a face to enhance auditory speech comprehension. For example, prior work has shown that the mouth movements reflect articulatory features of speech segments and durational information, while pitch and speech amplitude are primarily cued by eyebrow and head movements. Little is known about how the visual perception of segmental and prosodic speech information is influenced by linguistic experience. Using eye-tracking, we studied how perceivers' visual scanning of different regions on a talking face predicts accuracy in a task targeting both segmental versus prosodic information, and also asked how this was influenced by language familiarity. Twenty-four native English perceivers heard two audio sentences in either English or Mandarin (an unfamiliar, non-native language), which sometimes differed in segmental or prosodic information (or both). Perceivers then saw a silent video of a talking face, and judged whether that video matched either the first or second audio sentence (or whether both sentences were the same). First, increased looking to the mouth predicted correct responses only for non-native language trials. Second, the start of a successful search for speech information in the mouth area was significantly delayed in non-native versus native trials, but just when there were only prosodic differences in the auditory sentences, and not when there were segmental differences. Third, (in correct trials) the saccade amplitude in native language trials was significantly greater than in non-native trials, indicating more intensely focused fixations in the latter. Taken together, these results suggest that mouth-looking was generally more evident when processing a non-native versus native language in all analyses, but fascinatingly, when measuring perceivers' latency to fixate the mouth, this language effect was largest in trials where only prosodic information was useful for the task.

Seeing to understand better? The interplay between cognitive traits and nonverbal visual cues in L2 video comprehension

This study delves into the relationship between the cognitive traits of L2 learners and nonverbal visual scaffolds, specifically beat gestures and facial articulatory features, embedded in training materials. These scaffolds have been proven to enhance L1 comprehension. The research further investigates how this relationship impacts the effects of these scaffolds on learners' comprehension performance post-training. For this purpose, the study recruited 120 English as a Foreign Language (EFL) learners aged 20–24, all at the B2 level of English proficiency, for a 4-week hybrid instructional period. The findings suggest that working memory (WM) plays a significant role in L2 learners' processing of nonverbal visual cues. High-WM learners showed improved comprehension with facial cues, while low-WM individuals leaned towards treatments with fewer nonverbal cues. Additionally, learners’ modality preferences were pivotal for their comprehension. Auditory verbalizers—those who prefer spoken texts—favored treatments emphasizing auditory elements. In contrast, textual verbalizers—those inclined towards written texts and with high working memory—benefited more from exposure to rich nonverbal cues. This study highlights the importance of a balanced instructional strategy that integrates both verbal and nonverbal cues, tailored specifically for learners with distinct cognitive traits and preferences.

Improving Second Language Vowel Production With Hand Gestures Encoding Visible Articulation: Evidence From Picture‐Naming and Paragraph‐Reading Tasks

AbstractThis study investigates whether audiovisual phonetic training with hand gestures encoding visible or nonvisible articulation features has a differential impact on learning second language sounds. Ninety‐nine Catalan–Spanish bilingual students were trained to differentiate English /æ/ and /ʌ/, which differ in the visible lip aperture and nonvisible tongue position, with training involving no gestures, gestures representing the lip aperture, or gestures representing the tongue position. Before, immediately after, and 1 week after the training, participants’ perception of the targets was assessed through a word‐identification task, and their production was tested through paragraph‐reading, picture‐naming, and word‐imitation tasks. Although all participants improved in perception and production, the lip hand gesture was more effective in adjusting lip aperture than the other two conditions in the paragraph‐reading and picture‐naming tasks. These results suggest that hand gestures encoding visible rather than nonvisible articulation features are more effective for improving second language pronunciation.

BiLSTM-CNN with fixed weight approach for tracking speech articulatory features

Speech production is a complex sequential process which involve the coordination of various articulatory features. Among them tongue being a highly versatile active articulator responsible for shaping airflow to produce targeted speech sounds that are intellectual, clear, and distinct. This paper presents a novel approach for predicting tongue and lip articulatory features involved in a given speech acoustics using a stacked Bidirectional Long Short-Term Memory (BiLSTM) architecture, combined with a one-dimensional Convolutional Neural Network (CNN) for post-processing with fixed weights initialization. The proposed network is trained with two datasets consisting of simultaneously recorded speech and Electromagnetic Articulography (EMA) datasets, each introducing variations in terms of geographical origin, linguistic characteristics, phonetic diversity, and recording equipment. The performance of the model is assessed in Speaker Dependent (SD), Speaker Independent (SI), corpus dependent (CD) and cross corpus (CC) modes. Experimental results indicate that the proposed model with fixed weights approach outperformed the adaptive weights initialization with in relatively minimal number of training epochs. These findings contribute to the development of robust and efficient models for articulatory feature prediction, paving the way for advancements in speech production research and applications.

Speech Sounds Production, Narrative Skills, and Verbal Memory of Children with 22q11.2 Microdeletion.

22q11.2 deletion syndrome (22q11.2DS), the most frequent microdeletion syndrome in humans, is related to a high risk of developing neurodevelopmental disorders. About 95% of patients with 22q11.2DS have speech and language impairments. Global articulation, story generation, and verbal memory tests were applied to compare articulatory characteristics of speech sounds, spontaneous language abilities, and immediate verbal memory between four groups of Serbian-speaking children: patients with 22q11.2DS, children with clinical presentation of 22q11.2DS that do not have the microdeletion, children with non-syndromic congenital heart defects, and their peers with typical speech-sound development. The obtained results showed that children with this microdeletion have impaired articulation skills and expressive language abilities. However, we did not observe weaker receptive language skills and immediate verbal memory compared to healthy controls. Children with 22q11.2DS should be considered a risk category for the development of speech-sound pathology and expressive language abilities. Since speech intelligibility is an instrument of cognition and adequate peer socialization, and language impairment in school-aged children with 22q11DS might be an indicator of increased risk for later psychotic symptoms, patients with 22q11.2 microdeletion should be included in a program of early stimulation of speech-language development immediately after diagnosis is established.

Sound symbolism in manual and vocal responses: phoneme-response interactions associated with grasping as well as vertical and size dimensions of keypresses

It has been shown that reading the vowel [i] and consonant [t] facilitates precision grip responses, while [ɑ] and [k] are associated with faster power grip responses. A similar effect has been observed when participants perform responses with small or large response keys. The present study investigated whether the vowels and consonants could produce different effects with the grip responses and keypresses when the speech units are read aloud (Experiment 1) or silently (Experiment 2). As a second objective, the study investigated whether the recently observed effect, in which the upper position of a visual stimulus is associated with faster vocalizations of the high vowel and the lower position is associated with the low vowel, can be observed in manual responses linking, for example, the [i] with responses of the upper key and [ɑ] with lower responses. Firstly, the study showed that when the consonants are overtly articulated, the interaction effect can be observed only with the grip responses, while the vowel production was shown to systematically influence small/large keypresses, as well as precision/power grip responses. Secondly, the vowel [i] and consonant [t] were associated with the upper responses, while [ɑ] and [k] were associated with the lower responses, particularly in the overt articulation task. The paper delves into the potential sound-symbolic implications of these phonetic elements, suggesting that their acoustic and articulatory characteristics might implicitly align them with specific response magnitudes, vertical positions, and grip types.

Articulatory and acoustic differences between lyric and dramatic singing in Western classical music.

Within the realm of voice classification, singers could be sub-categorized by the weight of their repertoire, the so-called "singer's Fach." However, the opposite pole terms "lyric" and "dramatic" singing are not yet well defined by their acoustic and articulatory characteristics. Nine professional singers of different singers' Fach were asked to sing a diatonic scale on the vowel /a/, first in what the singers considered as lyric and second in what they considered as dramatic. Image recording was performed using real time magnetic resonance imaging (MRI) with 25 frames/s, and the audio signal was recorded via an optical microphone system. Analysis was performed with regard to sound pressure level (SPL), vibrato amplitude, and frequency and resonance frequencies as well as articulatory settings of the vocal tract. The analysis revealed three primary differences between dramatic and lyric singing: Dramatic singing was associated with greater SPL and greater vibrato amplitude and frequency as well as lower resonance frequencies. The higher SPL is an indication of voice source changes, and the lower resonance frequencies are probably caused by the lower larynx position. However, all these strategies showed a considerable individual variability. The singers' Fach might contribute to perceptual differences even for the same singer with regard to the respective repertoire.

Initial Stop Acquisition in Typically Developing Children Aged 1- to 2-Year-Olds

Objectives: This study investigated the articulatory characteristics of initial stops in the speech of 1-2-year-old children.Methods: A total of 35 typically developing children aged 1-2 years participated in the study. The participants performed a naming task using nine monosyllabic words containing initial stops. The accuracy of consonants in the initial stops, as well as errors in articulation position and voicing features were analyzed among children in the late 1-year-old and early 2-year-old age groups. Consonant accuracy, articulation position and voicing error ratios were compared between the two age groups. Additionally, the study investigated patterns of voicing errors observed in each age group in detail.Results: Findings revealed that the articulatory characteristics of initial stops were prominent in voicing errors observed in both age groups. Consonant accuracy was significantly higher in the early 2-year-old group compared to the late 1-year-old group. The most prevalent error observed in both late 1-year-old and early 2-year-old children was voicing errors. Examining the patterns of voicing errors, it was found that late 1-year-olds exhibited all possible error patterns in voicing features, while early 2-year-olds showed a reduction or disappearance of some voicing error patterns, excluding errors related to aspiration.Conclusion: This study investigated the articulatory characteristics of initial stops in the speech production of children in the late age of 1 and the early age of 2, presenting information on the acquisition process of initial stops. Error patterns in voicing features were commonly observed in the younger age group before the late age of 2.

Articulatory characteristics and vowel space analysis of Mandarin Chinese non-low vowels in Korean-speaking learners

Vowels are a fundamental component of Mandarin Chinese syllables. Accurate pronunciation of Mandarin Chinese relies on the distinct articulation of different vowels, especially the visually similar “ü” and “u”. Korean-speaking learners of Chinese Mandarin as a second language commonly exhibit non-standard pronunciation or errors in articulating “ü”, leading to confusion, particularly in non-low vowels centered around this sound. Using Praat to analyze the Formant values of non-low vowel sounds from six participants, a comparison between Korean L2 Chinese speakers and native Chinese speakers revealed vowel spaces differentiated through F2 values for /u/ and /o/, while /ɣ/ and /o/ are distinguished by F1 values. The sound similar to Korean “ㅟ” differs from /y/ in that the former is a gliding sound transitioning from low to high. The distinction between /y/ and /u/, /o/, /ɣ/ was found to be unclear, with /y/ often merging into sounds like “iu” and “io”. This aligns with historical vowel changes in Chinese “ü”, noted in previous linguistic studies, showing variations like “iu” and “io” over different periods. Korean L2 Chinese speakers tend to substitute /ɯ/ for /u/ and “ㅟ” (/ɯ/+/i/) for /y/, avoiding rounded lip movements in pronunciation, thus reducing effort and the need for additional phonemes. Furthermore, the F3 value for Korean L2 Chinese /y/ is notably lower, suggesting that the correct articulation should closely resemble the tongue position of /i/, combined with rounded lips.

Speech perception difficulty modulates theta-band encoding of articulatory synergies.

The human brain tracks available speech acoustics and extrapolates missing information such as the speaker's articulatory patterns. However, the extent to which articulatory reconstruction supports speech perception remains unclear. This study explores the relationship between articulatory reconstruction and task difficulty. Participants listened to sentences and performed a speech-rhyming task. Real kinematic data of the speaker's vocal tract were recorded via electromagnetic articulography (EMA) and aligned to corresponding acoustic outputs. We extracted articulatory synergies from the EMA data with principal component analysis (PCA) and employed partial information decomposition (PID) to separate the electroencephalographic (EEG) encoding of acoustic and articulatory features into unique, redundant, and synergistic atoms of information. We median-split sentences into easy (ES) and hard (HS) based on participants' performance and found that greater task difficulty involved greater encoding of unique articulatory information in the theta band. We conclude that fine-grained articulatory reconstruction plays a complementary role in the encoding of speech acoustics, lending further support to the claim that motor processes support speech perception.NEW & NOTEWORTHY Top-down processes originating from the motor system contribute to speech perception through the reconstruction of the speaker's articulatory movement. This study investigates the role of such articulatory simulation under variable task difficulty. We show that more challenging listening tasks lead to increased encoding of articulatory kinematics in the theta band and suggest that, in such situations, fine-grained articulatory reconstruction complements acoustic encoding.

Predicting vocal tract shape information from tongue contours and audio using neural networks

Midsagittal ultrasound imaging of the tongue is a portable and inexpensive way to provide articulatory information. However, although ultrasound images show a portion of the tongue surface, other vocal tract structures (e.g., palate) are not typically visible. This missing information may be useful for speech therapy and other applications, e.g., by characterizing vocal tract constrictions and informing how morphological variations affect speech patterns. Prediction of the vocal tract shape from information available during ultrasound imaging (e.g., tongue contours and audio recordings) is, thus, potentially valuable. Recent advancements in articulatory prediction from audio recordings (i.e., acoustic inversion) and speech recognition using combined articulatory and acoustic data have used neural network models. Inspired by these models, this study investigates how well fusion of articulatory and acoustic features in speaker-independent models can predict expanded articulatory information. Specifically, recurrent neural network models will be trained to predict the vocal tract shape based on partial tongue contours and acoustic features, during production of vowels and central approximants. Features will be extracted from simultaneously recorded audio and 2D MRI (USC 75-Speaker Database). Different acoustic features and network architectures will be compared, with the goal of refining future models to predict vocal tract shapes during ultrasound imaging.

The Application of Spectrogram in the Teaching of High-level Vocal Music Major Students

Abstract Vocal music teaching has developed to the present day, still following the method of oral transmission, but with the development of modern science and technology, sound images can be processed to clearly see the quality of sound used. Students’ vocal signals are used in the article to extract articulation features and construct a vocal spectrogram. The vocal spectrogram is used in vocal music teaching to enhance the students’ timbre resonance. In order to verify its application in dissecting students’ singing effects, the state of different students’ American soprano articulation area is concretized on the spectrogram in terms of students’ voice change differences, breathing aspects, and vocal resonance while comparing the differences between the manual evaluation and the evaluation of the vocal spectrogram analysis. It can be seen that the minimum fundamental frequency of student A in the first group is 191.7 lower than that of student B, which is 287.8, and the maximum fundamental frequency of 1033.2 is also significantly higher than that of student B, which is just the same as the conclusion drawn from the manual scale. To sum up, according to the sound spectrogram, the teacher can well analyze the students’ voice waveforms visually, and the students can adjust the vocal state through the voice waveforms so as to carry out the correct reinforcement training and add new means for the traditional mode of vocal teaching, so as to make the teaching of vocal music gradually realize the visualization and intuition.

Проблема палатальности и палатализованности в разносистемных языках Сибири (по данным МРТ)

The paper considers the phonetic characteristics of nasal palatalized and non-palatalized sounds of the German Mennonite language (self-name Plautdietsch) and the Altai language and nasal forelingual and oral backlingual sounds of the Russian language. Given the strong influence of Russian on Altai and Plautdietsch, the objective is to determine if Russian palatalization, which structures the whole consonant system and is a constitutive-differential feature, influences the other languages under study. Magnetic resonance imaging was used to identify the articulation features and reveal the constitutive differential parameters characteristic of the phonetic subsystem of palatalized phonemes. Nineteen tomograms with target sounds were described and analyzed following the methodology of the V. M. Nadelyaev Laboratory of Experimental Phonetic Researches of the Institute of Philology of the Siberian Branch of the Russian Academy of Sciences. The MRI images demonstrate that the articulation of palatalized sounds involves the elevation of the middle and back parts of the tongue towards the boundary between the hard and soft palate. The production of palatalized sounds in the Russian language is found to be characterized by the advancement of the tongue body to the front part of the oral cavity. The study findings indicate that palatalization may not always produce an acoustic effect of softness. This research contributes to the understanding of the articulatory mechanisms and phonetic features of palatalized sounds in the languages under consideration, highlighting the interplay between language contact and phonetic adaptation.