Vocal Tract Research Articles

An open-source toolbox for measuring vocal tract shape from real-time magnetic resonance images

Real-time magnetic resonance imaging (rtMRI) is a technique that provides high-contrast videographic data of human anatomy in motion. Applied to the vocal tract, it is a powerful method for capturing the dynamics of speech and other vocal behaviours by imaging structures internal to the mouth and throat. These images provide a means of studying the physiological basis for speech, singing, expressions of emotion, and swallowing that are otherwise not accessible for external observation. However, taking quantitative measurements from these images is notoriously difficult. We introduce a signal processing pipeline that produces outlines of the vocal tract from the lips to the larynx as a quantification of the dynamic morphology of the vocal tract. Our approach performs simple tissue classification, but constrained to a researcher-specified region of interest. This combination facilitates feature extraction while retaining the domain-specific expertise of a human analyst. We demonstrate that this pipeline generalises well across datasets covering behaviours such as speech, vocal size exaggeration, laughter, and whistling, as well as producing reliable outcomes across analysts, particularly among users with domain-specific expertise. With this article, we make this pipeline available for immediate use by the research community, and further suggest that it may contribute to the continued development of fully automated methods based on deep learning algorithms.

Mechanisms of sensorimotor adaptation in a hierarchical state feedback control model of speech.

Upon perceiving sensory errors during movements, the human sensorimotor system updates future movements to compensate for the errors, a phenomenon called sensorimotor adaptation. One component of this adaptation is thought to be driven by sensory prediction errors-discrepancies between predicted and actual sensory feedback. However, the mechanisms by which prediction errors drive adaptation remain unclear. Here, auditory prediction error-based mechanisms involved in speech auditory-motor adaptation were examined via the feedback aware control of tasks in speech (FACTS) model. Consistent with theoretical perspectives in both non-speech and speech motor control, the hierarchical architecture of FACTS relies on both the higher-level task (vocal tract constrictions) as well as lower-level articulatory state representations. Importantly, FACTS also computes sensory prediction errors as a part of its state feedback control mechanism, a well-established framework in the field of motor control. We explored potential adaptation mechanisms and found that adaptive behavior was present only when prediction errors updated the articulatory-to-task state transformation. In contrast, designs in which prediction errors updated forward sensory prediction models alone did not generate adaptation. Thus, FACTS demonstrated that 1) prediction errors can drive adaptation through task-level updates, and 2) adaptation is likely driven by updates to task-level control rather than (only) to forward predictive models. Additionally, simulating adaptation with FACTS generated a number of important hypotheses regarding previously reported phenomena such as identifying the source(s) of incomplete adaptation and driving factor(s) for changes in the second formant frequency during adaptation to the first formant perturbation. The proposed model design paves the way for a hierarchical state feedback control framework to be examined in the context of sensorimotor adaptation in both speech and non-speech effector systems.

Multimodal impressions of voice quality settings: the role of vocal and visual symbolism

This study considers instances of voice quality settings under a sound-symbolic and synesthetic perspective, focusing on the auditory impressions these settings might have on listeners' attributions of meaning effects and associations between vocal and visual features related to emotional expression. Three perceptual experiments are carried out. The first experiment examined the impressionistic effects of eight voice quality settings characterized by differences in pitch. The second experiment examined the impressionistic effects of seven voice quality settings characterized by productions with the presence or absence of turbulent airflow, irregularity, and tenseness. The third experiment investigated associations between facial expressions of basic emotions and voice quality characteristics. Data are considered in terms of acoustic (fundamental frequency values), articulatory (reduced or expanded length of the vocal tract), perceptual impressions of size (big/small), strength (strong/weak), brightness (dark, clear), and distinctiveness (muffled/distinct), and visual features (facial expressions of the basic emotions sadness, happiness, anger, disgust, fear, and neutrality). The results provide corroborating evidence of existing links between sound and meaning and are discussed in relation to the frequency, production, sirenic biological codes, phonetic metaphors, and the vocal and facial gestures involved in emotional expression.

Real-Time Vocal Tract Model for Elongation of Segment Lengths in a Waveguide Model

A vocal tract model based on a digital waveguide is presented in which the vocal tract has been decomposed into uniform cylindrical segments of variable lengths. We present a model for the real-time numerical solution of the digital waveguide equations in a uniform tube with the temporally varying cross section. In the current work, the uniform cylindrical segments of the vocal tract may have their different lengths, the time taken by the sound wave to propagate through a cylindrical segment in an axial direction may not be an integer multiple of each other. In such a case, the delay in an axial direction is necessarily a fractional delay. For the approximation of fractional-delay filters, Lagrange interpolation is used in the current model. Variable length of the individual segment of the vocal tract enables the model to produce realistic results. These results are validated with accurate benchmark model. The proposed model has been devised to elongate or shorten any arbitrary cylindrical segment by a suitable scaling factor. This model has a single algorithm and there is no need to make section of segments for elongation or shortening of the intermediate segments. The proposed model is about 23% more efficient than the previous model.

Transfer Characteristics of Vocal Tract Closed by Mask Cavity

This paper analyses the changes in transfer characteristics of the vocal tract when closed by a mask, i.e. a chamber. The analysis was performed in two ways: by analytical estimation and by measurements in the vocal tract physical model for the case of mask with inner volume V = 430 cm3, corresponding to the oxygen masks used in combat airplanes. It was shown that closing the vocal tract with a mask cavity increases the first formant frequency by about 10% in front and high vowels (/e/, /i/, and /u/) and the frequencies of the first two formants by about 5% in the remaining two vowels (/a/ and /o/). It was also revealed that longitudinal and transversal resonances in the mask chamber can lead to errors in the recognition of the vowel formant frequencies. The results point to the need for additional knowledge about resonances in mask application.

Influence of Overpressure Breathing on Vowel Formant Frequencies

Voice controlled management systems are based on speech recognition techniques. The use of such systems in combat aircraft is complex due to a number of critical factors which affect the accuracy of speech recognition, such as high level of ambient noise and vibration, use of oxygen masks, serious psycho-physical stress of speakers, etc. One of the specificity of the oxygen mask application is overpressure breathing. The results of the simulations presented in this paper show that the presence of overpressure on the order of 1000 Pa in the vocal tract has a significant influence on the first two formant frequencies. The formants discrimination field is significantly reduced when oxygen mask is used, influencing both perceptive and automatic discrimination of spoken vowels.

Non-uniform Rectilinear Grid in the Waveguide Modeling of the Vocal Tract

For many years, a digital waveguide model is being used for sound propagation in the modeling of the vocal tract with the structured and uniform mesh of scattering junctions connected by same delay lines. There are many varieties in the formation and layouts of the mesh grid called topologies. Current novel work has been dedicated to the mesh of two-dimensional digital waveguide models of sound propagation in the vocal tract with the structured and non-uniform rectilinear grid in orientation. In this work, there are two types of delay lines: one is called a smaller-delay line and other is called a larger-delay line. The larger-delay lines are the double of the smaller delay lines. The scheme of using the combination of both smaller- and larger-delay lines generates the non-uniform rectilinear two-dimensional waveguide mesh. The advantage of this approach is the ability to get a transfer function without fractional delay. This eliminates the need to get interpolation for the approximation of fractional delay and give efficient simulation for sound wave propagation in the two-dimensional waveguide modeling of the vocal tract. The simulation has been performed by considering the vowels /ɔ/, /a/, /i/ and /u/ in this work. By keeping the same sampling frequency, the standard two-dimensional waveguide model with uniform mesh is considered as our benchmark model. The results and efficiency of the proposed model have compared with our benchmark model.

Monitoring the Effect of Levodopa Using Sustained Phonemes in Parkinson's Disease Patients.

Parkinson's disease (PD) is a neurological disease identified by multiple symptoms, and levodopa is one of the most effective medications for treating the disease. To determine the dosage of levodopa, it is necessary to meet on a regular basis and observe motor function. The early detection and progression of the disease have been proposed using hypokinetic dysarthria. However, previous studies have not examined the effects of levodopa on speech rigorously and have provided inconsistent results. In this study, three sustained phonemes of PD patients were investigated for the effect of medication. A set of features characterizing vocal fold dynamics as well as the vocal tract coordinators were extracted from the sustained phonemes /of 28 PD patients during levodopa medication off and on states. All the features were statistically investigated and classified using a linear discriminant analysis (LDA) classifier. LDA classifier identified medication on from medication off based on the combined features from phoneme /a/, /o/ and /m/ with the accuracy=82.75% and F1-score=82.18%. Voice recording of PD patients during sustained phonemes /a/, /o/ and /m/ has the potential for identifying whether the patients are in On state or Off state of medication.Clinical Relevance- The outcomes of this study have the potential to monitor the effect and progress of levodopa on PD patients.

The Change of Vocal Tract Length in People with Parkinson's Disease.

Hypokinetic dysarthria is one of the early symptoms of Parkinson's disease (PD) and has been proposed for early detection and also for monitoring of the progression of the disease. PD reduces the control of vocal tract muscles such as the tongue and lips and, therefore the length of the active vocal tract is altered. However, the change in the vocal tract length due to the disease has not been investigated. The aim of this study was to determine the difference in the apparent vocal tract length (AVTL) between people with PD and age-matched control healthy people. The phoneme, /a/ from the UCI Parkinson's Disease Classification Dataset and the Italian Parkinson's Voice and Speech Dataset were used and AVTL was calculated based on the first four formants of the sustained phoneme (F1-F4). The results show a correlation between Parkinson's disease and an increase in vocal tract length. The most sensitive feature was the AVTL calculated using the first formants of sustained phonemes (F1). The other significant finding reported in this article is that the difference is significant and only appeared in the male participants. However, the size of the database is not sufficiently large to identify the possible confounding factors such as the severity and duration of the disease, medication, age, and comorbidity factors.Clinical relevance-The outcomes of this research have the potential to improve the identification of early Parkinsonian dysarthria and monitor PD progression.

Bora's high vowels involve a two-way dental contrast, not a three-way backness contrast

The Bora language of Peru has six phonemic vowels, conventionally transcribed as /i ε a o ɨ ɯ/. This inventory is noteworthy since it appears to exhibit a three-way backness distinction among three high unrounded vowels. In this paper we present audiovisual data which confirms two significant facts: (1) /ɯ/ is high, back, and unrounded; and (2) the only vowel produced with lip rounding is /o/. However, the experiment further reveals that the Bora segment traditionally written /ɨ/ is actually not central but rather a possibly novel type of front vowel, articulated with substantial contact between the tongue and both the upper and lower teeth. We use the methodology of Iskarous (2010) to estimate vocal tract area functions for Bora vowels based on their observed formant frequencies and amplitudes. The results of this procedure support our claim that /ɨ/ is front and /ɯ/ is back. Several phonological alternations also motivate this characterization. The opening of the mouth and dental contact visible in the production of /ɨ/ are correlated with its acoustic measurements, albeit in a previously undocumented way. We conclude that Bora’s /ɨ/ should be reclassified as /i̪/ – front and dental.p { line-height: 115%; text-align: left; orphans: 2; widows: 2; margin-bottom: 0.1in; direction: ltr; background: transparent }

Deriving Vocal Fold Oscillation Information from Recorded Voice Signals Using Models of Phonation.

During phonation, the vocal folds exhibit a self-sustained oscillatory motion, which is influenced by the physical properties of the speaker's vocal folds and driven by the balance of bio-mechanical and aerodynamic forces across the glottis. Subtle changes in the speaker's physical state can affect voice production and alter these oscillatory patterns. Measuring these can be valuable in developing computational tools that analyze voice to infer the speaker's state. Traditionally, vocal fold oscillations (VFOs) are measured directly using physical devices in clinical settings. In this paper, we propose a novel analysis-by-synthesis approach that allows us to infer the VFOs directly from recorded speech signals on an individualized, speaker-by-speaker basis. The approach, called the ADLES-VFT algorithm, is proposed in the context of a joint model that combines a phonation model (with a glottal flow waveform as the output) and a vocal tract acoustic wave propagation model such that the output of the joint model is an estimated waveform. The ADLES-VFT algorithm is a forward-backward algorithm which minimizes the error between the recorded waveform and the output of this joint model to estimate its parameters. Once estimated, these parameter values are used in conjunction with a phonation model to obtain its solutions. Since the parameters correlate with the physical properties of the vocal folds of the speaker, model solutions obtained using them represent the individualized VFOs for each speaker. The approach is flexible and can be applied to various phonation models. In addition to presenting the methodology, we show how the VFOs can be quantified from a dynamical systems perspective for classification purposes. Mathematical derivations are provided in an appendix for better readability.

MALip: Modal Amplification Lipreading based on reconstructed audio features

Lipreading is a type of human–computer interaction based on visual information. From the perspective of pronunciation principle, lip is only one of the vocal tract organs, and it is difficult to express the complete pronunciation process. It is very challenging to recognize speech contents solely through lip movements. As the shape of the vocal tract determines the final sound during the pronunciation process, we propose to improve the accuracy of lipreading by concurrently exploiting the lip features and reconstructed vocal tract features, and we call our method Modal Amplification Lipreading (MALip). We extend the U-Net model to learn the vocal tract features from Mel-Spectrogram features extracted from audio. Our model aims to reduce the computation complexity while ensuring the extended vocal tract features to have a good quality. We also introduce techniques to ensure the vocal tract features to be effective without being compromised by interference from noise or invalid audio. In addition, to facilitate the study with the incorporation of vocal tract features, we recorded a maximum sentence-level Chinese dataset ICSLR in an experimental environment, and verify the effectiveness of reconstructed audio features in improving the lip recognition accuracy for the first time. Through extensive experiments on ICSLR and publicly natural sentence dataset CMLR, we demonstrate the effectiveness of our MALip method compared with state-of-the-art counterparts.

Agreement and disagreement between major emotion recognition systems

The evaluation of systems that claim to recognize emotions expressed by human beings is a contested and complex task: The early pioneers in this field gave the impression that these systems will eventually recognize a flash of anger, suppressed glee/happiness, momentary disgust or contempt, lurking fear, or sadness in someone’s face or voice (Picard and Klein, 2002; Schuller et al., 2011). Emotion recognition systems are trained on ‘labelled’ databases — collection of video/audio recording comprising images and voices of humans enacting one emotional state. Machine learning programmes then regress the pixel distributions or wave forms against the labels. The system is then said to have learnt how to recognize and interpret human emotions and rated using information science metrics. These systems are adopted by the world at large for applications ranging from autistic spectrum communications to teaching and learning, and onwards to covert surveillance. The training databases depend upon human emotions recorded in ideal conditions — faces looking at the camera and centrally located, voices articulated through noise-cancelling microphones. Yet there are reports that the posed training data set, that is racially-skewed and gender unbalanced, does not prepare these systems to cope with data-in-the-wild and that expression-unrelated variations (like illumination, head pose, and identity bias (Li and Deng, 2020)) can impact their performance as well. Deployments of these systems tend to adopt one or other and apply it to data collected outside laboratory conditions and use the resulting classifications in subsequent processing. We have devised a testing method that helps to quantify the similarities and differences of facial emotion recognition systems (FER) and speech emotion recognition systems (SER). We report on the development of a data base comprising videos and sound track of 64 politicians and 7 government spokespersons (25 F, 46 M; 34 White Europeans, 19 East Asians, and 18 South Asians), ranging in age from 32–85 years, and each of the 71 has on average three 180 s videos; a total of 16.66 h of data. We have compared the performance of two FERs (Emotient and Affectiva) and two SERs (OpenSmile and Vokaturi) on our data by analysing emotions reported by these systems on a frame-by-frame basis. We have analysed the directly observable head movements, and the indirectly observable muscle movement parts of the face and for the muscle movements in the vocal tract. There was marked disagreement in emotions recognized, and the differences were exacerbated more women than for men, and more for South and East Asians than for White Europeans. Levels of agreement and disagreement on both high-level (i.e. emotion labels) and lower-level features (e.g. Euler angles of head movement) are shown. We show that inter-system disagreement may also be used as an effective response variable in reasoning about data features that influence disagreement. We argue that reliability of subsequent processing in approaches that adopt these systems may be enhanced by restricting action to cases where systems agree within a given tolerance level. This paper may be considered as a foray into the greater debate about the so-called algorithmic (un)fairness and data bias in the development and deployment of machine learning systems of which FERs and SERs are a good exemplar.

Penguins perceive variations of source- and filter-related vocal parameters of species-specific vocalisations.

Animal vocalisations encode a wide range of biological information about the age, sex, body size, and social status of the emitter. Moreover, vocalisations play a significant role in signalling the identity of the emitter to conspecifics. Recent studies have shown that, in the African penguin (Spheniscus demersus), acoustic cues to individual identity are encoded in the fundamental frequency (F0) and resonance frequencies (formants) of the vocal tract. However, although penguins are known to produce vocalisations where F0 and formants vary among individuals, it remains to be tested whether the receivers can perceive and use such information in the individual recognition process. In this study, using the Habituation-Dishabituation (HD) paradigm, we tested the hypothesis that penguins perceive and respond to a shift of ± 20% (corresponding to the natural inter-individual variation observed in ex-situ colonies) of F0 and formant dispersion (ΔF) of species-specific calls. We found that penguins were more likely to look rapidly and for longer at the source of the sound when F0 and formants of the calls were manipulated, indicating that they could perceive variations of these parameters in the vocal signals. Our findings provide the first experimental evidence that, in the African penguin, listeners can perceive changes in F0 and formants, which can be used by the receiver as potential cues for the individual discrimination of the emitter.

Comparison and Relation Between Vocal Tract Discomfort and Voice Handicap Index in Teachers With and Without Muscle Tension Dysphonia

Objective One of the most voice-related complaints in teachers is vocal tract discomfort (VTD) which can increase the voice handicap index (VHI) to different degrees. In teachers Muscle Tension Dysphonia (MTD), increases voice complaints and decreases the voice-related quality of life. The aims of this study were to evaluate and compare the frequency and severity subscales of the VTD score and total score of VHI and determine the relation between them in primary school teachers with and without MTD. Materials & Methods This study was cross-sectional and was performed on 80 primary female teachers with and without MTD. The participants in both groups were randomly selected after checking the inclusion criteria. Voice history, auditory-perceptual assessment of voice, palpation, and laryngeal video-stroboscopy were assessed on the teachers in two groups. Then, asked the participants to complete the Persian versions of the VTD and VHI scales. After determining the normal distribution of the data using the Kolmogorov-Smirnov test, the results of VTD, VHI, and their subscales were compared between two groups with an independent t-test. Also, the relationship between them was assessed with the Pearson correlation coefficient analysis. Results The mean score of the frequency subscale of VTD in teachers with MTD was more than in teachers without MTD (30.17±5.11, 8.22±2.26), respectively. The score of severity subscales of VTD in teachers with MTD was 39.12±4.94 and in teachers without MTD was 7.89±2.13. Also, the total score of the VHI questionnaire in teachers with MTD was significantly higher than teachers without MTD (P<0.05). Moreover, there was a significant positive correlation between the frequency and severity subscales of VTD and the total score of VHI in the two groups (P<0.05). Conclusion This study showed the frequency and severity scores of the VTD and the degree of VHI experienced by MTD are remarkably higher than teachers without MTD. The authors emphasize on the use of VTD and VHI scales in screening teachers who are at risk of voice problems, and the researchers pointed out the importance of paying attention to the reduction of discomfort feelings in the vocal tract and voice handicap in the voice therapy sessions of teachers with MTD.

Evaluation of Immediate Effects of Straw Phonation Exercise and Determination of the Ideal Performance Time in Healthy Adults

The ideal performance time of straw phonation exercise, one of the semi-occluded vocal tract exercise types, in which optimal effects are seen in healthy voices is still unknown. The aim of this study was to investigate the immediate effect of straw phonation exercise in healthy adults after 1, 3, 5, 7, and 10 minutes, considering: acoustic parameters, electroglottographic parameters, auditory-perceptual general voice quality, the participants' self-assessment regarding vocal effort, and laryngeal discomfort during exercise. This study included 69 participants (35 women, 34 men) with a healthy voice. Participants performed the exercise with straw for a total of 10 minutes. The data were collected before (m0) and immediately after exercises at the first (m1), third (m3), fifth (m5), seventh (m7), and tenth (m10) minutes of performance using sustained emission of vowel /a/ and a sentence. There was an increase in the fundamental frequency at m7 and a decrease in the shimmer from m5 in men. The sound pressure level increased at m5 and m10 in women and from m1 in men. The Acoustic Voice Quality Index score decreased at m5 in men. The mean average closed quotient decreased at m10 in women. Auditory-perceptual general voice quality was optimal at m3 in women and at m5 in men, it deteriorated at m7 in women and at m10 in men. The vocal effort during the exercise increased at m7 in women and at m10 in men. The laryngeal discomfort during the exercise increased for both sexes at m10. The straw phonation exercise in healthy individuals promoted optimal effects on voice quality after 5 and 7 minutes for men and 3 and 5 minutes for women. Prolongation of the exercise time causes negative effects on voice quality and an increase in vocal effort and laryngeal discomfort presented by the participants.

Comparison of Aerosol Emissions during Specific Speech Tasks

Recent investigations into the behavior of aerosolized emissions from the oral cavity have shown that particulate emissions do indeed occur during speech. To date, there is little information about the relative contribution of different speech sounds in producing particle emissions in a free field. This study compares airborne aerosol generation in participants producing isolated speech sounds: fricative consonants, plosive consonants, and vowel sounds. Prospective, reversal experimental design, where each participant served as their own control and all participants were exposed to all stimuli. While participants produced isolated speech tasks, a planar beam of laser light, a high-speed camera, and image software calculated the number of particulates detected over time. This study compared airborne aerosols emitted by human participants at a distance of 2.54cm between the laser sheet and the mouth. Statistically significant increases in particulate count over ambient dust distribution for all speech sounds. When collapsed across loudness levels, emitted particles in vowel sounds were statistically greater than consonants, suggesting that mouth opening, as opposed to the place of vocal tract constriction or manner of sound production, might also be influential in the degree to which particulates become aerosolized during speech. The results of this research will inform boundary conditions for computational models of aerosolized particulates during speech.

Comparison of Remote-Audiovisual Teaching and Face-to-Face Teaching on Factors Causing Teacher’s Voice Problems and Subjective Voice Evaluation

Objectives: Due to the global pandemic caused by the coronavirus (COVID-19), many remote-audiovisual teaching methods have been introduced. Remote-audiovisual teaching may lead to changes in teacher’s voice usage patterns. Therefore, it is necessary to compare between remote-audiovisual teaching and traditional face-to-face teaching methods in regards to the factors that affect teacher’s voice problems and subjective voice evaluation. Methods: This study investigated whether there was significant difference between two teaching methods on the risk factors (environment, physical, vocal use, psychological) causing voice problems and subjective voice evaluations (vocal effort ratings, vocal tract discomfort scale). Seventy-nine teachers in elementary, middle, and high schools participated in this study. Results: Face-to-face teaching had significantly higher scores when compared with remote-audiovisual teaching for environmental and voice use factors. For psychological factors, remote-audiovisual teaching showed significantly higher scores than face-to-face teaching. There were no significant differences in physical factors between the two teaching methods. In subjective voice evaluation, face-to-face teaching showed significantly higher scores compared to remote-audiovisual teaching in vocal effort ratings and frequency and severity of vocal tract discomfort scale. Conclusion: This study shows that teachers not only require more vocal loading and vocal effort in face-to-face teaching compared to remote-audiovisual teaching, but also experience high vocal discomfort. Therefore, that the results of this study suggest that the remote-audiovisual teaching method allows teachers to use their voice more comfortably in comparison to face-to-face teaching.

Estimating Sample Area Functions of Human Vocal Tracts in Emotional Speech Signals

Estimating Sample Area Functions of Human Vocal Tracts in Emotional Speech Signals

Image Cryptosystem in Optical Gyrator Transform Domain Using Audio Keys

Two remote sensing image encryption algorithms based on the randomness of audio channel sequences are proposed and their robustness is verified by many attack experiments. The first encryption algorithm uses the randomness of audio to encrypt image pixels in RMD. Compared with traditional image encryption algorithms, this algorithm has high randomness and security and can resist stronger password attacks. However, the encryption algorithm takes a long time. Considering that information sometimes needs to be transmitted urgently, a second encryption algorithm is proposed. By combining vocal tract and random phase to form new public and private keys, the number of computation amount and dislocations is reduced. The second algorithm is short in time but low in security.