Vocal Tract Characteristics Research Articles

Normalization of speech kinematic data for characterizing primary muscle tension dysphonia

Primary muscle tension dysphonia (pMTD) is a voice disorder of unknown etiology in which people have reduced volume and become easily fatigued. The tongue has biomechanical linkage to the laryngeal motor system that affects the voice, suggesting that tongue movement variability might be a marker for this disorder. Previous studies of healthy/disordered speech have reduced individual talker differences in physical vocal tract characteristics via normalization procedures. Here, we obtained tongue movement data for diadochokinetic sequences produced by healthy adult talkers and people with pMTD. We used an electromagnetic articulography system to test three healthy adult talkers and a participant with pMTD. We recorded vertical displacement of the tongue dorsum for sustained /a/ and 30 sec of rapidly repeated /pataka/. Displacement (mm) was scaled relative to each talker’s sustained vowel production, while movement time (msec) was normalized by dividing each /ka/ portion by the corresponding /pataka/ utterance length. The results suggest (1) normalization effectively reduced talker variability in tongue movement displacement and timing, and (2) the talker with pMTD showed markedly higher temporal variability, compared to the control talkers. We are currently testing this putative group difference using additional participants.

Transfer Characteristics of Vocal Tract Closed by Mask Cavity

This paper analyses the changes in transfer characteristics of the vocal tract when closed by a mask, i.e. a chamber. The analysis was performed in two ways: by analytical estimation and by measurements in the vocal tract physical model for the case of mask with inner volume V = 430 cm3, corresponding to the oxygen masks used in combat airplanes. It was shown that closing the vocal tract with a mask cavity increases the first formant frequency by about 10% in front and high vowels (/e/, /i/, and /u/) and the frequencies of the first two formants by about 5% in the remaining two vowels (/a/ and /o/). It was also revealed that longitudinal and transversal resonances in the mask chamber can lead to errors in the recognition of the vowel formant frequencies. The results point to the need for additional knowledge about resonances in mask application.

MALip: Modal Amplification Lipreading based on reconstructed audio features

Lipreading is a type of human–computer interaction based on visual information. From the perspective of pronunciation principle, lip is only one of the vocal tract organs, and it is difficult to express the complete pronunciation process. It is very challenging to recognize speech contents solely through lip movements. As the shape of the vocal tract determines the final sound during the pronunciation process, we propose to improve the accuracy of lipreading by concurrently exploiting the lip features and reconstructed vocal tract features, and we call our method Modal Amplification Lipreading (MALip). We extend the U-Net model to learn the vocal tract features from Mel-Spectrogram features extracted from audio. Our model aims to reduce the computation complexity while ensuring the extended vocal tract features to have a good quality. We also introduce techniques to ensure the vocal tract features to be effective without being compromised by interference from noise or invalid audio. In addition, to facilitate the study with the incorporation of vocal tract features, we recorded a maximum sentence-level Chinese dataset ICSLR in an experimental environment, and verify the effectiveness of reconstructed audio features in improving the lip recognition accuracy for the first time. Through extensive experiments on ICSLR and publicly natural sentence dataset CMLR, we demonstrate the effectiveness of our MALip method compared with state-of-the-art counterparts.

A study of correlation between physiological process of articulation and emotions on Mandarin Chinese

The goal of this work is to investigate the correlation between physiological process of articulation and emotion expressing modes via speech of Mandarin Chinese. A bimodal emotional speech dataset with simultaneous speech and articulatory data is developed for both acoustic and articulatory analysis to explore the intrinsic mechanism of speech emotion expression with the dynamic movement characteristics of vocal tract. The articulation information is presented by the electromagnetic articulography (EMA) signal, which reflects vocal tract information by collecting the movement of tongue, lip, etc. The correlation between the articulatory features and emotion expressing is verified with automatic emotion recognition performance on 4 typical emotions as neutral, joy, anger, and sadness, where the moderate and intense states on the last three are considered separately to make a total of 7 emotional states. Results show that the articulatory information itself can deliver a part of speech emotion clues, and with comparison to mono-modal speech signal, the emotion recognition performance by bimodal signal with both speech and EMA can be significantly improved with an improvement of 9.75%. This improvement proved that study on physiological process of articulation during voicing of emotional speech can be an effective breakthrough on emotional speech research.

Simulating vocal learning of spoken language: Beyond imitation

Computational approaches have an important role to play in understanding the complex process of speech acquisition, in general, and have recently been popular in studies of vocal learning in particular. In this article we suggest that two significant problems associated with imitative vocal learning of spoken language, the speaker normalisation and phonological correspondence problems, can be addressed by linguistically grounded auditory perception. In particular, we show how the articulation of consonant–vowel syllables may be learnt from auditory percepts that can represent either individual utterances by speakers with different vocal tract characteristics or ideal phonetic realisations. The result is an optimisation-based implementation of vocal exploration – incorporating semantic, auditory, and articulatory signals – that can serve as a basis for simulating vocal learning beyond imitation.

VocalPrint: A mmWave-Based Unmediated Vocal Sensing System for Secure Authentication

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VocalPrint</i> , a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VocalPrint</i> exploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the vocal tract and vocal source features and input them into an ensemble classifier for authentication. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VocalPrint</i> is practical as it allows the effortless transition to a smartphone while having sufficient usability due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VocalPrint</i> achieves over 96 percent authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Links between language, fluid dynamics, and the airflows that transport pathogens

Growing evidence shows that the airflows that accompany speech during social interactions contribute to the transport of pathogens, such as the SARS-CoV-2 virus. However, it is still elusive how the manners of articulation during speaking, such as time-varying airflow rate, and instantaneous teeth and lip movements, may affect the transport features of airflow exiting the mouth. We combine experimental and numerical approaches to investigate the flow patterns produced by representative vowels, such as /a/, /o/, and /i/, and consonants, such as /p/, /k/, /s/, and /h/, that have distinct articulatory characteristics. A 3D vocal tract is modeled with a temporally varying exit that captures key morphologic and kinematic features of the human vocal tract, including teeth and lips, during speaking. An incompressible flow solver based on a sharp-interface immersed-boundary-method (IBM) is employed to compute the resultant airflow. By comparing representative utterance pairs, like /apa/ and /aha/, we are able to isolate the effect of articulatory features and show significant differences in spatial-temporal patterns of airflow under the influences of a time-varying orifice, different orifice aspect ratios, and different relative lip-teeth positions. This work helps bring insight into the understanding of articulatory phonetics, and the links to different languages, from a fluid dynamics perspective.

Human Hyoid Descent is Adaptive rather than Maladaptive for Swallowing Performance: An Experimental and Computational Argument

PurposeDistinctive features of the human supralaryngeal vocal tract have been argued to be adaptive for speech and maladaptive for swallowing. However, this maladaptive hypothesis has not been thoroughly tested and in fact is refuted by studies of normal human feeding physiology. This study validates a computational model of hyolingual biomechanics derived from an previously validated experimental nonhuman primate model system and examines how changes in hyolingual and craniofacial morphology over the course of human evolution and ontogeny affect hyolingual biomechanics during swallowing. In particular, we examine superoanterior excursion of the hyoid, which is often reduced in human patients with dysphagia and therefore used here as a surrogate for swallowing performance.MethodsA computational model of hyolingual range of motion during swallowing in Macaca mulatta was developed in R using in vivo data. Model outputs were compared against in vivo macaque data to determine the model's goodness of fit using a receiver operating characteristics (ROC) curve; model predictive power was estimated using the area under the ROC curve. Several aspects of macaque craniofacial and hyoid morphology were then altered towards the human condition with consequent adjustments made to muscle lengths, and the model calculated hyoid displacement during swallowing for each condition. A model was considered viable if its range of motion included a target of hyoid excursion derived from in vivo macaque and human data.ResultsThe macaque model had a sensitivity of 0.79, specificity of 0.82, and accuracy of 88% compared to in vivo data. A humanoid model predicts hyoid excursion within normal limits of healthy humans. Altering macaque morphology to approach the human condition by lowering the hyoid and making the symphysis vertical had little effect on model performance. Isolated shortening of the mandible significantly decreased hyoid excursion unless the symphysis was also made more vertical. Moreover, as the model was made more humanoid, both hyoid descent and physiologically maximal muscle shortening were necessary to approximate target hyoid excursion.ConclusionsHyoid descent—which is commonly assumed to be detrimental to swallowing—may actually maintain swallowing performance as the human face has shortened in recent evolutionary history. Moreover, because mandibular morphology affects suprahyoid muscle length, selective pressure to maintain swallowing performance may have resulted in the emergence of humanity’s characteristic vertical mandibular symphysis. Geometric constraints require human suprahyoid muscles to perform at their physiological limits when the hyoid is descended. Such extreme performance requirements may leave humans vulnerable to dysphagia not because of airway morphology alone but because of physiologic constraints.

Hands-Free Wearable Electrolarynx using Linear Predictive Coding Residual Waves and Listening Evaluation

A conventional electrolarynx (EL), which is used by laryngectomees, produces monotonous sound and occupies a user's hand; hence, we developed a hands-free wearable device that improves voice quality. The proposed device estimates individual vocal tract features using linear predictive coding (LPC) and generates sound vibrations using an LPC inverse filter. Additionally, we reproduced the vibration sound using a transducer and amplified the first harmonic frequency and the second one. We conducted an objective experiment to compare the spectra of natural voice, a conventional EL, and the proposed device. We also conducted a subjective experiment in which we asked healthy subjects to listen to and evaluate the conventional EL and the proposed device. The results of the objective experiment demonstrated that our model was characterized by two formant peaks that were similar to the conventional EL and the natural voice. The results of the subjective experiment demonstrated that our model was more powerful and clearer than the conventional EL. These findings indicate that the voice of our device is spectrally close to human voice and gives the audience a more powerful and clearer sound.

Analyzing the vocal tract characteristics for out-of-breath speech.

In this work, vocal tract characteristic changes under the out-of-breath condition are explored. Speaking under the influence of physical exercise is called out-of-breath speech. The change in breathing pattern results in perceptual changes in the produced sound. For vocal tract, the first four formants show a lowering in their average frequency. The bandwidths BF1 and BF2 widen, whereas the other two get narrowed. The change in bandwidth is small for the last three. For a speaker, the change in frequency and bandwidth may not be uniform across formants. Subband analysis is carried out around formants for comparing the variation of the vocal tract with the source. A vocal tract adaptive empirical wavelet transform is used for extracting formant specific subbands from speech and source. The support vector machine performs the subband-based binary classification between the normal and out-of-breath speech. For all speakers, it shows an F1-score improvement of 4% over speech subbands. Similarly, a performance improvement of 5% can be seen for both male and female speakers. Furthermore, the misclassification amount is less for source compared to speech. These results suggest that physical exercise influences the source more than the vocal tract.

On the Speech Properties and Feature Extraction Methods in Speech Emotion Recognition.

Many speech emotion recognition systems have been designed using different features and classification methods. Still, there is a lack of knowledge and reasoning regarding the underlying speech characteristics and processing, i.e., how basic characteristics, methods, and settings affect the accuracy, to what extent, etc. This study is to extend physical perspective on speech emotion recognition by analyzing basic speech characteristics and modeling methods, e.g., time characteristics (segmentation, window types, and classification regions—lengths and overlaps), frequency ranges, frequency scales, processing of whole speech (spectrograms), vocal tract (filter banks, linear prediction coefficient (LPC) modeling), and excitation (inverse LPC filtering) signals, magnitude and phase manipulations, cepstral features, etc. In the evaluation phase the state-of-the-art classification method and rigorous statistical tests were applied, namely N-fold cross validation, paired t-test, rank, and Pearson correlations. The results revealed several settings in a 75% accuracy range (seven emotions). The most successful methods were based on vocal tract features using psychoacoustic filter banks covering the 0–8 kHz frequency range. Well scoring are also spectrograms carrying vocal tract and excitation information. It was found that even basic processing like pre-emphasis, segmentation, magnitude modifications, etc., can dramatically affect the results. Most findings are robust by exhibiting strong correlations across tested databases.

Robust vowel region detection method for multimode speech

The aim of this paper is to explore a robust method for vowel region detection from multimode speech. In realistic scenario, speech can be classified into three modes namely; conversation, extempore, and read. The existing method detects the vowel form the speech recorded in clean environment which may not be appropriate for the multimode speech tasks. To address this issue, we proposed an approach based on continuous wavelet transform coefficients and phone boundaries for detecting the vowel regions from different modes of the speech signal. For evaluation of the proposed vowel region (VR) detection technique, TIMIT (read speech) and Bengali (read, extempore, and conversation speech) corpora are used. The proposed VR detection technique is compared to the state-of-the-art methods. The experiments has recorded significant gain in the performance of the proposed technique than the state-of-the-art methods. The efficiency of the proposed technique is shown by extracting vocal tract and excitation source features from automatically detected VRs for developing the multilingual speech mode classification (MSMC) model. The evaluation results report that the performance of the MSMC model is significantly improved when features are extracted from the vowel regions than the entire speech utterance.

Speech Signal Analysis in Phase Domain

Speech signal analysis based on the Phase Spectrum (PS) in the recent years becomes more popular to the researchers due to its attractive properties. Though some problem arises during its processing but appropriate modification gives the useful result. In this study, we proposed a new method of speech signal analysis based on its phase spectral representation. After analyzing the PS we modified it vocal tract dominated spectrum by utilizing the minimum-phase stable component of the speech spectrum. The modified signal holding only the phase spectral component is used for the parametric modeling of the speech signal to estimate the characteristics of vocal tract perfectly. This extracted feature was found to provide complementary evidence from the magnitude spectrum of speech but with better resolution. According to perceptual analysis, the PS takes precedence over the magnitude spectrum. This research utilizes the Group Delay (GD) spectrum as a representative of the PS because of its meaningful characteristic. All pole modeling is performed on the signal evaluated from the GD spectrum to find the resonances of the vocal tract system accurately. The effectiveness of the method is tested by synthesizing some vowels over a range of pitch periods from low to high pitched speech. The validity of the proposed method is also verified by plotting the formant contour on the spectrogram of a sentence from the TIMIT database and standard F2-F1 plot of natural speech spoken by male and female speakers. The proposed method performs better than the state-of-the-art methods.

Effect of Music Training on Categorical Perception of Speech and Music.

Background and ObjectivesThe aim of this study is to evaluate the effect of music training on the characteristics of auditory perception of speech and music. The perception of speech and music stimuli was assessed across their respective stimulus continuum and the resultant plots were compared between musicians and non-musicians.Subjects and MethodsThirty musicians with formal music training and twenty-seven non-musicians participated in the study (age: 20 to 30 years). They were assessed for identification of consonant-vowel syllables (/da/ to /ga/), vowels (/u/ to /a/), vocal music note (/ri/ to /ga/), and instrumental music note (/ri/ to /ga/) across their respective stimulus continuum. The continua contained 15 tokens with equal step size between any adjacent tokens. The resultant identification scores were plotted against each token and were analyzed for presence of categorical boundary. If the categorical boundary was found, the plots were analyzed by six parameters of categorical perception; for the point of 50% crossover, lower edge of categorical boundary, upper edge of categorical boundary, phoneme boundary width, slope, and intercepts.ResultsOverall, the results showed that both speech and music are perceived differently in musicians and non-musicians. In musicians, both speech and music are categorically perceived, while in non-musicians, only speech is perceived categorically.ConclusionsThe findings of the present study indicate that music is perceived categorically by musicians, even if the stimulus is devoid of vocal tract features. The findings support that the categorical perception is strongly influenced by training and results are discussed in light of notions of motor theory of speech perception.

VocalLock

Recent years have witnessed the surge of biometric-based user authentication for mobile devices due to its promising security and convenience. As a natural and widely-existed behavior, human speaking has been exploited for user authentication. Existing voice-based user authentication explores the unique characteristics from either the voiceprint or mouth movements, which is vulnerable to replay attacks and mimic attacks. During speaking, the vocal tract, including the static shape and dynamic movements, also exhibits the individual uniqueness, and they are hardly eavesdropped and imitated by adversaries. Hence, our work aims to employ the individual uniqueness of vocal tract to realize user authentication on mobile devices. Moreover, most voice-based user authentications are passphrase-dependent, which significantly degrade the user experience. Thus, such user authentications are pressed to be implemented in a passphrase-independent manner while being able to resist various attacks. In this paper, we propose a user authentication system, VocalLock, which senses the whole vocal tract during speaking to identify different individuals in a passphrase-independent manner on smartphones leveraging acoustic signals. VocalLock first utilizes FMCW on acoustic signals to characterize both the static shape and dynamic movements of the vocal tract during speaking, and then constructs a passphrase-independent user authentication model based on the unique characteristics of vocal tract through GMM-UBM. The proposed VocalLock can resist various spoofing attacks, while achieving a satisfactory user experience. Extensive experiments in real environments demonstrate VocalLock can accurately authenticate user identity in a passphrase-independent manner and successfully resist various attacks.

Effects of Increase in Laryngeal Cross-sectional Area on Vocal Tract Characteristics in Human Whistling

Effects of Increase in Laryngeal Cross-sectional Area on Vocal Tract Characteristics in Human Whistling

Multilingual and multimode phone recognition system for Indian languages

The aim of this paper is to develop a flexible framework capable of automatically recognizing phonetic units present in a speech utterance of any language spoken in any mode. In this study, we considered two modes of speech: conversation and read modes in four Indian languages, namely, Telugu, Kannada, Odia, and Bengali. The proposed approach consists of two stages: (i) Automatic speech mode classification (SMC) and (ii) Automatic phoneme recognition using mode-specific multilingual phone recognition system (MMPRS). The vocal tract and excitation source features are considered for classifying speech modes using feed forward neural networks (FFNNs). The vocal tract, excitation source, and tandem features are used in training deep neural network (DNN)-based multilingual phone recognition systems (MPRSs). The performance of the proposed approach is compared with baseline mode-dependent and mode-independent MPRSs. Experimental results show that the proposed approach which combines both SMC and MMPRS into a single system outperforms the baseline phone recognition systems.

Detection and assessment of hypernasality in repaired cleft palate speech using vocal tract and residual features.

The presence of hypernasality in repaired cleft palate (CP) speech is a consequence of velopharyngeal insufficiency. The coupling of the nasal tract with the oral tract adds nasal formant and antiformant pairs in the hypernasal speech spectrum. This addition deviates the spectral and linear prediction (LP) residual characteristics of hypernasal speech compared to normal speech. In this work, the vocal tract constriction feature, peak to side-lobe ratio feature, and spectral moment features augmented by low-order cepstral coefficients are used to capture the spectral and residual deviations for hypernasality detection. The first feature captures the lower-frequencies prominence in speech due to the presence of nasal formants, the second feature captures the undesirable signal components in the residual signal due to the nasal antiformants, and the third feature captures the information about formants and antiformants in the spectrum along with the spectral envelope. The combination of three features gives normal versus hypernasal speech detection accuracies of 87.76%, 91.13%, and 93.70% for /a/, /i/, and /u/ vowels, respectively, and hypernasality severity detection accuracies of 80.13% and 81.25% for /i/ and /u/ vowels, respectively. The speech data are collected from 30 control normal and 30 repaired CP children between the ages of 7 and 12.

Automatic hypernasality grade assessment in cleft palate speech based on the spectral envelope method.

Due to velopharyngeal incompetence, airflow overflows from the oral cavity to the nasal cavity, which results in hypernasality. Hypernasality greatly reduces speech intelligibility and affects the daily communication of patients with cleft palate. Accurate assessment of hypernasality grades can provide assisted diagnosis for speech-language pathologists (SLPs) in clinical settings. Utilizing a support vector machine (SVM), this paper classifies speech recordings into four grades (normal, mild, moderate and severe hypernasality) based on vocal tract characteristics. Linear prediction (LP) analysis is widely used to model the vocal tract. Glottal source information may be included in the LP-based spectrum. The stabilized weighted linear prediction (SWLP) method, which imposes the temporal weights on the closed-phase interval of the glottal cycle, is a more robust approach for modeling the vocal tract. The extended weighted linear prediction (XLP) method weights each lagged speech signal separately, which achieves a finer time scale on the spectral envelope than the SWLP method. Tested speech recordings were collected from 60 subjects with cleft palate and 20 control subjects, and included a total of 4640 Mandarin syllables. The experimental results showed that the spectral envelope of normal speech decreases faster than that of hypernasal speech in the high-frequency part. The experimental results also indicate that the SWLP- and XLP-based methods have smaller correlation coefficients between normal and hypernasal speech than the LP method. Thus, the SWLP and XLP methods have better ability to distinguish hypernasal from normal speech than the LP method. The classification accuracies of the four hypernasality grades using the SWLP and XLP methods range from 83.86% to 97.47%. The selection of the model order and the size of the weight function are also discussed in this paper.

Unsupervised Phonetic Segmentation of Classical Arabic Speech Using Forward and Inverse Characteristics of the Vocal Tract

Automatic segmentation of speech is about identifying boundaries of phonemes in a given utterance. This paper presents a strategy driven by cosine distance similarity scores for identifying phoneme boundaries. The proposed strategy helps in the selection of appropriate feature extraction technique for speech segmentation applications. After assessing various state-of-the-art speech processing techniques, a novel combination of forward and inverse characteristics of vocal tract (FICV) is developed. The proposed technique is evaluated on Classical Arabic dataset. Extensive experiments are made to compare the proposed technique with state-of-the-art techniques, including the hidden Markov model-based forced alignment procedures. The results show that proposed technique has total error rate of 14.48%, while the accuracy is 85.2% within 10 ms alignment error. When compared with the existing state-of-the-art technique, the proposed technique outperforms by 12.29% and 22.73% in terms of error rates and alignment accuracies, respectively, which signifies the potential of using FICV in speech segmentation.