Acoustic Speech Signal Research Articles

On Short-Time Estimation of Vocal Tract Length from Formant Frequencies.

Vocal tract length is highly variable across speakers and determines many aspects of the acoustic speech signal, making it an essential parameter to consider for explaining behavioral variability. A method for accurate estimation of vocal tract length from formant frequencies would afford normalization of interspeaker variability and facilitate acoustic comparisons across speakers. A framework for considering estimation methods is developed from the basic principles of vocal tract acoustics, and an estimation method is proposed that follows naturally from this framework. The proposed method is evaluated using acoustic characteristics of simulated vocal tracts ranging from 14 to 19 cm in length, as well as real-time magnetic resonance imaging data with synchronous audio from five speakers whose vocal tracts range from 14.5 to 18.0 cm in length. Evaluations show improvements in accuracy over previously proposed methods, with 0.631 and 1.277 cm root mean square error on simulated and human speech data, respectively. Empirical results show that the effectiveness of the proposed method is based on emphasizing higher formant frequencies, which seem less affected by speech articulation. Theoretical predictions of formant sensitivity reinforce this empirical finding. Moreover, theoretical insights are explained regarding the reason for differences in formant sensitivity.

Speech perception of sine-wave signals by children with cochlear implants.

Children need to discover linguistically meaningful structures in the acoustic speech signal. Being attentive to recurring, time-varying formant patterns helps in that process. However, that kind of acoustic structure may not be available to children with cochlear implants (CIs), thus hindering development. The major goal of this study was to examine whether children with CIs are as sensitive to time-varying formant structure as children with normal hearing (NH) by asking them to recognize sine-wave speech. The same materials were presented as speech in noise, as well, to evaluate whether any group differences might simply reflect general perceptual deficits on the part of children with CIs. Vocabulary knowledge, phonemic awareness, and "top-down" language effects were all also assessed. Finally, treatment factors were examined as possible predictors of outcomes. Results showed that children with CIs were as accurate as children with NH at recognizing sine-wave speech, but poorer at recognizing speech in noise. Phonemic awareness was related to that recognition. Top-down effects were similar across groups. Having had a period of bimodal stimulation near the time of receiving a first CI facilitated these effects. Results suggest that children with CIs have access to the important time-varying structure of vocal-tract formants.

Left dorsal speech stream components and their contribution to phonological processing

Models propose an auditory-motor mapping via a left-hemispheric dorsal speech-processing stream, yet its detailed contributions to speech perception and production are unclear. Using fMRI-navigated repetitive transcranial magnetic stimulation (rTMS), we virtually lesioned left dorsal stream components in healthy human subjects and probed the consequences on speech-related facilitation of articulatory motor cortex (M1) excitability, as indexed by increases in motor-evoked potential (MEP) amplitude of a lip muscle, and on speech processing performance in phonological tests. Speech-related MEP facilitation was disrupted by rTMS of the posterior superior temporal sulcus (pSTS), the sylvian parieto-temporal region (SPT), and by double-knock-out but not individual lesioning of pars opercularis of the inferior frontal gyrus (pIFG) and the dorsal premotor cortex (dPMC), and not by rTMS of the ventral speech-processing stream or an occipital control site. RTMS of the dorsal stream but not of the ventral stream or the occipital control site caused deficits specifically in the processing of fast transients of the acoustic speech signal. Performance of syllable and pseudoword repetition correlated with speech-related MEP facilitation, and this relation was abolished with rTMS of pSTS, SPT, and pIFG. Findings provide direct evidence that auditory-motor mapping in the left dorsal stream causes reliable and specific speech-related MEP facilitation in left articulatory M1. The left dorsal stream targets the articulatory M1 through pSTS and SPT constituting essential posterior input regions and parallel via frontal pathways through pIFG and dPMC. Finally, engagement of the left dorsal stream is necessary for processing of fast transients in the auditory signal.

Left dorsal speech stream components and their contribution to phonological processing.

Models propose an auditory-motor mapping via a left-hemispheric dorsal speech-processing stream, yet its detailed contributions to speech perception and production are unclear. Using fMRI-navigated repetitive transcranial magnetic stimulation (rTMS), we virtually lesioned left dorsal stream components in healthy human subjects and probed the consequences on speech-related facilitation of articulatory motor cortex (M1) excitability, as indexed by increases in motor-evoked potential (MEP) amplitude of a lip muscle, and on speech processing performance in phonological tests. Speech-related MEP facilitation was disrupted by rTMS of the posterior superior temporal sulcus (pSTS), the sylvian parieto-temporal region (SPT), and by double-knock-out but not individual lesioning of pars opercularis of the inferior frontal gyrus (pIFG) and the dorsal premotor cortex (dPMC), and not by rTMS of the ventral speech-processing stream or an occipital control site. RTMS of the dorsal stream but not of the ventral stream or the occipital control site caused deficits specifically in the processing of fast transients of the acoustic speech signal. Performance of syllable and pseudoword repetition correlated with speech-related MEP facilitation, and this relation was abolished with rTMS of pSTS, SPT, and pIFG. Findings provide direct evidence that auditory-motor mapping in the left dorsal stream causes reliable and specific speech-related MEP facilitation in left articulatory M1. The left dorsal stream targets the articulatory M1 through pSTS and SPT constituting essential posterior input regions and parallel via frontal pathways through pIFG and dPMC. Finally, engagement of the left dorsal stream is necessary for processing of fast transients in the auditory signal.

Chapter 8 - Neural basis of speech perception

The functional neuroanatomy of speech processing has been difficult to characterize. One major impediment to progress has been the failure to consider task effects when mapping speech-related processing systems. We summarize a dual-stream model of speech processing that addresses this situation. In this model, a ventral stream processes speech signals for comprehension, and a dorsal stream maps acoustic speech signals to parietal and frontal-lobe articulatory networks. The model assumes that the ventral stream is largely bilaterally organized, although there are important computational differences between the left- and right-hemisphere systems, whereas the dorsal stream is strongly left-hemisphere-dominant.

What is the McGurk effect?

What is the McGurk effect?

Audio-Visual Speech Recognition for People with Speech Disorders

recognition of disorder people is a difficult task due to the lack of motor-control of the speech articulators. Multimodal speech recognition can be used to enhance the robustness of disordered speech. This paper introduces an automatic speech recognition system for people with dysarthria speech disorder based on both speech and visual components. The Mel-Frequency Cepestral Coefficients (MFCC) is used as features representing the acoustic speech signal. For the visual counterpart, the Discrete Cosine Transform (DCT) Coefficients are extracted from the speaker's mouth region. Face and mouth regions are detected using the Viola-Jones algorithm. The acoustic and visual input features are then concatenated on one feature vector. Then, the Hidden Markov Model (HMM) classifier is applied on the combined feature vector of acoustic and visual components. The system is tested on isolated English words spoken by disorder speakers from UA-Speech data. Results of the proposed system indicate that visual features are highly effective and can improve the accuracy to reach 7.91% for speaker dependent experiments and 3% for speaker independent experiments.

Priming in speech perception through captions and sign language

When listeners know the content of the message they are about to hear, distorted or partially masked speech appears dramatically more intelligible compared to when the content is unknown. In the current research this priming phenomenon was investigated quantitatively using a same-different task where the prime and auditory message match on only 50% of trials. The first experiment was concerned with the effect of the timing of typed captions relative to the auditory message. For nonsense sentences subjected to four different types of distortion and masking, optimum performance was achieved when the initiation of the text preceded the acoustic speech signal by 800 ms. Performance was slightly poorer with simultaneous delivery and much poorer when the auditory signal preceded the caption. The second experiment investigated whether priming effects could be observed when the prime was delivered by sign language rather than captions. Preliminary results with normal-hearing signers indicate that some benefits in same-different performance can be triggered by sign language primes. Confirmation of this result and extension to listeners with hearing impairment who use sign and speech together in simultaneous communication (SimCom) may inform best practices for the use of SimCom in educational and other settings. [Work supported by NIDCD 01625 and The Hearing Health Foundation.]

Objective evaluation of chronic laryngeal dysphonia by spectro- temporal analysis

In this paper we develop system dedicated to the objective characterization of dysphonia chronic laryngeal origin. The purpose of this system is threefold: diagnosis, treatment and monitoring of patient. . For this we proceed initially to the remote recording and archiving of an acoustic speech signal voiced in this case a sustained for three seconds. We then apply at the Otorhinolaryngology (ORL) department of the University Hospital of Tlemcen, different algorithms of objective assessment of three parameters in this case the fundamental frequency, Disturbance of short-term speech signal (jitter) and Short-Time discrete Fourier Transform (ST.DFT) that allow experts to assess the development of chronic dysphonia of tumor or inflammatory origin (larynx cancer, inflammatory polyp of the vocal cords, chronic laryngitis).

Bodo pametni nadzorni sistemi prisluhnili, razumeli in spregovorili slovensko?

Članek obravnava tehnologije govorjenega jezika, ki bi lahko omogočile t. i. pametnim nadzornim sistemom, da bi nekoč prisluhnili, razumeli in spregovorili slovensko. Tovrstni sistemi se z uporabo senzorjev in naprednih računalniških metod umetnega zaznavanja in razpoznavanja vzorcev do neke mere zavedajo okolja ter prisotnosti ljudi in drugih pojavov, ki bi lahko bili predmet varnostnega nadzora. Med tovrstne pojave spada tudi govor, ki lahko predstavlja ključni vir informacije pri določenih varnostnonadzornih okoliščinah. Tehnologije, ki omogočajo samodejno razpoznavanje in tvorjenje govora ter samodejno razpoznavanje govorcev in njihovega psihofizičnega stanja s pomočjo napredne računalniške analize govornega zvočnega signala, odpirajo povsem nove dimenzije razvoja pametnih nadzornih sistemov. Samodejno razpoznavanje varnostno sumljivih govornih izjav, kričanja in klicev na pomoč ter samodejno zaznavanje varnostno sumljivega psihofizičnega stanja govorcev tovrstnim sistemom doda pridih umetne inteligence. Članek predstavlja trenutno stanje razvoja omenjenih tehnologij in možnosti njihove uporabe za slovenski govorjeni jezik ter različne varnostnonadzorne scenarije uporabe tovrstnih sistemov. Naslovljena so tudi širša pravna in etična vprašanja, ki jih odpira razvoj in uporaba tovrstnih tehnologij. Govorni nadzor je namreč eno najbolj občutljivih vprašanj varstva zasebnosti.

Acoustic features mediating height estimation from human speech

The current experiment was aimed at providing the first direct evidence regarding the role of subglottal resonances in height discrimination. Past research has investigated whether or not listeners are able to discriminate which of two talkers is taller (Rendall et al., 2007), but has not established what parameters of the acoustic speech signal listeners use to distinguish speaker height. In the current study, we examined the role of subglottal resonances in height discrimination. Subglottal resonances generally refer to resonances of the lower airways starting in the lungs and terminating at the glottis. Subglottal resonances would be a good candidate for use in height estimation because they remain relatively stable across vowels and are relatively unaffected by other factors influencing the supra-laryngeal vocal tract; instead they are influenced nearly exclusively by the lower airway acoustic properties. Listeners participated in two tasks, the first task was a two-alternative forced choice (2AFC) height discrimination test, in which listeners heard sentences produced by two talkers of the same gender and were asked to determine which of the two was the tallest. The second task involved listening to five talkers (all same gender) sentences, and then ranking those individuals from tallest to shortest. Findings indicate that listeners are able to discriminate and rank speakers heights better than chance, and the role of subglottal resonances will be reported.

Speech Science: Technique, Concept, Theory

The purpose of this paper is to outline a history and a possible future of speech science as a discipline. Two major and sometimes opposing views of speech science are discussed. One view is inspired by Stetson's work, and holds that study of speech movements is most likely to reveal the essentials of speech motor control. The other view regards the speech acoustic signal as part of the control mechanism; in this view, movements and the signals produced by them are intertwined as part of the control mechanism. The importance of understanding these two views of speech science is in the application of speech science skills to clinical disorders of speech motor control. The paper uses motor speech disorders as an example of how the two views can be joined empirically and theoretically for a maximally productive application of speech science to an understanding of disorders of speech motor control.

Acoustic-visual synthesis technique using bimodal unit-selection

This paper presents a bimodal acoustic-visual synthesis technique that concurrently generates the acoustic speech signal and a 3D animation of the speaker’s outer face. This is done by concatenating bimodal diphone units that consist of both acoustic and visual information. In the visual domain, we mainly focus on the dynamics of the face rather than on rendering. The proposed technique overcomes the problems of asynchrony and incoherence inherent in classic approaches to audiovisual synthesis. The different synthesis steps are similar to typical concatenative speech synthesis but are generalized to the acoustic-visual domain. The bimodal synthesis was evaluated using perceptual and subjective evaluations. The overall outcome of the evaluation indicates that the proposed bimodal acoustic-visual synthesis technique provides intelligible speech in both acoustic and visual channels.

Speech sensorimotor learning through a virtual vocal tract

Studies of speech sensorimotor learning often manipulate auditory feedback by modifying isolated acoustic parameters such as formant frequency or fundamental frequency using near real-time resynthesis of a participant’s speech. An alternative approach is to engage a participant in a total remapping of the sensorimotor working space using a virtual vocal tract. To support this approach for studying speech sensorimotor learning, we have developed a system to control an articulatory synthesizer using electromagnetic articulography data. Articulator movement data from the NDI Wave System are streamed to a Maeda articulatory synthesizer. The resulting synthesized speech provides auditory feedback to the participant. This approach allows the experimenter to generate novel articulatory-acoustic mappings. Moreover, the acoustic output of the synthesizer can be perturbed using acoustic resynthesis methods. Since no robust speech-acoustic signal is required from the participant, this system will allow for the study of sensorimotor learning in any individuals, even those with severe speech disorders. In the current work, we present preliminary results that demonstrate that typically functioning participants can use a virtual vocal tract to produce diphthongs within a novel articulatory-acoustic workspace. Once sufficient baseline performance is established, perturbations to auditory feedback (formant shifting) can elicit compensatory and adaptive articulatory responses.

Vowel Phoneme Segmentation for Speaker Identification Using an ANN-Based Framework

AbstractVowel phonemes are a part of any acoustic speech signal. Vowel sounds occur in speech more frequently and with higher energy. Therefore, vowel phoneme can be used to extract different amounts of speaker discriminative information in situations where acoustic information is noise corrupted. This article presents an approach to identify a speaker using the vowel sound segmented out from words spoken by the speaker. The work uses a combined self-organizing map (SOM)- and probabilistic neural network (PNN)-based approach to segment the vowel phoneme. The segmented vowel is later used to identify the speaker of the word by matching the patterns with a learning vector quantization (LVQ)-based code book. The LVQ code book is prepared by taking features of clean vowel phonemes uttered by the male and female speakers to be identified. The proposed work formulates a framework for the design of a speaker-recognition model of the Assamese language, which is spoken by ∼3 million people in the Northeast Indian state of Assam. The experimental results show that the segmentation success rates obtained using a SOM-based technique provides an increase of at least 7% compared with the discrete wavelet transform-based technique. This increase contributes to the improvement in overall performance of speaker identification by ∼3% compared with earlier related works.

The New Mode to Neurohumoral Regulation in Respiratory and Circulatory Systems

We defined the sophisticated organic whole by placing oxygen (along with CO2 and [H+] Trinity) as the core. The regulation through integrative nervous system is still unclear. In order for us to easily comprehend the working mode of respiratory and circulatory regulations, we use an analogy of ¡°sound system regulation mode¡±: 1. The wave signal generated by the circulating blood in the circulatory system due to pulmonary ventilation is compared to the speech acoustic signals produced by person. 2. Multiple peripheral chemoreceptors and baroreceptors are compared to the sonic microphones that sense sound waves; they collect and upload distinct signals. 3. The central nervous system is equivalent to the volume amplifier control system. 4. The central chemoreceptors act as a volume‐adjusting knob; it balances the central incoming/outgoing integration. 5. The respiratory muscle and thorax lung jointly play the role of a speaker.This concept explains why we can stably maintain our respiration by breath‐by‐breath pattern via respiratory muscles contraction/diastole conversion. Similarly, our circulation can be maintain stable by beat‐by‐beat pattern.

CALL Visual Feedback for Pronunciation of Vowels

-

Audiovisuelle Sprechererkennung durch linguistisch naive Personen

Human speech perception is not only based on acoustic speech signals but also on visual cues like lip or jaw movements. Based on this assumption we used a between-subject design to test listeners’ speaker identification ability in a voice line-up after they were familiarized with a speaker under either of the following condition: (a) visual and degraded acoustic information, (b) degraded acoustic information only, and (c) visual information only. The results from this experiment indicate that listeners are able to perform the identification task to a considerable degree under all three experimental conditions. We conclude that listeners’ identification ability of speakers based on degraded acoustic material is about as good as their identification ability based on visual speech cues. The combination of acoustic and visual cues does not enhance listeners’ performance.

Telemonitoring of Three Characteristic Parameters of Acoustics Vocal Signal in Patients with Tumor or Inflammatory Chronic Dysphonia

The evaluation of voice quality and the perception of its degradation through various acoustic clues are major concerns for voice professionals involved in the process of vocal rehabilitation. In this context, it is necessary to pay particular attention to the set of indices capable of delivering relevant information to help diagnose and assess the effects of vocal re-education proposed. Therefore, the contribution of experimental phonetics in clinical practice is a proven fact. If the study we propose is part of the evaluation of voice quality, our problem is to develop a system dedicated to the objective characterization of chronic dysphonia of tumor or inflammatory origin. The aim of the telemedicine device that we develop is threefold: diagnosis, treatment and monitoring of disease. For this we proceed initially to the remote recording and archiving of an acoustic speech signal voiced in this case “a” sustained for three seconds. We then apply at the ENT department of the University Hospital of Tlemcen, different algorithms of objective assessment of three parameters in this case the fundamental frequency, jitter and DFT-RD that allow experts to assess the development of chronic dysphonia of tumor or inflammatory origin (larynx cancer, inflammatory polyp of the vocal cords, chronic laryngitis).

Multimodal behavior and interaction as indicators of cognitive load

High cognitive load arises from complex time and safety-critical tasks, for example, mapping out flight paths, monitoring traffic, or even managing nuclear reactors, causing stress, errors, and lowered performance. Over the last five years, our research has focused on using the multimodal interaction paradigm to detect fluctuations in cognitive load in user behavior during system interaction. Cognitive load variations have been found to impact interactive behavior: by monitoring variations in specific modal input features executed in tasks of varying complexity, we gain an understanding of the communicative changes that occur when cognitive load is high. So far, we have identified specific changes in: speech, namely acoustic, prosodic, and linguistic changes; interactive gesture; and digital pen input, both interactive and freeform. As ground-truth measurements, galvanic skin response, subjective, and performance ratings have been used to verify task complexity. The data suggest that it is feasible to use features extracted from behavioral changes in multiple modal inputs as indices of cognitive load. The speech-based indicators of load, based on data collected from user studies in a variety of domains, have shown considerable promise. Scenarios include single-user and team-based tasks; think-aloud and interactive speech; and single-word, reading, and conversational speech, among others. Pen-based cognitive load indices have also been tested with some success, specifically with pen-gesture, handwriting, and freeform pen input, including diagraming. After examining some of the properties of these measurements, we present a multimodal fusion model, which is illustrated with quantitative examples from a case study. The feasibility of employing user input and behavior patterns as indices of cognitive load is supported by experimental evidence. Moreover, symptomatic cues of cognitive load derived from user behavior such as acoustic speech signals, transcribed text, digital pen trajectories of handwriting, and shapes pen, can be supported by well-established theoretical frameworks, including O'Donnell and Eggemeier's workload measurement [1986] Sweller's Cognitive Load Theory [Chandler and Sweller 1991], and Baddeley's model of modal working memory [1992] as well as McKinstry et al.'s [2008] and Rosenbaum's [2005] action dynamics work. The benefit of using this approach to determine the user's cognitive load in real time is that the data can be collected implicitly that is, during day-to-day use of intelligent interactive systems, thus overcomes problems of intrusiveness and increases applicability in real-world environments, while adapting information selection and presentation in a dynamic computer interface with reference to load.