Acoustic Patterns Of Speech Research Articles

Language-Independent Sleepy Speech Detection.

Prolonged sleepiness can lead to impairment of cognitive and physical performance and may cause unfortunate accidents. Speech signals are easily accessible using a simple microphone or other means, hence, automated approaches for accurate sleepiness detection from speech signals are desired to prevent degradation in human performance and accidental injury. Sleepiness is known to affect acoustic patterns of speech so that they are different from those of normal speech, and this change is also independent of the language being spoken. To date, there have been no studies examining linguistic-independent sleepy speech detection. We used two different languages, English and German, to detect sleepy speech, where the former was used to train/validate and the latter to test the effectiveness of machine and deep learning models. Specifically, we trained ResNet50, a deep learning model, and five machine learning models with relevant vocal features. Speech data segments from three English-speaking subjects were used for training the model and segments from an English-speaking subject were used for validation. We then tested ResNet50 and the five different machine-learning models using speech data segments from one German-speaking subject. Deep learning far outperformed all of the machine learning approaches. The accuracy, sensitivity, specificity, and geometric mean values were found to be 0.96, 0.92, 0.99, and 0.95, respectively, using ResNet50 on the test data. Our preliminary results suggest that sleepiness can be accurately detected independently from linguistic speech. Clinical Relevance-It is not known if sleepiness can be detected regardless of the language spoken. Our results show the feasibility of accurate sleepiness detection using deep learning even when tested with a different language than trained on.

Recognizing speech in the context of talker variability

The relationship of the acoustic patterns of speech to phonetic categories varies across talkers. A change in talker therefore requires listeners use this relationship which depends on determining the acoustic-phonetic mapping for a new talker. The work from our group has demonstrated that tuning the perceptual system to the acoustic-phonetic system of a new talker is associated with a redirection of attention and a momentary increased load on working memory. These empirical findings support the hypothesis that speech perception is best thought of as an active cognitive process, where the ambiguity of the acoustic signal determines the mobilization of cognitive resources needed to support accurate perception. In this talk, I will (1) discuss the implication of these results for both cognitive and neural models of speech perception and (2) make the argument that extant theories of speech perception must address recognition in the context of acoustic-linguistic pattern variability, such as talker variability, as it is central to the computational problem of understanding speech perception.

Beyond lexical meaning: The effect of emotional prosody on spoken word recognition.

This study employs an auditory-visual associative priming paradigm to test whether non-emotional words uttered in emotional prosody (e.g., pineapple spoken in angry prosody or happy prosody) facilitate recognition of semantically emotional words (e.g., mad, upset or smile, joy). The results show an affective priming effect between emotional prosody and emotional words independent of lexical carriers of the prosody. Learned acoustic patterns in speech (e.g., emotional prosody) map directly to social concepts and representations, and this social information influences the spoken word recognition process.

"Is voice a marker for Autism spectrum disorder? A systematic review and meta-analysis".

Individuals with Autism Spectrum Disorder (ASD) tend to show distinctive, atypical acoustic patterns of speech. These behaviors affect social interactions and social development and could represent a non-invasive marker for ASD. We systematically reviewed the literature quantifying acoustic patterns in ASD. Search terms were: (prosody OR intonation OR inflection OR intensity OR pitch OR fundamental frequency OR speech rate OR voice quality OR acoustic) AND (autis* OR Asperger). Results were filtered to include only: empirical studies quantifying acoustic features of vocal production in ASD, with a sample size >2, and the inclusion of a neurotypical comparison group and/or correlations between acoustic measures and severity of clinical features. We identified 34 articles, including 30 univariate studies and 15 multivariate machine-learning studies. We performed meta-analyses of the univariate studies, identifying significant differences in mean pitch and pitch range between individuals with ASD and comparison participants (Cohen's d of 0.4-0.5 and discriminatory accuracy of about 61-64%). The multivariate studies reported higher accuracies than the univariate studies (63-96%). However, the methods used and the acoustic features investigated were too diverse for performing meta-analysis. We conclude that multivariate studies of acoustic patterns are a promising but yet unsystematic avenue for establishing ASD markers. We outline three recommendations for future studies: open data, open methods, and theory-driven research. Autism Res 2017, 10: 384-407. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.

Talker variability in audio-visual speech perception.

A change in talker is a change in the context for the phonetic interpretation of acoustic patterns of speech. Different talkers have different mappings between acoustic patterns and phonetic categories and listeners need to adapt to these differences. Despite this complexity, listeners are adept at comprehending speech in multiple-talker contexts, albeit at a slight but measurable performance cost (e.g., slower recognition). So far, this talker variability cost has been demonstrated only in audio-only speech. Other research in single-talker contexts have shown, however, that when listeners are able to see a talker’s face, speech recognition is improved under adverse listening (e.g., noise or distortion) conditions that can increase uncertainty in the mapping between acoustic patterns and phonetic categories. Does seeing a talker’s face reduce the cost of word recognition in multiple-talker contexts? We used a speeded word-monitoring task in which listeners make quick judgments about target word recognition in single- and multiple-talker contexts. Results show faster recognition performance in single-talker conditions compared to multiple-talker conditions for both audio-only and audio-visual speech. However, recognition time in a multiple-talker context was slower in the audio-visual condition compared to audio-only condition. These results suggest that seeing a talker’s face during speech perception may slow recognition by increasing the importance of talker identification, signaling to the listener a change in talker has occurred.

The socially weighted encoding of spoken words: a dual-route approach to speech perception

Spoken words are highly variable. A single word may never be uttered the same way twice. As listeners, we regularly encounter speakers of different ages, genders, and accents, increasing the amount of variation we face. How listeners understand spoken words as quickly and adeptly as they do despite this variation remains an issue central to linguistic theory. We propose that learned acoustic patterns are mapped simultaneously to linguistic representations and to social representations. In doing so, we illuminate a paradox that results in the literature from, we argue, the focus on representations and the peripheral treatment of word-level phonetic variation. We consider phonetic variation more fully and highlight a growing body of work that is problematic for current theory: words with different pronunciation variants are recognized equally well in immediate processing tasks, while an atypical, infrequent, but socially idealized form is remembered better in the long-term. We suggest that the perception of spoken words is socially weighted, resulting in sparse, but high-resolution clusters of socially idealized episodes that are robust in immediate processing and are more strongly encoded, predicting memory inequality. Our proposal includes a dual-route approach to speech perception in which listeners map acoustic patterns in speech to linguistic and social representations in tandem. This approach makes novel predictions about the extraction of information from the speech signal, and provides a framework with which we can ask new questions. We propose that language comprehension, broadly, results from the integration of both linguistic and social information.

Moving to the Speed of Sound: Context Modulation of the Effect of Acoustic Properties of Speech

Suprasegmental acoustic patterns in speech can convey meaningful information and affect listeners' interpretation in various ways, including through systematic analog mapping of message-relevant information onto prosody. We examined whether the effect of analog acoustic variation is governed by the acoustic properties themselves. For example, fast speech may always prime the concept of speed or a faster response. Alternatively, the effect may be modulated by the context-dependent interpretation of those properties; the effect of rate may depend on how listeners construe its meaning in the immediate linguistic or communicative context. In two experiments, participants read short scenarios that implied, or did not imply, urgency. Scenarios were followed by recorded instructions, spoken at varying rates. The results show that speech rate had an effect on listeners' response speed; however, this effect was modulated by discourse context. Speech rate affected response speed following contexts that emphasized speed, but not without such contextual information.

Anatomic development of the vocal tract: Implications for speech motor control and acoustic properties of speech

It has been proposed that models of speech production should be developed with consideration of age and gender differences. Although some important age differences in anatomy and acoustic patterns of vocalization have been described between infants and adults, much less attention has been given to the anatomic development of the speech production system between infancy and adulthood. This talk reviews the anatomic development of the craniofacial, oral, and laryngeal systems of speech production between birth and young adulthood. Implications of the developmental patterns are considered for the ontogeny of speech production, especially speech motor control and acoustic patterns of speech. Consideration also is given to gender and race as they relate to vocal tract anatomy and its development in children. Topics to be reviewed include: craniofacial skeleton, nasopharynx, tongue, lips, and larynx. The overall pattern of development will be summarized for each system and intervals of especially rapid growth will be identified. A major hypothesis to be evaluated is whether growth is harmonious across the major anatomic systems.

Auditory processing of the acoustic patterns of speech.

The auditory processing of the acoustic patterns of speech relevant for the perception of phonetic units is discussed in terms of a three-stage generic model of phonetic perception. It is argued that the auditory system analyzes each brief segment of the acoustic waveform of speech for tonal components and the shape of the "spectral envelope." The presence and pattern of tonal components is used in the perception of voicing and pitch. The shapes of spectral envelopes and their dynamics induce the perception of phonetic units. The important attributes of the spectral-envelope patterns are identified.

The perceptual similarity of mirror-image acoustic patterns in speech

Prevocalic and postvocalic (unreleased) occurrences of a stop consonant differ in acoustic shape, but are not unrelated. In particular, the formant transitions taking place at release of a stop consonant approximately mirror in time the formant transitions occurring during closure, assuming that the vowel is the ~same. Several experiments have been performed using brief two-component tone burst approximations to the second and third formant transitions’ that occur in prevocalic and postvocalic allophones of/b, d, g/in order to determine whether such mirror-image acoustic patterns are perceptually related. Listener judgments of similarity within triads of these stimuli indicate that mirror-image patterrts representing the same place of articulation are less similar to each other than to patterns representing different places of articulation. Implications for the child who is acquiring language of the fact that mirror-image patterns in speech do not have inherent perceptual similarity are discussed.