Speech Segments Research Articles

The Mason-Alberta Phonetic Segmenter: a forced alignment system based on deep neural networks and interpolation.

Given an orthographic transcription, forced alignment systems automatically determine boundaries between segments in speech, facilitating the use of large corpora. In the present paper, we introduce a neural network-based forced alignment system, the Mason-Alberta Phonetic Segmenter (MAPS). MAPS serves as a testbed for two possible improvements we pursue for forced alignment systems. The first is treating the acoustic model as a tagger, rather than a classifier, motivated by the common understanding that segments are not truly discrete and often overlap. The second is an interpolation technique to allow more precise boundaries than the typical 10 ms limit in modern systems. During testing, all system configurations we trained significantly outperformed the state-of-the-art Montreal Forced Aligner in the 10 ms boundary placement tolerance threshold. The greatest difference achieved was a 28.13 % relative performance increase. The Montreal Forced Aligner began to slightly outperform our models at around a 30 ms tolerance. We also reflect on the training process for acoustic modeling in forced alignment, highlighting how the output targets for these models do not match phoneticians' conception of similarity between phones and that reconciling this tension may require rethinking the task and output targets or how speech itself should be segmented.

Graph Neural Network-Based Speech Emotion Recognition: A Fusion of Skip Graph Convolutional Networks and Graph Attention Networks

In speech emotion recognition (SER), our research addresses the critical challenges of capturing and evaluating node information and their complex interrelationships within speech data. We introduce Skip Graph Convolutional and Graph Attention Network (SkipGCNGAT), an innovative model that combines the strengths of skip graph convolutional networks (SkipGCNs) and graph attention networks (GATs) to address these challenges. SkipGCN incorporates skip connections, enhancing the flow of information across the network and mitigating issues such as vanishing gradients, while also facilitating deeper representation learning. Meanwhile, the GAT in the model assigns dynamic attention weights to neighboring nodes, allowing SkipGCNGAT to focus on both the most relevant local and global interactions within the speech data. This enables the model to capture subtle and complex dependencies between speech segments, thus facilitating a more accurate interpretation of emotional content. It overcomes the limitations of previous single-layer graph models, which were unable to effectively represent these intricate relationships across time and in different speech contexts. Additionally, by introducing a pre-pooling SkipGCN combination technique, we further enhance the ability of the model to integrate multi-layer information before pooling, improving its capacity to capture both spatial and temporal features in speech. Furthermore, we rigorously evaluated SkipGCNGAT on the IEMOCAP and MSP-IMPROV datasets, two benchmark datasets in SER. The results demonstrated that SkipGCNGAT consistently achieved state-of-the-art performance. These findings highlight the effectiveness of the proposed model in accurately recognizing emotions in speech, offering valuable insights and a solid foundation for future research on capturing complex relationships within speech signals for emotion recognition.

Enhanced Speech Processing for Air Traffic Control Using an Optimized Generative Adversarial Network

In order to improve the quality of voice calls during air traffic control, an improved SEGAN air traffic control speech enhancement algorithm is proposed. Aiming at the problem that the traditional speech enhancement algorithm based on generative adversarial network (SEGAN) is drowned under low signal-to-noise ratio conditions, a multi-stage, multi-mapping, multi-dimensional output generator and a multi-scale, multi-discriminator network model are proposed based on the SEGAN network model. First, the speech semantic features are extracted based on the deep neural network structure to complete the semantic segmentation of air traffic control speech. Secondly, multiple generators are set to further optimize the speech signal. Then, a downsampling module is added to the convolution layer to improve the model's utilization of speech information and reduce the loss of speech information. Finally, multi-scale, multiple discriminators are used to learn the distribution law and information of speech samples in multiple directions. The results show that under low signal-to-noise ratio conditions, the improved SEGAN model improves the short-term objective intelligibility (STOI) and the perceptual evaluation of speech quality (PESQ) by and respectively , which can quickly and effectively perform air traffic control speech enhancement and provide preparation for subsequent air traffic control speech recognition.

Towards imagined speech: Identification of brain states from EEG signals for BCI-based communication systems

BackgroundThe electroencephalogram (EEG) based brain-computer interface (BCI) system employing imagined speech serves as a mechanism for decoding EEG signals to facilitate control over external devices or communication with the external world at the moment the user desires. To effectively deploy such BCIs, it is imperative to accurately discern various brain states from continuous EEG signals when users initiate word imagination. New methodThis study involved the acquisition of EEG signals from 15 subjects engaged in four states: resting, listening, imagined speech, and actual speech, each involving a predefined set of 10 words. The EEG signals underwent preprocessing, segmentation, spatio-temporal and spectral analysis of each state, and functional connectivity analysis using the phase locking value (PLV) method. Subsequently, five features were extracted from the frequency and time-frequency domains. Classification tasks were performed using four machine learning algorithms in both pair-wise and multiclass scenarios, considering subject-dependent and subject-independent data. ResultsIn the subject-dependent scenario, the random forest (RF) classifier achieved a maximum accuracy of 94.60 % for pairwise classification, while the artificial neural network (ANN) classifier achieved a maximum accuracy of 66.92 % for multiclass classification. In the subject-independent scenario, the random forest (RF) classifier achieved maximum accuracies of 81.02 % for pairwise classification and 55.58 % for multiclass classification. Moreover, EEG signals were classified based on frequency bands and brain lobes, revealing that the theta (θ) and delta (δ) bands, as well as the frontal and temporal lobes, are sufficient for distinguishing between brain states. ConclusionThe findings promise to develop a system capable of automatically segmenting imagined speech segments from continuous EEG signals.

Introduction

Welcome to the first issue of the Journal of Connected Speech. We intend the journal to be a home for research and discussion in the wide area of communication beyond the single speech segment.

Dual-stream Noise and Speech Information Perception based Speech Enhancement

In real-world scenarios, dynamic ambient noise often degrades speech quality, highlighting the need for advanced speech enhancement techniques. Traditional methods, which rely on static embeddings as auxiliary features, struggle to address the complexities of varying noise conditions. To overcome this, we propose a Dual-stream Noise and Speech Information Perception (DNSIP) approach that dynamically detects and processes both noise and speech through innovative information extraction and suppression mechanisms. Initially, non-speech segments predominantly contain environmental noise, while speech segments carry information about the intended speaker. To handle this dynamic nature, real-time voice activity detection (VAD) is employed to accurately differentiate between speech and noise components. Building on VAD estimates, we propose an innovative information extraction framework that selectively extracts relevant noise and speech features from the noisy input, establishing a dual-stream network for concurrent noise and speech learning. To account for the temporal and spectral variability of noise and speech, a frequency-sequence attention mechanism is integrated, enhancing the model’s ability to learn contextual and spectral dependencies. Additionally, an information suppression module is introduced to minimize cross-stream interference by attenuating noise within the speech stream and suppressing speech content within the noise stream. The derived noise and speech spectrograms are then utilized to formulate a minimum mean square error log-spectral amplitude (MMSE-LSA) estimator for robust speech enhancement. Experimental evaluations on the WSJ0 and VCTK+DEMAND datasets demonstrate that our DNSIP approach surpasses existing state-of-the-art methods, underscoring its efficacy in challenging acoustic environments.

Zygosity Effects on Human Voice: Fundamental Frequency Analysis of Brazilian Twins' Speech.

Voice production can be influenced by interindividual variations related to genetic, physiological, behavioral, and several environmental factors. Here we examined the effect of zygosity on speaking fundamental frequency (F0) statistical descriptors. Our aims were: (1) to determine whether the genetic similarity between monozygotic (MZ) and dizygotic (DZ) twins affects F0 characteristics, and (2) to quantify the contribution of genetic factors to these characteristics. The study involved 79 same-sex twin pairs of Brazilian Portuguese speakers, comprising 65 MZ and 14 DZ twins, aged 18 to 66 years (31.7 ± 11.6 years), with 21 male and 58 female pairs. Participants were recorded while uttering a greeting phrase and the Brazilian Portuguese version of the 'Happy Birthday to You' song. Speech segments were analyzed using Praat free software, and F0 measures were automatically extracted in both Hertz and semitone scales. Statistical descriptors, including centrality, dispersion, and extreme values of F0 were examined, and the ACE model (i.e., total genetic effects, A; shared environmental influences, C; and nonshared environmental influences, E) was employed to estimate the additive effect;ts of monozygosity. As anticipated, we observed a zygosity effect on several F0 parameters, with more similarity between MZ twins compared to DZ twins. We discuss the genetic influences on F0 parameters and the absence of a monozygosity effect in two of them. Additionally, we briefly address potential biases associated with the selected measurement scale for statistical modeling. Finally, we explore the influence of genetic factors on F0 patterns, as well as environmental, life history and linguistic factors, particularly concerning F0 variation in speech.

Consonant lengthening marks the beginning of words across a diverse sample of languages.

Speech consists of a continuous stream of acoustic signals, yet humans can segment words and other constituents from each other with astonishing precision. The acoustic properties that support this process are not well understood and remain understudied for the vast majority of the world's languages, in particular regarding their potential variation. Here we report cross-linguistic evidence for the lengthening of word-initial consonants across a typologically diverse sample of 51 languages. Using Bayesian multilevel regression, we find that on average, word-initial consonants are about 13 ms longer than word-medial consonants. The cross-linguistic distribution of the effect indicates that despite individual differences in the phonology of the sampled languages, the lengthening of word-initial consonants is a widespread strategy to mark the onset of words in the continuous acoustic signal of human speech. These findings may be crucial for a better understanding of the incremental processing of speech and speech segmentation.

Real-time detection of spoken speech from unlabeled ECoG signals: A pilot study with an ALS participant.

Brain-Computer Interfaces (BCIs) hold significant promise for restoring communication in individuals with partial or complete loss of the ability to speak due to paralysis from amyotrophic lateral sclerosis (ALS), brainstem stroke, and other neurological disorders. Many of the approaches to speech decoding reported in the BCI literature have required time-aligned target representations to allow successful training - a major challenge when translating such approaches to people who have already lost their voice. In this pilot study, we made a first step toward scenarios in which no ground truth is available. We utilized a graph-based clustering approach to identify temporal segments of speech production from electrocorticographic (ECoG) signals alone. We then used the estimated speech segments to train a voice activity detection (VAD) model using only ECoG signals. We evaluated our approach using held-out open-loop recordings of a single dysarthric clinical trial participant living with ALS, and we compared the resulting performance to previous solutions trained with ground truth acoustic voice recordings. Our approach achieves a median error rate of around 0.5 seconds with respect to the actual spoken speech. Embedded into a real-time BCI, our approach is capable of providing VAD results with a latency of only 10 ms. To the best of our knowledge, our results show for the first time that speech activity can be predicted purely from unlabeled ECoG signals, a crucial step toward individuals who cannot provide this information anymore due to their neurological condition, such as patients with locked-in syndrome. ClinicalTrials.gov, registration number NCT03567213.

Le Petit Prince Hong Kong (LPPHK): Naturalistic fMRI and EEG data from older Cantonese speakers

Currently, the field of neurobiology of language is based on data from only a few Indo-European languages. The majority of this data comes from younger adults neglecting other age groups. Here we present a multimodal database which consists of task-based and resting state fMRI, structural MRI, and EEG data while participants over 65 years old listened to sections of the story The Little Prince in Cantonese. We also provide data on participants’ language history, lifetime experiences, linguistic and cognitive skills. Audio and text annotations, including time-aligned speech segmentation and prosodic information, as well as word-by-word predictors such as frequency and part-of-speech tagging derived from natural language processing (NLP) tools are included in this database. Both MRI and EEG data diagnostics revealed that the data has good quality. This multimodal database could advance our understanding of spatiotemporal dynamics of language comprehension in the older population and help us study the effects of healthy aging on the relationship between brain and behaviour.

Automatic Speech Recognition in Primary Progressive Apraxia of Speech.

Transcribing disordered speech can be useful when diagnosing motor speech disorders such as primary progressive apraxia of speech (PPAOS), who have sound additions, deletions, and substitutions, or distortions and/or slow, segmented speech. Since transcribing speech can be a laborious process and requires an experienced listener, using automatic speech recognition (ASR) systems for diagnosis and treatment monitoring is appealing. This study evaluated the efficacy of a readily available ASR system (wav2vec 2.0) in transcribing speech of PPAOS patients to determine if the word error rate (WER) output by the ASR can differentiate between healthy speech and PPAOS and/or among its subtypes, whether WER correlates with AOS severity, and how the ASR's errors compare to those noted in manual transcriptions. Forty-five patients with PPAOS and 22 healthy controls were recorded repeating 13 words, 3 times each, which were transcribed manually and using wav2vec 2.0. The WER and phonetic and prosodic speech errors were compared between groups, and ASR results were compared against manual transcriptions. Mean overall WER was 0.88 for patients and 0.33 for controls. WER significantly correlated with AOS severity and accurately distinguished between patients and controls but not between AOS subtypes. The phonetic and prosodic errors from the ASR transcriptions were also unable to distinguish between subtypes, whereas errors calculated from human transcriptions were. There was poor agreement in the number of phonetic and prosodic errors between the ASR and human transcriptions. This study demonstrates that ASR can be useful in differentiating healthy from disordered speech and evaluating PPAOS severity but does not distinguish PPAOS subtypes. ASR transcriptions showed weak agreement with human transcriptions; thus, ASR may be a useful tool for the transcription of speech in PPAOS, but the research questions posed must be carefully considered within the context of its limitations. https://doi.org/10.23641/asha.26359417.

Level-Dependent Subcortical Electroencephalography Responses to Continuous Speech.

The auditory brainstem response (ABR) is a measure of subcortical activity in response to auditory stimuli. The wave V peak of the ABR depends on the stimulus intensity level, and has been widely used for clinical hearing assessment. Conventional methods estimate the ABR average electroencephalography (EEG) responses to short unnatural stimuli such as clicks. Recent work has moved toward more ecologically relevant continuous speech stimuli using linear deconvolution models called temporal response functions (TRFs). Investigating whether the TRF waveform changes with stimulus intensity is a crucial step toward the use of natural speech stimuli for hearing assessments involving subcortical responses. Here, we develop methods to estimate level-dependent subcortical TRFs using EEG data collected from 21 participants listening to continuous speech presented at 4 different intensity levels. We find that level-dependent changes can be detected in the wave V peak of the subcortical TRF for almost all participants, and are consistent with level-dependent changes in click-ABR wave V. We also investigate the most suitable peripheral auditory model to generate predictors for level-dependent subcortical TRFs and find that simple gammatone filterbanks perform the best. Additionally, around 6 min of data may be sufficient for detecting level-dependent effects and wave V peaks above the noise floor for speech segments with higher intensity. Finally, we show a proof-of-concept that level-dependent subcortical TRFs can be detected even for the inherent intensity fluctuations in natural continuous speech.

End-to-end elevator voice command recognition system based on improved convolutional neural network

Abstract As a common life scenario, the use of voice control in elevator operation can effectively reduce public health risks and improve user experience in the post-epidemic era. However, due to the characteristics of its closure and public space, it is difficult to deploy the elevator. To solve this problem, an end-to-end speech command recognition algorithm based on a new deep neural network is proposed. The algorithm uses the self-built corpus for training, and uses the speech segment interception algorithm to obtain audio segments under the premise of speech stream as input, puts them into the network model in real time for reasoning, and drives the elevator to run according to the output results. The results show that compared with the recognition model using MFCC-CNN, the network has achieved about 10% improvement in accuracy under the premise of smaller computation, and compared with the DTW algorithm, the network has improved the accuracy by about 25%. Finally, a typical deployment environment is constructed to prove the correctness and effectiveness of the method in practical application.

Dissociating prosodic from syntactic delta activity during natural speech comprehension

Decoding human speech requires the brain to segment the incoming acoustic signal into meaningful linguistic units, ranging from syllables and words to phrases. Integrating these linguistic constituents into a coherent percept sets the root of compositional meaning and hence understanding. One important cue for segmentation in natural speech is prosodic cues, such as pauses, but their interplay with higher-level linguistic processing is still unknown. Here, we dissociate the neural tracking of prosodic pauses from the segmentation of multi-word chunks using magnetoencephalography (MEG). We find that manipulating the regularity of pauses disrupts slow speech-brain tracking bilaterally in auditory areas (below 2Hz) and in turn increases left-lateralized coherence of higher-frequency auditory activity at speech onsets (around 25-45Hz). Critically, we also find that multi-word chunks-defined as short, coherent bundles of inter-word dependencies-are processed through the rhythmic fluctuations of low-frequency activity (below 2Hz) bilaterally and independently of prosodic cues. Importantly, low-frequency alignment at chunk onsets increases the accuracy of an encoding model in bilateral auditory and frontal areas while controlling for the effect of acoustics. Our findings provide novel insights into the neural basis of speech perception, demonstrating that both acoustic features (prosodic cues) and abstract linguistic processing at the multi-word timescale are underpinned independently by low-frequency electrophysiological brain activity in the delta frequency range.

Speechformer-CTC: Sequential modeling of depression detection with speech temporal classification

Speech-based automatic depression detection systems have been extensively explored over the past few years. Typically, each speaker is assigned a single label (Depressive or Non-depressive), and most approaches formulate depression detection as a speech classification task without explicitly considering the non-uniformly distributed depression pattern within segments, leading to low generalizability and robustness across different scenarios. However, depression corpora do not provide fine-grained labels (at the phoneme or word level) which makes the dynamic depression pattern in speech segments harder to track using conventional frameworks. To address this, we propose a novel framework, Speechformer-CTC, to model non-uniformly distributed depression characteristics within segments using a Connectionist Temporal Classification (CTC) objective function without the necessity of input–output alignment. Two novel CTC-label generation policies, namely the Expectation-One-Hot and the HuBERT policies, are proposed and incorporated in objectives on various granularities. Additionally, experiments using Automatic Speech Recognition (ASR) features are conducted to demonstrate the compatibility of the proposed method with content-based features. Our results show that the performance of depression detection, in terms of Macro F1-score, is improved on both DAIC-WOZ (English) and CONVERGE (Mandarin) datasets. On the DAIC-WOZ dataset, the system with HuBERT ASR features and a CTC objective optimized using HuBERT policy for label generation achieves 83.15% F1-score, which is close to state-of-the-art without the need for phoneme-level transcription or data augmentation. On the CONVERGE dataset, using Whisper features with the HuBERT policy improves the F1-score by 9.82% on CONVERGE1 (in-domain test set) and 18.47% on CONVERGE2 (out-of-domain test set). These findings show that depression detection can benefit from modeling non-uniformly distributed depression patterns and the proposed framework can be potentially used to determine significant depressive regions in speech utterances.

Accuracy of Automatic Processing of Speech-Language Pathologist and Child Talk During School-Based Therapy Sessions.

This study examines the accuracy of Interaction Detection in Early Childhood Settings (IDEAS), a program that automatically transcribes audio files and estimates linguistic units relevant to speech-language therapy, including part-of-speech units that represent features of language complexity, such as adjectives and coordinating conjunctions. Forty-five video-recorded speech-language therapy sessions involving 27 speech-language pathologists (SLPs) and 56 children were used. The F measure determines the accuracy of IDEAS diarization (i.e., speech segmentation and speaker classification). Two additional evaluation metrics, namely, median absolute relative error and correlation, indicate the accuracy of IDEAS for the estimation of linguistic units as compared with two conditions, namely, Oracle (manual diarization) and Voice Type Classifier (existing diarizer with acceptable accuracy). The high F measure for SLP talk data suggests high accuracy of IDEAS diarization for SLP talk but less so for child talk. These differences are reflected in the accuracy of IDEAS linguistic unit estimates. IDEAS median absolute relative error and correlation values for nine of the 10 SLP linguistic unit estimates meet the accuracy criteria, but none of the child linguistic unit estimates meet these criteria. The type of linguistic units also affects IDEAS accuracy. IDEAS was tailored to educational settings to automatically convert audio recordings into text and to provide linguistic unit estimates in speech-language therapy sessions and classroom settings. Although not perfect, IDEAS is reliable in automatically capturing and returning linguistic units, especially in SLP talk, that are relevant in research and practice. The tool offers a way to automatically measure SLP talk in clinical settings, which will support research seeking to understand how SLP talk influences children's language growth.

Разум, язык, культура: как сегодня прочитывать концепцию «арабского разума» ал-Джа̄бири

In his four-volume Critique of Arab Reason the late Moroccan philosopher Muḥammad ‘Ābid al-Jābirī (1935–2010) argues that human history witnessed “Greco-European” and “Arab” variants of reason, and not the single one. He challenges the confidence of philosophers in the universality of reason which disclosed itself in the course of European history. Reading al-Jābirī’s theory in Kantian optics will prove fruitful in view of significant number of observations accumulated in Arabic philosophical studies and directly related to the question of the type of rationality that grounds Arab-Muslim culture. The Arabic linguistic tradition bases itself not on the being of substance but on the flux of action. The indivisible segment of the Arabic speech is the flow of sound between the two “sides” (ḥ arf, pl. ḥ urūf). The copulative function ensures the gluing of the subject and predicate into a single statement due to the action of “leaning” (isnād) between them, which excludes the use of the “to be”-like copula and removes “being” from the list of the fundamental metaphysical categories. All Arabic words, with few exceptions, are “branches” growing from “roots” represented by names of actions. The “root – branches” (’aṣl – furū‘) model, developed by the representatives of the Arabic linguistic tradition and Islamic jurists, explains the diversity of Islamic culture which allows for direct contradictions, legitimizing them by tracing down to a single “root”, which contrasts with the law of contradiction fundamental to European rationality. Islamic ethics is concerned not with the moral perfection of a single substantial individual, but with maintaining a network of interactions between human being and God and between members of society, constituting society not as a hierarchy of strata and classes, but as multiple networks of interactions. The concept of the “Arab reason” by al-Jābirī, read in Kantian optics on the basis of the achievements of philosophical Arabic studies, allows to substantiate the hypothesis of the plurality of types of rationality developed through the history of humankind.

Effects of note-taking on the accuracy and fluency of consecutive interpreters' immediate free recall of source texts: A three-stage developmental study

Note-taking for consecutive interpreting (CI) is a special type of note-taking in that selective notes are taken in a highly multi-tasking condition under time pressure during a CI speech segment but are used to enable a complete recall of the source speech immediately. Given the specialty of interpreters' note-taking, it is still unclear in what ways note-taking affects interpreters' memory performance. Moreover, the effect of note-taking is theoretically assumed to depend on interpreting expertise, which has not been substantiated by empirical research. This study addresses these questions by investigating the effect of note-taking on the accuracy and fluency of interpreters' memory of the source text at different training stages. An oral recall experiment was administered to student interpreters at three critical note-taking training stages: the initial stage (with one-month training in note-taking), after the critical development stage (with six-month training in note-taking), and the advanced stage (with two-year training in note-taking), with a sample size of 53, 53 and 35 at each stage respectively. Participants listened to and immediately recalled speech segments consecutively in two note-taking conditions: with and without note-taking. Recall accuracy was measured by completeness of information recalled, and recall fluency was measured by oral fluency during recall. Results showed that note-taking significantly enhanced both accuracy and fluency of interpreters' recall of the source text at all the three stages, revealing a facilitative effect of note-taking on source text memory. The size of the note-taking effect increased as interpreting training experience accumulated, indicating that interpreting training enhanced the facilitative role of note-taking. However, the developmental trajectories of the note-taking effect on recall accuracy and fluency differed. As interpreting training proceeded, the note-taking effect on recall fluency was first improved (from Stage 1 to Stage 2), and later was the note-taking effect on recall accuracy (from Stage 1 to Stage 3). The findings shed light on the cognitive mechanisms of how note-taking functions for consecutive interpreters. A potential practical implication for interpreter training is that a step-by-step note-taking training approach may be beneficial.

Infants show systematic rhythmic motor responses while listening to rhythmic speech.

Rhythm is known to play an important role in infant language acquisition, but few infant language development studies have considered that rhythm is multimodal and shows strong connections between speech and the body. Based on the observation that infants sometimes show rhythmic motor responses when listening to auditory rhythms, the present study asked whether specific rhythm cues (pitch, intensity, or duration) would systematically increase infants' spontaneous rhythmic body movement, and whether their rhythmic movements would be associated with their speech processing abilities. We used pre-existing experimental and video data of 148 German-learning 7.5- and 9.5-month-old infants tested on their use of rhythm as a cue for speech segmentation. The infants were familiarized with an artificial language featuring syllables alternating in pitch, intensity, duration, or none of these cues. Subsequently, they were tested on their recognition of bisyllables based on perceived rhythm. We annotated infants' rhythmic movements in the videos, analyzed whether the rhythmic moving durations depended on the perceived rhythmic cue, and correlated them with the speech segmentation performance. The result was that infants' motor engagement was highest when they heard a duration-based speech rhythm. Moreover, we found an association of the quantity of infants' rhythmic motor responses and speech segmentation. However, contrary to the predictions, infants who exhibited fewer rhythmic movements showed a more mature performance in speech segmentation. In sum, the present study provides initial exploratory evidence that infants' spontaneous rhythmic body movements while listening to rhythmic speech are systematic, and may be linked with their language processing. Moreover, the results highlight the need for considering infants' spontaneous rhythmic body movements as a source of individual differences in infant auditory and speech perception.

Audiovisual Perception of Lexical Stress: Beat Gestures and Articulatory Cues.

Human communication is inherently multimodal. Auditory speech, but also visual cues can be used to understand another talker. Most studies of audiovisual speech perception have focused on the perception of speech segments (i.e., speech sounds). However, less is known about the influence of visual information on the perception of suprasegmental aspects of speech like lexical stress. In two experiments, we investigated the influence of different visual cues (e.g., facial articulatory cues and beat gestures) on the audiovisual perception of lexical stress. We presented auditory lexical stress continua of disyllabic Dutch stress pairs together with videos of a speaker producing stress on the first or second syllable (e.g., articulating VOORnaam or voorNAAM). Moreover, we combined and fully crossed the face of the speaker producing lexical stress on either syllable with a gesturing body producing a beat gesture on either the first or second syllable. Results showed that people successfully used visual articulatory cues to stress in muted videos. However, in audiovisual conditions, we were not able to find an effect of visual articulatory cues. In contrast, we found that the temporal alignment of beat gestures with speech robustly influenced participants' perception of lexical stress. These results highlight the importance of considering suprasegmental aspects of language in multimodal contexts.