Prosodic Entrainment Research Articles

Speech Prosody Serves Temporal Prediction of Language via Contextual Entrainment.

Temporal prediction assists language comprehension. In a series of recent behavioral studies, we have shown that listeners specifically employ rhythmic modulations of prosody to estimate the duration of upcoming sentences, thereby speeding up comprehension. In the current human magnetoencephalography (MEG) study on participants of either sex, we show that the human brain achieves this function through a mechanism termed entrainment. Through entrainment, electrophysiological brain activity maintains and continues contextual rhythms beyond their offset. Our experiment combined exposure to repetitive prosodic contours with the subsequent presentation of visual sentences that either matched or mismatched the duration of the preceding contour. During exposure to prosodic contours, we observed MEG coherence with the contours, which was source-localized to right-hemispheric auditory areas. During the processing of the visual targets, activity at the frequency of the preceding contour was still detectable in the MEG; yet sources shifted to the (left) frontal cortex, in line with a functional inheritance of the rhythmic acoustic context for prediction. Strikingly, when the target sentence was shorter than expected from the preceding contour, an omission response appeared in the evoked potential record. We conclude that prosodic entrainment is a functional mechanism of temporal prediction in language comprehension. In general, acoustic rhythms appear to endow language for employing the brain's electrophysiological mechanisms of temporal prediction.

A Study of Prosodic Entrainment and Social Factors in Mandarin Conversations

In conversations, interlocutors usually adopt prosody to that of their partner, and they become similar in prosodic production for successful communication. This phenomenon of prosodic entrainment is related to complex factors. This study aims to explore the relationship between prosodic entrainment and social factors. Two analyses are accomplished: the analysis of prosodic entrainment and gender, and the analysis of prosodic entrainment and role. In terms of prosodic entrainment and gender, it is found that the most prosodic features are entrained in female-male conversations, and the least in male-male conversations. In terms of prosodic entrainment and roles, it is found that different roles have influence on the entrainment degree, and information givers entrain more to followers in conversation.

The Influence of Conversation Role on Prosodic Entrainment in Mandarin Interactions

The aim of this study is to find out how conversation role affects prosodic entrainment in Mandarin interactions. Tongji Games Corpus are adopted for this study, in which two interlocutors play unequal roles in Picture Ordering Games (information giver and information follower), and equal roles in Picture Classifying Games. Based on this corpus, two tests are conducted in this study, the Role Influence Test and Role Direction Test. In the analysis of Role Influence Test, it is found that the entrainment degree in Picture Ordering Games is significantly bigger than that in the Picture Classifying Games. In the analysis of Role Direction Test, it is found that information givers entrain more to followers in conversation. These findings provide evidences that conversation roles, as one type of the social factors, have influence on the degree and direction of prosodic entrainment in Mandarin interactions.

Measuring Prosodic Entrainment in Conversation: A Review and Comparison of Different Methods.

This study aims to further our understanding of prosodic entrainment and its different subtypes by analyzing a single corpus of conversations with 12 different methods and comparing the subsequent results. Entrainment on three fundamental frequency features was analyzed in a subset of recordings from the LUCID corpus (Baker & Hazan, 2011) using the following methods: global proximity, global convergence, local proximity, local convergence, local synchrony (Levitan & Hirschberg, 2011), prediction using linear mixed-effects models (Schweitzer & Lewandowski, 2013), geometric approach (Lehnert-LeHouillier, Terrazas, & Sandoval, 2020), time-aligned moving average (Kousidis et al., 2008), HYBRID method (De Looze et al., 2014), cross-recurrence quantification analysis (e.g., Fusaroli & Tylén, 2016), and windowed, lagged cross-correlation (Boker et al., 2002). We employed entrainment measures on a local timescale (i.e., on adjacent utterances), a global timescale (i.e., over larger time frames), and a time series-based timescale that is larger than adjacent utterances but smaller than entire conversations. We observed variance in results of different methods. Results suggest that each method may measure a slightly different type of entrainment. The complex implications this has for existing and future research are discussed.

Assessing verbal interaction of adult learners in computer‐supported collaborative problem solving

AbstractThe objective of this study is to explore new ways of assessing collaborative problem solving (CPS) processes based on different modalities of audio data and their combination. The data collection took place in an educational lab setting during an experiment with adult teams from professional contexts who collaboratively solved multiple problems as part of a CPS training. From audio data, both verbal (ie, speech) and non‐verbal (ie, pitch) aspects were extracted. Four analysis methods were used, including (a) content analysis; (b) linguistic inquiry and word count; (c) verbal entrainment analysis; and (d) acoustic–prosodic entrainment based on pitch data. Insights are given into the CPS processes of the participating groups using these measures and relevant relationships between some of these measures are further investigated. Based on content analysis, it was found that most of the interactions during the CPS process are task oriented, whereas team‐oriented interactions are less present. Second, three measures of proportion of contribution in CPS were investigated and clear differences in participation patterns between and within teams were found. We suggest that a combination of utterance count and words per sentence could provide valuable insights for quantity and equality of participation. Third, the study explored pronoun use and found that the most frequently used personal pronouns were first‐person singular. Next, the results indicated a relationship between pronoun use and the relative frequency of interactions. Fourth, a rather weak relationship between lexical entrainment measures and the acoustic–prosodic measures were found, suggesting that these measures are indicative of separate communicative aspects in CPS. This study contributes to a better understanding of which type of audio‐based data is most informative to teachers and students as a feedback or assessment tool. This study complements previous research as it focuses on spoken human‐to‐human communication collected in an authentic context. Practitioner notesWhat is already known about this topic Support and guidance systems for learning coaches, teachers and learners are needed to foster the educational quality of collaborative problem solving (CPS) activities. CPS is a complex process and measuring the quality of CPS processes remains challenging. Multimodal learning analytics, focusing on verbal and non‐verbal data sources and using content analysis, linguistic inquiry and word count and verbal and acoustic entrainment measures could be valuable to measure the quality of CPS. What this paper adds The majority of interactions during CPS processes are task oriented or cognitive of nature, whereas team‐oriented interactions are less present. Utterance count and words per sentence should be used in combination, as they are indicative of different aspects. Pronoun use in learners' discourse is related to the types of CPS interactions. Lexical entrainment measures and acoustic–prosodic are indicative of distinctive communicative aspects in CPS. Implications for practice and/or policy Quality indicators of CPS processes should include both verbal and non‐verbal measures of students' interactions. Educational researchers and the (Edtech) industry should further leverage their forces to foster the development of (semi‐)automated systems for measuring the quality of CPS processes. It should be further investigated how quality indicators of CPS processes can be most meaningful to trainers, teachers and learners, for example, through the use of dashboards.

Investigating prosodic entrainment from global conversations to local turns and tones in Mandarin conversations

Previous research on acoustic entrainment has paid less attention to tones than to other prosodic features. This study sets a hierarchical framework by three layers of conversations, turns and tone units, investigates prosodic entrainment in Mandarin spontaneous dialogues at each level, and compares the three. Our research has found that (1) global and local entrainment exist independently, and local entrainment is more evident than global; (2) variation exists in prosodic features’ contribution to entrainment at three levels: amplitude features exhibiting more prominent entrainment at both global and local levels, and speaking-rate and F0 features showing more prominence at the local levels; and (3) no convergence is found at the conversational level, at the turn level or over tone units.

Second language acoustic–prosodic entrainment in conversation and storytelling

Prosodic entrainment is the tendency for individuals to modify their acoustic-prosodic speech behaviors to converge with the behaviors of their interlocutors. Evidence of entrainment in the native language (L1) is robust, yet research regarding its development in the second language (L2) is sparse. To examine prosodic entrainment in spoken dialogues, we developed a natural speech corpus in which spontaneous conversations of 30 young adults were collected. Data from 15 dyadic groups of participants in the conversation and the story-telling sessions in both their native language (Mandarin Chinese) and L2 (English) were collected and analyzed. Prosodic entrainment has been examined with the low-level prosodic features, a bundle of the acoustic measure (i.e., f0, speaking rate), and voice measures (i.e., jitter and NHR). Individual difference measures (i.e., attention to details, communication skills) are also examined, and their correlations with the prosodic features are reported. The research is expected to contribute to the study of L2 prosody acquisition and human–machine interaction as well as future research in turn-taking, conversational analysis, and individual differences in L1 and L2 speech behaviors.

Verbal entrainment in autism spectrum disorder and first-degree relatives

Entrainment, the unconscious process leading to coordination between communication partners, is an important dynamic human behavior that helps us connect with one another. Difficulty developing and sustaining social connections is a hallmark of autism spectrum disorder (ASD). Subtle differences in social behaviors have also been noted in first-degree relatives of autistic individuals and may express underlying genetic liability to ASD. In-depth examination of verbal entrainment was conducted to examine disruptions to entrainment as a contributing factor to the language phenotype in ASD. Results revealed distinct patterns of prosodic and lexical entrainment in individuals with ASD. Notably, subtler entrainment differences in prosodic and syntactic entrainment were identified in parents of autistic individuals. Findings point towards entrainment, particularly prosodic entrainment, as a key process linked to social communication difficulties in ASD and reflective of genetic liability to ASD.

Rhythm Perception, Speaking Rate Entrainment, and Conversational Quality: A Mediated Model.

Acoustic-prosodic entrainment, defined as the tendency for individuals to modify their speech behaviors to more closely align with the behaviors of their conversation partner, plays an important role in successful interaction. From a mechanistic perspective, acoustic-prosodic entrainment is, by its very nature, a rhythmic activity. Accordingly, it is highly plausible that an individual's rhythm perception abilities play a role in their ability to successfully entrain. Here, we examine the impact of rhythm perception in speaking rate entrainment and subsequent conversational quality. A round-robin paradigm was used to collect 90 dialogues from neurotypical adults. Additional assessments determined participants' rhythm perception abilities, social competence, and partner familiarity (i.e., whether the conversation partners knew each other prior to the interaction. Mediation analysis was used to examine the relationships between rhythm perception scores, speaking rate entrainment (using a measure of static local synchrony), and a measure of conversational success (i.e., conversational quality) based on third-party listener observations. Findings were compared to the same analysis with three additional predictive factors: participant gender, partner familiarity, and social competence. Results revealed a relationship between rhythm perception and speaking rate entrainment. In unfamiliar conversation partners, there was a relationship between speaking rate entrainment and conversational quality. The relationships between entrainment and each of the three additional factors (i.e., gender, partner familiarity, and social competence) were nonsignificant. In unfamiliar conversation partners, better rhythm perception abilities were indicative of increased conversational quality mediated by higher levels of speaking rate entrainment. These results support theoretical postulations specifying rhythm perception abilities as a component of acoustic-prosodic entrainment, which, in turn, facilitates conversational success. Knowledge of this relationship contributes to the development of a causal framework for considering a mechanism by which rhythm perception deficits in clinical populations may impact conversational success.

Prosodic entrainment in individuals with autism spectrum disorder

Abstract Entrainment is the tendency of people to behave similarly during an interaction. It occurs on different levels of behaviour, including speech, and has been associated with pro-social behaviour and increased rapport. This review paper outlines the current understanding of linguistic entrainment, particularly at the speech level, in individuals with autism spectrum disorder (ASD), a disorder that is associated with social difficulties and unusual prosody. Aberrant entrainment patterns in individuals with ASD could thus contribute to both their perceived unusual prosody and their social difficulties. Studying the relationship between speech entrainment and ASD holds great potential for applied benefits in utilizing this knowledge for pre-screening or diagnosis, monitoring progress longitudinally, and intervention practices. Our findings suggest that research on entrainment in ASD is sparse and exploratory, and the ecological validity of experimental paradigms varies. Moreover, there is little consistency in methodology and results vary between studies, which highlights the need for standardized methods in entrainment research. A promising way to standardize methods, facilitate their use, and extend them to everyday clinical practice, is by implementing automatic methods for speech analysis and adhering to open-science principles.

Pitch It Right: Using Prosodic Entrainment to Improve Robot-Assisted Foreign Language Learning in School-Aged Children

Robot-assisted language learning (RALL) is a promising application when employing social robots to help both children and adults acquire a language and is an increasingly widely studied area of child–robot interaction. By introducing prosodic entrainment, i.e., converging the robot’s pitch with that of the learner, the present study aimed to provide new insights into RALL as a facilitative method for interactive tutoring. It is hypothesized that pitch-level entrainment by a Nao robot during a word learning task in a foreign language will result in increased learning in school-aged children. The results indicate that entrainment has no significant effect on participants’ learning, contra the hypothesis. Research on the implementation of entrainment in the context of RALL is new. This study highlights constraints in currently available technologies for voice generation and methodological limitations that should be taken into account in future research.

Prosodic Entrainment in Conversations of Verbal Children and Teens on the Autism Spectrum.

Unusual speech prosody has long been recognized as a characteristic feature of the speech of individuals diagnosed with Autism Spectrum Disorders (ASD). However, research to determine the exact nature of this difference in speech prosody is still ongoing. Many individuals with verbal autism perform well on tasks testing speech prosody. Nonetheless, their expressive prosody is judged to be unusual by others. We propose that one aspect of this perceived difference in speech prosody in individuals with ASD may be due to a deficit in the ability to entrain—or become more similar—to their conversation partners in prosodic features over the course of a conversation. In order to investigate this hypothesis, 24 children and teens between the ages of 9 and 15 years participated in our study. Twelve of the participants had previously been diagnosed with ASD and the other 12 participants were matched to the ASD participants in age, gender, and non-verbal IQ scores. All participants completed a goal-directed conversation task, which was subsequently analyzed acoustically. Our results suggest (1) that youth diagnosed with ASD entrain less to their conversation partners compared to their neurotypical peers—in fact, children and teens diagnosed with ASD tend to dis-entrain from their conversation partners while their neurotypical peers tend to converge to their conversation partners’ prosodic features. (2) Although age interacts differently with prosodic entrainment in youth with and without ASD, this difference is attributable to the entrainment behavior of the conversation partners rather than to those with ASD. (3) Better language skill is negatively correlated with prosodic entrainment for both youth with and without ASD. The observed differences in prosodic entrainment in children and teens with ASD may not only contribute to the perceived unusual prosody in youth with ASD but are also likely to be indicative of their difficulties in social communication, which constitutes a core challenge for individuals with ASD.

Effects of adapting to user pitch on rapport perception, behavior, and state with a social robotic learning companion

Social robots such as learning companions, therapeutic assistants, and tour guides are dependent on the challenging task of establishing a rapport with their users. People rarely communicate with just words alone; facial expressions, gaze, gesture, and prosodic cues like tone of voice and speaking rate combine to help individuals express their words and convey emotion. One way that individuals communicate a sense of connection with one another is entrainment, where interaction partners adapt their way of speaking, facial expressions, or gestures to each other; entrainment has been linked to trust, liking, and task success and is thought to be a vital phenomenon in how people build rapport. In this work, we introduce a social robot that combines multiple channels of rapport-building behavior, including forms of social dialog and prosodic entrainment. We explore how social dialog and entrainment contribute to both self-reported and behavioral rapport responses. We find prosodic adaptation enhances perceptions of social dialog, and that social dialog and entrainment combined build rapport. Individual differences indicated by gender mediate these social responses; an individual’s underlying rapport state, as indicated by their verbal rapport behavior, is exhibited and triggered differently depending on gender. These results have important repercussions for assessing and modeling a user’s social responses and designing adaptive social agents.

Universals of listening: Equivalent prosodic entrainment in tone and non-tone languages

In English and Dutch, listeners entrain to prosodic contours to predict where focus will fall in an utterance. Here, we ask whether this strategy is universally available, even in languages with very different phonological systems (e.g., tone versus non-tone languages). In a phoneme detection experiment, we examined whether prosodic entrainment also occurs in Mandarin Chinese, a tone language, where the use of various suprasegmental cues to lexical identity may take precedence over their use in salience. Consistent with the results from Germanic languages, response times were facilitated when preceding intonation predicted high stress on the target-bearing word, and the lexical tone of the target word (i.e., rising versus falling) did not affect the Mandarin listeners' response. Further, the extent to which prosodic entrainment was used to detect the target phoneme was the same in both English and Mandarin listeners. Nevertheless, native Mandarin speakers did not adopt an entrainment strategy when the sentences were presented in English, consistent with the suggestion that L2 listening may be strained by additional functional load from prosodic processing. These findings have implications for how universal and language-specific mechanisms interact in the perception of focus structure in everyday discourse.

The Relationship between Prosodic Ability and Conversational Prosodic Entrainment.

Conversational entrainment or alignment-the convergence of conversation partners over the course of a conversation in a variety of linguistic features-is a well-attested conversational phenomenon. The research on prosodic entrainment has shown correlations between prosodic entrainment and several social dimensions of rapport between conversation partners. However, little is known about how skill-level in the entrainment domain affects the ability to converge during a conversation. The goal of the current study was to investigate whether skill-level of a speaker in receptive and expressive word, sentence, and emotional prosody is correlated with the amount of prosodic entrainment contributed at the conversational level. Twenty native speakers of American English were paired into ten dyads of seven female/female and three female/male conversation pairs. Conversations for each pair were recorded and analyzed. Test scores measuring word, sentence, and emotional prosody were correlated with the amount of fundamental frequency entrainment during conversations. The results indicate that a negative correlation exists between expressive prosody skill and the amount of f0 entrainment contributed by a speaker. This suggests that speakers with better expressive prosodic skills at the word and sentence level entrain less to their conversation partners. Receptive prosody ability was not correlated with conversational prosodic entrainment.

Entrainment profiles: Comparison by gender, role, and feature set

We examine prosodic entrainment in cooperative game dialogs for new feature sets describing register, pitch accent shape, and rhythmic aspects of utterances. For these as well as for established features we present entrainment profiles to detect within- and across-dialog entrainment by the speakers’ gender and role in the game. It turned out, that feature sets undergo entrainment in different quantitative and qualitative ways, which can partly be attributed to their different functions. Furthermore, interactions between speaker gender and role (describer vs. follower) suggest gender-dependent strategies in cooperative solution-oriented interactions: female describers entrain most, male describers least. Our data suggests a slight advantage of the latter strategy on task success.