Speakers In Dialogue Research Articles

MALN: Multimodal Adversarial Learning Network for Conversational Emotion Recognition

Multimodal emotion recognition in conversations (ERC) aims to identify the emotional state of constituent utterances expressed by multiple speakers in dialogue from multimodal data. Existing multimodal ERC approaches focus on modeling the global context of the dialogue and neglect to mine the characteristic information from the corresponding utterances expressed by the same speaker. Additionally, information from different modalities exhibits commonality and diversity for emotional expression. The commonality and diversity of multimodal information are compensated for each other but not effectively exploited in previous multimodal ERC works. To tackle these issues, we propose a novel Multimodal Adversarial Learning Network (MALN). MALN first mines the speaker’s characteristics from context sequences and then incorporate them with the unimodal features. Afterward, we design a novel adversarial module AMDM to exploit both commonality and diversity from the unimodal features. Finally, AMDM fuses different modalities to generate refined utterance representations for emotion classification. Extensive experiments are conducted on two public multimodal ERC datasets, IEMOCAP and MELD. Through the experiments, MALN shows its superiority over the state-of-the-art methods.

MMATERIC: Multi-Task Learning and Multi-Fusion for AudioText Emotion Recognition in Conversation

The accurate recognition of emotions in conversations helps understand the speaker’s intentions and facilitates various analyses in artificial intelligence, especially in human–computer interaction systems. However, most previous methods need more ability to track the different emotional states of each speaker in a dialogue. To alleviate this dilemma, we propose a new approach, Multi-Task Learning and Multi-Fusion AudioText Emotion Recognition in Conversation (MMATERIC) for emotion recognition in conversation. MMATERIC can refer to and combine the benefits of two distinct tasks: emotion recognition in text and emotion recognition in speech, and production of fused multimodal features to recognize the emotions of different speakers in dialogue. At the core of MATTERIC are three modules: an encoder with multimodal attention, a speaker emotion detection unit (SED-Unit), and a decoder with speaker emotion detection Bi-LSTM (SED-Bi-LSTM). Together, these three modules model the changing emotions of a speaker at a given moment in a conversation. Meanwhile, we adopt multiple fusion strategies in different stages, mainly using model fusion and decision stage fusion to improve the model’s accuracy. Simultaneously, our multimodal framework allows features to interact across modalities and allows potential adaptation flows from one modality to another. Our experimental results on two benchmark datasets show that our proposed method is effective and outperforms the state-of-the-art baseline methods. The performance improvement of our method is mainly attributed to the combination of three core modules of MATTERIC and the different fusion methods we adopt in each stage.

Stylistic alignment in natural conversation involving second language speakers

Abstract To date, a growing body of second language (L2) research has investigated linguistic alignment as a pedagogical intervention, focusing on L2 learners’ alignment behaviors in task-based interactions (e.g., Jung, YeonJoo, YouJin Kim & John Murphy. 2017. The role of task repetition in learning word-stress patterns through auditory priming tasks. Studies in Second Language Acquisition 39(2). 319–346; Kim, YouJin, YeonJoo Jung & Stephen Skalicky. 2019. Linguistic alignment, learner characteristics, and the production of stranded prepositions in relative clauses: Comparing FTF and SCMC contexts. Studies in Second Language Acquisition 41(5). 937–969). Linguistic alignment refers to a tendency where one speaker’s utterances align with particular language features of those of the other speaker in dialogue. The current study investigated how L2 speakers’ alignment behaviors differ in natural dialogues between L2-L1 and L2-L2 dyads in terms of language style (i.e., stylistic alignment) and the role of non-linguistic factors in the occurrence of stylistic alignment. The study analyzed a corpus of 360 texts using a computational tool. Results showed that stylistic alignment occurred to a greater extent in the L2-L2 dyad than in the L2-L1 dyad with respect to the word range, word frequency, word imageability, and proportion of bigrams proportion produced by the interlocutors. Furthermore, findings demonstrated the degree of stylistic alignment on each of the four selected lexical features was affected by numerous factors including age, group membership, nonnative speaker status, familiarity between interlocutors, and linguistic distance between L1 and L2. The effect of each factor on stylistic alignment in conversation is discussed in detail.

Alignment of code switching varies with proficiency in second language learning dialogue

Speakers in dialogue tend to adopt the language patterns of the other, aligning their language to their interlocutor. This can happen at many levels of communication, including the tendency to code switch (CS), or change to another language. Alignment has often been considered the result of an unconscious automatic process that facilitates speakers' mutual understanding. In dialogues with a second language (L2) learner, alignment is constrained by the proficiency of the learner, and additional non-automatic processes will be at play, namely the individual pedagogical goals of learner and tutor. In this study, we investigate alignment in dialogues between Spanish/Catalan learners of English and their tutors. We analyse CS incidence, whether code switching can be explained as automatic alignment between speakers, and whether this is independent of other, non-automatic factors related to speakers’ goals. We find that alignment of code switching is present, varies with learner proficiency, and that code switching can additionally be triggered by lexical overlap and turn taking asymmetry, which we attribute to conscious pedagogical choices on the part of both tutor, at lower levels, and learner, at higher levels of student proficiency.

Interaction Promotes the Adaptation of Referential Conventions to the Communicative Context.

Coordination between speakers in dialogue requires balancing repetition and change, the old and the new. Interlocutors tend to reuse established forms, relying on communicative precedents. Yet linguistic interaction also necessitates adaptation to changing contexts or dynamic tasks, which might favor abandoning existing precedents in favor of better communicative alternatives. We explored this tension using a maze game task in which individual participants and interacting pairs had to describe figures and their positions in one of two possible maze types: a regular maze, in which the grid-like structure of the maze is highlighted, and an irregular maze, in which specific parts of the maze are salient. Participants repeated this task several times. Both individuals and interacting pairs were affected by the different maze layouts, initially using more idiosyncratic description schemes for irregular mazes and more systematic schemes for regular mazes. Interacting pairs, but not individuals, abandoned their unsystematic initial descriptions in favor of a more systematic approach, which was better adapted for repeated interaction. Our results show communicative conventions are initially shaped by context, but interaction opens up the possibility for change if better alternatives are available.

Speech Planning at Turn Transitions in Dialog Is Associated With Increased Processing Load.

Speech planning is a sophisticated process. In dialog, it regularly starts in overlap with an incoming turn by a conversation partner. We show that planning spoken responses in overlap with incoming turns is associated with higher processing load than planning in silence. In a dialogic experiment, participants took turns with a confederate describing lists of objects. The confederate's utterances (to which participants responded) were pre-recorded and varied in whether they ended in a verb or an object noun and whether this ending was predictable or not. We found that response planning in overlap with sentence-final verbs evokes larger task-evoked pupillary responses, while end predictability had no effect. This finding indicates that planning in overlap leads to higher processing load for next speakers in dialog and that next speakers do not proactively modulate the time course of their response planning based on their predictions of turn endings. The turn-taking system exerts pressure on the language processing system by pushing speakers to plan in overlap despite the ensuing increase in processing load.

Going North: Glenn Gould, Stars and the Authentic Self

On December 28th, 1967, CBC aired an hour long documentary labelled the The Idea of North as part of its yearlong Canadian centennial celebration. Conceived by the well-known Canadian pianist Glenn Gould, the piece puts the voices of five speakers in dialogue by splicing and juxtaposing the text of the five, creating an aural collage which highlights common themes. The end result forms what Gould termed “Contrapuntal radio,” a format he would employ again in the next two installment of The Solitude Trilogy. A brief fragment from this documentary is later sampled by Canadian indie bad Stars, to open their 2012 album The North. Whether or not Stars were overtly influenced by Glenn Gould’s sound documentary is unclear. What I would like to suggests is that like The Idea of North, the album participates in a long Canadian tradition of engaging with northern themes, and thus contribute to the discursive formation of the North. Similar to the way in Gould juxtaposed the voices of his subjects, I will be taking the opening citation of The North as an invitation to put the two works in dialogue and argue that by employing a Northern discourse, often steeped in nostalgia, Stars are able to position themselves as an authentic indie band.

On the repetition of words with the potential for metaphoric extension in conversations between native and non-native speakers of English

Although quite a lot is known about the way that non-native speakers of English may interpret and produce metaphors in their second language, we know little about metaphor use in face-to-face conversation between primary and secondary speakers of English. In this article we explore the use of metaphors in two types of conversational data: one elicited in a semi-structured interview format, the other consisting of naturally occurring conversations involving one non-native speaker in dialogue with various native speakers. We found that although native speakers’ use of metaphor was occasionally problematic for the interaction, metaphor also afforded opportunities for topic development in these conversations. The repetition of a word with the potential for metaphoric extension was a particularly valuable strategy used by non-native speakers in these conversations in constructing their coherent contributions to the discourse. In contrast, the use of phraseological metaphors (often the focus of activities aimed at fostering second language learners’ mastery of conventional English metaphors) did not contribute to the joint construction of meanings in these circumstances. We discuss the role of high frequency vocabulary in these conversations and some implications for further research.

ROLE RELATIONSHIP ENACTMENT BETWEEN ENGLISH NON-NATIVE SPEAKERS IN AN ENGLISH DIALOGUE

This study aims at describing the enactment of role relationships between English non-native speakers in an English dialogue. The speakers are an Indonesian student and a Dutchman who met by accident in a public place in Semarang. The dialogue was analyzed by describing the clause and speech function productions by the speakers. The result of the study shows that in an equal turn right to talk the Dutchman performs as the dominant speaker, while the Indonesian Student as the marginal one. The Indonesian student plays the role of interaction stirrer by dominating the opening speech functions with questions. On the other hand, the Dutchman plays the role of interaction supporter by dominating the continuing and responding speech functions. The lack of confrontational speech functions by the speakers in this dialogue is probably caused by the low contact between them, since it is their first time of face-to-face meeting. Key words: role relationship, speech function, dominant speaker, marginal speaker

Introducing information in dialogues: forms of introduction chosen by young speakers and the responses elicited from young listeners

Young speakers in dialogue must establish mutual knowledge. Traditionally, researchers have focused on how children used indefinite and definite articles to signal novel and shared information. In this study of 170 children aged seven to thirteen, the form of introduction chosen, whether question or statement, is more significant than the type of article used. No developmental effects on article use emerge, with statement + indefinite always the least common choice. However, young speakers use question introductions significantly less than older children. There is also a developmental interaction between speakers' and listeners' behaviour. Question introduction elicit effective listener responses at all ages, but only twelve-year-olds respond reliably to statements. Unfortunately, young speakers prefer just those forms of introduction to which young listeners are least likely to provide informative feedback. Thus two separate developmental effects appear to combine to hamper young subjects' chances of achieving successful communication in dialogues.