Dialogue Segmentation Research Articles

Towards automated transcribing and coding of embodied teamwork communication through multimodal learning analytics

AbstractEffective collaboration and teamwork skills are critical in high‐risk sectors, as deficiencies in these areas can result in injuries and risk of death. To foster the growth of these vital skills, immersive learning spaces have been created to simulate real‐world scenarios, enabling students to safely improve their teamwork abilities. In such learning environments, multiple dialogue segments can occur concurrently as students independently organise themselves to tackle tasks in parallel across diverse spatial locations. This complex situation creates challenges for educators in assessing teamwork and for students in reflecting on their performance, especially considering the importance of effective communication in embodied teamwork. To address this, we propose an automated approach for generating teamwork analytics based on spatial and speech data. We illustrate this approach within a dynamic, immersive healthcare learning environment centred on embodied teamwork. Moreover, we evaluated whether the automated approach can produce transcriptions and epistemic networks of spatially distributed dialogue segments with a quality comparable to those generated manually for research objectives. This paper makes two key contributions: (1) it proposes an approach that integrates automated speech recognition and natural language processing techniques to automate the transcription and coding of team communication and generate analytics; and (2) it provides analyses of the errors in outputs generated by those techniques, offering insights for researchers and practitioners involved in the design of similar systems.Practitioner notesWhat is currently known about this topic Immersive learning environments simulate real‐world situations, helping students improve their teamwork skills. In these settings, students can have multiple simultaneous conversations while working together on tasks at different physical locations. The dynamic nature of these interactions makes it hard for teachers to assess teamwork and communication and for students to reflect on their performance. What this paper adds We propose a method that employs multimodal learning analytics for automatically generating teamwork‐related insights into the content of student conversations. This data processing method allows for automatically transcribing and coding spatially distributed dialogue segments generated from students working in teams in an immersive learning environment and enables downstream analysis. This approach uses spatial analytics, natural language processing and automated speech recognition techniques. Implications for practitioners Automated coding of dialogue segments among team members can help create analytical tools to assist in evaluating and reflecting on teamwork. By analysing spatial and speech data, it is possible to apply learning analytics advancements to support teaching and learning in fast‐paced physical learning spaces where students can freely engage with one another.

Novel framework for dialogue summarization based on factual-statement fusion and dialogue segmentation.

The explosive growth of dialogue data has aroused significant interest among scholars in abstractive dialogue summarization. In this paper, we propose a novel sequence-to-sequence framework called DS-SS (Dialogue Summarization with Factual-Statement Fusion and Dialogue Segmentation) for summarizing dialogues. The novelty of the DS-SS framework mainly lies in two aspects: 1) Factual statements are extracted from the source dialogue and combined with the source dialogue to perform the further dialogue encoding; and 2) A dialogue segmenter is trained and used to separate a dialogue to be encoded into several topic-coherent segments. Thanks to these two aspects, the proposed framework may better encode dialogues, thereby generating summaries exhibiting higher factual consistency and informativeness. Experimental results on two large-scale datasets SAMSum and DialogSum demonstrate the superiority of our framework over strong baselines, as evidenced by both automatic evaluation metrics and human evaluation.

IRWoZ: Constructing an Industrial Robot Wizard-of-OZ Dialoguing Dataset

Enabling a flexible and natural human-robot interaction (HRI) for industrial robots is a critical yet challenging task that can be facilitated by the use of conversational artificial intelligence (AI). Prior research has concentrated on strengthening interactions through the deployment of social robots, while disregarding the capabilities required to boost the flexibility and user experience associated with human-robot collaboration (HRC) on manufacturing tasks. One of the main challenges is the lack of publicly available industrial-oriented dialogue datasets for the training of conversational AI. In this work, we present an industrial robot wizard-of-Oz dialoguing dataste (IRWoZ) focused on enabling HRC in manufacturing tasks. The dataset covers four domains: assembly, transportation, position, and relocation. It is created using the Wizard-of-Oz technique to be less noisy. We manually constructed, annotated and validated dialogue segments (e.g., intentions, slots, annotations), as well as the responses. Building upon the proposed dataset, we benchmark it on the state-of-the-art (SoTA) language models, generative pre-trained (GPT-2) models, on dialogue state tracking and response generation tasks. We expect that the IRWoZ dataset will facilitate exciting ongoing dialogue research and we provide it freely accessible at https://github.com/lcroy/ToD4IR/tree/main/dataset.

Seeing Through the Character’s Eyes: Examining Phenomenological Experiences of Perspective-Taking During Reading

ABSTRACT Despite substantive work on the cognitive processes underlying comprehension, little research has examined the “phenomenological” nature of reading. We investigated how readers’ experiences of visual and verbal thought related to their transportation into the narrative text and whether these were influenced by perspective-taking. Specifically, readers reported the nature of their thoughts while reading and then completed a transportation assessment. Study 1 (n = 147) manipulated perspective-taking via explicit instructions, whereas Study 2 (n = 200) varied point of view within the text. Additionally, Study 2 examined whether reports varied across descriptive and dialogue text segments. Results suggested that visual and verbal reports were consistently negatively correlated. Further, transportation was positively associated with visual reports and negatively associated with verbal reports. We found no differences in thought reports for dialogue and descriptive text segments or across the perspective-taking manipulations. These findings suggest that phenomenological reading experiences are stable across a variety of reading situations.

Why does Lee say what he says the way he says it?

Abstract This paper employs Kecskes’ socio-cognitive approach to analyze the varied speech styles and cognitive dynamics of the Chinese character (Lee) in John Steinbeck’s East of Eden. The discussion of the novelistic dialogue segments has shown that the Chinese interlocutor’s verbal strategies vary from pidgin to English or a combination of the two, which are predominantly hearer-centered and marked by deliberate and conscious attempts on the part of the speaker to meet the cooperation principle. Lee’s movement between different communication modes is partly predetermined by the disparate power relations between the interlocutors and partly determined by his own communicative needs, thus producing a unique pattern that governs his language use in the given intercultural communicative process. In particular, pidgin is used as a self-protection mechanism, a buffer and a way of identification by the Chinese character, which informs the wider socio-historical context of Chinese immigrants’ victimization of racial discrimination in the American society at the turn of the twentieth century. Just like his shifting verbal strategies in intercultural communication, Lee’s cultural identity is also characterized by fluidity in the in-between space of two cultures.

The Analysis of Conversational Implicature from Violation of Politeness Principle in Big Bang Theory

The impolite language and conversational implicature produced by the violation of the politeness principle are gradually becoming common, which can sometimes achieve unexpected effects. This study selects the dialogue segments between main characters in Big Bang Theory to make text analysis, which aims to analyze reasons and contexts of intended violation, contents as well as effects of the conversational implicature. The results indicate that the contradiction of the politeness principle itself may cause impolite behavior and people often make use of the conversational implicature produced by deliberate violation of politeness principles to create a humorous atmosphere or achieve their real communicative intentions.

Smart Enough to Talk With Us? Foundations and Challenges for Dialogue Capable AI Systems

Smart Enough to Talk With Us? Foundations and Challenges for Dialogue Capable AI Systems

Easy Screening for Mild Alzheimer's Disease and Mild Cognitive Impairment from Elderly Speech.

This study presents a novel approach for early detection of cognitive impairment in the elderly. The approach incorporates the use of speech sound analysis, multivariate statistics, and data-mining techniques. We have developed a speech prosody-based cognitive impairment rating (SPCIR) that can distinguish between cognitively normal controls and elderly people with mild Alzheimer's disease (mAD) or mild cognitive impairment (MCI) using prosodic signals extracted from elderly speech while administering a questionnaire. Two hundred and seventy-three Japanese subjects (73 males and 200 females between the ages of 65 and 96) participated in this study. The authors collected speech sounds from segments of dialogue during a revised Hasegawa's dementia scale (HDS-R) examination and talking about topics related to hometown, childhood, and school. The segments correspond to speech sounds from answers to questions regarding birthdate (T1), the name of the subject's elementary school (T2), time orientation (Q2), and repetition of three-digit numbers backward (Q6). As many prosodic features as possible were extracted from each of the speech sounds, including fundamental frequency, formant, and intensity features and mel-frequency cepstral coefficients. They were refined using principal component analysis and/or feature selection. The authors calculated an SPCIR using multiple linear regression analysis. In addition, this study proposes a binary discrimination model of SPCIR using multivariate logistic regression and model selection with receiver operating characteristic curve analysis and reports on the sensitivity and specificity of SPCIR for diagnosis (control vs. MCI/mAD). The study also reports discriminative performances well, thereby suggesting that the proposed approach might be an effective tool for screening the elderly for mAD and MCI.

A Preliminary Study of Speech Prosody-Based Relationship with HDS-R Scores Toward Early Detection of Cognitive Impairment in Elderly Using Speech Prosody

This paper presents a new trial approach to early detection of cognitive impairment in the elderly with the use of speech sound analysis and multivariate statistical technique. In this paper, we focus on the prosodic features from speech sound. Japanese 115 subjects (32 males and 83 females between ages of 38 and 99) participated in this study. We collected speech sound in a few segments of dialogue of HDS-R examination. The segments corresponds to speech sound that is answering for questions on time orientation and number backward count. Firstly, 130 prosodic features have been extracted from each of the speech sounds. These prosodic features consist of spectral and pitch features (53), formant features (56), intensity features (19), and speech rate and response time (2). Secondly, these features are refined by principal component analysis and/or feature selection. Lastly, we have calculated speech prosody-based cognitive impairment rating (SPCIR) by multiple linear regression analysis. The results indicated that there is moderately significant correlation between HDS-R score and synthesis of several selected prosodic features. Consequently, adjusted coefficient of determination R2=0.50 suggests that prosody-based speech sound analysis has possibility to screen the elderly with cognitive impairment.

Discussion of Sexual Risk Behavior in HIV Care is Infrequent and Appears Ineffectual: A Mixed Methods Study

Consensus guidelines in HIV care call for clinicians to provide a brief sexual risk behavioral intervention in each visit. Studies based on participant reports find this occurs infrequently, but studies based on direct observation of clinical encounters are lacking. We conducted a mixed method study that used audio recordings of 116 routine outpatient visits by 58 different patients with HIV, in five different practice sites. Transcripts of the visits were coded and analyzed using a quantitative system. In addition, we conducted a qualitative analysis of the dialogue segments in which sexual risk behaviors arose as a topic. Discussion of sexual risk behavior occurred in 10 visits, and was generally quite brief. Two visits featured substantial counseling about sexual risk reduction; two others included substantial discussion which was not evidently directed at the patient's changing behavior. Cues suggesting a need or opportunity for such discussion that physicians did not follow up on occurred in seven additional visits. Interactions about sexual risk had less patient engagement than interactions about other health behaviors. Physicians seldom provide sexual risk reduction counseling in HIV care, even where specific indications are present.

Multifunctionality in dialogue

This paper studies the multifunctionality of dialogue utterances, i.e. the phenomenon that utterances in dialogue often have more than one communicative function. It is argued that this phenomenon can be explained by analyzing the participation in dialogue as involving the performance of several types of activity in parallel, relating to different dimensions of communication. The multifunctionality of dialogue utterances is studied by (1) redefining the notion of ‘utterance’ in a rigorous manner (calling the revised notion ‘functional segment’), and (2) empirically investigating the multifunctionality of functional segments in a corpus of dialogues, annotated with a rich, multidimensional annotation schema. It is shown that, when communicative functions are assigned to functional segments, thereby eliminating every form of segmentation-related multifunctionality, an average multifunctionality is found between 1.8 and 3.6, depending on what is considered to count as a segment's communicative function. Moreover, a good understanding of the nature of the relations among the various multiple functions that a segment may have, and of the relations between functional segments and other units in dialogue segmentation, opens the way for defining a multidimensional computational update semantics for dialogue interpretation.

Frequency of occurrence of Cantonese word initials, finals and tones

AbstractThirty 10-minute spontaneous speech and dialogue segments were randomly recorded from live radio and TV programmes broadcast in Hong Kong. The speech samples were carefully phonetically transcribed by two experienced investigators. Only those speech samples transcribed consistently by the two investigators were used to measure the frequency of occurrence of all Cantonese word initials, finals and tones. Results indicated a similar pattern of distribution of Cantonese initials and finals to that reported previously, with the exceptions of /n/ and /1/, /kwh/ and /kh/ and /kw/ and /k/. The differences are believed to be related to the sociolinguistic change of language use in the current generation in Hong Kong. Findings in tones also revealed a similar distribution as reported previously.

From Tiberias to Meḥoza: Redactorial and Editorial Processes in Amoraic Babylonia

The literary history of amoraic material-its formulation, editing, and transmission from one amoraic academy to the other-is a central issue in talmudic research. These processes are highlighted in amoraic dialogue segments, which are generally longer and more involved than apodictic material. Amoraic dialogue material from the Land of Israel as presented in the Babylonian Talmud constitutes the best sample for study of these editorial phenomena, since their inclusion in the Babylonian corpus required special editorial processes. The Bavli contains thirty-three sugyot of disputation dialogue between R. Yoḥanan ben Nafḥa and R. Shimʿon b. Laqish, set in the format of [inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="01i" /] which form the single largest body of amoraic dialogue material from the Land of Israel in the Babylonian Talmud. Clearly, these segments have been edited for inclusion in the Bavli, since the editorial terminology [inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="02i" /] is Babylonian Aramaic. Previous scholarly research has pointed to a special relationship between the teachings of R. Yoḥanan and the Amoraim of Meḥoza, especially Rava bar Yosef bar Ḥama. In this article, the author studies the Babylonian editing of the R. Yoḥanan-R. Shimʿon b. Laqish dialogue segments in the [inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="03i" /] format in light of their parallels in the Yerushalmi, and identifies Rava and his fourth and fifth generation colleagues in Meḥoza as the primary editors, to whom the terminology [inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="04i" /] can be attributed

Dialogue Acts, Synchronizing Units, and Anaphora Resolution

In this paper, we present the results of a corpus analysis, and a model of anaphora resolution in spontaneous spoken dialogues. The main finding of our corpus analysis is that less than half the pronouns and demonstratives have NP antecedents in the preceding text; 22% have sentential antecedents and the remainder have no identifiable linguistic antecedents. As part of the corpus analysis we present the results of inter-annotat or agreement tests. These were carried out for the annotation of anaphor types and their antecedents, and for the segmentation of the dialogues into dialogue acts. The results of the inter-annotator agreement tests indicate that our classification method is reliable and that the annotated dialogues can be used as a standard against which to measure the performance of the anaphor resolution algorithm. The algorithm, based on Strube (1998), is capable of classifying pronouns and demonstratives, and co-indexing anaphors with NP and sentential antecedents. The domain from which potential antecedents for both individual and discourse-deictic anaphors can be elicited is defined in terms of dialogue acts. The dialogue segmentation method uses dialogue acts to form Synchronizing Units, which reflect the achievement of common ground (Stalnaker 1974, 1979). We show that predicate information, NP form, and dialogue structure can be successfully used in the resolution process.