Map Task Dialogues Research Articles

Speech Pauses and Pronominal Anaphors

This paper addresses the usefulness of speech pauses for determining whether third person neuter gender singular pronouns refer to individual or abstract entities in Danish spoken language. The annotations of dyadic map task dialogues and spontaneous first encounters are analyzed and used in machine learning experiments act to automatically identify the anaphoric functions of pronouns and the type of abstract reference. The analysis of the data shows that abstract reference is more often performed by marked (stressed or demonstrative pronouns) than by unmarked personal pronouns in Danish speech as in English, and therefore previous studies of abstract reference in the former language are corrected. The data also show that silent and filled pauses precede significantly more often third person singular neuter gender pronouns when they refer to abstract entities than when they refer to individual entities. Since abstract entities are not the most salient ones and referring to them is cognitively more hard than referring to individual entities, pauses signal this complex processes. This is in line with perception studies, which connect pauses with the expression of abstract or complex concepts. We also found that unmarked pronouns referring to an entity type usually referred to by a marked pronoun are significantly more often preceded by a speech pause than marked pronouns with the same referent type. This indicates that speech pauses can also signal that the referent of a pronoun of a certain type is not the most expected one. Finally, language models were produced from the annotated map task and first encounter dialogues in order to train machine learning experiments to predict the function of third person neuter gender singular pronouns as a first step toward the identification of the anaphoric antecedents. The language models from the map task dialogues were also used for training classifiers to determine the referent type (speech act, event, fact or proposition) of abstract anaphors. In all cases, the best results were obtained by a multilayer perceptron with an F1-score between 0.52 and 0.67 for the three-class function prediction task and of 0.73 for the referential type prediction.

Adaptação acústico-prosódica local em Português Europeu

This paper presents an acoustic-prosodic analysis of entrainment in map-task dialogues in European Portuguese. Our main goal is to analyze how turn-by-turn entrainment varies with distinct structural metadata events: types of sentence-like units (SUs) in consecutive turns (e.g. interrogatives followed by declaratives, or both declaratives), and with the presence of discourse markers, affirmative cue words, and disfluencies in the beginning of turns. Results show that entrainment at turn-exchanges occurs in terms of pitch, energy, duration, and voice quality. Considering SUs types, question-answer pairs are the ones with stronger similarity, as declarative-interrogative turns are the ones where less entrainment occurs. Moreover, in question-answer pairs, Yes/No and Tag questions present stronger evidences of entrainment than Wh- questions. Regarding turn-initial structures, there are evidences of (i) stronger entrainment when the second turn begins with an affirmative cue word, (ii) less strong with ambiguous structures (such as ‘OK’), emphatic affirmative answers, and negative cue words; (iii) and scarce with disfluencies and discourse markers. Different degrees of local entrainment may be related to the informative structure of distinct structural metadata events.

Adaptação acústico-prosódica entre falantes

This paper presents a global analysis of entrainment in map-task dialogues in European Portuguese, including 48 dialogues, between 24 speakers. Our main goal is to analyze the acoustic-prosodic similarities between speaker pairs, namely if there are global entrainment cues displayed in the dialogues, if entrainment is manifested in distinct sets of features shared amongst the speakers, if entrainment depends on the gender and role of the speaker (giver or follower), and if speakers tend to entrain more with specific interlocutors regardless of the role. Results show that globally speakers tend to be more similar to their partners than to their own speech in the majority of the analyzed features, a strong evidence for entrainment. Moreover, almost all the pairs of speakers display cues of global entrainment, even though in different degrees (speakers entrain but in distinct features). Additionally, the role and gender effects tend to be less striking than the specific interlocutor effect. Our results support the fact that all prosodic parameters are monitored by the speakers in our corpus, contrarily to studies for other languages, which indicate that the main cues are energy related.

Cross-domain analysis of discourse markers in European Portuguese

This paper presents an analysis of discourse markers in two spontaneous speech corpora for European Portuguese - university lectures and map-task dialogues - and also in a collection of tweets, aiming at contributing to their categorization, scarcely existent for European Portuguese. Our results show that the selection of discourse markers is domain and speaker dependent. We also found that the most frequent discourse markers are similar in all three corpora, despite tweets containing discourse markers not found in the other two corpora. In this multidisciplinary study, comprising both a linguistic perspective and a computational approach, discourse markers are also automatically discriminated from other structural metadata events, namely sentence-like units and disfluencies. Our results show that discourse markers and disfluencies tend to co-occur in the dialogue corpus, but have a complementary distribution in the university lectures. We used three acoustic-prosodic feature sets and machine learning to automatically distinguish between discourse markers, disfluencies and sentence-like units. Our in-domain experiments achieved an accuracy of about 87% in university lectures and 84% in dialogues, in line with our previous results. The eGeMAPS features, commonly used for other paralinguistic tasks, achieved a considerable performance on our data, especially considering the small size of the feature set. Our results suggest that turn-initial discourse markers are usually easier to classify than disfluencies, a result also previously reported in the literature. We conducted a cross-domain evaluation in order to evaluate the robustness of the models across domains. The results achieved are about 11%-12% lower, but we conclude that data from one domain can still be used to classify the same events in the other. Overall, despite the complexity of this task, these are very encouraging state-of-the-art results. Ultimately, using exclusively acoustic-prosodic cues, discourse markers can be fairly discriminated from disfluencies and SUs. In order to better understand the contribution of each feature, we have also reported the impact of the features in both the dialogues and the university lectures. Pitch features are the most relevant ones for the distinction between discourse markers and disfluencies, namely pitch slopes. These features are in line with the wide pitch range of discourse markers, in a continuum from a very compressed pitch range to a very wide one, expressed by total deaccented material or H+L* L* contours, with upstep H tones.

The Mechanism for Updating Mutual Belief in Map Task Dialogue

The Mechanism for Updating Mutual Belief in Map Task Dialogue

Classificação prosódica de marcadores discursivos

This work describes the discourse markers present in two corpora for European Portuguese, in different domains (university lectures and map-task dialogues). In this study, we also perform a multiclass automatic classification task based on prosodic features to verify in both corpora which words are discourse markers, which are disfluencies, and which are sentence like-units (SUs). Results show that the selection of discourse markers varies across domain and between speakers. As for the classification task, results show that the discourse markers are better classified in the lectures corpus (87%) than in the dialogue corpus (84%). However, cross-domain experiments evidenced that data trained with the dialogue corpus predicts better the events in the lecture corpus, since this domain displays more speakers and therefore complex patterns. In both corpora, markers are more easily classified as SUs than as disfluencies.

Classificação prosódica de marcadores discursivos

This work describes the discourse markers present in two corpora for European Portuguese, in different domains (university lectures and map-task dialogues). In this study, we also perform a multiclass automatic classification task based on prosodic features to verify in both corpora which words are discourse markers, which are disfluencies, and which are sentence like-units (SUs). Results show that the selection of discourse markers varies across domain and between speakers. As for the classification task, results show that the discourse markers are better classified in the lectures corpus (87%) than in the dialogue corpus (84%). However, cross‑domain experiments evidenced that data trained with the dialogue corpus predicts better the events in the lecture corpus, since this domain displays more speakers and therefore complex patterns. In both corpora, markers are more easily classified as SUs than as disfluencies.

Speaking style effects in the production of disfluencies

This work explores speaking style effects in the production of disfluencies. University lectures and map-task dialogues are analyzed in order to evaluate if the prosodic strategies used when uttering disfluencies vary across speaking styles. Our results show that the distribution of disfluency types is not arbitrary across lectures and dialogues. Moreover, although there is a statistically significant cross-style strategy of prosodic contrast marking (pitch and energy increases) between the region to repair and the repair of fluency, this strategy is displayed differently depending on the specific speech task. The overall patterns observed in the lectures, with regularities ascribed for speaker and disfluency types, do not hold with the same strength for the dialogues, due to underlying specificities of the communicative purposes. The tempo patterns found for both speech tasks also confirm their distinct behaviour, evidencing the more dynamic tempo characteristics of dialogues. In university lectures, prosodic cues are given to the listener both for the units inside disfluent regions and between these and the adjacent contexts. This suggests a stronger prosodic contrast marking of disfluency–fluency repair when compared to dialogues, as if teachers were monitoring the different regions – the introduction to a disfluency, the disfluency itself and the beginning of the repair – demarcating them in very contrastive ways.

Data-driven models for timing feedback responses in a Map Task dialogue system

Traditional dialogue systems use a fixed silence threshold to detect the end of users’ turns. Such a simplistic model can result in system behaviour that is both interruptive and unresponsive, which in turn affects user experience. Various studies have observed that human interlocutors take cues from speaker behaviour, such as prosody, syntax, and gestures, to coordinate smooth exchange of speaking turns. However, little effort has been made towards implementing these models in dialogue systems and verifying how well they model the turn-taking behaviour in human–computer interactions. We present a data-driven approach to building models for online detection of suitable feedback response locations in the user's speech. We first collected human–computer interaction data using a spoken dialogue system that can perform the Map Task with users (albeit using a trick). On this data, we trained various models that use automatically extractable prosodic, contextual and lexico-syntactic features for detecting response locations. Next, we implemented a trained model in the same dialogue system and evaluated it in interactions with users. The subjective and objective measures from the user evaluation confirm that a model trained on speaker behavioural cues offers both smoother turn-transitions and more responsive system behaviour.

Head movements, facial expressions and feedback in conversations: empirical evidence from Danish multimodal data

This article deals with multimodal feedback in two Danish multimodal corpora, i.e., a collection of map-task dialogues and a corpus of free conversations in first encounters between pairs of subjects. Machine learning techniques are applied to both sets of data to investigate various relations between the non-verbal behaviour—more specifically head movements and facial expressions—and speech with regard to the expression of feedback. In the map-task data, we study the extent to which the dialogue act type of linguistic feedback expressions can be classified automatically based on the non-verbal features. In the conversational data, on the other hand, non-verbal and speech features are used together to distinguish feedback from other multimodal behaviours. The results of the two sets of experiments indicate in general that head movements, and to a lesser extent facial expressions, are important indicators of feedback, and that gestures and speech disambiguate each other in the machine learning process.

The intonation of polar questions in Italian: Where is the rise?

Earlier studies on Standard Italian describe polar questions as being characterised by a terminal rise, as opposed to a terminal fall for statements, where a low/falling accentual movement precedes the terminal part of the contour in both sentence types. The same is generally claimed for the Northern and Central Italian varieties (including Florentine, i.e. the variety from which Standard Italian stems), whereas Southern accents are characterised by an accentual rise followed by a terminal fall, being therefore the primary cue for question in non-terminal position. However, a closer look at the existing literature on regional Italian question intonation reveals that such a geographical distribution of intonational features across Italian accents is not that clear-cut. A reason for this discrepancy might be the different speaking styles – here intended as the broad spontaneous vs. read distinction – of the spoken productions analysed. The aim of this paper is to call into question the claim that a terminal rise preceded by an accentual low/fall is the most widespread intonational feature for marking questioning across Italian accents. The goal is to provide a clearer picture of question intonation in Italian by looking at the distribution of the rise as either on terminal or non-terminal position across a large number of varieties, where speech materials have been elicited with the same methodology, and they are therefore homogeneous with respect to speaking style. Intonation analysis has been carried out on spontaneous yes–no questions extracted from the Map Task dialogues collected in the CLIPS national corpus (Corpora e Lessici di Italiano Parlato e Scritto – Corpora and Lexicons of Spoken and Written Italian) covering 15 varieties of Italian. Results of this analysis on the Northern, Central, and Southern polar questions reveals that the accentual rise prevails, and that the distribution of the rise across varieties is independent of the geography.

Marking up Japanese Map Task Dialogue Corpus

This presentation reports an outline of marking up the Japanese Map Task Dialogue Corpus. The project was conducted by an independent group of researchers from the faculties of Chiba University and other institutions. This corpus contains 128 dialogues by 64 Japanese native speakers. The basic design of the dialogues and recordings conform to the original HCRC Map Task Corpus of the University of Edinburgh. Two speakers participated in the map task dialogues; an instruction giver who has a map with a route and an instruction follower who has a map without a route. The giver verbally instructs the follower to draw the route on his/her map. The corpus is marked up according to XML‐based TEI (Text Encoding Initiative) P5 format, which is developed in an effort to provide interchangeable and shareable text data. The transcriptions of TEI format are viewed as ‘‘tags‘‘ that describe the start and end times of utterances, the duration of pauses, nonverbal events, and synchronization of overlapping utterances. This corpus will be available through NII‐SRC (Speech Resources Consortium).

Audio channel constraints in video-mediated communication

This study investigated the effects of two types of audio channels upon the effectiveness of task-based interactions in a video-mediated context (VMC). Forty undergraduates completed a collaborative task (The Map Task) using either a full or half-duplex audio channel. Their performance was compared to face-to-face interactions, taken from the Human Communication Research Centre corpus of Map Task Dialogues. Effects of varying the audio channel were explored by comparing task performance, patterns of speech, and establishment of mutual understanding. Users of the full-duplex VMC made insufficient allowance for the VMC context; they completed the task less accurately than face-to-face participants, and interrupted each other more frequently than other participants. Participants in the half-duplex VMC however performed as well as face-to-face participants. They made sensible adaptations to the constraints imposed by the half-duplex VMC context, producing longer dialogues, with more explicit turn-taking management, and taking greater care in establishing mutual knowledge.

Adapting communicative strategies to computer‐mediated communication: an analysis of task performance and dialogue structure

AbstractThis study investigates how people adapt to text‐based computer‐mediated communication (CMC), specifically focusing on how novice users change their communicative strategies over time to achieve effective communication. Twenty pairs of university students completed three collaborative problem‐solving tasks (The Map Task; Brown et al., 1984), over a series of days. Their performance was compared with the results from spoken interactions, taken from the Human Communications Research Centre corpus of Map Task dialogues. Task performance by CMC participants was initially poorer when compared to the performance of spoken interlocutors, but improved as they gained experience of the CMC context. Detailed analysis of the process of communication, using dialogue measures and Conversational Games Analysis (Kowtko et al., 1992), shows that effective communication and collaboration was achieved by users of the CMC system after a relatively moderate amount of experience. Participants in the CMC context changed the way in which they gave instructions and interacted over time. They adopted a precise, highly specified style of giving directions which required little interpretation by addressees. This concise style of communication is rarely found in face‐to‐face or spoken interactions. Copyright © 2003 John Wiley & Sons, Ltd.

High-Rising Terminals and Fall-Rise Tunes in Australian English

It is usually claimed that statement high rises in Australian English are more or less phonetically identical to yes/no question rises. In this paper, statement high rises and question rises were examined in a corpus of controlled spontaneous speech (i.e. map task dialogues) to see if this is the case. It appears that speakers in this study used different kinds of rises for declaratives and questions. The majority of statement high rises were realized with a low pitch accent onset, whereas almost all question rises were produced with high pitch accent onsets. High-range fall-rises also appeared to be used by some speakers in the same way as statement high rises. Implications of these findings for the current ToBI analysis of Australian English are considered.