Modelling situated human-robot interaction using IrisTK

Gabriel Skantze,Martin Johansson

doi:10.18653/v1/w15-4624

Abstract

In this demonstration we show how situated multi-party human-robot interaction can be modelled using the open source framework IrisTK. We will demonstrate the capabilities of IrisTK by showing an application where two users are playing a collaborative card sorting game together with the robot head Furhat, where the cards are shown on a touch table between the players. The application is interesting from a research perspective, as it involves both multi-party interaction, as well as joint attention to the objects under discussion.

Highlights

There has been an increased interest in understanding and modelling multi-party, situated interaction between humans and robots (Bohus & Horvitz, 2011; Mutlu et al, 2012: Johansson et al, 2014; Al Moubayed et al, 2014)
While there are many examples of research systems that can engage in situated interaction (Bohus & Horvitz, 2011; Mutlu et al, 2012), the combination of all these techniques together with spoken dialog technology is not trivial, and it might be hard for a novice to put such systems together
The knowledge to develop and put together the necessary modules is of a very interdisciplinary nature. This calls for a dialog system toolkit for multi-party face-to-face interaction, which provides necessary modules for multimodal input and output and allows the developer or researcher to author the dialog flow in a way that is simple to understand for the novice, yet powerful enough to model more sophisticated behaviours

Summary

Introduction

There has been an increased interest in understanding and modelling multi-party, situated interaction between humans and robots (Bohus & Horvitz, 2011; Mutlu et al, 2012: Johansson et al, 2014; Al Moubayed et al, 2014). The task could be to sort a set of inventions in the order they were invented, or a set of animals by how fast they can run This is a collaborative game, which means that the visitors have to discuss the solution together with Furhat. Furhat’s role in the interaction is similar to that of the visitors, as opposed to for example a tutor role which is often given to robots in similar settings (cf. Al Moubayed et al, 2014)

Overview of IrisTK

Vision and Situation modelling

Speech processing

IrisFlow

System output

Discussion