A Diagrammatic Framework for Intuitive Human Robot Interaction: An Augmented Reality Approach

Abstract

This thesis explores the area of human robot interaction and develops a framework which is designed to allow ordinary people to control and program complex robots in an intuitive way that is both reliable and accurate. Our framework bridges the experience gap which is typically found between non-technical experts and complex robotic control technologies. The principle mechanism behind our framework is Augmented Reality (AR) and we use this to provide a diagrammatic service to users. The service uses a range of diagrammatic markers including marker-less AR objects, which can be created and connected together and used to command and control the robot and engage in two way communications in the form of a diagram. The diagrams are expressed as three-dimensional augmented reality objects, so that users can annotate arbitrary locations in a given environment with instructions through a video see-through camera. The robots automatically pick up these instructions and perform the respective actions at the specified locations. A key feature of this framework is its ability to translate diagrams into robot-related action tasks, giving each task a context through spatial references. Other features of the framework include scalability and the generic nature where it is conjectured to be applicable for a range of different robots and multimedia devices. In later chapters we report on two case studies where our framework is applied to a set of command and control tasks to measure performance across situation-awareness, task completion-time and cognitive-load. Our results show that our model leads to greater situation awareness and improved task completion times, when compared to conventional interaction methods, such as a gamepad controller. Finally, we conclude the thesis with an overview of our contributions and an account of future work which will integrate our model into multi-modal hybrids and extend the case studies to compare against other interaction methods.