Robot Programming by Demonstration

Abstract

Robot programming by demonstration (PbD) has become a central topic of robotics that spans across general research areas such as human-robot interaction, machine learning, machine vision and motor control. Robot PbD started about 30 years ago, and has grown importantly during the past decade. The rationale for moving from purely preprogrammed robots to very flexible user-based interfaces for training robots to perform a task is three-fold. First and foremost, PbD, also referred to as imitation learning, is a powerful mechanism for reducing the complexity of search spaces for learning. When observing either good or bad examples, one can reduce the search for a possible solution, by either starting the search from the observed good solution (local optima), or conversely, by eliminating from the search space what is known as a bad solution. Imitation learning is, thus, a powerful tool for enhancing and accelerating learning in both animals and artifacts. Second, imitation learning offers an implicit means of training a machine, such that explicit and tedious programming of a task by a human user can be minimized or eliminated (Fig. 59.1). Imitation learning is thus a natural means of interacting with a machine that would be accessible to lay people. Left: A robot learns how to make a chess move (namely moving the queen forward) by generalizing across different demonstrations of the task performed in slightly different situations (different starting positions of the hand). The robot records its jointsʼ trajectories and learns to extract what-to-imitate, i.e. that the task constraints are reduced to a subpart of the motion located in a plane defined by the three chess pieces. Right: The robot reproduces the skill in a new context (for different initial position of the chess piece) by finding an appropriate controller that satisfies both the task constraints and constraints relative to its body limitation (how-to-imitate problem), adapted from [59.1] Third, studying and modeling the coupling of perception and action, which is at the core of imitation learning, helps us to understand the mechanisms by which the self-organization of perception and action could arise during development. The reciprocal interaction of perception and action could explain how competence in motor control can be grounded in rich structure of perceptual variables, and vice versa, how the processes of perception can develop as means to create successful actions. PbD promises were thus multiple. On the one hand, one hoped that it would make learning faster, in contrast to tedious reinforcement learning methods or trials-and-error learning. On the other hand, one expected that the methods, being user-friendly, would enhance the application of robots in human daily environments. Recent progresses in the field, which we review in this chapter, show that the field has made a leap forward during the past decade toward these goals. In addition, we anticipate that these promises may be fulfilled very soon. Section 59.1 presents a brief historical overview of robot Programming by Demonstration (PbD), introducing several issues that will be discussed later in this chapter. Section 59.2 reviews engineering approaches to robot PbD with an emphasis on machine learning approaches that provide the robot with the ability to adapt the learned skill to different situations (Sect. 59.2.1). This section discusses also the different types of representation that one may use to encode a skill and presents incremental learning techniques to refine the skill progressively (Sect. 59.2.4). Section 59.2.3 emphasizes the importance to give the teacher an active role during learning and presents different ways in which the user can convey cues to the robot to help it to improve its learning. Section 59.2.4 discusses how PbD can be jointly used with other learning strategies to overcome some limitations of PbD. Section 59.3 reviews works that take a more biological approach to robot PbD and develops models of either the cognitive or neural processes of imitation learning in primates. Finally, Sect. 59.4 lists various open issues in robot PbD that have yet been little explored by the field. KeywordsHide Markov ModelReinforcement LearningAugmented RealityGaussian Mixture ModelHumanoid RobotThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.