Incremental Learning Introspective Movement Primitives From Multimodal Unstructured Demonstrations

Hongmin Wu,Wu Yan,Xuefeng Zhou,Qianxin Su,Zhihao Xu,Shuai Li,Taobo Cheng

doi:10.1109/access.2019.2947529

Abstract

Learning movement primitive from unstructured demonstrations has become a popular topic in recent years, which provides a natural way to endow human-inspired skills to robots. The main idea of movement primitives is that should suffice to reconstruct a large set of complex manipulation tasks. However, conventional learning methods mostly focus on the kinesthetic variables and ignore those critical introspective capacities in manipulation such as movement generalization and assessment of the sensory signals. In this paper, we investigate the association of generalization, fault detection, fault diagnoses, and task exploration during manipulation task, and call such movement primitives augmented with introspective capacities Introspective Movement Primitives (IMP). With our previous work, this paper mainly addresses how IMPs can be acquired by assessing the quality of multimodal sensory data of unstructured demonstrations and how they can incrementally create manipulation task by reverse execution and human interaction. Experimental evaluation on a human-robot collaborative packaging task with a Rethink Baxter robot, results indicate that our proposed method can effectively increase robustness towards external perturbations and adaptive exploration during robot manipulation task.

Highlights

With the rapid development of collaborative robots [1], robots are increasingly moving forward from the traditional structured environment of industrial manufacturing to the unstructured, dynamical, and shared work-space of co-existence, collaboration and integration with human beings
A switching vector auto-regressive (VAR) is considered to model multimodal observation sequences for improving the consistency during robot manipulation task for strengthening the temporal coherency [31], and its observation at each time step should consist of features extracted from an force and torque sensor, robot joint encoders, and a tactile sensor mounted on the end-effector
WORK In our previous work, we focus on the robot introspective capabilities in robotics, which only take the kinematic variables into consideration such that the task is restricted to be applied in human-robot collaborative scenarios that generalization as a desirable characteristic

Summary

INTRODUCTION

With the rapid development of collaborative robots [1], robots are increasingly moving forward from the traditional structured environment of industrial manufacturing to the unstructured, dynamical, and shared work-space of co-existence, collaboration and integration with human beings. A switching vector auto-regressive (VAR) is considered to model multimodal observation sequences for improving the consistency during robot manipulation task for strengthening the temporal coherency [31], and its observation at each time step should consist of features extracted from an force and torque sensor, robot joint encoders, and a tactile sensor mounted on the end-effector For this reason, a nonparametric Bayesian Hidden Markov Model is used to parse the demonstrations into segments (i.e. movement primitives) that can be explained by a set of the latent state zi, i ∈ {1, 2, . We attempt to keep the dynamical phenomenon consistency for many time-steps by increasing the expected probability of self-transition

COMPLEX TASK REPRESENTATION

INTROSPECTIVE CAPACITIES

REVERSE EXECUTION

HUMAN INTERACTION

CONCLUSION AND FUTURE WORK