An Incremental Learning Framework to Enhance Teaching by Demonstration Based on Multimodal Sensor Fusion.

Jie Li,Junpei Zhong,Jingfeng Yang,Chenguang Yang

doi:10.3389/fnbot.2020.00055

Abstract

Though a robot can reproduce the demonstration trajectory from a human demonstrator by teleoperation, there is a certain error between the reproduced trajectory and the desired trajectory. To minimize this error, we propose a multimodal incremental learning framework based on a teleoperation strategy that can enable the robot to reproduce the demonstration task accurately. The multimodal demonstration data are collected from two different kinds of sensors in the demonstration phase. Then, the Kalman filter (KF) and dynamic time warping (DTW) algorithms are used to preprocessing the data for the multiple sensor signals. The KF algorithm is mainly used to fuse sensor data of different modalities, and the DTW algorithm is used to align the data in the same timeline. The preprocessed demonstration data are further trained and learned by the incremental learning network and sent to a Baxter robot for reproducing the task demonstrated by the human. Comparative experiments have been performed to verify the effectiveness of the proposed framework.

Highlights

With the development of control theory and sensor technology, robots have been widely applied in various fields, especially in industry and social service
The purpose of this work is to investigate the practical effect of the proposed method on robot teaching by demonstration (TbD), as well as to explore the impact on the result considering the fusion of multiple modality information
We propose an incremental learning framework to learn demonstration features by integrating different modality data

Summary

Introduction

With the development of control theory and sensor technology, robots have been widely applied in various fields, especially in industry and social service. It plays an increasingly vital role in human daily life, such as entertainment, education, and home service, etc. TbD is an efficient approach to reduce the complexity of teaching a robot to perform new tasks (Billard et al, 2008; Yang et al, 2018). It is essential to take account of some learning methods to learn much more useful features effectively In this sense, robot learning contains two tasks: motion perception based on multiple sensors and features learning with efficient methods. Different modalities of sensors can enable obtaining an accurate description of the target motions

Objectives

Methods

Results

Discussion

Conclusion