Learning, Improving, and Generalizing Motor Skills for the Peg-in-Hole Tasks Based on Imitation Learning and Self-Learning

Nam Jun Cho,Il Hong Suh,Jong Bok Kim,Sang Hyoung Lee

doi:10.3390/app10082719

Nam Jun Cho, Il Hong Suh + Show 2 more

Open Access

https://doi.org/10.3390/app10082719

Copy DOI

Abstract

We propose a framework based on imitation learning and self-learning to enable robots to learn, improve, and generalize motor skills. The peg-in-hole task is important in manufacturing assembly work. Two motor skills for the peg-in-hole task are targeted: “hole search” and “peg insertion”. The robots learn initial motor skills from human demonstrations and then improve and/or generalize them through reinforcement learning (RL). An initial motor skill is represented as a concatenation of the parameters of a hidden Markov model (HMM) and a dynamic movement primitive (DMP) to classify input signals and generate motion trajectories. Reactions are classified as familiar or unfamiliar (i.e., modeled or not modeled), and initial motor skills are improved to solve familiar reactions and generalized to solve unfamiliar reactions. The proposed framework includes processes, algorithms, and reward functions that can be used for various motor skill types. To evaluate our framework, the motor skills were performed using an actual robotic arm and two reward functions for RL. To verify the learning and improving/generalizing processes, we successfully applied our framework to different shapes of pegs and holes. Moreover, the execution time steps and path optimization of RL were evaluated experimentally.

Highlights

In this paper, we propose a framework for learning, improving, and generalizing the motor skills for a peg-in-hole task
The main contribution of this paper is to propose a framework that enables robots to learn, improve, and generalize motor skills for the peg-in-hole task using a mixture of imitation learning and self-learning
We performed human demonstrations on the rectangle shape (Figure 5a) to learn and acquire the initial motor skills based on imitation learning

Summary

Introduction

We propose a framework for learning, improving, and generalizing the motor skills for a peg-in-hole task. We define a tuple of model parameters that can perform both classification of input signals and generation of appropriate motion trajectories as a motor skill. This task plays an important role in assembly work and is frequently encountered in the manufacturing industry [1]. The peg-in-hole task is often performed in conditions where the exact positions/postures of a hole or peg are unknown due to the errors in vision sensors and robot actuators To solve this problem, robots need to continuously perform the repetition of reaction classification and reaction generation, while the peg and hole maintain contact until their task completion. We focus on obtaining these optimal motor skills for the peg-in-hole task

Objectives

Results

Discussion

Conclusion