Robot Learning From Human Demonstration of Activities of Daily Living (ADL) Tasks

Abstract

Abstract The redundant kinematic structure of the human upper body provides sufficient dexterity in performing daily living activities, such as drinking, eating, and other manipulation tasks. Most of these Activities of Daily Living (ADL) tasks can be performed using various trajectories; however, healthy humans employ some possible trajectories to accomplish these tasks. Assistive robots can explore these Cartesian trajectories and joint angles’ trajectories from human demonstrations and utilize artificial intelligence techniques to recognize, learn, and perform ADLs using robotic arms. This paper aims to develop a platform for Assistive Robots to identify and perform ADLs through human demonstration. In this paper, two Mask Regional Convolutional Neural Networks (Mask R-CNN) were developed to predict human intention for a given task, and the Reinforcement Learning approach was implemented for optimum trajectory planning. The First Mask R-CNN network, which was used for the detection of an object, was trained on a novel dataset. This dataset comprises images captured by the research team as well as gathered images from the internet. The first Mask R-CNN is expected to achieve two main objectives. The first objective is related to structure learning, which detects an object in a cluttered environment. The second objective is to estimate parameters that maximize the weight and detection rate of diverse types of similar objects. The first Mask R-CNN tries to detect the object with its location in the image and provides a soft mask on it. The second Mask R-CNN, which tries to detect human posture, was trained on a pre-existing dataset comprising human posture images. The output of these two Mask R-CNN’s then feed-forward to recognize human intensions that can be achieved through human-pose key points. Prediction of human intention was performed if the detected object and human pose match a range of pre-stored values specific to ADLs. Once the human intention has been identified via Mask R-CNN networks, the system starts planning for an optimal trajectory. For the trajectory planning, the Q-learning approach was implemented, which uses the concept of reward and penalty while exploring the unstructured environment in order to find an optimal trajectory. Experiments were conducted to determine the level of accuracy and reliability provided by our assistive platform in predicting human actions and performing trajectory planning. Results show that our assistive platform successfully recognized human actions such as water pouring, cereal making, and opening the door, as well as finding optimized trajectories to perform those actions.