Human Demonstration Data Research Articles

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment.

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

Research on LFD System of Humanoid Dual-Arm Robot

Although robots have been widely used in a variety of fields, the idea of enabling them to perform multiple tasks in the same way that humans do remains a difficulty. To solve this, we investigate the learning from demonstration (LFD) system with our independently designed symmetrical humanoid dual-arm robot. We present a novel action feature matching algorithm. This algorithm accurately transforms human demonstration data into task models that robots can directly execute, considerably improving LFD’s generalization capabilities. In our studies, we used motion capture cameras to capture human demonstration actions, which included combinations of simple actions (the action layer) and a succession of complicated operational tasks (the task layer). For the action layer data, we employed Gaussian mixture models (GMM) for processing and constructing an action primitive library. As for the task layer data, we created a “keyframe” segmentation method to transform this data into a series of action primitives and build another action primitive library. Guided by our algorithm, the robot successfully imitated complex human tasks. Results show its excellent task learning and execution, providing an effective solution for robots to learn from human demonstrations and significantly advancing robot technology.

One-Shot Domain-Adaptive Imitation Learning via Progressive Learning Applied to Robotic Pouring

Traditional deep learning-based visual imitation learning techniques require a large amount of demonstration data for model training, and the pre-trained models are difficult to adapt to new scenarios. To address these limitations, we propose a unified framework using a novel progressive learning approach comprised of three phases: i) a coarse learning phase for concept representation, ii) a fine learning phase for action generation, and iii) an imaginary learning phase for domain adaptation. Overall, this approach leads to a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">one-shot domain-adaptive imitation learning</i> framework. We use robotic pouring as an example task to evaluate its effectiveness. Our results show that the method has several advantages over contemporary end-to-end imitation learning approaches, including an improved success rate for task execution and more efficient training for deep imitation learning. In addition, the generalizability to new domains is improved, as demonstrated here with novel backgrounds, target containers, and granule combinations in the experiment. We believe that the proposed method is broadly applicable to various industrial or domestic applications that involve deep imitation learning for robotic manipulation, and where the target scenarios are diverse and human demonstration data is limited. For project video, please check our website:. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this paper is to develop a progressive learning framework, which can be used for both service and industrial robots to learn from human demonstrations, and then transfer the learned skill to different scenarios with ease. We use the robotic pouring task as an example to demonstrate the effectiveness of our proposed method, since pouring is an essential skill for service robots to assist humans’ daily lives, and can benefit robot automation in wet-lab industries. The aim of this research is to enable robots to obtain visuomotor skills (such as the pouring skill), and accomplish the tasks with a high success rate using our proposed progressive learning method. We conducted experiments to show that the proposed method has good performance, high data efficiency and evident generalizability. This is significant for intelligent robots working in various practical applications.

Learning compliant dynamical system from human demonstrations for stable force control in unknown environments

Force control in unknown environments poses a significant challenge in robotics due to the necessity of adjusting impedance to accommodate environmental uncertainties. One potential solution is to transfer human variable impedance skills for delicate force control to robots. However, due to the short duration and small displacement of the contact transients, it is difficult to collect demonstration data and learn the control strategy used by human. This paper focuses on learning a stable force control policy from human demonstration during contact transients. We first propose a human-in-the-loop method for transferring human variable impedance skills during contact transient in unknown environments. Then, we analyze human demonstration data and conclude two observations: (1) Demonstrators adjust the end-effector velocity of the robot according to the force error, such that the velocity magnitude reduces nonlinearly as the force error decreases; (2) Demonstrators modify the slope of the velocity–force error curve according to environmental stiffnesses, such that the slope magnitude decreases as environmental stiffness increases. Finally, based on these observations, we proposed a novel human-inspired force control strategy called compliant dynamical system (CDS), which requires only actual contact force and rough estimation of environmental stiffness to achieve desired force control in contact tasks. The stability of the proposed CDS is rigorously proved using Lyapunov stability theory to guarantee accurate force control performance. The effectiveness of the proposed method is validated by simulation and real-world experiments involving single-arm contact with unknown environments and dual-arm cooperative transportation of an object.

Anthropomorphic Grasping of Complex-Shaped Objects Using Imitation Learning

This paper presents an autonomous grasping approach for complex-shaped objects using an anthropomorphic robotic hand. Although human-like robotic hands have a number of distinctive advantages, most of the current autonomous robotic pickup systems still use relatively simple gripper setups such as a two-finger gripper or even a suction gripper. The main difficulty of utilizing human-like robotic hands lies in the sheer complexity of the system; it is inherently tough to plan and control the motions of the high degree of freedom (DOF) system. Although data-driven approaches have been successfully used for motion planning of various robotic systems recently, it is hard to directly apply them to high-DOF systems due to the difficulty of acquiring training data. In this paper, we propose a novel approach for grasping complex-shaped objects using a high-DOF robotic manipulation system consisting of a seven-DOF manipulator and a four-fingered robotic hand with 16 DOFs. Human demonstration data are first acquired using a virtual reality controller with 6D pose tracking and individual capacitive finger sensors. Then, the 3D shape of the manipulation target object is reconstructed from multiple depth images recorded using the wrist-mounted RGBD camera. The grasping pose for the object is estimated using a residual neural network (ResNet), K-means clustering (KNN), and a point-set registration algorithm. Then, the manipulator moves to the grasping pose following the trajectory created by dynamic movement primitives (DMPs). Finally, the robot performs one of the object-specific grasping motions learned from human demonstration. The suggested system is evaluated by an official tester using five objects with promising results.

Goal-aware generative adversarial imitation learning from imperfect demonstration for robotic cloth manipulation

Generative Adversarial Imitation Learning (GAIL) can learn policies without explicitly defining the reward function from demonstrations. GAIL has the potential to learn policies with high-dimensional observations as input, e.g., images. By applying GAIL to a real robot, perhaps robot policies can be obtained for daily activities like washing, folding clothes, cooking, and cleaning. However, human demonstration data are often imperfect due to mistakes, which degrade the performance of the resulting policies. We address this issue by focusing on the following features: (1) many robotic tasks are goal-reaching tasks, and (2) labeling such goal states in demonstration data is relatively easy. With these in mind, this paper proposes Goal-Aware Generative Adversarial Imitation Learning (GA-GAIL), which trains a policy by introducing a second discriminator to distinguish the goal state in parallel with the first discriminator that indicates the demonstration data. This extends a standard GAIL framework to more robustly learn desirable policies even from imperfect demonstrations through a goal-state discriminator that promotes achieving the goal state. Furthermore, GA-GAIL employs the Entropy-maximizing Deep P-Network (EDPN) as a generator, which considers both the smoothness and causal entropy in the policy update, to achieve stable policy learning from two discriminators. Our proposed method was successfully applied to two real-robotic cloth-manipulation tasks: turning a handkerchief over and folding clothes. We confirmed that it learns cloth-manipulation policies without task-specific reward function design. Video of the real experiments are available at this URL.

Human-Tailored Data-Driven Control System of Autonomous Vehicles

Path Tracking Controller (PTC) plays a key role in achieving improved dynamic behaviour of Autonomous Vehicle (AV) while it is mainly responsible for implementing the planned paths. In this work, a learning-based Model Predictive Controller (MPC) is proposed to integrate a data-driven approach to the path tracking task using human driving demonstrations. The objective is to reproduce customised vehicle motion profile that is statistically comparable to a human driver with a proper trade-off between vehicle speed-acceleration profile and tracking accuracy. A combined longitudinal and lateral MPC controller with a novel feature-based parametric cost function is designed to perform the path tracking task under different road conditions. A high precision data logging system is developed to collect human driving demonstration data. From the analysis of the collected data, relevant features are chosen based on their applicability to produce a customised motion profile while performing AV’s path tracking task. Next, a bi-level optimisation-based Inverse Optimal Controller (IOC) is designed to learn the cost function’s parameters using the human demonstration data. The outcomes of the study show that the proposed controller can effectively regenerate some characteristics of human driving, including speed, longitudinal and lateral accelerations, and yaw rate.

Learning manipulation skills with demonstrations for the swing process control of dredgers

The Cutter Suction Dredger (CSD) is one of the key equipment dedicated to the construction and maintenance projects of harbours, ports and navigational channels. Among the dredging manipulations, the swing process is the most tedious and recurring work for human operators, which often leads to accidents because of carelessness or fatigue of the operators. This paper aims at producing a learning approach for the intelligent control of the swing process of a CSD so as to release human operators from such a boring and heavy task. To this end, the swing process control is formulated as a sequential decision making problem, and Deep Reinforcement Learning (DRL) is employed to design the learning approach based on deterministic policy gradient. The novel feature of the proposed approach is that the manipulation skills are obtained via trial-and-error interactions with a predicting network constructed by human demonstration data. In our approach, human demonstrations can provide a channel to predict state transitions, and also can regulate the exploration procedure for the learning agent. In addition, we carry out empirical studies to investigate how to treat the demonstration data with regard to self exploration, and the experimental results show that the proposed approach provides an effective means of controlling the swing process of CSDs.

Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning

Autonomous exploration is an important application of multi-vehicle systems, where a team of networked robots are coordinated to explore an unknown environment collaboratively. This technique has earned significant research interest due to its usefulness in search and rescue, fault detection and monitoring, localization and mapping, etc. In this paper, a novel cooperative exploration strategy is proposed for multiple mobile robots, which reduces the overall task completion time and energy costs compared to conventional methods. To efficiently navigate the networked robots during the collaborative tasks, a hierarchical control architecture is designed which contains a high-level decision making layer and a low-level target tracking layer. The proposed cooperative exploration approach is developed using dynamic Voronoi partitions, which minimizes duplicated exploration areas by assigning different target locations to individual robots. To deal with sudden obstacles in the unknown environment, an integrated deep reinforcement learning based collision avoidance algorithm is then proposed, which enables the control policy to learn from human demonstration data and thus improve the learning speed and performance. Finally, simulation and experimental results are provided to demonstrate the effectiveness of the proposed scheme.

Force control design for robots based on correlation in human demonstration data

Force control design for robots based on correlation in human demonstration data

Teaching Tasks to Multiple Small Robots by Classifying and Splitting a Human Example

[abstFig src='/00290002/14.jpg' width='300' text='Two robots performing the task of opening the folding chair lying on the floor' ] In this study, we present a novel framework to address the problem of teaching manipulation tasks performed by a single human to a set of multiple small robots in a short period. First, we focused on classifying the manipulation style used during a human-performed task. An allocator process is proposed to determine the type and number of robots to be taught based on the capabilities of available robots. Then, according to the detected task requirements, robot behaviors are generated to create robot programs by splitting human demonstration data. Small robots were used to evaluate our approach in four defined tasks that were taught by a single human. Experiments demonstrated the efficiency of the method to classify and judge whether the division of a task is necessary or not. Moreover, robot programs were generated for manipulating selected objects either individually or in a cooperative manner.

バリ取り作業におけるヒューマンスキルの教示方法に関する検討

This paper discusses a new method for teaching a deburring robot based on demonstration of human skillful motion. The robot is programmed to adjust the tool feedrate in accordance with the varying burr characteristics, such as burr size and material properties. This dynamic change of tool feedrate is motivated by the effective human skill in performing a deburring task. The relationship between the tool feedrate and burr characteristics is obtained from human demonstration data and stored in a computer as an associative memory. This associative memory enables the robot to select the tool feedrate that well matches the burr characteristics. Therefore, the robot motion is always effective in removing burrs and generating smooth finish of workpiece surface without severe tool wear. In order to identify burr characteristics, a laser displacement sensor has been used for direct burr height measurement, and a deburring process model has been applied for material property differentiation. The learned associative memory is stored and represented by a neural network, which can be easily incorporated into robot programming. Experimental results show that a robot can perform a deburring task in a manner similar to its human teacher.