Robotic Manipulation Tasks Research Articles

Learning context-adaptive task constraints for robotic manipulation

Constraint-based control approaches offer a flexible way to specify robotic manipulation tasks and execute them on robots with many degrees of freedom. However, the specification of task constraints and their associated priorities usually requires a human-expert and often leads to tailor-made solutions for specific situations. This paper presents our recent efforts to automatically derive task constraints for a constraint-based robot controller from data and adapt them with respect to previously unseen situations (contexts). We use a programming-by-demonstration approach to generate training data in multiple variations (context changes) of a given task. From this data we learn a probabilistic model that maps context variables to task constraints and their respective soft task priorities. We evaluate our approach with 3 different dual-arm manipulation tasks on an industrial robot and show that it performs better than comparable approaches with respect to reproduction accuracy in previously unseen contexts.

On the Impact of Gravity Compensation on Reinforcement Learning in Goal-Reaching Tasks for Robotic Manipulators

Advances in machine learning technologies in recent years have facilitated developments in autonomous robotic systems. Designing these autonomous systems typically requires manually specified models of the robotic system and world when using classical control-based strategies, or time consuming and computationally expensive data-driven training when using learning-based strategies. Combination of classical control and learning-based strategies may mitigate both requirements. However, the performance of the combined control system is not obvious given that there are two separate controllers. This paper focuses on one such combination, which uses gravity-compensation together with reinforcement learning (RL). We present a study of the effects of gravity compensation on the performance of two reinforcement learning algorithms when solving reaching tasks using a simulated seven-degree-of-freedom robotic arm. The results of our study demonstrate that gravity compensation coupled with RL can reduce the training required in reaching tasks involving elevated target locations, but not all target locations.

Hierarchical, Dense and Dynamic 3D Reconstruction Based on VDB Data Structure for Robotic Manipulation Tasks.

This paper presents a novel approach to implement hierarchical, dense and dynamic reconstruction of 3D objects based on the VDB (Variational Dynamic B + Trees) data structure for robotic applications. The scene reconstruction is done by the integration of depth-images using the Truncated Signed Distance Field (TSDF). The proposed reconstruction method is based on dynamic trees in order to provide similar reconstruction results to the current state-of-the-art methods (i.e., complete volumes, hashing voxels and hierarchical volumes) in terms of execution time but with a direct multi-level representation that remains real-time. This representation provides two major advantages: it is a hierarchical and unbounded space representation. The proposed method is optimally implemented to be used on a GPU architecture, exploiting the parallelism skills of this hardware. A series of experiments will be presented to prove the performance of this approach in a robot arm platform.

Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

Deep learning has provided new ways of manipulating, processing and analyzing data. It sometimes may achieve results comparable to, or surpassing human expert performance, and has become a source of inspiration in the era of artificial intelligence. Another subfield of machine learning named reinforcement learning, tries to find an optimal behavior strategy through interactions with the environment. Combining deep learning and reinforcement learning permits resolving critical issues relative to the dimensionality and scalability of data in tasks with sparse reward signals, such as robotic manipulation and control tasks, that neither method permits resolving when applied on its own. In this paper, we present recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Despite these continuous improvements, currently, the challenges of learning robust and versatile manipulation skills for robots with deep reinforcement learning are still far from being resolved for real-world applications.

Suction-based Grasp Point Estimation in Cluttered Environment for Robotic Manipulator Using Deep Learning-based Affordance Map

Perception and manipulation tasks for robotic manipulators involving highly-cluttered objects have become increasingly indemand for achieving a more efficient problem solving method in modern industrial environments. But, most of the available methods for performing such cluttered tasks failed in terms of performance, mainly due to inability to adapt to the change of the environment and the handled objects. Here, we propose a new, near real-time approach to suction-based grasp point estimation in a highly cluttered environment by employing an affordance-based approach. Compared to the state-of-the-art, our proposed method offers two distinctive contributions. First, we use a modified deep neural network backbone for the input of the semantic segmentation, to classify pixel elements of the input red, green, blue and depth (RGBD) channel image which is then used to produce an affordance map, a pixel-wise probability map representing the probability of a successful grasping action in those particular pixel regions. Later, we incorporate a high speed semantic segmentation to the system, which makes our solution have a lower computational time. This approach does not need to have any prior knowledge or models of the objects since it removes the step of pose estimation and object recognition entirely compared to most of the current approaches and uses an assumption to grasp first then recognize later, which makes it possible to have an object-agnostic property. The system was designed to be used for household objects, but it can be easily extended to any kind of objects provided that the right dataset is used for training the models. Experimental results show the benefit of our approach which achieves a precision of 88.83%, compared to the 83.4% precision of the current state-of-the-art.

Hindsight Goal Ranking on Replay Buffer for Sparse Reward Environment

This paper proposes a method for prioritizing the replay experience referred to as Hindsight Goal Ranking (HGR) in overcoming the limitation of Hindsight Experience Replay (HER) that generates hindsight goals based on uniform sampling. HGR samples with higher probability on the states visited in an episode with larger temporal difference (TD) error, which is considered as a proxy measure of the amount which the RL agent can learn from an experience. The actual sampling for large TD error is performed in two steps: first, an episode is sampled from the relay buffer according to the average TD error of its experiences, and then, for the sampled episode, the hindsight goal leading to larger TD error is sampled with higher probability from future visited states. The proposed method combined with Deep Deterministic Policy Gradient (DDPG), an off-policy model-free actor-critic algorithm, accelerates learning significantly faster than that without any prioritization on four challenging simulated robotic manipulation tasks. The empirical results show that HGR uses samples more efficiently than previous methods across all tasks.

Dynamics Learning With Object-Centric Interaction Networks for Robot Manipulation

Understanding the physical interactions of objects with environments is critical for multi-object robotic manipulation tasks. A predictive dynamics model can predict the future states of manipulated objects, which is used to plan plausible actions that enable the objects to achieve desired goal states. However, most current approaches on dynamics learning from high-dimensional visual observations have limitations. These methods either rely on a large amount of real-world data or build a model with a fixed number of objects, which makes them difficult to generalize to unseen objects. This paper proposes a Deep Object-centric Interaction Network (DOIN) which encodes object-centric representations for multiple objects from raw RGB images and reasons about the future trajectory for each object in latent space. The proposed model is trained only on large amounts of random interaction data collected in simulation. The learned model combined with a model predictive control framework enables a robot to search action sequences that manipulate objects to the desired configurations. The proposed method is evaluated both in simulation and real-world experiments on multi-object pushing tasks. Extensive simulation experiments show that DOIN can achieve high prediction accuracy in different scenes with different numbers of objects and outperform state-of-the-art baselines in the manipulation tasks. Real-world experiments demonstrate that the model trained on simulated data can be transferred to the real robot and can successfully perform multi-object pushing tasks for previously-unseen objects with significant variations in shape and size.

Combining Self-Organizing and Graph Neural Networks for Modeling Deformable Objects in Robotic Manipulation.

Modeling deformable objects is an important preliminary step for performing robotic manipulation tasks with more autonomy and dexterity. Currently, generalization capabilities in unstructured environments using analytical approaches are limited, mainly due to the lack of adaptation to changes in the object shape and properties. Therefore, this paper proposes the design and implementation of a data-driven approach, which combines machine learning techniques on graphs to estimate and predict the state and transition dynamics of deformable objects with initially undefined shape and material characteristics. The learned object model is trained using RGB-D sensor data and evaluated in terms of its ability to estimate the current state of the object shape, in addition to predicting future states with the goal to plan and support the manipulation actions of a robotic hand.

State Primitive Learning to Overcome Catastrophic Forgetting in Robotics

People can learn continuously a wide range of tasks without catastrophic forgetting. To mimic this functioning of continual learning, current methods mainly focus on studying a one-step supervised learning problem, e.g., image classification. They aim to retain the performance of previous image classification results when neural networks are sequentially trained on new images. In this paper, we concentrate on solving multi-step robotic tasks sequentially with the proposed architecture called state primitive learning. By projecting the original state space into a low-dimensional representation, meaningful state primitives can be generated to describe tasks. Under two kinds of different constraints on the generation of state primitives, control signals corresponding to different robotic tasks can be separately addressed only with an efficient linear regression. Experiments on several robotic manipulation tasks demonstrate the new method efficacy to learn control signals under the scenario of continual learning, delivering substantially improved performance over the other comparison methods.

Natural Human–Machine Interface With Gesture Tracking and Cartesian Platform for Contactless Electromagnetic Force Feedback

In this article, a novel human–machine interface, in which two Leap Motion (LM) controllers and a coil are attached to a Cartesian platform to provide contactless electromagnetic force feedback for enhancing the accuracy and efficiency of human–robot manipulation tasks is presented. To implement such an interface, an interval Kalman filter, an improved particle filter, and a mean filter are integrated to estimate accurately the position and orientation of the hand gesture tracked by the two LM controllers, and to smoothen the movement of the Cartesian platform. The back propagation neural network is employed to regulate the electric currents of the coil attached to the Cartesian platform for accurate force feedback. A series of comparative experiments are performed, and the results show that the presented interface greatly improved the efficiency and accuracy of human–robot manipulation tasks in comparison with existing methods, indicating its great potentials for many industry scenarios.

SkillMaN — A skill-based robotic manipulation framework based on perception and reasoning

One of the problems that service robotics deals with is to bring mobile manipulators to work in semi-structured human scenarios, which requires an efficient and flexible way to execute every-day tasks, like serve a cup in a cluttered environment. Usually, for those tasks, the combination of symbolic and geometric levels of planning is necessary, as well as the integration of perception models with knowledge to guide both planning levels, resulting in a sequence of actions or skills which, according to the current knowledge of the world, may be executed. This paper proposes a planning and execution framework, called SkillMaN, for robotic manipulation tasks, which is equipped with a module with experiential knowledge (learned from its experience or given by the user) on how to execute a set of skills, like pick-up, put-down or open a drawer, using workflows as well as robot trajectories. The framework also contains an execution assistant with geometric tools and reasoning capabilities to manage how to actually execute the sequence of motions to perform a manipulation task (which are forwarded to the executor module), as well as the capacity to store the relevant information to the experiential knowledge for further usage, and the capacity to interpret the actual perceived situation (in case the preconditions of an action do not hold) and to feed back the updated state to the planner to resume from there, allowing the robot to adapt to non-expected situations. To evaluate the viability of the proposed framework, an experiment has been proposed involving different skills performed with various types of objects in different scene contexts.

Learning Structured Representations of Spatial and Interactive Dynamics for Trajectory Prediction in Crowded Scenes

Context plays a significant role in the generation of motion for dynamic agents in interactive environments. This work proposes a modular method that utilises a learned model of the environment for motion prediction. This modularity explicitly allows for unsupervised adaptation of trajectory prediction models to unseen environments and new tasks by relying on unlabelled image data only. We model both the spatial and dynamic aspects of a given environment alongside the per agent motions. This results in more informed motion prediction and allows for performance comparable to the state-of-the-art. We highlight the model's prediction capability using a benchmark pedestrian prediction problem and a robot manipulation task and show that we can transfer the predictor across these tasks in a completely unsupervised way. The proposed approach allows for robust and label efficient forward modelling, and relaxes the need for full model re-training in new environments.

Using Human Gaze to Improve Robustness Against Irrelevant Objects in Robot Manipulation Tasks

Deep imitation learning enables the learning of complex visuomotor skills from raw pixel inputs. However, this approach suffers from the problem of overfitting to the training images. The neural network can easily be distracted by task-irrelevant objects. In this letter, we use the human gaze measured by a head-mounted eye tracking device to discard task-irrelevant visual distractions. We propose a mixture density network-based behavior cloning method that learns to imitate the human gaze. The model predicts gaze positions from raw pixel images and crops images around the predicted gazes. Only these cropped images are used to compute the output action. This cropping procedure can remove visual distractions because the gaze is rarely fixated on task-irrelevant objects. This robustness against irrelevant objects can improve the manipulation performance of robots in scenarios where task-irrelevant objects are present. We evaluated our model on four manipulation tasks designed to test the robustness of the model to irrelevant objects. The results indicate that the proposed model can predict the locations of task-relevant objects from gaze positions, is robust to task-irrelevant objects, and exhibits impressive manipulation performance especially in multi-object handling.

AugGAIL : 로봇 조작 작업을 위한 증강된 생성적 적대 모방 학습

AugGAIL : 로봇 조작 작업을 위한 증강된 생성적 적대 모방 학습

Perception of cloth in assistive robotic manipulation tasks

Assistive robots need to be able to perform a large number of tasks that imply some type of cloth manipulation. These tasks include domestic chores such as laundry handling or bed-making, among others, as well as dressing assistance to disabled users. Due to the deformable nature of fabrics, this manipulation requires a strong perceptual feedback. Common perceptual skills that enable robots to complete their cloth manipulation tasks are reviewed here, mainly relying on vision, but also resorting to touch and force. The use of such basic skills is then examined in the context of the different cloth manipulation tasks, be them garment-only applications in the line of performing domestic chores, or involving physical contact with a human as in dressing assistance.

Realtime Simulation of Thin-Shell Deformable Materials Using CNN-Based Mesh Embedding

We address the problem of accelerating thin-shell deformable object simulations by dimension reduction. We present a new algorithm to embed a high-dimensional configuration space of deformable objects in a low-dimensional feature space, where the configurations of objects and feature points have approximate one-to-one mapping. Our key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies and a new mesh embedding approach based on physics-inspired loss term. We have applied our approach to accelerate high-resolution thin shell simulations corresponding to cloth-like materials, where the configuration space has tens of thousands of degrees of freedom. We show that our physics-inspired embedding approach leads to higher accuracy compared with prior mesh embedding methods. Finally, we show that the temporal evolution of the mesh in the feature space can also be learned using a recurrent neural network (RNN) leading to fully learnable physics simulators. After training our learned simulator runs $500-10000\times$ faster and the accuracy is high enough for robot manipulation tasks.

A Reinforcement Learning-Based Framework for Robot Manipulation Skill Acquisition

This paper studies robot manipulation skill acquisition based on a proposed reinforcement learning framework. Robot can learn policy autonomously by interacting with environment with a better learning efficiency. Aiming at the manipulator operation task, a reward function design method based on objects configuration matching (OCM) is proposed. It is simple and suitable for most Pick and Place skills learning. Integrating robot and object state, high-level action set and the designed reward function, the Markov model of robot manipulator is built. An improved Proximal Policy Optimize algorithm with manipulation set as the output of Actor (MAPPO) is proposed as the main structure to construct the robot reinforcement learning framework. The framework combines with the Markov model to learn and optimize the skill policy. A same simulation environment as the real robot is set up, and three robot manipulation tasks are designed to verify the effectiveness and feasibility of the reinforcement learning framework for skill acquisition.

4-DOF Robot Arm for Pick and Place Process

The more increase in the number of industries in developing countries, the more laborers or workers are required. To reduce the cost of the labor force and to increase the manufacturing capacity of industries, advanced robot arms are more needed. This paper aims to eliminate the control of the robot arm for picking and placing the object. The arm is constructed in this research with four joints, four links, and four Dc motors to drive the robot arm. Arduino Microcontroller is used to generate the required control signals for DC motors. This paper involves the PID-based kinematic control system which is used for doing successful robotic manipulation tasks in its workspace. Standard DH parameter convention is used to calculate forward kinematics. An analytical method for a closed-formed solution of inverse kinematics is applied to solve the unknown joint angles required for the autonomous position of a robot arm. Four DC motors of four joints are controlled using PID techniques for position control. And for the interface control system via PC, a MATLAB GUI is used to interactively communicate with Arduino controller. The control of this pick and place robot arm system is fully tested in real-time, and the results show that the system has a precise control system for obtaining a good system performance.

A Survey of Methods and Strategies for High-Precision Robotic Grasping and Assembly Tasks—Some New Trends

Grasping and assembly are essential tasks in high-precision robotic manipulation for industrial manufacturing as well as for home service applications. Many efforts have been devoted to this area in an attempt to meet the increasing precision requirement of the task. However, it remains a problematic objective to fulfill high precision, high reliability, high speed, and high flexibility all at once during one robotic manipulation task. To find answers to the above-mentioned problem, this article tries to categorize, review, and compare the recent works focusing on robotic grasping and assembly tasks to reveal some potential trends in this research area. The approaches will be divided into five groups based on the difference in the utilization of sensing or constraints. For each part, robotic grasping and assembly will be treated as practical cases to illustrate the concrete work in that area. This article could give the readers some knowledge on the current developments in robotic manipulation, and provide new thoughts on future direction in this area—inspiring new designs, structures, and systems to meet new requirements in applications in industrial manufacturing and home service.

Comparison of end-to-end and hybrid deep reinforcement learning strategies for controlling cable-driven parallel robots

Deep reinforcement learning (DRL) has been proven effective in learning policies of high-dimensional states and actions. Recently, a variety of robot manipulation tasks have been accomplished using end-to-end DRL strategies. An end-to-end DRL strategy accomplishes a robot manipulation task as a black box. On the other hand, a robot manipulation task can be divided into multiple subtasks and accomplished by non-learning-based approaches. A hybrid DRL strategy integrates DRL with non-learning-based approaches. The hybrid DRL strategy accomplishes some subtasks of a robot manipulation task by DRL and the rest subtasks by non-learning-based approaches. However, the effects of integrating DRL with non-learning-based approaches on the learning speed and the robustness of DRL to model uncertainties have not been discussed. In this study, an end-to-end DRL strategy and a hybrid DRL strategy are developed and compared in controlling a cable-driven parallel robot. This study shows that, by integrating DRL with non-learning-based approaches, the hybrid DRL strategy learns faster and is more robust to model uncertainties than the end-to-end DRL strategy. This study demonstrates that, by taking advantages of both learning and non-learning-based approaches, the hybrid DRL strategy provides an alternative to accomplish a robot manipulation task.