Grasping Point Research Articles

Center Direction Network for Grasping Point Localization on Cloths

Center Direction Network for Grasping Point Localization on Cloths

Sim-to-Real Surgical Robot Learning and Autonomous Planning for Internal Tissue Points Manipulation Using Reinforcement Learning

Indirect simultaneous positioning (ISP), where internal tissue points are placed at desired locations indirectly through the manipulation of boundary points, is a type of subtask frequently performed in robotic surgeries. Although challenging due to complex tissue dynamics, automating the task can potentially reduce the workload of surgeons. This paper presents a sim-to-real framework for learning to automate the task without interacting with a real environment, and for planning preoperatively to find the grasping points that minimize local tissue deformation. A control policy is learned using deep reinforcement learning (DRL) in the FEM-based simulation environment and transferred to real-world situation. Grasping points are planned in the simulator by utilizing the trained policy using Bayesian optimization (BO). Inconsistent simulation performance is overcome by formulating the problem as a state augmented Markov decision process (MDP). Experimental results show that the learned policy places the internal tissue points accurately, and that the planned grasping points yield small tissue deformation among the trials. The proposed learning and planning scheme is able to automate internal tissue point manipulation in surgeries and has the potential to be generalized to complex surgical scenarios.

Grasping point optimization for sheet metal part based on GSA-Kriging model in a multi-robot assembly system

Grasping point optimization for sheet metal part based on GSA-Kriging model in a multi-robot assembly system

Multiple Workpiece Grasping Point Localization Method Based on Deep Learning

Multiple Workpiece Grasping Point Localization Method Based on Deep Learning

Grasp Point Optimization and Leakage-Compliant Dimensioning of Energy-Efficient Vacuum-Based Gripping Systems

Vacuum-based handling, used in many applications and industries, offers great flexibility and fast handling processes. However, due to significant energy conversion losses from electrical energy to the useable suction flow, vacuum-based handling is highly energy-inefficient. In preliminary work, we showed that our grasp optimization method offers the potential to save at least 50% of energy by a targeted placement of individual suction cups on the part to be handled. By considering the leakage between gripper and object, this paper aims to extend the grasp optimization method by predicting the effective compressed air consumption depending on object surface roughness, gripper diameter and gripper count. Through balancing of the target pressure difference and the leakage tolerance in combination with the gripper count and gripper diameter, significant reductions of the compressed air, use and therefore the overall energy consumption, can be achieved. With knowledge about the gripper-specific leakage behavior, in the future it will be straightforward for system integrators to minimize the need for oversizing due to process-related uncertainties and therefore to provide application-specific and energy-optimized handling solutions to their customers.

Affordance-Based Grasping Point Detection Using Graph Convolutional Networks for Industrial Bin-Picking Applications.

Grasping point detection has traditionally been a core robotic and computer vision problem. In recent years, deep learning based methods have been widely used to predict grasping points, and have shown strong generalization capabilities under uncertainty. Particularly, approaches that aim at predicting object affordances without relying on the object identity, have obtained promising results in random bin-picking applications. However, most of them rely on RGB/RGB-D images, and it is not clear up to what extent 3D spatial information is used. Graph Convolutional Networks (GCNs) have been successfully used for object classification and scene segmentation in point clouds, and also to predict grasping points in simple laboratory experimentation. In the present proposal, we adapted the Deep Graph Convolutional Network model with the intuition that learning from n-dimensional point clouds would lead to a performance boost to predict object affordances. To the best of our knowledge, this is the first time that GCNs are applied to predict affordances for suction and gripper end effectors in an industrial bin-picking environment. Additionally, we designed a bin-picking oriented data preprocessing pipeline which contributes to ease the learning process and to create a flexible solution for any bin-picking application. To train our models, we created a highly accurate RGB-D/3D dataset which is openly available on demand. Finally, we benchmarked our method against a 2D Fully Convolutional Network based method, improving the top-1 precision score by 1.8% and 1.7% for suction and gripper respectively.

Suction-based Grasp Point Estimation in Cluttered Environment for Robotic Manipulator Using Deep Learning-based Affordance Map

Perception and manipulation tasks for robotic manipulators involving highly-cluttered objects have become increasingly indemand for achieving a more efficient problem solving method in modern industrial environments. But, most of the available methods for performing such cluttered tasks failed in terms of performance, mainly due to inability to adapt to the change of the environment and the handled objects. Here, we propose a new, near real-time approach to suction-based grasp point estimation in a highly cluttered environment by employing an affordance-based approach. Compared to the state-of-the-art, our proposed method offers two distinctive contributions. First, we use a modified deep neural network backbone for the input of the semantic segmentation, to classify pixel elements of the input red, green, blue and depth (RGBD) channel image which is then used to produce an affordance map, a pixel-wise probability map representing the probability of a successful grasping action in those particular pixel regions. Later, we incorporate a high speed semantic segmentation to the system, which makes our solution have a lower computational time. This approach does not need to have any prior knowledge or models of the objects since it removes the step of pose estimation and object recognition entirely compared to most of the current approaches and uses an assumption to grasp first then recognize later, which makes it possible to have an object-agnostic property. The system was designed to be used for household objects, but it can be easily extended to any kind of objects provided that the right dataset is used for training the models. Experimental results show the benefit of our approach which achieves a precision of 88.83%, compared to the 83.4% precision of the current state-of-the-art.

RGB-D 이미지를 활용한 CNN 기반 실시간 파지점 탐색

본 논문에서는 RGB-D 이미지를 활용한 CNN 기반의 실시간 파지점 탐색 알고리즘을 제안한다. 파지점 탐색이란 로봇의 손으로 물체의 어느 지점을 잡았을 때 안정적으로 물체를 들어 올릴 수 있는지 결정하는 기술로 단순한 이미지 내의 물체 탐지 기법과는 차이가 있다. 파지의 위치가 불안정할 경우 로봇의 손으로 물체를 들어 올리는 과정 중 물체를 떨어뜨리거나 동작을 수행할 수 없는 경우가 발생하기에 정확하고 안정적인 파지점 탐색이 필요하다. 인식된 물체의 사전 정보 없이 주어진 RGB-D 영상으로부터 파지점을 탐색하며, 파지점 탐색 알고리즘의 정량적인 평가를 위해 빠른 Rectangle Metric 측정 알고리즘을 제안한다. 본 논문에서 제안하는 알고리즘은 통해 이미지 처리가 아닌 학습을 통한 인경신경망 모델을 사용함으로서 단순한 형태의 물체뿐만이 아닌 복잡한 형태의 물체의 파지점 탐색까지도 가능하다.

Grasping Point Detection of Randomly Placed Fruit Cluster Using Adaptive Morphology Segmentation and Principal Component Classification of Multiple Features

Precise and rapid grasping point detection based on machine vision is one of the challenging problems in automatic sorting of randomly placed fruit clusters by robot. Grasping stalk of fruit cluster can improve grasping success probability and reduce fruit damage. For the problem that the segmentation of stalk candidates for randomly placed fruit cluster based on existing morphology algorithm tends to be low precision, an improved image segmentation algorithm based on adaptive morphology is proposed. According to edge distances defined based on minimum distance between edge point and unconnected components in minimum domain, the adaptive convolution kernel is constructed. In addition, a run analysis method with different and unordered labels is designed to reduce calculation time of edge distances. For the problem that it is difficult to describe and classify unconstraint stalk by existing features, an improved region classification algorithm based on principal components of multiple features is proposed. The descriptors based on features of object region are designed and principal components of multiple features are extracted based on variance contribution to improve precision and speed of stalk extraction. The proposed grasping point detection method of randomly placed fruit cluster based on improved morphology image segmentation and region classification algorithms is verified by experiments with grape clusters based on parallel robot sorting system. The results show that, compared with existing methods, the average precisions of segmentation and extraction for stalk increase by 9.89% and 2.17% respectively, the average precision and time of grasping point detection reach 94.50% and 2.01s respectively.

Safe path planning for free-floating space robot to approach noncooperative spacecraft

In order to eliminate plume impingement on the target satellite, a free-floating space robot (also known as a chaser), which has the advantage of executing on-orbit service, is always used. This paper develops a path planning method for a safe rendezvous of chaser with a noncooperative target satellite in orbital coordinates. Safety principles for rendezvous in terminal approaching phase are proposed. Grasp points on the target satellite are analyzed and classified into two categories, and a moving ellipse trajectory is adopted to approach a rotating and uncontrolled target satellite. This method guarantees that the chaser can successfully escape if unexpected error occurs or capture fails. The simulation results show that, with this novel autonomous rendezvous method, the chaser can approach the noncooperative target satellite along the designated trajectory in any quadrant of the orbital plane.

An Integrated Approach of Sensors to Detect Grasping Point for Unstructured 3-D Parts

An Integrated Approach of Sensors to Detect Grasping Point for Unstructured 3-D Parts

Grasp points in different tasks are switched by each task demand: An investigation by a task-change paradigm

Grasp points in different tasks are switched by each task demand: An investigation by a task-change paradigm

Grasp Point of an Object is Switched between Central and Peripheral Visual Field Conditions

The purpose of this study is to investigate whether grasp point of each target object is switched between central and peripheral visual field conditions. We measured grasp points of a lift-up task (LT) and a pinch task (PT), that did not lift from a table, under the three visual field conditions. As a result, in the normal condition, the grasp points of LT and PT were different and the grasp points of LT were almost the centers of gravity of objects. While, the grasp points of LT were switched to those of PT under the central visual field condition and the grasp points of PT were switched to those of LT under the peripheral visual field condition. These results indicate that in the human brain there are at least two types of grasp point computation and the central and peripheral vision systems respectively contribute to these computations of PT and LT.

Do Humans Prefer to See Their Grasping Points?

ABSTRACT To grasp an object the digits need to be placed at suitable positions on its surface. The selection of such grasping points depends on several factors. Here the authors examined whether being able to see 1 of the selected grasping points is such a factor. Subjects grasped large cylinders or oriented blocks that would normally be grasped with the thumb continuously visible and the final part of the index finger's trajectory occluded by the object in question. An opaque screen that hid the thumb's usual grasping point was used to examine whether individuals would choose a grip that was oriented differently to maintain vision of the thumb's grasping point. A transparent screen was used as a control. Occluding the thumb's grasping point made subjects move more carefully (adopting a larger grip aperture) and choose a slightly different grip orientation. However, the change in grip orientation was much too small to keep the thumb visible. The authors conclude that humans do not particularly aim for visible grasping points.

Haptic and Visual Influences on Grasp Point Selection

ABSTRACT When using precision grip to pick up objects, there are many possible pairs of grasp points that permit the thumb and index finger to exert opposed forces for secure grip. Previously, it was shown that individuals select grasp points so that the line between them (grasp axis) passes through or near the center of mass (CoM), thus minimizing the torque around the grasp axis during lifting. The accuracy of grasp axis selection depended on object spatial symmetry, indicating the importance of vision. The authors investigated how grasp point selection is influenced by haptic as well as visual information. Ten participants lifted cuboids whose CoM was located either symmetrically in the geometric center or asymmetrically toward one end. Results for the asymmetric cuboid revealed that grasp points migrated toward the asymmetric CoM from the geometric center. This was more pronounced in the presence of visual cues that reliably indicated the location of CoM. The results suggest that grasp point selection is influenced by a multimodal representation of CoM.

Classification of Phalaenopsis Plantlet Parts and Identification of Suitable Grasping Point for Automatic Transplanting Using Machine Vision

Classification of Phalaenopsis Plantlet Parts and Identification of Suitable Grasping Point for Automatic Transplanting Using Machine Vision