Grasp Success Rate Research Articles

A Reconfigurable Design for Omni-adaptive Grasp Learning

The engineering design of robotic grippers presents an ample design space for optimization towards robust grasping. In this paper, we adopt the reconfigurable design of the robotic gripper using a novel soft finger structure with omni-directional adaptation, which generates a large number of possible gripper configurations by rearranging these fingers. Such reconfigurable design with these omni-adaptive fingers enables us to systematically investigate the optimal arrangement of the fingers towards robust grasping. Furthermore, we adopt a learning-based method as the baseline to benchmark the effectiveness of each design configuration. As a result, we found that a 3-finger and 4-finger radial configuration is the most effective one achieving an average 96\% grasp success rate on seen and novel objects selected from the YCB dataset. We also discussed the influence of the frictional surface on the finger to improve the grasp robustness.

Research on manipulator grasping method based on vision

Aiming at the problem that the manipulator cannot grasp the object accurately when the robot changes the position and pose of the object in the process of static grasping, a set of manipulator grasping method research based on vision is proposed. Firstly, a set of camera detection and robot grasping system model is built. Secondly, each coordinate system is created for the grasping system, and the transformation relation between each coordinate system, matrix model and the quantity that needs to be calibrated are introduced in detail. Thirdly, the trajectory function from the image coordinate system to the coordinates of the manipulator is obtained through the direct linear method calibration experiment. Finally, the experimental platform for camera detection and manipulator grasping objects based on xavis was built, and the grasping success rate of the experimental platform was tested. The experimental results show that the grasping error rate of the experimental platform is within the control range. Therefore, the manipulator grasping method based on vision is of reference significance for engineering applications.

Myoelectric Control of a Soft Hand Exoskeleton Using Kinematic Synergies.

Soft hand exoskeletons offer a lightweight, low-profile alternative to rigid rehabilitative robotic systems, enabling their use to restore activities of daily living (ADL) in those with hand paresis due to stroke or other conditions. The hand exoskeleton with embedded synergies (HEXOES) is a soft cable-driven hand exoskeleton capable of independently actuating and sensing 10 degrees of freedom (DoF) of the hand. Control of the 10 DoF exoskeleton is dimensionally reduced using three manually defined synergies in software corresponding to thumb, index, and 3-finger flexion and extension. In this paper, five healthy subjects control HEXOES using a neural network which decodes synergy weights from contralateral electromyography (EMG) activity. The three synergies are manipulated in real time to grasp and lift 15 ADL objects of various sizes and weights. The neural network's training and validation mean squared error, object grasp time, and grasp success rate were measured for five healthy subjects. The final training error of the neural network was 4.8 ± 1.8% averaged across subjects and tasks, with 8.3 ± 3.4% validation error. The time to reach, grasp, and lift an object was 11.15 ± 4.35s on average, with an average success rate of 66.7% across all objects. The complete system demonstrates real time use of biosignals and machine learning to allow subjects to operate kinematic synergies to grasp objects using a wearable hand exoskeleton. Future work and applications are further discussed, including possible design improvements and enrollment of individuals with stroke.

Autonomous Vision-Based Aerial Grasping for Rotorcraft Unmanned Aerial Vehicles.

Autonomous vision-based aerial grasping is an essential and challenging task for aerial manipulation missions. In this paper, we propose a vision-based aerial grasping system for a Rotorcraft Unmanned Aerial Vehicle (UAV) to grasp a target object. The UAV system is equipped with a monocular camera, a 3-DOF robotic arm with a gripper and a Jetson TK1 computer. Efficient and reliable visual detectors and control laws are crucial for autonomous aerial grasping using limited onboard sensing and computational capabilities. To detect and track the target object in real time, an efficient proposal algorithm is presented to reliably estimate the region of interest (ROI), then a correlation filter-based classifier is developed to track the detected object. Moreover, a support vector regression (SVR)-based grasping position detector is proposed to improve the grasp success rate with high computational efficiency. Using the estimated grasping position and the UAV?Äôs states, novel control laws of the UAV and the robotic arm are proposed to perform aerial grasping. Extensive simulations and outdoor flight experiments have been implemented. The experimental results illustrate that the proposed vision-based aerial grasping system can autonomously and reliably grasp the target object while working entirely onboard.

Densely Supervised Grasp Detector (DSGD)

This paper presents Densely Supervised Grasp Detector (DSGD), a deep learning framework which combines CNN structures with layer-wise feature fusion and produces grasps and their confidence scores at different levels of the image hierarchy (i.e., global-, region-, and pixel-levels). Specifically, at the global-level, DSGD uses the entire image information to predict a grasp. At the region-level, DSGD uses a region proposal network to identify salient regions in the image and uses a grasp prediction network to generate segmentations and their corresponding grasp poses of the salient regions. At the pixel-level, DSGD uses a fully convolutional network and predicts a grasp and its confidence at every pixel. During inference, DSGD selects the most confident grasp as the output. This selection from hierarchically generated grasp candidates overcomes limitations of the individual models. DSGD outperforms state-of-the-art methods on the Cornell grasp dataset in terms of grasp accuracy. Evaluation on a multi-object dataset and real-world robotic grasping experiments show that DSGD produces highly stable grasps on a set of unseen objects in new environments. It achieves 97% grasp detection accuracy and 90% robotic grasping success rate with real-time inference speed.

Learning robust, real-time, reactive robotic grasping

We present a novel approach to perform object-independent grasp synthesis from depth images via deep neural networks. Our generative grasping convolutional neural network (GG-CNN) predicts a pixel-wise grasp quality that can be deployed in closed-loop grasping scenarios. GG-CNN overcomes shortcomings in existing techniques, namely discrete sampling of grasp candidates and long computation times. The network is orders of magnitude smaller than other state-of-the-art approaches while achieving better performance, particularly in clutter. We run a suite of real-world tests, during which we achieve an 84% grasp success rate on a set of previously unseen objects with adversarial geometry and 94% on household items. The lightweight nature enables closed-loop control of up to 50 Hz, with which we observed 88% grasp success on a set of household objects that are moved during the grasp attempt. We further propose a method combining our GG-CNN with a multi-view approach, which improves overall grasp success rate in clutter by 10%. Code is provided at https://github.com/dougsm/ggcnn

In vivo human-like robotic phenotyping of leaf traits in maize and sorghum in greenhouse

In plant phenotyping, leaf-level physiological and chemical trait measurements are needed to investigate and monitor the condition of plants. The manual measurement of these properties is time consuming, error prone, and laborious. The use of robots is a new approach to accomplish such endeavors, enabling automated monitoring with minimal human intervention. In this paper, a plant phenotyping robotic system was developed to realize automated measurement of plant leaf properties. The robotic system comprised of a four Degree of Freedom (DOF) robotic manipulator and a Time-of-Flight (TOF) camera. A robotic gripper was developed to integrate an optical fiber cable (coupled to a portable spectrometer) for leaf spectral reflectance measurement, and a thermistor for leaf temperature measurement. A MATLAB program along with a Graphical User Interface (GUI) was developed to control the robotic system and its components, and for acquiring and recording data obtained from the sensors. The system was tested in a greenhouse using maize and sorghum plants. The results showed that leaf temperature measurements by the phenotyping robot were significantly correlated with those measured manually by a human researcher (R2 = 0.58 for maize and 0.63 for sorghum). The leaf spectral measurements by the phenotyping robot predicted leaf chlorophyll, water content and potassium with moderate success (R2 ranged from 0.52 to 0.61), whereas the prediction for leaf nitrogen and phosphorus were poor. The total execution time to grasp and take measurements from one leaf was 35.5 ± 4.4 s for maize and 38.5 ± 5.7 s for sorghum. Furthermore, the test showed that the grasping success rate was 78% for maize and 48% for sorghum. The phenotyping robot can be useful to complement the traditional image-based high-throughput plant phenotyping in greenhouses by collecting in vivo leaf-level physiological and biochemical trait measurements.

Fast geometry-based computation of grasping points on three-dimensional point clouds

Industrial and service robots deal with the complex task of grasping objects that have different shapes and which are seen from diverse points of view. In order to autonomously perform grasps, the robot must calculate where to place its robotic hand to ensure that the grasp is stable. We propose a method to find the best pair of grasping points given a three-dimensional point cloud with the partial view of an unknown object. We use a set of straightforward geometric rules to explore the cloud and propose grasping points on the surface of the object. We then adapt the pair of contacts to a multi-fingered hand used in experimentation. We prove that, after performing 500 grasps of different objects, our approach is fast, taking an average of 17.5 ms to propose contacts, while attaining a grasp success rate of 85.5%. Moreover, the method is sufficiently flexible and stable to work with objects in changing environments, such as those confronted by industrial or service robots.

Real-World Multiobject, Multigrasp Detection

A deep learning architecture is proposed to predict graspable locations for robotic manipulation. It considers situations where no, one, or multiple object(s) are seen. By defining the learning problem to be classified with null hypothesis competition instead of regression, the deep neural network with red, green, blue and depth (RGB-D) image input predicts multiple grasp candidates for a single object or multiple objects, in a single shot. The method outperforms state-of-the-art approaches on the Cornell dataset with 96.0% and 96.1% accuracy on imagewise and object-wise splits, respectively. Evaluation on a multiobject dataset illustrates the generalization capability of the architecture. Grasping experiments achieve 96.0% grasp localization and 89.0% grasping success rates on a test set of household objects. The real-time process takes less than 0.25 s from image to plan.

A novel hybrid closed-loop control approach for dexterous prosthetic hand based on myoelectric control and electrical stimulation

PurposeThe purpose of this study is to present a novel hybrid closed-loop control method together with its performance validation for the dexterous prosthetic hand.Design/methodology/approachThe hybrid closed-loop control is composed of a high-level closed-loop control with the user in the closed loop and a low-level closed-loop control for the direct robot motion control. The authors construct the high-level control loop by using electromyography (EMG)-based human motion intent decoding and electrical stimulation (ES)-based sensory feedback. The human motion intent is decoded by a finite state machine, which can achieve both the patterned motion control and the proportional force control. The sensory feedback is in the form of transcutaneous electrical nerve stimulation (TENS) with spatial-frequency modulation. To suppress the TENS interfering noise, the authors propose biphasic TENS to concentrate the stimulation current and the variable step-size least mean square adaptive filter to cancel the noise. Eight subjects participated in the validation experiments, including pattern selection and egg grasping tasks, to investigate the feasibility of the hybrid closed-loop control in clinical use.FindingsThe proposed noise cancellation method largely reduces the ES noise artifacts in the EMG electrodes by 18.5 dB on average. Compared with the open-loop control, the proposed hybrid closed-loop control method significantly improves both the pattern selection efficiency and the egg grasping success rate, both in blind operating scenarios (improved by 1.86 s, p < 0.001, and 63.7 per cent, p < 0.001) or in common operating scenarios (improved by 0.49 s, p = 0.008, and 41.3 per cent, p < 0.001).Practical implicationsThe proposed hybrid closed-loop control method can be implemented on a prosthetic hand to improve the operation efficiency and accuracy for fragile objects such as eggs.Originality/valueThe primary contribution is the proposal of the hybrid closed-loop control, the spatial-frequency modulation method for the sensory feedback and the noise cancellation method for the integrating of the myoelectric control and the ES-based sensory feedback.

A model-based scooping grasp for the autonomous picking of unknown objects with a two-fingered gripper

Grasping objects used in daily activities is not an easy task for a robot: the diversity of shapes and volumes of objects renders specific grasping methods inefficient. In this paper, we propose a novel model-based scooping grasp for the picking of thin objects lying on a flat surface, which are typically elusive to common grippers and grasping strategies. A robotic work cell composed of a serial arm, a commercially available gripper and a 3D camera overlooking the workspace is used to demonstrate and test the algorithm. Since a commercial gripper is used, the robot is capable of grasping a large variety of objects, in addition to the targeted thin objects. An experiment based on a test set of 80 objects results in an overall grasp success rate of 84%, which demonstrates the potential of the novel scooping grasp to extend the capabilities of existing grippers.

Grasp Pose Detection in Point Clouds

Recently, a number of grasp detection methods have been proposed that can be used to localize robotic grasp configurations directly from sensor data without estimating object pose. The underlying idea is to treat grasp perception analogously to object detection in computer vision. These methods take as input a noisy and partially occluded RGBD image or point cloud and produce as output pose estimates of viable grasps, without assuming a known CAD model of the object. Although these methods generalize grasp knowledge to new objects well, they have not yet been demonstrated to be reliable enough for wide use. Many grasp detection methods achieve grasp success rates (grasp successes as a fraction of the total number of grasp attempts) between 75% and 95% for novel objects presented in isolation or in light clutter. Not only are these success rates too low for practical grasping applications, but the light clutter scenarios that are evaluated often do not reflect the realities of real-world grasping. This paper proposes a number of innovations that together result in an improvement in grasp detection performance. The specific improvement in performance due to each of our contributions is quantitatively measured either in simulation or on robotic hardware. Ultimately, we report a series of robotic experiments that average a 93% end-to-end grasp success rate for novel objects presented in dense clutter.

Supporting Reaching Movements of Robotic Hands Subject to Communication Delay by Displaying End Effector Position Using Three Orthogonal Rays

In this paper, we propose an operation support system, which shows the spatial relation between a robotic hand and an object, to overcome reduced usability due to communication delay. A communication delay of four seconds occurs during remote operation between the earth and the moon. Previous studies have used a method that feeds back the predicted position of a robotic hand after the communication delay occurs. However, this does not allow an accurate reaching motion necessary for grasping. Therefore, we try to show the spatial relation between a robotic hand and an object using three orthogonal rays, where each is annotated with 1 cm increments. Based on the results of our experiments, the grasping success rate of the proposed method increases by three times. As a result, the proposed method is effective in making accurate reaching motions for grasping in the visual-servoing remote control between the earth and the moon.

Robotic grasping using visual and tactile sensing

Visual and tactile sensing are complementary factors in the task of robotic grasping. In this paper, a grasp detection deep network is first proposed to detect the grasp rectangle from the visual image， then a new metric using tactile sensing is designed to assess the stability of the grasp. By means of this scheme, a THU grasp dataset， which includes the visual information, corresponding tactile and grasp configurations， is collected to train the proposed deep network. Experiments results have demonstrated that the proposed grasp detection deep networks outperform other mainstream approaches in a public grasp dataset. Furthermore， the grasp success rate can be improved significantly in real world scenarios. The trained model has also been successfully implemented in a new robotic platform to perform the robotic grasping task in a cluttered scenario.

Autonomous Sweet Pepper Harvesting for Protected Cropping Systems

In this letter, we present a new robotic harvester (Harvey) that can autonomously harvest sweet pepper in protected cropping environments. Our approach combines effective vision algorithms with a novel end-effector design to enable successful harvesting of sweet peppers. Initial field trials in protected cropping environments, with two cultivar, demonstrate the efficacy of this approach achieving a 46% success rate for unmodified crop, and 58% for modified crop. Furthermore, for the more favourable cultivar we were also able to detach 90% of sweet peppers, indicating that improvements in the grasping success rate would result in greatly improved harvesting performance.

Electrical neuromodulation of the cervical spinal cord facilitates forelimb skilled function recovery in spinal cord injured rats

Enabling motor control by epidural electrical stimulation of the spinal cord is a promising therapeutic technique for the recovery of motor function after a spinal cord injury (SCI). Although epidural electrical stimulation has resulted in improvement in hindlimb motor function, it is unknown whether it has any therapeutic benefit for improving forelimb fine motor function after a cervical SCI. We tested whether trains of pulses delivered at spinal cord segments C6 and C8 would facilitate the recovery of forelimb fine motor control after a cervical SCI in rats. Rats were trained to reach and grasp sugar pellets. Immediately after a dorsal funiculus crush at C4, the rats showed significant deficits in forelimb fine motor control. The rats were tested to reach and grasp with and without cervical epidural stimulation for 10weeks post-injury. To determine the best stimulation parameters to activate the cervical spinal networks involved in forelimb motor function, monopolar and bipolar currents were delivered at varying frequencies (20, 40, and 60Hz) concomitant with the reaching and grasping task. We found that cervical epidural stimulation increased reaching and grasping success rates compared to the no stimulation condition. Bipolar stimulation (C6– C8+ and C6+ C8–) produced the largest spinal motor-evoked potentials (sMEPs) and resulted in higher reaching and grasping success rates compared with monopolar stimulation (C6– Ref+ and C8– Ref+). Forelimb performance was similar when tested at stimulation frequencies of 20, 40, and 60Hz. We also found that the EMG activity in most forelimb muscles as well as the co-activation between flexor and extensor muscles increased post-injury. With epidural stimulation, however, this trend was reversed indicating that cervical epidural spinal cord stimulation has therapeutic potential for rehabilitation after a cervical SCI.

Changing Task Objectives for Improved Sweet Pepper Detection for Robotic Harvesting

This letter presents a method to improve detection for robotic harvesting by changing the task objective in real time in an adaptive thresholding algorithm. The adaptive thresholding algorithm includes three main parts: 3-D adaptive thresholding, object detection, and fusion. Optimal local 3-D thresholds that were previously determined according to changing illumination conditions were expanded in this research to include also changing task objectives. The task objectives describe the relationships between false positive rate, true positive rate, and accuracy in the location. The first task objective aims to maximize detection and minimize false alarms so as to ensure the arm is directed only towards real fruits. The second task objective focuses on high accuracy in the detection. Intensive evaluations were conducted on databases, which contained 240 images acquired in the field with various artificial illumination setups. The difference between the two tasks objectives was on average 0.09 in detection rates and 0.66 cm in the accuracy. Robotic experiments resulted in 26.6% difference in pepper grasping success rate with two different task objectives indicating the importance of changing the task objectives for the fruit detection task.

Physically Based Grasp Quality Evaluation Under Pose Uncertainty

Although there has been great progress in robot grasp planning, automatically generated grasp sets using a quality metric are not as robust as human-generated grasp sets when applied to real problems. Most previous research on grasp quality metrics has focused on measuring the quality of established grasp contacts after grasping, but it is difficult to reproduce the same planned final grasp configuration with a real robot hand, which makes the quality evaluation less useful in practice. In this study, we focus more on the grasping process, which usually involves changes in contact and object location, and explore the efficacy of using dynamic simulation in estimating the likely success or failure of a grasp in the real environment. Among many factors that can possibly affect the result of grasping, we particularly investigated the effect of considering object dynamics and pose uncertainty on the performance in estimating the actual grasp success rates measured from experiments. We observed that considering both dynamics and uncertainty improved the performance significantly, and when applied to automatic grasp set generation, this method generated more stable and natural grasp sets compared with a commonly used method based on kinematic simulation and force-closure analysis.

Does a nonprimate mammal, the Northern tree shrew (Tupaia belangeri), exhibit paw preference in two forms of a grasping task?

Tree shrews represent a relevant model to study the evolution of primate manual laterality as they are phylogenetically close to primates, they are able to grasp despite having a nonopposable thumb, and they possess a well-developed visual system. In this study, we examined the paw laterality and grasping success rate of 30 Tupaia belangeri (15 males, 15 females) in 2 forced-food grasping tasks (i.e., in a forced-food grasping experiment, the animal has to use paws instead of mouth for food retrieval). We also attempted to determine whether paw usage would be affected by the availability of visual cues using both a visual task (transparent tube) and a nonvisual task (identical but opaque tube). In both tasks, tree shrews showed paw preferences at an individual but not at a population level. Paw laterality (direction and strength) did not differ between tasks. Moreover, in the specific task that we used, grasping success rate was not affected by an absence of visual cues, indicating that tree shrews did not rely on visual guidance to direct their grasps in this forced-food grasping experiment. Our findings suggest that, in contrast to primates, paw usage in tree shrews may result from a modification of a fixed motor pattern in which the preferred direction may be learned. This basic motor organization might be a first step in the evolution of manual laterality, which eventually became controlled by vision in the primate lineage.

A control study of two methods in oronasal conversion of nose biliary duct

Objective To compare the success rate, the one-time success rate, the average required time and the incidence of complications of two methods in oronasal conversion of nose biliary duct in naso-biliary drainage. Methods 60 cases of patients with ENBD in the ERCP in our department from March,2008 to March, 2010 were divided into the study group and the control group by random. The patients with sleeving the gastric tube by the guidewire during oronasal conversion of nose biliary duct were considered as the study group, and the patients using the traditional method were considered as the control group. The success rate of sleeving gastric tube, the one-time success rate of grasping, the average required time and the adverse reactions of the two groups were compared. Results The success rate of the study group was 100%, and the control group was 90%, there was no statistical significance. The one-time success rate of the study group was 76.7%, higher than 46.7% of the control group. The average required time of the study group were (1.3±0.3) min, shorter than (5.1±0.7) min of the control group. The incidence of complications in the study group were 13.3%, lower than 93.3% of the control group. Conclusions To sleeve the gastric tube by the guidewire can increase the one-time success rate of grasping the gastric tube, shorten time of grasping the gastric tube and significantly reduce discomfort of patients. Key words: Guidewire; Nose biliary duct; Drainage