Graspable Objects Research Articles

Do graspable objects always leave a motor signature? A study on memory traces

Several studies have reported the existence of reciprocal interactions between the type of motor activity physically performed on objects and the conceptual knowledge that is retained of them. Whether covert motor activity plays a similar effect is less clear. Certainly, objects are strong triggers for actions, and motor components can make the associated concepts more memorable. However, addition of an action-related memory trace may not always be automatic and could rather depend on ‘how’ objects are encountered. To test this hypothesis, we compared memory for objects that passive observers experienced as verbal labels (the word describing them), visual images (color photographs) and actions (pantomimes of object use). We predicted that the more direct the involvement of action-related representations the more effective would be the addition of a motor code to the experience and the more accurate would be the recall. Results showed that memory for objects presented as words i.e., a format that might only indirectly prime the sensorimotor system, was generally less accurate compared to memory for objects presented as photographs or pantomimes, which are more likely to directly elicit motor simulation processes. In addition, free recall of objects experienced as pantomimes was more accurate when these items afforded actions performed towards one’s body than actions directed away from the body. We propose that covert motor activity can contribute to objects’ memory, but the beneficial addition of a motor code to the experience is not necessarily automatic. An advantage is more likely to emerge when the observer is induced to take a first-person stance during the encoding phase, as may happen for objects affording actions directed towards the body, which obviously carry more relevance for the actor.

Get to grips with motivation: Slipping and gripping movements are biased by approach-avoidance context

People are better at approaching appetitive cues signaling reward and avoiding aversive cues signaling punishment than vice versa. This action bias has previously been shown in approach-avoidance tasks involving arm movements in response to appetitive or aversive cues. It is not known whether appetitive or aversive stimuli also bias more distal dexterous actions, such as gripping and slipping, in a similar manner. To test this hypothesis, we designed a novel task involving grip force control (gripping and slipping) to probe gripping-related approach and avoidance behavior. 32 male volunteers, aged 18-40 years, were instructed to either grip ("approach") or slip ("avoid") a grip-force device with their right thumb and index finger at the sight of positive or negative images. In one version of this pincer grip task, participants were responding to graspable objects and in another version of the task they were responding to happy or angry faces. Bayesian repeated measures Analysis of variance revealed extreme evidence for an interaction between response type and cue valence (Bayes factor = 296). Participants were faster to respond in affect-congruent conditions ("approach appetitive," "avoid aversive") than in affect-incongruent conditions ("approach aversive," "avoid appetitive"). This bias toward faster response times for affect-congruent conditions was present regardless of whether it was a graspable object or a face signaling valence. Since our results mirror the approach and avoidance effects previously observed for arm movements, we conclude that a tendency favoring affectively congruent cue-response mappings is an inherent feature of motor control and thus also includes precision grip.

GSG: A Granary-Shaped Soft Gripper With Mechanical Sensing via Snap-Through Structure

Soft robotic grippers have attracted considerable attention in terms of the advantages of the high compliance and robustness to variance in object geometry; however, they are still limited by the corresponding sensing capabilities. We propose a novel soft gripper that looks like a ‘granary’ in the geometrical shape with a snap-through bistable mechanism fabricated by an ordered mold technology, which consists of a palm chamber, shell, cap and three fingers. It can achieve ‘sensing’ mechanically and perform pinching, enveloping and caging grasps for objects with various profiles. In particular, the snap-through bistable mechanism in the proposed gripper allows us to reduce the complexity of the mechanism, control, and sensing designs. The grasping behavior is activated once the gripper's deformation or perceived pressure arrives at a certain value. First, after the theoretical model for snap-through behavior is constructed, the modularized design of the gripper is described in detail. Then, the ordered molding method is employed to fabricate the proposed gripper. Finally, the finite element (FE) simulations are conducted to verify the built theoretical model. Further, a series of grasping experiments are carried out to assess the performance of the proposed gripper on grasping and sensing. The experimental results illustrate that the proposed gripper can manipulate a variety of soft and rigid objects and remain stable even though it undergoes external disturbances.

Does a look of fear prompt to act? The effects of gaze and face emotional expression on manipulable objects.

Gaze direction is an important social cue for understanding the intentions of other people. Indeed, interacting with others requires the ability to encode their current focus of attention in order to predict their future actions. Previous studies have showed that when asked to detect or identify a target, people are faster if shown a gaze oriented toward rather than away from that target. Most importantly, there is evidence that the emotion conveyed by the face with the averted gaze matters. We further tested the interplay between gaze and face emotion in the context of manipulable objects to understand whether and to what extent other people's gaze influences our own actions toward objects. Participants judged whether a target graspable object was upright or inverted after viewing a face cue with a central or averted gaze. Importantly, the target's handle could be oriented toward the gazed-at location or the opposite side such that gaze and handle were corresponding or non-corresponding in space. Furthermore, we manipulated the expression of the cue by using neutral and fearful faces. Results showed a handle-response (H-R) compatibility effect (i.e., a facilitation when the response key is on the same side as the object's handle) only with fearful cues with a central gaze.

Decoding sounds depicting hand-object interactions in primary somatosensory cortex.

Neurons, even in the earliest sensory regions of cortex, are subject to a great deal of contextual influences from both within and across modality connections. Recent work has shown that primary sensory areas can respond to and, in some cases, discriminate stimuli that are not of their target modality: for example, primary somatosensory cortex (SI) discriminates visual images of graspable objects. In the present work, we investigated whether SI would discriminate sounds depicting hand-object interactions (e.g. bouncing a ball). In a rapid event-related functional magnetic resonance imaging experiment, participants listened attentively to sounds from 3 categories: hand-object interactions, and control categories of pure tones and animal vocalizations, while performing a one-back repetition detection task. Multivoxel pattern analysis revealed significant decoding of hand-object interaction sounds within SI, but not for either control category. Crucially, in the hand-sensitive voxels defined from an independent tactile localizer, decoding accuracies were significantly higher for hand-object interactions compared to pure tones in left SI. Our findings indicate that simply hearing sounds depicting familiar hand-object interactions elicit different patterns of activity in SI, despite the complete absence of tactile stimulation. These results highlight the rich contextual information that can be transmitted across sensory modalities even to primary sensory areas.

GR-ConvNet v2: A Real-Time Multi-Grasp Detection Network for Robotic Grasping.

We propose a dual-module robotic system to tackle the problem of generating and performing antipodal robotic grasps for unknown objects from the n-channel image of the scene. We present an improved version of the Generative Residual Convolutional Neural Network (GR-ConvNet v2) model that can generate robust antipodal grasps from n-channel image input at real-time speeds (20 ms). We evaluated the proposed model architecture on three standard datasets and achieved a new state-of-the-art accuracy of 98.8%, 95.1%, and 97.4% on Cornell, Jacquard and Graspnet grasping datasets, respectively. Empirical results show that our model significantly outperformed the prior work with a stricter IoU-based grasp detection metric. We conducted a suite of tests in simulation and the real world on a diverse set of previously unseen objects with adversarial geometry and household items. We demonstrate the adaptability of our approach by directly transferring the trained model to a 7 DoF robotic manipulator with a grasp success rate of 95.4% and 93.0% on novel household and adversarial objects, respectively. Furthermore, we validate the generalization capability of our pixel-wise grasp prediction model by validating it on complex Ravens-10 benchmark tasks, some of which require closed-loop visual feedback for multi-step sequencing.

Perching and Grasping Mechanism Inspired by a Bird’s Claw

In nature, birds can freely observe and rest on the surface of objects such as tree branches, mainly due to their flexible claws, thus this paper is inspired by bird perching and shows two imitation bird claw perching grasping mechanisms in the shape of “three in front and one at the back”. One is articulated, the other is resilient, the difference being that the former has a pin-articulated claw structure and uses a double fishing line to perform the grasping and resetting action, while the latter uses a resilient linking piece, a single fishing line and resilient linking piece to perform the grasping and resetting action. To verify the grasping effect, experiments were designed to grasp objects of different shapes and maximum grasping weight load. The results show that the two types of perching grasping mechanism can reach a large degree of toe bending, have good passive bending deformation ability, can grasp different types of objects, including the articulated type has a stronger deformation ability, and can grasp branches with a diameter in the range of 12.5–55.8 mm. The elastic reset type is smoother than the articulated type toe bending curve, and the maximum graspable object weight is about three times the overall weight of the grasping mechanism. The maximum gripping weight is about three times the overall weight of the gripping mechanism and the load capacity is about two times that of the articulated type.

Learning-Based Slip Detection for Robotic Fruit Grasping and Manipulation under Leaf Interference.

Robotic harvesting research has seen significant achievements in the past decade, with breakthroughs being made in machine vision, robot manipulation, autonomous navigation and mapping. However, the missing capability of obstacle handling during the grasping process has severely reduced harvest success rate and limited the overall performance of robotic harvesting. This work focuses on leaf interference caused slip detection and handling, where solutions to robotic grasping in an unstructured environment are proposed. Through analysis of the motion and force of fruit grasping under leaf interference, the connection between object slip caused by leaf interference and inadequate harvest performance is identified for the first time in the literature. A learning-based perception and manipulation method is proposed to detect slip that causes problematic grasps of objects, allowing the robot to implement timely reaction. Our results indicate that the proposed algorithm detects grasp slip with an accuracy of 94%. The proposed sensing-based manipulation demonstrated great potential in robotic fruit harvesting, and could be extended to other pick-place applications.

Few-Shot Instance Grasping of Novel Objects in Clutter

Instance grasping, which aims to grasp a specific object out of clutter, is a fundamental task within robotics. However, allowing a robot to quickly learn to perform instance grasping for new, previously unseen objects remains challenging. In this work, we present an instance grasping meta-learning framework (IGML), a simple yet effective end-to-end approach that not only teaches robots to identify novel objects but also how to grasp them. Given only a few examples to specify the grasping point of the target object, our IGML can quickly learn to recognize the target object and grasp it at the demonstrated grasping point by leveraging prior experience. Experimental results on the test sets show that IGML achieves decent success rates in cluttered environments, significantly surpassing state-of-the-art methods. Then we deployed IGML on a UR5 robot arm to handle pick-and-place scenarios and achieved a precision rate of 93.4% and a recall rate of 87.1%.

When Transformer Meets Robotic Grasping: Exploits Context for Efficient Grasp Detection

In this letter, we present a transformer-based architecture, namely TF-Grasp, for robotic grasp detection. The developed TF-Grasp framework has two elaborate designs making it well suitable for visual grasping tasks. The first key design is that we adopt the local window attention to capture local contextual information and detailed features of graspable objects. Then, we apply the cross window attention to model the long-term dependencies between distant pixels. Object knowledge, environmental configuration, and relationships between different visual entities are aggregated for subsequent grasp detection. The second key design is that we build a hierarchical encoder-decoder architecture with skip-connections, delivering shallow features from the encoder to decoder to enable a multi-scale feature fusion. Due to the powerful attention mechanism, TF-Grasp can simultaneously obtain the local information (i.e., the contours of objects), and model long-term connections such as the relationships between distinct visual concepts in clutter. Extensive computational experiments demonstrate that TF-Grasp achieves competitive results versus state-of-art grasping convolutional models and attains a higher accuracy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$97.99 \%$</tex-math></inline-formula> and 94.6% on Cornell and Jacquard grasping datasets, respectively. Real-world experiments using a 7DoF Franka Emika Panda robot also demonstrate its capability of grasping unseen objects in a variety of scenarios. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/WangShaoSUN/grasp-transformer</uri> .

Autonomous Robotic Manipulation: Real-Time, Deep-Learning Approach for Grasping of Unknown Objects

Recent advancement in vision-based robotics and deep-learning techniques has enabled the use of intelligent systems in a wider range of applications requiring object manipulation. Finding a robust solution for object grasping and autonomous manipulation became the focus of many engineers and is still one of the most demanding problems in modern robotics. This paper presents a full grasping pipeline proposing a real-time data-driven deep-learning approach for robotic grasping of unknown objects using MATLAB and convolutional neural networks. The proposed approach employs RGB-D image data acquired from an eye-in-hand camera centering the object of interest in the field of view using visual servoing. Our approach aims at reducing propagation errors and eliminating the need for complex hand tracking algorithm, image segmentation, or 3D reconstruction. The proposed approach is able to efficiently generate reliable multi-view object grasps regardless of the geometric complexity and physical properties of the object in question. The proposed system architecture enables simple and effective path generation and a real-time tracking control. In addition, our system is modular, reliable, and accurate in both end effector path generation and control. We experimentally justify the efficacy and effectiveness of our overall system on the Barrett Whole Arm Manipulator.

Lazy Rearrangement Planning in Confined Spaces

Object rearrangement is important for many applications but remains challenging, especially in confined spaces, such as shelves, where objects cannot be accessed from above and they block reachability to each other. Such constraints require many motion planning and collision checking calls, which are computationally expensive. In addition, the arrangement space grows exponentially with the number of objects. To address these issues, this work introduces a lazy evaluation framework with a local monotone solver and a global planner. Monotone instances are those that can be solved by moving each object at most once. A key insight is that reachability constraints at the grasps for objects' starts and goals can quickly reveal dependencies between objects without having to execute expensive motion planning queries. Given that, the local solver builds lazily a search tree that respects these reachability constraints without verifying that the arm paths are collision free. It only collision checks when a promising solution is found. If a monotone solution is not found, the non-monotone planner loads the lazy search tree and explores ways to move objects to intermediate locations from where monotone solutions to the goal can be found. Results show that the proposed framework can solve difficult instances in confined spaces with up to 16 objects, which state-of-the-art methods fail to solve. It also solves problems faster than alternatives, when the alternatives find a solution. It also achieves high-quality solutions, i.e., only 1.8 additional actions on average are needed for non-monotone instances.

Emotions Modulate Affordances-Related Motor Responses: A Priming Experiment.

Traditionally, research on affordances and emotions follows two separate routes. For the first time, this article explicitly links the two phenomena by investigating whether, in a discrimination task (artifact vs. natural object), the motivational states induced by emotional images can modulate affordances-related motor response elicited by dangerous and neutral graspable objects. The results show faster RTs: (i) for both neutral and dangerous objects with neutral images; (ii) for dangerous objects with pleasant images; (iii) for neutral objects with unpleasant images. Overall, these data support a significant effect of emotions on affordances. The article also proposes a brain neural network underlying emotions and affordance interplay.

Emotions in motion: affective valence can influence compatibility effects with graspable objects.

Previous studies showed that affective valence (positive, negative) influences Stimulus-Response Compatibility (SRC) effects elicited by both relevant and irrelevant spatial dimensions. We tested whether valence influences SRC effects when the irrelevant spatial dimension rather than being conveyed by the entire stimulus location is conveyed by the location of the stimulus' graspable part, i.e., the Handle-Response (H-R) compatibility effect. Participants saw objects with either a flower, a spider or nothing on their handle and categorized them as kitchen utensils or garage tools through button presses. In Experiment 1, a random presentation of valenced stimuli was used, whereas in Experiment 2 differently valenced stimuli were arranged in different blocks. Furthermore, participants in Experiment 2 could be spider-fearful or not. In Experiment 1, an H-R compatibility effect occurred for response latencies, regardless of whether stimuli presented a negative, positive or no element on their handle. In Experiment 2 the effect occurred only when a positive element was shown on the object's handle. In addition, spider-fearful individuals showed significantly slower responses when the element appearing on the object's handle had a negative valence. These results suggest that the SRC effect observed with pictures of graspable objects may be sensitive to the affective characteristics of stimuli and that approach/avoidance response tendencies may also depend on individual differences (being spider-fearful or not).

Bioinspired Multimodal Multipose Hybrid Fingers for Wide-Range Force, Compliant, and Stable Grasping.

The increasing demand for grasping diverse objects in unstructured environments poses severe challenges to the existing soft/rigid robotic fingers due to the issues in balancing force, compliance, and stability, and hence has given birth to several hybrid designs. These hybrid designs utilize the advantages of rigid and soft structures and show better performance, but they are still suffering from narrow output force range, limited compliance, and rarely reported stability. Owing to its rigid-soft coupling structure with flexible switched multiple poses, human finger, as an excellent hybrid design, shows wide-range output force, excellent compliance, and stability. Inspired by human finger, we propose a hybrid finger with multiple modes and poses, coupled by a soft actuator (SA) and a rigid actuator (RA) in parallel. The multiple actuation modes formed by a pneumatic-based rigid-soft collaborative strategy can selectively enable the RA's high force and SA's softness, whereas the multiple poses derived from the specially designed underactuated RA skeleton can be flexibly switched with tasks, thus achieving high compliance. Such hybrid fingers also proved to be highly stable under external stimuli or gravity. Furthermore, we modularize and configure these fingers into a series of grippers with excellent grasping performance, for example, wide graspable object range (diverse from 0.1 g potato chips to 27 kg dumbbells for a 420 g two-finger gripper), high compliance (tolerate objects with 94% gripper span size and 4 cm offset), and high stability. Our study highlights the potential of fusing rigid-soft technologies for robot development, and potentially impacts future bionics and high-performance robot development.

Using High-Density Electroencephalography to Explore Spatiotemporal Representations of Object Categories in Visual Cortex.

Visual object perception involves neural processes that unfold over time and recruit multiple regions of the brain. Here, we use high-density EEG to investigate the spatiotemporal representations of object categories across the dorsal and ventral pathways. In , human participants were presented with images from two animate object categories (birds and insects) and two inanimate categories (tools and graspable objects). In , participants viewed images of tools and graspable objects from a different stimulus set, one in which a shape confound that often exists between these categories (elongation) was controlled for. To explore the temporal dynamics of object representations, we employed time-resolved multivariate pattern analysis on the EEG time series data. This was performed at the electrode level as well as in source space of two regions of interest: one encompassing the ventral pathway and another encompassing the dorsal pathway. Our results demonstrate shape, exemplar, and category information can be decoded from the EEG signal. Multivariate pattern analysis within source space revealed that both dorsal and ventral pathways contain information pertaining to shape, inanimate object categories, and animate object categories. Of particular interest, we note striking similarities obtained in both ventral stream and dorsal stream regions of interest. These findings provide insight into the spatio-temporal dynamics of object representation and contribute to a growing literature that has begun to redefine the traditional role of the dorsal pathway.

Unknown Object Grasping Based on Adaptive Dynamic Force Balance

Unknown Object Grasping Based on Adaptive Dynamic Force Balance

Deep Dexterous Grasping of Novel Objects From a Single View

Dexterous grasping of a novel object given a single view is an open problem. This paper makes several contributions to its solution. First, we present a simulator for generating and testing dexterous grasps. Second, we present a dataset, generated by this simulator, of 2.4 million simulated dexterous grasps of variations of 294 base objects drawn from 20 categories. Third, we combine an existing approach to learn a grasp generation model with three different learned evaluative models employing ResNet-50 or VGG16 as their visual backbone. Fourth, we train, and evaluate 17 variants of the resulting generative-evaluative architectures on the simulated dataset, showing improvement from 69.53% grasp success rate to 90.49%. Fifth, we present a real robot implementation and evaluate the four most promising variants, executing 196 real robot grasps in total. We show that our best architectural variant achieves a grasp success rate of 87.8% on real novel objects seen from a single view, improving on a baseline of 57.1%. Finally, we explore the inner workings of our best evaluative model and perform an extensive analysis of its results on the simulated dataset.

Comparing Single Touch to Dynamic Exploratory Procedures for Robotic Tactile Object Recognition

Recognizing objects by touch is a very useful skill for robots to be employed in both structured and unstructured environments. While in some applications it is useful to recognize an object from a single touch, in other scenarios specific robot movements can be used to obtain more information about the object, making recognition easier. In this letter, we show how this can be obtained through the combination of: (i) a recently developed tactile sensor that measures both normal and shear forces on multiple contact points, and (ii) an exploratory procedure that involves dynamic shaking of the gripped object. We compare the recognition accuracy in three conditions: static (i.e. single touch), short dynamic (i.e. using a small fraction of the exploratory procedure), and dynamic (i.e. using the entire exploratory procedure). We report experiments with six different machine learning techniques, and several combinations of tactile features, to recognize ten objects. Overall, our results demonstrate that: (i) the sensor we use is well suited for recognizing grasped objects with high accuracy, and (ii) the dynamic exploratory procedure provides a 38&#x0025; improvement over single touch recognition. We make our data and code publicly available, to encourage reproduction of our results.

The visual encoding of graspable unfamiliar objects.

We explored by eye-tracking the visual encoding modalities of participants (N = 20) involved in a free-observation task in which three repetitions of ten unfamiliar graspable objects were administered. Then, we analysed the temporal allocation (t = 1500ms) of visual-spatial attention to objects' manipulation (i.e., the part aimed at grasping the object) and functional (i.e., the part aimed at recognizing the function and identity of the object) areas. Within the first 750ms, participants tended to shift their gaze on the functional areas while decreasing their attention on the manipulation areas. Then, participants reversed this trend, decreasing their visual-spatial attention to the functional areas while fixing the manipulation areas relatively more. Crucially, the global amount of visual-spatial attention for objects' functional areas significantly decreased as an effect of stimuli repetition while remaining stable for the manipulation areas, thus indicating stimulus familiarity effects. These findings support the action reappraisal theoretical approach, which considers object/tool processing as abilities emerging from semantic, technical/mechanical, and sensorimotor knowledge integration.