Object Affordances Research Articles

Social and Robust Navigation for Indoor Robots Based on Object Semantic Grid and Topological Map

For the indoor navigation of service robots, human–robot interaction and adapting to the environment still need to be strengthened, including determining the navigation goal socially, improving the success rate of passing doors, and optimizing the path planning efficiency. This paper proposes an indoor navigation system based on object semantic grid and topological map, to optimize the above problems. First, natural language is used as a human–robot interaction form, from which the target room, object, and spatial relationship can be extracted by using speech recognition and word segmentation. Then, the robot selects the goal point from the target space by object affordance theory. To improve the navigation success rate and safety, we generate auxiliary navigation points on both sides of the door to correct the robot trajectory. Furthermore, based on the topological map and auxiliary navigation points, the global path is segmented into each topological area. The path planning algorithm is carried on respectively in every room, which significantly improves the navigation efficiency. This system has demonstrated to support autonomous navigation based on language interaction and significantly improve the safety, efficiency, and robustness of indoor robot navigation. Our system has been successfully tested in real domestic environments.

Post-stroke object affordances: An EEG investigation

Rehabilitating upper limb function after stroke is a key therapeutic goal. In healthy brains, objects, especially tools, are said to cause automatic motoric ‘affordances’; affecting our preparation to handle objects. For example, the N2 event-related potential has been shown to correlate with the functional properties of objects in healthy adults during passive viewing. We posited that such an affordance effect might also be observed in chronic-stage stroke survivors. With either dominant or non-dominant hand forward, we presented three kinds of stimuli in stereoscopic depth; grasp objects affording a power-grip, pinch objects affording a thumb and forefinger precision-grip and an empty desk, affording no action. EEG data from 10 stroke survivors and 15 neurologically healthy subjects were analysed for the N1 and N2 ERP components. Both components revealed differences between the two object stimuli categories and the empty desk for both groups, suggesting the presence of affordance-related motor priming from around 100 to 370 ms after stimulus onset. Hence, we speculate that stroke survivors with loss of upper limb function may benefit from object presentation regimes designed to maximise motor priming when attempting movements with manipulable objects. However, further investigation would be necessary with acute stage patients, especially those diagnosed with apraxia.

The Grasp Strategy of a Robot Passer Influences Performance and Quality of the Robot-Human Object Handover.

Task-aware robotic grasping is critical if robots are to successfully cooperate with humans. The choice of a grasp is multi-faceted; however, the task to perform primes this choice in terms of hand shaping and placement on the object. This grasping strategy is particularly important for a robot companion, as it can potentially hinder the success of the collaboration with humans. In this work, we investigate how different grasping strategies of a robot passer influence the performance and the perceptions of the interaction of a human receiver. Our findings suggest that a grasping strategy that accounts for the subsequent task of the receiver improves substantially the performance of the human receiver in executing the subsequent task. The time to complete the task is reduced by eliminating the need of a post-handover re-adjustment of the object. Furthermore, the human perceptions of the interaction improve when a task-oriented grasping strategy is adopted. The influence of the robotic grasp strategy increases as the constraints induced by the object's affordances become more restrictive. The results of this work can benefit the wider robotics community, with application ranging from industrial to household human-robot interaction for cooperative and collaborative object manipulation.

What triggers a gesture? Exploring affordance compatibility effects in representational gesture production.

What are the mechanisms responsible for spontaneous cospeech gesture production? Driven by the close connection between cospeech gestures and object-related actions, recent research suggests that cospeech gestures originate in perceptual and motoric simulations that occur while speakers process information for speaking (Hostetter & Alibali, 2008). Here, we test this claim by highlighting object affordances during a communication task, inspired by the classic stimulus-response compatibility paradigm by Tucker and Ellis (1998). We compared cospeech gestures in situations where target objects were oriented toward the speakers' dominant hand (grasping potential enhanced), with situations where they were oriented toward the nondominant hand. Before the main experiment, we conducted a replication attempt of Tucker and Ellis' (1998: Experiment 1) to (re)establish the effect of stimulus compatibility, using contemporary items. Contrary to expectations, we could not replicate the original findings. Furthermore, compatibly with our replication results, the gesture data showed that enhancing grasping potential did not increase the amount of cospeech gestures produced. Vertical orientation nevertheless did, with upright objects eliciting more cospeech gestures than inverted ones, which does suggest a relation between affordance and gesture production. Our results challenge the automaticity of affordance effects, both in a classic stimulus-response compatibility experiment as well as in a more interactive dialogue setting and suggest that previous findings on cospeech gestures emerge from thinking and communicating about action-evoking content rather than from the affordance-compatibility of the presented objects. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

A Sociocultural Theory of Creativity: Bridging the Social, the Material, and the Psychological

The present article gives an overview of sociocultural approaches to creativity and advances a particular theory of the creative process grounded in the notions of difference, position, perspective, dialogue, and affordance. If sociocultural psychology challenges old dichotomies between mind and body, individual and society, then creativity is ideally placed to demonstrate their interdependence. While sociocultural thinking in creativity research has traditionally emphasized the social or collaborative nature of creative processes, recovering old scholarship and reviewing it in light of current empirical developments shows how socio-materiality can properly inform psychological theory in this area. The article starts with an outline of sociocultural principles before considering their application to creativity. It then formulates four propositions regarding the creative process: (a) differences of perspective increase creative potential; (b) exchanging positions and perspectives, within and between individuals, fosters creative processes; (c) these exchanges result in perspectives that reveal previously unperceived affordances; and (d) oftentimes, it is the affordances of material objects or of unique idea combinations that guide the development of novel perspectives in creative work. Evidence supporting these key hypotheses of the perspective-affordance theory of creativity (PAT) comes from research conducted in a variety of areas within psychology and in related fields. In the end, the methodological and practical implications of considering creativity as a process of recognizing differences, exchanging positions, developing perspectives and discovering affordances will be discussed, as well as the broader implications of building theories that bring together, rather than keep separate, the social, the material, and the psychological.

Voluntary and involuntary dynamics of perception-action processing

The current study investigated the role of both involuntary motor activations and voluntary stimulus-response (S-R) mappings (i.e., the subjects voluntarily learned altered S-R mappings) in perception-action processing. Involuntary motor activations were triggered by graspable objects, and voluntary S-R mappings were manipulated by adjusting the proportion of compatible trials (PC; 80%, 50%, and 20%) for which the potential motor properties of objects were compatible with the responses. The results showed large positive compatible effects when regular S-R associations were strong (80% PC) but negative compatible effects when reversed S-R associations were established (20% PC). Moreover, the size and developing speed of the compatible effects were larger with 80% PC than with 20% PC. The findings suggest that the direction, magnitude, and developing speed of object affordances are codriven by both involuntary motor activations and voluntary S-R mappings. Specifically, when these two factors are consistent with each other, motor activations will be amplified, and the magnitude and developing speed of affordance effects will thus be increased. However, when they are inconsistent, they will compete with each other, and affordance effects will be decreased or reversed with an increasing proportion of trials within the S-R mappings. The codriven model of perception-action processes reflects the optimization of our cognitive control system for processing a complicated environment.

New perspectives on person-centered care: an affordance-based account.

Despite the growing interest and supporting evidence for person-centered care (PCC), there is still a fundamental disagreement about what makes healthcare person-centered. In this article, we define PCC as operating with three fundamental conditions: personal, participatory and holistic. To further understand these concepts, we develop a framework based on the theory of affordances, which we apply to the healthcare case of rehabilitation and a concrete experiment on social interactions between persons with cerebral palsy and physio- and occupational therapists. Based on the application of the theory, we argue that in order for healthcare to be considered as PCC, professionals need to adopt a personalistic attitude in their care, defined (at the how-level) in terms of mutual affordances: how the professional and the person of care acknowledges each other as a person in an interaction. In opposition, we define (at the what level) the functionalistic attitude in terms of object affordances, those related to a concrete goal. We show that PCC requires a balance of personalistic and functionalistic attitudes, since this contributes to a participatory and holistic conception of, and interaction with, the person of care.

Visual and Haptic Perception of Affordances of Feelies.

Most objects have well-defined affordances. Investigating perception of affordances of objects that were not created for a specific purpose would provide insight into how affordances are perceived. In addition, comparison of perception of affordances for such objects across different exploratory modalities (visual vs. haptic) would offer a strong test of the lawfulness of information about affordances (i.e., the invariance of such information over transformation). Along these lines, "feelies"- objects created by Gibson with no obvious function and unlike any common object-could shed light on the processes underlying affordance perception. This study showed that when observers reported potential uses for feelies, modality significantly influenced what kind of affordances were perceived. Specifically, visual exploration resulted in more noun labels (e.g., "toy") than haptic exploration which resulted in more verb labels (i.e., "throw"). These results suggested that overlapping, but distinct classes of action possibilities are perceivable using vision and haptics. Semantic network analyses revealed that visual exploration resulted in object-oriented responses focused on object identification, whereas haptic exploration resulted in action-oriented responses. Cluster analyses confirmed these results. Affordance labels produced in the visual condition were more consistent, used fewer descriptors, were less diverse, but more novel than in the haptic condition.

Segmenting areas of potential contamination for adaptive robotic disinfection in built environments

Mass-gathering built environments such as hospitals, schools, and airports can become hot spots for pathogen transmission and exposure. Disinfection is critical for reducing infection risks and preventing outbreaks of infectious diseases. However, cleaning and disinfection are labor-intensive, time-consuming, and health-undermining, particularly during the pandemic of the coronavirus disease in 2019. To address the challenge, a novel framework is proposed in this study to enable robotic disinfection in built environments to reduce pathogen transmission and exposure. First, a simultaneous localization and mapping technique is exploited for robot navigation in built environments. Second, a deep-learning method is developed to segment and map areas of potential contamination in three dimensions based on the object affordance concept. Third, with short-wavelength ultraviolet light, the trajectories of robotic disinfection are generated to adapt to the geometries of areas of potential contamination to ensure complete and safe disinfection. Both simulations and physical experiments were conducted to validate the proposed methods, which demonstrated the feasibility of intelligent robotic disinfection and highlighted the applicability in mass-gathering built environments.

Learning Object Affordances from Sensory-Motor Interaction via Bayesian Networks with Auto-Encoder Features

In this paper, we study learning relationships between objects, actions, and effects. “Affordance” is an ecological psychology concept, which addresses how humans learn these relationships and which is also studied in cognitive robotics to transform the same ability to robots. Our model is built on top of two existing models in this field and use their strengths to introduce a novel system, where an anthropomorphic robot observes its environment and changes in that environment after executing pre-learned actions. Robot transforms these observations to object and effect properties in the same space and object affordances are learned using Bayesian Networks. The dimensions of features are decreased through autoencoders to achieve a compact network. Usage of a probabilistic model helps our system to deal with missing information or to make predictions for object properties and actions along with effect properties. We illustrate the advantages of our model by comparing it with the two aforementioned models.

Beyond the Self: Using Grounded Affordances to Interpret and Describe Others’ Actions

We propose a developmental approach that allows a robot to interpret and describe the actions of human agents by reusing previous experience. The robot first learns the association between words and object affordances by manipulating the objects in its environment. It then uses this information to learn a mapping between its own actions and those performed by a human in a shared environment. It finally fuses the information from these two models to interpret and describe human actions in light of its own experience. In our experiments, we show that the model can be used flexibly to do inference on different aspects of the scene. We can predict the effects of an action on the basis of object properties. We can revise the belief that a certain action occurred, given the observed effects of the human action. In an early action recognition fashion, we can anticipate the effects when the action has only been partially observed. By estimating the probability of words given the evidence and feeding them into a pre-defined grammar, we can generate relevant descriptions of the scene. We believe that this is a step towards providing robots with the fundamental skills to engage in social collaboration with humans.

Human-centric metrics for indoor scene assessment and synthesis

In recent years, many approaches have been proposed to analyze the affordance of 3D objects and to facilitate the synthesis of aesthetic and realistic indoor scenes. However, how to assess the quality of such synthesized 3D scenes is still a challenging problem. To address this, we present a novel approach through so-called Human-Centric Metrics (HCMs) for quantitatively evaluating the layout quality of certain objects, and thus to facilitate indoor scene synthesis. Our HCMs consider both the human-object factors to assess the functional accessibility of indoor objects, and the human-environment factors to assess the subjective comfort in the scene. Given a 3D scene with arranged furniture objects, our method automatically places human agents and then use HCMs to assess both the object arrangement and indoor environment based on the position/direction of a certain human agent. The conducted user study shows that the assessment capability of HCMs well conforms to the interior design knowledge. We also use the HCMs based assessment results to synthesize 3D indoor scenes by suggesting indoor objects and their layout given an empty room. A series of experimental results and comparisons are presented to validate the usability and effectiveness of our HCMs for virtual scene assessment and synthesis.

Intention-Related Natural Language Grounding via Object Affordance Detection and Intention Semantic Extraction.

Similar to specific natural language instructions, intention-related natural language queries also play an essential role in our daily life communication. Inspired by the psychology term “affordance” and its applications in Human-Robot interaction, we propose an object affordance-based natural language visual grounding architecture to ground intention-related natural language queries. Formally, we first present an attention-based multi-visual features fusion network to detect object affordances from RGB images. While fusing deep visual features extracted from a pre-trained CNN model with deep texture features encoded by a deep texture encoding network, the presented object affordance detection network takes into account the interaction of the multi-visual features, and reserves the complementary nature of the different features by integrating attention weights learned from sparse representations of the multi-visual features. We train and validate the attention-based object affordance recognition network on a self-built dataset in which a large number of images originate from MSCOCO and ImageNet. Moreover, we introduce an intention semantic extraction module to extract intention semantics from intention-related natural language queries. Finally, we ground intention-related natural language queries by integrating the detected object affordances with the extracted intention semantics. We conduct extensive experiments to validate the performance of the object affordance detection network and the intention-related natural language queries grounding architecture.

Faster recognition of graspable targets defined by orientation in a visual search task.

Peri-hand space is the area surrounding the hand. Objects within this space may be subject to increased visuospatial perception, increased attentional prioritization, and slower attentional disengagement compared to more distal objects. This may result from kinesthetic and visual feedback about the location of the hand that projects from the reach and grasp networks of the dorsal visual stream back to occipital visual areas, which in turn, refines cortical visual processing that can subsequently guide skilled motor actions. Thus, we hypothesized that visual stimuli that afford action, which are known to potentiate activity in the dorsal visual stream, would be associated with greater alterations in visual processing when presented near the hand. To test this, participants held their right hand near or far from a touchscreen that presented a visual array containing a single target object that differed from 11 distractor objects by orientation only. The target objects and their accompanying distractors either strongly afforded grasping or did not. Participants identified the target among the distractors by reaching out and touching it with their left index finger while eye-tracking was used to measure visual search times, target recognition times, and search accuracy. The results failed to support the theory of enhanced visual processing of graspable objects near the hand as participants were faster at recognizing graspable compared to non-graspable targets, regardless of the position of the right hand. The results are discussed in relation to the idea that, in addition to potentiating appropriate motor responses, object affordances may also potentiate early visual processes necessary for object recognition.

Review of Jens Kjaerulff. “Situating time: New technologies at work, a perspective from Alfred Gell’s oeuvre”

Review of Jens Kjaerulff. “Situating time: New technologies at work, a perspective from Alfred Gell’s oeuvre”

A knowledge-driven layered inverse reinforcement learning approach for recognizing human intents

ABSTRACT There is a rising trend in exploring the capability of inverse reinforcement learning (IRL) in high dimensional demonstrations. Our aim is to recognise human intents from video data within an IRL framework. For this, we present a two-layered maximum likelihood IRL model. The usefulness of knowledge representation (KR) schemes and availability of advisors at different layers is exploited through this model. Two main aspects are addressed: a. the importance of having abstract high-level information to the IRL framework in terms of semantic object affordance and b. deductively exploring the utility of a state at different temporal abstractions. The effectiveness of the proposed model has been evaluated with the help of standard Cornell Activity Dataset (CAD-120).

Deep sensorimotor learning for RGB-D object recognition

Research findings in cognitive neuroscience establish that humans, early on, develop their understanding of real-world objects by observing others interact with them or by performing active exploration and physical interactions with them. This fact has motivated the so-called “sensorimotor” learning approach, where the object appearance information (sensory) is combined with the object affordances (motor), i.e. the types of actions a human can perform with the object. In this work, the aforementioned paradigm is adopted, and a neuro-biologically inspired two-stream model for RGB-D object recognition is investigated. Both streams are realized as state-of-the-art deep neural networks that process and fuse appearance and affordance information in multiple ways. In particular, three model variants are developed to efficiently encode the spatio-temporal nature of the hand–object interaction, while an attention mechanism that relies on the appearance stream confidence is also investigated. Additionally, a suitable auxiliary loss is proposed for model training, utilized to further optimize both information streams. Experiments on the challenging SOR3D dataset, which consists of 14 object types and 13 object affordances, demonstrate the efficacy of the proposed model in RGB-D object recognition. Overall, the best performing developed model achieves 90.70% classification accuracy, which is further increased to 91.98% when trained using the auxiliary loss. The latter corresponds to 46% relative error reduction compared to the appearance-only classifier performance. Finally, a cross-view analysis on the SOR3D dataset provides valuable feedback for the viewpoint impact on the affordance information.

Toward Affordance Detection and Ranking on Novel Objects for Real-World Robotic Manipulation

This letter presents a framework to detect and rank affordances of novel objects to assist with robotic manipulation tasks. The framework segments the affordance map of unseen objects using region-based affordance segmentation. Detected affordances define an initial state from which to generate action primitives for manipulation via the planning domain definition language (PDDL). The proposed category-agnostic affordance segmentation approach generalizes learned affordances to unseen objects by utilizing binary classification on proposed instance masks. The predicted pixel-wise level affordances are ranked by KL-divergence, augmenting the available affordance choices for manipulation tasks with non-primary affordances of an object. Experimental results show that the proposed method achieves state-of-the-art performance on affordance segmentation of novel objects, and outperforms baselines on affordance ranking. Actual robotic manipulation scenarios demonstrate the use of affordance detection with PDDL-generated action primitives for task execution. Prediction of ranked affordances on unseen objects provides flexibility to accomplish goal-oriented tasks.

A Survey on Predicting Resident Intentions Using Contextual Modalities in Smart Home

The Smart Home is an environment that enables the resident to interact with home appliances which provide resident intended services. In recent years, predicting resident intention based on the contextual modalities like activity, speech, emotion, object affordances, and physiological parameters have increased importance in the field of pervasive computing. Contextual modality is the feature through which resident interacts with the home appliances like TVs, lights, doors, fans, etc. These modalities assist the appliances in predicting the resident intentions making them recommend resident intended services like opening and closing doors, turning on and off televisions, lights, and fans. Resident-appliance interaction can be achieved by embedding artificial intelligence-based machine learning algorithms into the appliances. Recent research works on the contextual modalities and associated machine learning algorithms which are required to build resident intention prediction system have been surveyed in this article. A classification taxonomy of contextual modalities is also discussed.

Learning Grasp Affordance Reasoning Through Semantic Relations

Reasoning about object affordances allows an autonomous agent to perform generalised manipulation tasks among object instances. While current approaches to grasp affordance estimation are effective, they are limited to a single hypothesis. We present an approach for detection and extraction of multiple grasp affordances on an object via visual input. We define semantics as a combination of multiple attributes, which yields benefits in terms of generalisation for grasp affordance prediction. We use Markov Logic Networks to build a knowledge base graph representation to obtain a probability distribution of grasp affordances for an object. To harvest the knowledge base, we collect and make available a novel dataset that relates different semantic attributes. We achieve reliable mappings of the predicted grasp affordances on the object by learning prototypical grasping patches from several examples. We show our method's generalisation capabilities on grasp affordance prediction for novel instances and compare with similar methods in the literature. Moreover, using a robotic platform, on simulated and real scenarios, we evaluate the success of the grasping task when conditioned on the grasp affordance prediction.