Sensorimotor Contingency Theory Research Articles

Using enactive robotics to think outside of the problem-solving box: How sensorimotor contingencies constrain the forms of emergent autononomous habits.

We suggest that the influence of biology in 'biologically inspired robotics' can be embraced at a deeper level than is typical, if we adopt an enactive approach that moves the focus of interest from how problems are solved to how problems emerge in the first place. In addition to being inspired by mechanisms found in natural systems or by evolutionary design principles directed at solving problems posited by the environment, we can take inspiration from the precarious, self-maintaining organization of living systems to investigate forms of cognition that are also precarious and self-maintaining and that thus also, like life, have their own problems that must be be addressed if they are to persist. In this vein, we use a simulation to explore precarious, self-reinforcing sensorimotor habits as a building block for a robot's behavior. Our simulations of simple robots controlled by an Iterative Deformable Sensorimotor Medium demonstrate the spontaneous emergence of different habits, their re-enactment and the organization of an ecology of habits within each agent. The form of the emergent habits is constrained by the sensory modality of the robot such that habits formed under one modality (vision) are more similar to each other than they are to habits formed under another (audition). We discuss these results in the wider context of: (a) enactive approaches to life and mind, (b) sensorimotor contingency theory, (c) adaptationist vs. structuralist explanations in biology, and (d) the limits of functionalist problem-solving approaches to (artificial) intelligence.

Affordances of musical instruments: Conceptual consideration.

While the concept of affordances has been applied in music research, it has not been satisfyingly developed regarding musical instruments. The resulting vagueness restricts the potential of the concept to guide exploration, discussion, and development of new approaches towards musical learning. Also, the concept of affordances comes with strong ontological claims and thus prompts the researcher to be careful when merging it with other theoretical domains or applying it in empirical studies. Consequently, the present article aims at contributing to a conceptualization of affordances of musical instruments by highlighting and discussing components that are necessary to consider in such a project. The first part consists of an overview of key elements of ecological psychology and more recent theoretical contributions, which are of relevance to the aim of the article: Material Engagement Theory, Skilled Intentionality Framework, and Sensorimotor Contingency Theory. A brief review of examples on how the concept of affordances has been applied in music research is presented. The main section of the article discusses four components, vital to further theoretical developments on affordances of musical instruments: the musical niche, spatial networks, sensorimotor relationship, and the amodal nature of affordances. Central to the argument is an understanding of affordances as relational, limited in scope and bound up with the physical interaction between musician and instrument. Accordingly, it is proposed that analytical focus in studies of musical instruments should be the sensorimotor relationship, spatiotemporally unfolding through a musical event. The article is concluded with comments upon educational implications of the presented perspective and suggestions on further research on the topic.

A Formal Account of Structuring Motor Actions With Sensory Prediction for a Naive Agent.

For naive robots to become truly autonomous, they need a means of developing their perceptive capabilities instead of relying on hand crafted models. The sensorimotor contingency theory asserts that such a way resides in learning invariants of the sensorimotor flow. We propose a formal framework inspired by this theory for the description of sensorimotor experiences of a naive agent, extending previous related works. We then use said formalism to conduct a theoretical study where we isolate sufficient conditions for the determination of a sensory prediction function. Furthermore, we also show that algebraic structure found in this prediction can be taken as a proxy for structure on the motor displacements, allowing for the discovery of the combinatorial structure of said displacements. Both these claims are further illustrated in simulations where a toy naive agent determines the sensory predictions of its spatial displacements from its uninterpreted sensory flow, which it then uses to infer the combinatorics of said displacements.

A critical approach to sensorimotor contingency theory: brain as agent and conscious mind as a guide of action

I present and consider critically O'Regan and Noë's sensorimotor contingency theory, proposed as an alternative to solve the explanatory gap problem. I start with the criticism that these authors address the current conception of representation, according to which conscious experiences are representations of the external world produced by the brain. Afterward, I summarize the way the sensorimotor contingency theory addresses the problem of the explanatory gap, explaining the existence, form, and content of visual consciousness in terms of an "exploratory activity" mediated by sensorimotor contingency laws. Finally, in agreement with criticisms addressed to O'Regan and Noë's solution, I propose a way to face the problem of the explanatory gap, which, recognizing the relevance of the body and the external environment to the existence, form and content of visual consciousness, but privileging the role of the brain as an organ of visual consciousness, and as an agent who uses visual consciousness as a guide to initiate and maintain embodied and situated adaptive actions in the world.

Where Do I Move My Sensors? Emergence of a Topological Representation of Sensors Poses From the Sensorimotor Flow

This article deals with the perception of mobile robotic systems within the framework of interactive perception, and inspired by the sensorimotor contingencies (SMCs) theory. These approaches state that perception arises from the active exploration of an environment. In the SMC theory, it is postulated that information about the structure of space could be recovered from a quasi-uninterpreted sensorimotor flow. In a recent article, the authors have provided a mathematical framework for the construction of a sensorimotor representation of the interaction between the sensors and the body of a naive agent, provided that the sensory inputs come from the agent's own body. An extension of these results, with stimulations coming from an unknown changing environment, is proposed in this article. More precisely, it is demonstrated that, through repeated explorations of its motor configurations, the perceived sensory invariants can be exploited to build a topologically accurate internal representation of the relative poses of the agent's sensors in the physical world. Precise theoretical considerations are provided as well as an experimental framework assessed in simulated but challenging environments.

Identification of Invariant Sensorimotor Structures as a Prerequisite for the Discovery of Objects.

Perceiving the surrounding environment in terms of objects is useful for any general purpose intelligent agent. In this paper, we investigate a fundamental mechanism making object perception possible, namely the identification of spatio-temporally invariant structures in the sensorimotor experience of an agent. We take inspiration from the Sensorimotor Contingencies Theory to define a computational model of this mechanism through a sensorimotor, unsupervised and predictive approach. Our model is based on processing the unsupervised interaction of an artificial agent with its environment. We show how spatio-temporally invariant structures in the environment induce regularities in the sensorimotor experience of an agent, and how this agent, while building a predictive model of its sensorimotor experience, can capture them as densely connected subgraphs in a graph of sensory states connected by motor commands. Our approach is focused on elementary mechanisms, and is illustrated with a set of simple experiments in which an agent interacts with an environment. We show how the agent can build an internal model of moving but spatio-temporally invariant structures by performing a Spectral Clustering of the graph modeling its overall sensorimotor experiences. We systematically examine properties of the model, shedding light more globally on the specificities of the paradigm with respect to methods based on the supervised processing of collections of static images.

Discovering space — Grounding spatial topology and metric regularity in a naive agent’s sensorimotor experience

In line with the sensorimotor contingency theory, we investigate the problem of the perception of space from a fundamental sensorimotor perspective. Despite its pervasive nature in our perception of the world, the origin of the concept of space remains largely mysterious. For example in the context of artificial perception, this issue is usually circumvented by having engineers pre-define the spatial structure of the problem the agent has to face. We here show that the structure of space can be autonomously discovered by a naive agent in the form of sensorimotor regularities, that correspond to so called compensable sensory experiences: these are experiences that can be generated either by the agent or its environment. By detecting such compensable experiences the agent can infer the topological and metric structure of the external space in which its body is moving. We propose a theoretical description of the nature of these regularities and illustrate the approach on a simulated robotic arm equipped with an eye-like sensor, and which interacts with an object. Finally we show how these regularities can be used to build an internal representation of the sensor’s external spatial configuration.

An Open-Ended Approach to Piagetian Development of Adaptive Behavior

In this paper, we are interested in the open-ended development of adaptive behavior by infant humans in the context of embodied, enactive cognitive science. We focus on the sensorimotor development of an infant child from gestation to toddler and discuss what aspects of the body, brain, and environment could allow for the sort of leaps of complexity observed in the developing infant that has heretofore not been replicable by artificial means. We use the backdrops of Piagetian developmental principles and Sensorimotor Contingency Theory to discuss this process in terms of skill proficiency, and discuss biologically plausible means for achieving it by referring to predictive processing and the free energy principle. We also refer to the theory of affordances to examine the selection of appropriate behaviors in a complex environment, and investigate phenomenological accounts to discuss the intentionality inherent in the purposeful behaviors that develop. Throughout this paper we develop a functional account of infant development which is based on the aforementioned theories and which leads to a biologically realistic explanation for the theory laid out by Piaget consistent with the embodied and enactive views.

Grounding the experience of a visual field through sensorimotor contingencies

Artificial perception is traditionally handled by hand-designing task specific algorithms. However, a truly autonomous robot should develop perceptive abilities on its own, by interacting with its environment, and adapting to new situations. The sensorimotor contingencies theory proposes to ground the development of those perceptive abilities in the way the agent can actively transform its sensory inputs. We propose a sensorimotor approach, inspired by this theory, in which the agent explores the world and discovers its properties by capturing the sensorimotor regularities they induce. This work presents an application of this approach to the discovery of a so-called visual field as the set of regularities that a visual sensor imposes on a naive agent’s experience. A formalism is proposed to describe how those regularities can be captured in a sensorimotor predictive model. Finally, the approach is evaluated on a simulated system coarsely inspired from the human retina.

Learning New Sensorimotor Contingencies: Effects of Long-Term Use of Sensory Augmentation on the Brain and Conscious Perception.

Theories of embodied cognition propose that perception is shaped by sensory stimuli and by the actions of the organism. Following sensorimotor contingency theory, the mastery of lawful relations between own behavior and resulting changes in sensory signals, called sensorimotor contingencies, is constitutive of conscious perception. Sensorimotor contingency theory predicts that, after training, knowledge relating to new sensorimotor contingencies develops, leading to changes in the activation of sensorimotor systems, and concomitant changes in perception. In the present study, we spell out this hypothesis in detail and investigate whether it is possible to learn new sensorimotor contingencies by sensory augmentation. Specifically, we designed an fMRI compatible sensory augmentation device, the feelSpace belt, which gives orientation information about the direction of magnetic north via vibrotactile stimulation on the waist of participants. In a longitudinal study, participants trained with this belt for seven weeks in natural environment. Our EEG results indicate that training with the belt leads to changes in sleep architecture early in the training phase, compatible with the consolidation of procedural learning as well as increased sensorimotor processing and motor programming. The fMRI results suggest that training entails activity in sensory as well as higher motor centers and brain areas known to be involved in navigation. These neural changes are accompanied with changes in how space and the belt signal are perceived, as well as with increased trust in navigational ability. Thus, our data on physiological processes and subjective experiences are compatible with the hypothesis that new sensorimotor contingencies can be acquired using sensory augmentation.

Acquisition of a space representation by a naive agent from sensorimotor invariance and proprioceptive compensation

In this article, we present a simple agent which learns an internal representation of space without a priori knowledge of its environment, body, or sensors. The learned environment is seen as an internal space representation. This representation is isomorphic to the group of transformations applied to the environment. The model solves certain theoretical and practical issues encountered in previous work in sensorimotor contingency theory. Considering the mathematical description of the internal representation, analysis of its properties and simulations, we prove that this internal representation is equivalent to knowledge of space.

Extended Consciousness: an Interim Report

AbstractAdvocates of extended cognition hold that the physical machinery of mind sometimes extends beyond the skull and skin. In the first part of this paper, I explain why, and more specifically the precise sense in which, consciousness presents such theorists with an extra hurdle to be cleared. The key challenge is posed by phenomenal consciousness, the what‐it's‐like‐ness of experience. I consider two arguments for the claim that the physical machinery of phenomenal consciousness sometimes extends beyond the skull and skin. The first – the argument from sensory substitution – suggests that acceptance of extended phenomenal consciousness should follow from a careful analysis of the phenomenon in which technological augmentation enables one sensory modality, for instance touch, to support the kind of environmental access and interaction ordinarily supported by a different sensory modality, for instance vision. The second argument – the argument from the relational character of experience – suggests that acceptance of extended phenomenal consciousness should follow from a particular conception of conscious experience that is mandated by sensorimotor contingency theory. I conclude that neither argument is decisive.

Sensorimotor strategies for recognizing geometrical shapes: a comparative study with different sensory substitution devices.

The sensorimotor approach proposes that perception is constituted by the mastery of lawful sensorimotor regularities or sensorimotor contingencies (SMCs), which depend on specific bodily characteristics and on actions possibilities that the environment enables and constrains. Sensory substitution devices (SSDs) provide the user information about the world typically corresponding to one sensory modality through the stimulation of another modality. We investigate how perception emerges in novice adult participants equipped with vision-to-auditory SSDs while solving a simple geometrical shape recognition task. In particular, we examine the distinction between apparatus-related SMCs (those originating mostly in properties of the perceptual system) and object-related SMCs (those mostly connected with the perceptual task). We study the sensorimotor strategies employed by participants in three experiments with three different SSDs: a minimalist head-mounted SSD, a traditional, also head-mounted SSD (the vOICe) and an enhanced, hand-held echolocation device. Motor activity and fist-person data are registered and analyzed. Results show that participants are able to quickly learn the necessary skills to distinguish geometric shapes. Comparing the sensorimotor strategies utilized with each SSD we identify differential features of the sensorimotor patterns attributable mostly to the device, which account for the emergence of apparatus-based SMCs. These relate to differences in sweeping strategies between SSDs. We identify, also, components related to the emergence of object-related SMCs. These relate mostly to exploratory movements around the border of a shape. The study provides empirical support for SMC theory and discusses considerations about the nature of perception in sensory substitution.

The worldly constituents of perceptual presence.

The worldly constituents of perceptual presence.

Extending sensorimotor contingency theory: prediction, planning, and action generation

One of the main assertions of sensorimotor contingency theory is that sensory experience is not generated by activating an internal representation of the outside world through sensory signals, but corresponds to a mode of exploration and hence is an active process. Perception and sensory awareness emerge from using the structure of changes in the sensory input resulting from these exploratory actions, called sensorimotor contingencies (SMCs), for planning, reasoning, and goal achievement. Using a previously developed computational model of SMCs we show how an artificial agent can plan ahead with SMCs and use them for action guidance. Our main assumption is that SMCs are associated with a utility for the agent, and that the agent selects actions that maximize this utility. We analyze the properties of the resulting actions in a robot that is endowed with several sensory modalities and controlled by our model in a simple environment. The results demonstrate that its actions avoid aversive events, and that it can achieve a low-level form of spatial awareness that is resilient to the complete loss of a sensory modality.

Knowing What? Radical Versus Conservative Enactivism

The binary divide between traditional cognitivist and enactivist paradigms is tied to their respective commitments to understanding cognition as based on knowing that as opposed to knowing how. Using O'Regan's and Noe's landmark sensorimotor contingency theory of perceptual experience as a foil, I demonstrate how easy it is to fall into conservative thinking. Although their account is advertised as decidedly ‘skill-based’, on close inspection it shows itself to be riddled with suppositions threatening to reduce it to a rules-and-representations approach. To remain properly enactivist it must be purged of such commitments and indeed all commitment to mediating knowledge: it must embrace a more radical enactivism

Does sensorimotor contingency theory account for perceptual-motor dissociations?

We review studies that indicate a dissociation between the perceptual estimate and the resulting cognitive representation of given properties of a seen object, on the one hand, and the motor action exerted on the same object. We propose that there exist multiple levels of organization of sensorimotor loops and that internal models may be made accessible to one level of organization while remaining inaccessible to another level.

Re-presenting the case for representation

O'Regan & Noë (O&N) present the most radical departure yet from traditional approaches to visual perception. However, internal representation cannot yet be abandoned. I will discuss: (1) recent evidence for very short-term pictorial representation of each fixation; (2) the possibility of abstract representation, largely unconsidered by the authors; and (3) that sensorimotor contingency theory requires internal visual retention and comparison.