Action-effect Binding Research Articles

Theoretical Perspective on an Ideomotor Brain-Computer Interface: Toward a Naturalistic and Non-invasive Brain-Computer Interface Paradigm Based on Action-Effect Representation.

Recent years have been marked by the fulgurant expansion of non-invasive Brain-Computer Interface (BCI) devices and applications in various contexts (medical, industrial etc.). This technology allows agents “to directly act with thoughts,” bypassing the peripheral motor system. Interestingly, it is worth noting that typical non-invasive BCI paradigms remain distant from neuroscientific models of human voluntary action. Notably, bidirectional links between action and perception are constantly ignored in BCI experiments. In the current perspective article, we proposed an innovative BCI paradigm that is directly inspired by the ideomotor principle, which postulates that voluntary actions are driven by the anticipated representation of forthcoming perceptual effects. We believe that (1) adapting BCI paradigms could allow simple action-effect bindings and consequently action-effect predictions and (2) using neural underpinnings of those action-effect predictions as features of interest in AI methods, could lead to more accurate and naturalistic BCI-mediated actions.

Reward strengthens action–effect binding.

According to ideomotor theory, anticipatory representations of action consequences are the basis for voluntary action control. A previous study suggested that rewards strengthen the acquisition of action effect links and hence ideomotor learning. Participants in our experiments (total N = 231) first learned to associate 4 manual actions with unique sound effects. Two sound effects were additionally predictive of a monetary reward. In a subsequent test phase, the former sound effects were presented as response primes in a speeded reaction time task. Response times were higher when the primes preceded a response other than the one to which they were linked in the preceding learning phase, an index of response−effect learning. Response priming was stronger for previously rewarded actions. Critically, this effect was not observed in a control condition with previously punished actions that produced a monetary loss. Overall, the results suggest that relations to rewarding consequences enhance associations between actions and sensory effects, a process that may facilitate reinforcement learning.

Connecting action control and agency: Does action-effect binding affect temporal binding?

The sense of agency, i.e., the notion that we, as agents, are in control of our own actions and can affect our environment by acting, is an integral part of human volition. Recent work has attempted to ground agency in basic mechanisms of human action control. Along these lines, action-effect binding has been shown to affect explicit judgments of agency. Here, we investigate if such action-effect bindings are also related to temporal binding which is often used as an implicit measure of agency. In two experiments, we found evidence for the establishment of short-term action-effect bindings as well as temporal binding effects. However, the two phenomena were not associated with each other. This finding suggests that the relation of action control and agency is not a simple one, and it adds to the evidence in favor of a dissociation between subjective agency and perceptual biases such as temporal binding.

Supplemental Material for Reward Strengthens Action–Effect Binding

Supplemental Material for Reward Strengthens Action–Effect Binding

Revisiting intersubjective action-effect binding: No evidence for social moderators.

Effect-based accounts of human action control have recently highlighted the possibility of representing one's own actions in terms of anticipated changes in the behavior of social interaction partners. In contrast to action effects that pertain to the agent's body or the agent's physical environment, social action effects have been proposed to come with peculiarities inherent to their social nature. Here, we revisit the currently most prominent demonstration of such a peculiarity: the role of eye contact for action-effect learning in social contexts (Sato & Itakura, 2013, Cognition, 127, 383-390). In contrast to the previous demonstration of action-effect learning, a conceptual and a direct replication both yielded evidence for the absence of action-effect learning in the proposed design, irrespective of eye contact. Bayesian statistics supported this claim by demonstrating evidence in favor of the null hypothesis of no effect. These results suggest a limited generalizability of the original findings-for example, due to limitations that are inherent in the proposed study design or due to cultural differences.

Action-effect binding and agency.

The sense of agency is a pervasive phenomenon that accompanies conscious acting and extends to the consequences of one's actions in the environment. Subjective feelings of agency are typically explained in terms of predictive processes, based on internal forward models inherent to the sensorimotor system, and postdictive processes, i.e., explicit, retrospective judgments by the agent. Only recently, research has begun to elucidate the link between sense of agency and more basic processes of human action control. The present study was conducted in this spirit and explored the relation between short-term action-effect binding and explicit agency judgments. We found evidence for such a link in that the participants' short-term action-effect binding predicted subsequent agency ratings. This offers a new perspective on the sense of agency, providing an additional mechanism (together with predictive and postdictive processes) that may underlie its formation.

The Age Effects on the Cognitive Processes of Intention-Based and Stimulus-Based Actions: An ERP Study.

The functional decline in action among older adults is caused not only by physical weakness but also by cognitive decline. In this study, we aimed to compare the cognitive effects of age between intention-based and stimulus-based action modes electrophysiologically. Because age-related declines in cognitive function might proceed distinctly according to specific action modes and processes, four specific cognitive processes, action-effect binding, stimulus-response linkage, action-effect feedback control, and effect-action retrieval, were investigated. We recorded event-related potentials (ERPs) during a modified acquisition-test paradigm in young (mean age = 21, SD = 2) and old (mean age = 69, SD = 5) groups. A temporal bisection task and a movement pre-cuing task were used during the acquisition and test phases, respectively. Using ERP indices including readiness potential (RP), P3, N2 and contingent negative variation (CNV) to identify these four specific processes for the two action modes, we revealed the effects of age on each ERP index. The results showed similar patterns of waveforms but consistently decreasing amplitudes of all four ERP indices in the old age group compared with the young age group, which indicates not only generally declining functions of action preparation in older adults but also age effects specific to the action modes and processes that might otherwise be mixed together under confounding experimental conditions. Particularly, an interference effect indexed by the differences in the amplitudes of CNV between congruent and incongruent tasks was observed in the young age group, which is consistent with previous behavioral reports. However, this effect was absent in the old age group, indicating a specific age-related deficit in the effect-action retrieval process of intention-based action, which might be caused by an age-related deficit in associative memory. In sum, this study investigated the cognitive processes of two action modes from a developmental perspective and suggests the importance of adding associative memory training to interventions for older adults with the aim of improving intention-based action.

A HTML5 open source tool to conduct studies based on Libet's clock paradigm.

Libet’s clock is a well-known procedure in experiments in psychology and neuroscience. Examples of its use include experiments exploring the subjective sense of agency, action-effect binding, and subjective timing of conscious decisions and perceptions. However, the technical details of the apparatus used to conduct these types of experiments are complex, and are rarely explained in sufficient detail as to guarantee an exact replication of the procedure. With this in mind, we developed Labclock Web, a web tool designed to conduct online and offline experiments using Libet’s clock. After describing its technical features, we explain how to configure specific experiments using this tool. Its degree of accuracy and precision in the presentation of stimuli has been technically validated, including the use of two cognitive experiments conducted with voluntary participants who performed the experiment both in our laboratory and via the Internet. Labclock Web is distributed without charge under a free software license (GPLv3) since one of our main objectives is to facilitate the replication of experiments and hence the advancement of knowledge in this area.

Spatial action-effect binding.

The temporal interval between an action and its ensuing effect is perceptually compressed. Specifically, the perceived onset of actions is shifted towards their effects in time and, vice versa, the perceived onset of effects is shifted towards their causing actions. In four experiments, we report evidence showing that action-effect binding also occurs in the spatial domain. Participants controlled the location of a visual stimulus by performing stylus movements before they judged either the position of the stylus or the position of the visual stimulus. The results yielded spatial binding between the perceived stylus position and the perceived stimulus position when the stimulus was under full control of the hand movement compared to control conditions without direct control.

Intentional Binding is independent of the validity of the action effect's identity

When an action produces an effect, both events are perceived to be shifted in time toward each other. This shift is called Intentional Binding (IB) effect. First evidence shows that this shift does not depend on the statistical predictability of the produced effect's identity (Desantis, Hughes, & Waszak, 2012). We confirm this result by comparing the perceived duration of action–effect intervals before valid and invalid action effects using the method of constant stimuli. The perceived duration of action–effect intervals did not differ for valid and invalid effects. This result was true for different durations of the action–effect interval (Experiments 1–4: 250ms, Experiments 1 & 2: 400ms), different effect modalities (Experiments 1 & 3: visual, Experiments 2–4: auditive), and two types of validity variations (Experiments 1 & 2: 80% valid, Experiments 3 & 4: 100% valid vs. random). We validated our results by using a clock paradigm and a numerical duration estimation task (Experiment 4). We conclude that the IB effect is not the result of internal prediction due to action–effect bindings, but might rely on higher-order processes.

From outcome prediction to action selection: developmental change in the role of action–effect bindings

Ideomotor theory considers bidirectional action-effect associations to be the fundamental building blocks for intentional action. The present study employed a novel pupillometric and oculomotor paradigm to study developmental changes in the role of action-effects in the acquisition of voluntary action. Our findings suggest that both 7- and 12-month-olds (and adults) can use acquired action-effect bindings to predict action outcomes but only 12-month-olds (and adults) showed evidence for employing action-effects to select actions. This dissociation supports the idea that infants acquire action-effect knowledge before they have developed the cognitive machinery necessary to make use of that knowledge to perform intentional actions.

Intersubjective action-effect binding: Eye contact modulates acquisition of bidirectional association between our and others’ actions

In everyday social life, we predict others’ actions in response to our own actions. Subsequently, on the basis of these predictions, we control our actions to attain desired social outcomes and/or adjust our actions to accommodate the anticipated actions of the others. Representation of the bidirectional association between our and others’ actions, that is, intersubjective action-effect binding, could make such intersubjective action control easier and smoother. The present study investigated not only whether or not intersubjective action-effect binding was acquired but also whether or not eye contact modulated it. Experiment 1 showed that after a repeated experience during which participants’ finger movements triggered a target female individual’s mouth gesture, observing the target’s mouth gestures came to automatically trigger the participants’ finger movements. Experiments 2 and 3 revealed that this effect was not observed when the target’s gaze direction was averted (Experiment 2) or when the target’s eyes were closed (Experiment 3) throughout the acquisition phase. These results indicate that intersubjective action-effect binding occurs and that an ostensive signal, that is, eye contact modulates it.

Action‐effect binding is decreased in motor conversion disorder: Implications for sense of agency

The abnormal movements seen in motor conversion disorder are affected by distraction and entrainment, similar to voluntary movement. Unlike voluntary movement, however, patients lack a sense of control for the abnormal movements, a failure of "self-agency." The action-effect binding paradigm has been used to quantify the sense of self-agency, because subjective contraction of time between an action and its effect only occurs if the patient feels that they are the agent responsible for the action. We used this paradigm, coupled with emotional stimuli, to investigate the sense of agency with voluntary movements in patients with motor conversion disorder. Twenty patients with motor conversion disorder and 20 age-matched and sex-matched healthy volunteers used a rotating clock to judge the time of their own voluntary key presses (action) and a subsequent auditory tone (effect) after they completed conditioning blocks in which high, medium, and low tones were coupled to images of happy, fearful, and neutral faces. The results replicated those produced previously: it was reported that an effect after a voluntary action occurred earlier, and the preceding action occurred later, compared with trials that used only key presses or tones. Patients had reduced overall binding scores relative to healthy volunteers, suggesting a reduced sense of agency. There was no effect of the emotional stimuli (faces) or other interaction effects. Healthy volunteers with subclinical depressive symptoms had higher overall binding scores. We demonstrate that patients with motor conversion disorder have decreased action-effect binding for normal voluntary movements compared with healthy volunteers, consistent with the greater experience of lack of control.

Instant attraction: immediate action-effect bindings occur for both, stimulus- and goal-driven actions.

Flexible behavior is only possible if contingencies between own actions and following environmental effects are acquired as quickly as possible; and recent findings indeed point toward an immediate formation of action-effect bindings already after a single coupling of an action and its effect. The present study explored whether these short-term bindings occur for both, stimulus- and goal-driven actions (“forced-choice actions” vs. “free-choice actions”). Two experiments confirmed that immediate action-effect bindings are formed for both types of actions and affect upcoming behavior. These findings support the view that action-effect binding is a ubiquitous phenomenon which occurs for any type of action.

Action-Effect Bindings and Ideomotor Learning in Intention- and Stimulus-Based Actions

According to ideomotor theory, action-effect associations are crucial for voluntary action control. Recently, a number of studies started to investigate the conditions that mediate the acquisition and application of action-effect associations by comparing actions carried out in response to exogenous stimuli (stimulus-based) with actions selected endogenously (intention-based). There is evidence that the acquisition and/or application of action-effect associations is boosted when acting in an intention-based action mode. For instance, bidirectional action-effect associations were diagnosed in a forced choice test phase if participants previously experienced action-effect couplings in an intention-based but not in a stimulus-based action mode. The present study aims at investigating effects of the action mode on action-effect associations in more detail. In a series of experiments, we compared the strength and durability of short-term action-effect associations (binding) immediately following intention- as well as stimulus-based actions. Moreover, long-term action-effect associations (learning) were assessed in a subsequent test phase. Our results show short-term action-effect associations of equal strength and durability for both action modes. However, replicating previous results, long-term associations were observed only following intention-based actions. These findings indicate that the effect of the action mode on long-term associations cannot merely be a result of accumulated short-term action-effect bindings. Instead, only those episodic bindings are selectively perpetuated and retrieved that integrate action-relevant aspects of the processing event, i.e., in case of intention-based actions, the link between action and ensuing effect.

On the Influence of Reward on Action-Effect Binding

Ideomotor theory states that the formation of anticipatory representations about the perceptual consequences of an action [i.e., action-effect (A-E) binding] provides the functional basis of voluntary action control. A host of studies have demonstrated that A-E binding occurs fast and effortlessly, yet little is known about cognitive and affective factors that influence this learning process. In the present study, we sought to test whether the motivational value of an action modulates the acquisition of A-E associations. To this end, we linked specific actions with monetary incentives during the acquisition of novel A-E mappings. In a subsequent test phase, the degree of binding was assessed by presenting the former effect stimuli as task-irrelevant response primes in a forced-choice response task, absent reward. Binding, as indexed by response priming through the former action-effects, was only found for reward-related A-E mappings. Moreover, the degree to which reward associations modulated the binding strength was predicted by individuals’ trait sensitivity to reward. These observations indicate that the association of actions and their immediate outcomes depends on the motivational value of the action during learning, as well as on the motivational disposition of the individual. On a larger scale, these findings also highlight the link between ideomotor theories and reinforcement-learning theories, providing an interesting perspective for future research on anticipatory regulation of behavior.

Action-effect binding by observational learning

The acquisition of bidirectional action–effect associations plays a central role in the ability to intentionally control actions. Humans learn about actions not only through active experience, but also through observing the actions of others. In Experiment 1, we examined whether action–effect associations can be acquired by observational learning. To this end, participants observed how a model repeatedly pressed two buttons during an observation phase. Each of the buttonpresses led to a specific tone (action effect). In a subsequent test phase, the tones served as target stimuli to which the participants had to respond with buttonpresses. Reaction times were shorter if the stimulus–response mapping in the test phase was compatible with the action–effect association in the observation phase. Experiment 2 excluded the possibility that the impact of perceived action effects on own actions was driven merely by an association of spatial features with the particular tones. Furthermore, we demonstrated that the presence of an agent is necessary to acquire novel action–effect associations through observation. Altogether, the study provides evidence for the claim that bidirectional action–effect associations can be acquired by observational learning. Our findings are discussed in the context of the idea that the acquisition of action–effect associations through observation is an important cognitive mechanism subserving the human ability for social learning.

How learning to shake a rattle affects 8-month-old infants’ perception of the rattle's sound: Electrophysiological evidence for action-effect binding in infancy

Bidirectional action-effect associations play a fundamental role in intentional action control and the development of the mirror neuron system. However, it has been questioned if infants are able to acquire bidirectional action-effect associations (i.e., are able to intentionally control their actions). To investigate this, we trained 8-month-old infants for one week to use a novel rattle that produced a specific sound when shaken. Infants were also presented with another sound, which was not related to an action. Thereafter, infants’ EEG responses to these two sounds and to an additional, unfamiliar sound were recorded. Infants displayed a stronger mu-desynchronization above cortical motor sites (i.e., motor resonance) when listening to the action-related sound than when hearing other sounds. Our results provide therefore electrophysiological evidence that infants as young as 8 months are able to acquire bidirectional action-effect associations and parallel findings of audiovisual mirror neurons in the monkey brain.

Ketamine administration in healthy volunteers reproduces aberrant agency experiences associated with schizophrenia

Introduction. Aberrant experience of agency is characteristic of schizophrenia. An understanding of the neurobiological basis of such experience is therefore of considerable importance for developing successful models of the disease. We aimed to characterise the effects of ketamine, a drug model for psychosis, on sense of agency (SoA). SoA is associated with a subjective compression of the temporal interval between an action and its effects: This is known as “intentional binding”. This action–effect binding provides an indirect measure of SoA. Previous research has found that the magnitude of binding is exaggerated in patients with schizophrenia. We therefore investigated whether ketamine administration to otherwise healthy adults induced a similar pattern of binding. Methods. 14 right-handed healthy participants (8 female; mean age 22.4 years) were given low-dose ketamine (100 ng/mL plasma) and completed the binding task. They also underwent structured clinical interviews. Results. Ketamine mimicked the performance of schizophrenia patients on the intentional binding task, significantly increasing binding relative to placebo. The size of this effect also correlated with aberrant bodily experiences engendered by the drug. Conclusions. These data suggest that ketamine may be able to mimic certain aberrant agency experiences that characterise schizophrenia. The link to individual changes in bodily experience suggests that the fundamental change produced by the drug has wider consequences in terms of individuals' experiences of their bodies and movements.

Where do action goals come from? Evidence for spontaneous action-effect binding in infants.

One of the great questions in psychology concerns how we develop to become intentional agents. Ideomotor theory suggests that intentional actions depend on, and emerge from the automatic acquisition of bidirectional action–effect associations: perceiving an action–effect sequence creates an integrated representation that can be employed for action control in the opposite order, selecting an action by anticipating its effect. We provide first evidence for the spontaneous acquisition of bidirectional action–effect associations in 9- 12-, and 18-month-olds, suggesting that the mechanism underlying action–effect integration is in place at the latest around 9 months of age.