Visual Action Effects Research Articles

Suppression of mutually incompatible proprioceptive and visual action effects in tool use

Movements of a tool typically diverge from the movements of the hand manipulating that tool, such as when operating a pivotal lever where tool and hand move in opposite directions. Previous studies suggest that humans are often unaware of the position or movements of their effective body part (mostly the hand) in such situations. It has been suggested that this might be due to a “haptic neglect” of bodily sensations to decrease the interference of representations of body and tool movements. However, in principle this interference could also be decreased by neglecting sensations regarding the tool and focusing instead on body movements. While in most tool use situations the tool-related action effects are task-relevant and thus suppression of body-related rather than tool-related sensations is more beneficial for successful goal achievement, we manipulated this task-relevance in a controlled experiment. The results showed that visual, tool-related effect representations can be suppressed just as proprioceptive, body-related ones in situations where effect representations interfere, given that task-relevance of body-related effects is increased relative to tool-related ones.

Spatial\u2013numerical associations in the presence of an avatar

When we interact with other people or avatars, they often provide an alternative spatial frame of reference compared to our own. Previous studies introduced avatars into stimulus–response compatibility tasks and demonstrated compatibility effects as if the participant was viewing the task from the avatar’s point of view. However, the origin of this effect of perspective taking remained unclear. To distinguish changes in stimulus coding from changes in response coding, caused by the avatar, two experiments were conducted that combined a SNARC task and a spontaneous visual perspective taking task to specify the role of response coding. We observed compatibility effects that were based on the avatar’s perspective rather than the participants’ own. Because number magnitude was independent of the avatar’s perspective, the observed changes in compatibility caused by different perspectives indicate changes in response coding. These changes in response coding are only significant when they are accompanied by visual action effects.

Imitation of action-effects increases social affiliation

Imitating someone’s actions influences social-affective evaluations and motor performance for the action model and the imitator alike. Both phenomena are explained by the similarity between the sensory and motor representations of the action. Importantly, however, theoretical accounts of action control hold that actions are represented in terms of their sensory effects, which encompass features of the movement but also features of an action’s consequence in the outside world. This suggests that social-affective consequences of imitation should not be limited to situations in which the imitator copies the model’s body movements. Rather, the present study tested whether copying the perceived action-effects of another person without imitating the eventual body movements increases the social-affective evaluation of this person. In three experiments, participants produced visual action-effects while observing videos of models who performed either the same or a different movement and produced either the same or a different action-effect. If instructions framed the action in terms of the movement, participants preferred models with similar movements (Experiment 1). However, if instructions framed the action in terms of the to-be produced action-effect in the environment, participants preferred models with similar action-effects (Experiments 2 and 3). These results extend effect-based accounts of action control like the ideomotor framework and suggest a close link between action control and affective processing in social interactions.

Electrophysiological evidence for action-effect prediction.

A central function of the human brain is prediction. Influential theoretical views suggest that we form predictions about forthcoming sensory outcomes that follow from our own actions and that anticipation of these outcomes is fundamental for action control. However, little is known about how predicted outcomes are represented at the neural level. We hypothesized that neural activity reflecting the anticipation of outcomes should precede action initiation. Following computational theories on action control, anticipation should be reflected in an outcome-selective reduction in neural activity. To test this, the present study recorded scalp electroencephalography to show that brain activity reflects action-effect specific representations before response initiation. We harnessed frequency entrainment to tag neural activity specific for particular action-effects. Participants engendered visual action-effects that were presented in flickering frequencies of 6 or 10 Hz, either according to a fixed cue-response mapping (Experiment 1) or freely chosen (Experiment 2). Results showed that perception of action-effects entrained a steady-state visual brain response specific to the perceived action-effect at either 6 or 10 Hz. More important, neural pattern reflecting action-effect specific representations emerged even before a response was given (i.e., without any visual stimulation). In line with theoretical accounts, perception of action-effects led to a specific increase in neural activity, while prediction of action-effects led to a specific decrease in neural activity. These findings provide neural evidence for the anticipation of action-effects before response initiation, supporting major theories of action control. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Do we see it or not? Sensory attenuation in the visual domain.

Sensory consequences of an agent's actions are perceived less intensely than sensory stimuli that are not caused (and thus not predicted) by the observer. This effect of sensory attenuation has been discussed as a key principle of perception, potentially mediating various crucial functions such as agency and the discrimination of self-caused sensory stimulation from stimuli caused by external factors. Precise models describe the theoretical underpinnings of this phenomenon across a variety of modalities, especially the auditory, tactile, and visual domain. Despite these strong claims, empirical evidence for sensory attenuation in the visual domain is surprisingly sparse and ambiguous. In the present article, the authors therefore aim to clarify the role of sensory attenuation for learned visual action effects. To this end, the authors present a comprehensive replication effort including 3 separate, high-powered experiments on sensory attenuation in the visual domain with 1 direct and 2 preregistered, conceptual replication attempts of an influential study on this topic (Cardoso-Leite et al., 2010). Signal detection analyses were targeted to distinguish between true visual sensitivity and response bias. Contrary to previous assumptions and despite high statistical power, however, the authors found no evidence for sensory attenuation of learned visual action effects. Bayesian analyses further supported the null hypothesis of no effect, thus constraining theories that promote sensory attenuation as an immediate and necessary consequence of voluntary actions. (PsycINFO Database Record

Intentional binding of visual effects.

When an action produces an effect, the effect is perceived earlier in time compared to a stimulus without preceding action. This temporal bias is called intentional binding (IB) and serves as an implicit measure of sense of agency. Typically, IB is investigated by presenting a rotating clock hand while participants execute an action and perceive a resulting tone. Participants are asked to estimate the time point of tone onset by referring to the clock hand position. This timepoint estimate is compared to a timepoint estimate of a tone in a condition in which the tone occurs without preceding action. Studies employing this classic clock paradigm employed auditory action effects. We modified this paradigm to investigate potential IB of visual action effects, and, additionally, to investigate how IB differs for visual action effects (Experiment 1) in comparison to auditory action effects (Experiment 2). Our results show that, like the IB of anauditory effect, the time point of a visual action effect is shifted toward the causing action, and that the size of the IB depends on the delay durationof the effect. Comparable to auditory action effects, earlier action effects showed stronger IB compared to later action effects. Yet overall IB of the visual effects was weaker than IB of the auditory effects. As IB is seen as an indicator of sense of agency, this may have important implications for the design of human-machine interfaces.

The role of feedback delay in dual-task performance.

Doing two things at once is hard, and it is probably hard for various reasons. Here we aim to demonstrate that one so far barely considered reason is the monitoring of sensory action feedback, which detracts from processing of other concurrent tasks. To demonstrate this, we engaged participants in a psychological refractory period paradigm. The responses in the two tasks produced visual action effects. These effects occurred either immediately or they were delayed for the first of the two responses. We assumed that delaying these effects would engage a process of monitoring visual feedback longer, and delay a concurrent task more, as compared to immediate effects. This prediction was confirmed in two experiments. We discuss the reasons for feedback monitoring and its possible contribution to dual tasking.

The role of effect grouping in free-choice response selection

Which motor actions are preferred to replace an initially planned but momentary not executable action? Previous research (Khan, Mourton, Buckolz, Adams, & Hayes, 2010, Acta Psychologica) suggests that anatomical constraints seem to be a major determinant for such choices: For example, participants more frequently chose to respond with the finger homologous to the prepared one. We argue that in this case finger homology is confounded with action effect similarity, and action effects have been ascribed a crucial role in action selection. We report two experiments. Experiment 1 replicated the results obtained by Khan et al. In Experiment 2, we introduced visual action effects in the paradigm. Results from this experiment clearly point to a role of effect similarity in addition to mere finger homology status for the choice frequency effect.

Intra- and intermodal integration of discrepant visual and proprioceptive action effects

Integration of discrepant visual and proprioceptive action effects puts high demands on the human information processing system. The present study aimed to examine the integration mechanisms for the motor (Exp. 1) and visual modality (Exp. 2). According to theories of common coding, we assumed that visual as well as proprioceptive information is represented within the same cognitive domain and is therefore likely to affect each other (multisensory cross talk). Thus, apart from the often-confirmed visual dominance in multisensory integration, we asked about intra- and intermodal recall of either proprioceptive or visual information and whether there were any differences between the motor and visual modality. In a replication paradigm, we perturbed the relation between hand movements and cursor movements. The task required the (intra- vs. intermodal) replication of an initially performed (seen) hand (cursor) movement in a subsequent motor (visual) replication phase. First, mechanisms of integration were found to be dependent on the output modality. Visual action effects interfered the motor modality, but proprioceptive action effects did not have any effects on the visual modality. Second, however, intermodal integration was more susceptible to interference, and this was found to be independent from the output modality. Third, for the motor modality, the locus of perturbation (perturbation of cursor amplitude or perturbation of hand amplitude) was irrelevant, but for the visual modality, perturbation of hand amplitudes reduced the cross talk. Tool use is one field of application of these kinds of results, since the optimized integration of conflicting action effects is a precondition for using tools successfully.

ERP correlates of action effect prediction and visual sensory attenuation in voluntary action

Sensory attenuation of voluntary action effects has been widely reported in both somatosensory and auditory domains. However, relatively little research has focused on physiological measures of sensory attenuation of visual action effects. One previous study found, perhaps surprisingly, that both auditory and visual sensory attenuation were manifested as decreased ERP amplitude over the vertex. The present study aimed to extend these findings using a novel paradigm in which voluntary actions were either associated with a visual action effect or to no effect. Crucially, this allowed us to explore both sensory attenuation (by comparing ERPs to action-triggered versus externally triggered stimuli) and action effect prediction (by comparing actions that triggered a stimulus with actions that did not). With regard to sensory attenuation, we found that attenuation of cortical responses to visual action effects was manifested in a reduced activation of a frontoparietal network, from 150ms after stimulus. Differences between actions that produced an effect and those that did not were observed in lateralized motor potentials and may reflect the cortical correlates of the action effect prediction. We also observed a re-activation of lateralized motor activity following onset of the action effect, suggesting a common representation of action effects in visual and motor cortices. Taken together, these findings help to elucidate the cortical mechanisms of voluntary action as well as their sensory consequences and inform how our interaction with the external world is processed and controlled.

Letters as visual action-effects in skilled typing

In the present study, action-(visual)-effect associations in participants who type using the 10-finger-system and participants who do hunt-and-peck typing were investigated. Following responses (on either a keyboard or an external response device) to colored squares, participants were presented response corresponding or non-corresponding letters. In Experiment 1, evidence for action activation by anticipated action-effects was found. In Experiment 2, evidence that actions can activate the corresponding action-effects was obtained, but only when participants responded on a keyboard and only in the first half of the experimental trials. In both experiments, effects were only present in participants who type using the 10-finger-system. Thus, results provided evidence for bidirectional action–effect associations in 10-finger-system typists. Anticipated visual action-effects are important for action selection in expert typists. Visual action-effects also have a function for performance monitoring, but only in an appropriate context. The functional use of existing action–effect associations is flexible.