Visual adaptation dominates bimodal visual-motor action adaptation.

Stephan De La Rosa,Ylva Ferstl,Heinrich H Bülthoff

doi:10.1038/srep23829

Stephan De La Rosa, Ylva Ferstl + Show 1 more

Open Access

https://doi.org/10.1038/srep23829

Copy DOI

Abstract

A long standing debate revolves around the question whether visual action recognition primarily relies on visual or motor action information. Previous studies mainly examined the contribution of either visual or motor information to action recognition. Yet, the interaction of visual and motor action information is particularly important for understanding action recognition in social interactions, where humans often observe and execute actions at the same time. Here, we behaviourally examined the interaction of visual and motor action recognition processes when participants simultaneously observe and execute actions. We took advantage of behavioural action adaptation effects to investigate behavioural correlates of neural action recognition mechanisms. In line with previous results, we find that prolonged visual exposure (visual adaptation) and prolonged execution of the same action with closed eyes (non-visual motor adaptation) influence action recognition. However, when participants simultaneously adapted visually and motorically – akin to simultaneous execution and observation of actions in social interactions - adaptation effects were only modulated by visual but not motor adaptation. Action recognition, therefore, relies primarily on vision-based action recognition mechanisms in situations that require simultaneous action observation and execution, such as social interactions. The results suggest caution when associating social behaviour in social interactions with motor based information.

Highlights

How do humans visually recognize the actions of others? Two physiologically plausible accounts have been proposed to explain visual action recognition
The behavioural adaptation paradigm is able to measure the effect of motor and vision-based recognition processes on visual action recognition
The results indicate that social action categorization is influenced by motor adaptation supporting the idea that the motor system affects action recognition when participants merely observe an action

Summary

Introduction

How do humans visually recognize the actions of others? Two physiologically plausible accounts have been proposed to explain visual action recognition. Much of the evidence for either account has been demonstrated by showing that either visual or motor action information alone influence action recognition[16,17,18,19,20]. This is in contrast to action recognition in social interactions, where humans simultaneously execute and observe actions. We investigated the interaction of vision and motor-based action recognition processes using a behavioural adaptation paradigm. Participants adapted to a ‘fist bump’ or ‘punch’ action in separate conditions and subsequently categorized an ambiguous action as either a ‘punch’ or ‘fist bump’ (test) (Fig. 1B–D). This dynamic ambiguous test was a video of a morphed action that displayed a weighted linear average of limb joint angles of a punch and a fist bump action

Methods

Results

Conclusion