Learned audio-visual cross-modal associations in observed piano playing activate the left planum temporale. An fMRI study

Abstract

Lip reading is known to activate the planum temporale (PT), a brain region which may integrate visual and auditory information. To find out whether other types of learned audio-visual integration occur in the PT, we investigated “key-touch reading” using functional magnetic resonance imaging (fMRI). As well-trained pianists are able to identify pieces of music by watching the key-touching movements of the hands, we hypothesised that the visual information of observed sequential finger movements is transformed into the auditory modality during “key-touch reading” as is the case during lip reading. We therefore predicted activation of the PT during key-touch reading. Twenty-six healthy right-handed volunteers were recruited for fMRI. Of these, 7 subjects had never experienced piano training (naı̈ve group), 10 had a little experience of piano playing (less trained group), and the remaining 9 had been trained for more than 8 years (well trained group). During task periods, subjects were required to view the bimanual hand movements of a piano player making key presses. During control periods, subjects viewed the same hands sliding from side to side without tapping movements of the fingers. No sound was provided. Sequences of key presses during task periods consisted of pieces of familiar music, unfamiliar music, or random sequences. Well-trained subjects were able to identify the familiar music, whereas less-trained subjects were not. The left PT of the well-trained subjects was equally activated by observation of familiar music, unfamiliar music, and random sequences. The naı̈ve and less trained groups did not show activation of the left PT during any of the tasks. These results suggest that PT activation reflects a learned process. As the activation was elicited by viewing key pressing actions regardless of whether they constituted a piece of music, the PT may be involved in processes that occur prior to the identification of a piece of music, that is, mapping the complex sequence structure of hand movements onto the sequence of sounds.