Multisensory integration for expert musicians investigated using eye tracking

Abstract

Musical sight-reading requires to process simultaneously multimodal information: visual, auditory, motor. Does the expertise in music rely on an efficient cross-modal integration? This talk investigates this issue with 2 experiments. In the first one, 30 expert and 31 non-expert musicians were required to report whether two successively presented fragments of classical music were same or different. In half the conditions the 61 participants received the fragments in the same modal presentation (visual/visual), in the other half they received the fragments in cross-modal presentation (auditory/visual). Analysis of Response Time and Errors showed that more experienced musicians seemed to be better able to transfer the information from one modality to another. In a second experiment using eye-tracking, 64 participants, 26 expert and 38 non-expert musicians were required also to report whether two successively presented fragments of classical music were same or different but in cross-modal way only. Visual and auditory cues were used to investigate whether a kind of expert memory using retrieval cues was acting for more expert musicians. An accent mark, emphasis placed on a note contributing to the prosody of the musical phrase, was put in a congruent or incongruent way, during the auditory and reading phases. As expected, the analysis of fixations and mistakes validated the hypothesis of modal independence for expert musicians, observed in the first experiment. Moreover analyses validated the cross-modal ability of expert memory, using accent marks as retrieval cues. Results are discussed in terms of amodal memory for expert musicians that can be in support of theoretical work by Ericsson and Kintsch (1995): more experienced performers better integrate knowledge across modalities using retrieval cues.