Abstract
To date, few studies have focused on the behavioural differences between the learning of multisensory auditory-visual and intra-modal associations. More specifically, the relative benefits of novel auditory-visual and verbal-visual associations for learning have not been directly compared. In Experiment 1, 20 adult volunteers completed three paired associate learning tasks: non-verbal novel auditory-visual (novel-AV), verbal-visual (verbal-AV; using pseudowords), and visual-visual (shape-VV). Participants were directed to make a motor response to matching novel and arbitrarily related stimulus pairs. Feedback was provided to facilitate trial and error learning. The results of Signal Detection Theory analyses suggested a multisensory enhancement of learning, with significantly higher discriminability measures (d-prime) in both the novel-AV and verbal-AV tasks than the shape-VV task. Motor reaction times were also significantly faster during the verbal-AV task than during the non-verbal learning tasks. Experiment 2 (n = 12) used a forced-choice discrimination paradigm to assess whether a difference in unisensory stimulus discriminability could account for the learning trends in Experiment 1. Participants were significantly slower at discriminating unisensory pseudowords than the novel sounds and visual shapes, which was notable given that these stimuli produced superior learning. Together the findings suggest that verbal information has an added enhancing effect on multisensory associative learning in adults.
Highlights
Effective perception of our everyday environment requires that specific associations are learnt between the multiple representations of objects and events that occur across different sensory modalities
The results of Experiment 1 demonstrated a significant advantage for the learning of both verbal and non-verbal auditory-visual stimulus associations over intra-modal visual-visual associations
A follow-up experiment evaluated whether unequal unisensory stimulus discriminability could explain differences in learning trends between the verbal-AV, novel-AV and shape-VV tasks
Summary
Effective perception of our everyday environment requires that specific associations are learnt between the multiple representations of objects and events that occur across different sensory modalities. Such multisensory associative learning, between the auditory and visual modalities, plays an important role in many social and cognitive processes, including object identification as well as lexical and semantic processing[1,2,3]. Barutchu et al.[6] recently demonstrated that superior auditory-visual multisensory abilities were associated with above average general intellectual abilities in children. Not unexpectedly, learning by experience and prior knowledge have been shown to influence multisensory processes at both a behavioural and neural level[1,13,14,15,16,17,18,19]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
