AbstractReview of psychological and language acquisition research into seeing faces while listening, seeing gesture while listening, illustrated text, reading while listening, and same language subtitled video, confirms that bi-modal input has a consistently positive effect on language learning over a variety of input types. This effect is normally discussed using a simple additive model where bi-modal input increases the total amount of data and adds redundancy to duplicated input thus increasing comprehension and then learning. Parallel studies in neuroscience suggest that bi-modal integration is a general effect using common brain areas and following common neural paths. Neuroscience also shows that bi-modal effects are more complex than simple addition, showing early integration of inputs, a learning/developmental effect, and a superadditive effect for integrated bi-modal input. The different bodies of research produce a revised model of bi-modal input as a learned, active system. The implications for language learning are that bi- or multi-modal input can powerfully enhance language learning and that the learning benefits of such input will increase alongside the development of neurological integration of the inputs.
Read full abstract