How well do Humans Capture the Sounds of Speech in Writing?

Abstract

A large body of research on connections between sensory modalities has shown that deep connections exist between sound and vision, such that people have a tendency to associate certain sounds with certain visual properties, including line-drawn shapes. While recognising the role of written language in audio-visual associations, previous research has largely considered written language a potential source of bias rather than a means of gaining deeper in sights into underlying audio-visual associations. We looked to ancient and unfamiliar writing systems spanning recorded human history, to explore whether humans have tried to encode certain characteristics of speech sounds in the letters they created to represent them. Our findings have revealed that modern humans can correctly identify unfamiliar letters at levels higher than would be obtained by chance, and that scripts which encode a particular sound with a particular set of visual characteristics tend to have more correct guesses. This suggests that humans share certain correspondences between sound and sight, which transcend both geographical space and historical time. The present multisensory demonstration aims to provide an interactive experience of the powerful connection between sounds and written letters through a series of activities integrating vision, audition, touch and imagination.