Colour–Touch Cross-Modal Correspondence and Its Impact on Single-Modal Judgement in Multimodal Perception

Abstract

Abstract This study explored the colour–touch cross-modal correspondence and its impact on colour–touch multisensory perception. Two laboratory experiments were conducted based on a pre-experiment. In the first experiment, participants chose the colour according to their tactile sense against the vibration generated by the smartphone simulator. A positive cross-modal correspondence was obtained between the vibration amplitude and the colour chroma, indicating the existence of a colour–touch correspondence. The correspondence between vibration parameters and colour hue was preliminarily explored. The impact of the correspondence on the multisensory senses was investigated in the second experiment. Different stimulus combinations were played on a smartphone. The participants were asked to choose the perceived stimulus level. The data illustrated that a better correspondence improved judgement accuracy and reduced reactiontime. Overall, this study offers evidence for colour–touch cross-modal correspondence and presents its potential influence on the single-modal judgement in multimodal perception. The results can be applied in further cross-modal studies, assisting barrier-free society construction and multimodal interface design.