How many categories are there in crossmodal correspondences? A study based on exploratory factor analysis.

Abstract

Humans naturally associate stimulus features of one sensory modality with those of other modalities, such as associating bright light with high-pitched tones. This phenomenon is called crossmodal correspondence and is found between various stimulus features, and has been suggested to be categorized into several types. However, it is not yet clear whether there are differences in the underlying mechanism between the different kinds of correspondences. This study used exploratory factor analysis to address this question. Through an online experiment platform, we asked Japanese adult participants (Experiment 1: N = 178, Experiment 2: N = 160) to rate the degree of correspondence between two auditory and five visual features. The results of two experiments revealed that two factors underlie the subjective judgments of the audiovisual crossmodal correspondences: One factor was composed of correspondences whose auditory and visual features can be expressed in common Japanese terms, such as the loudness-size and pitch-vertical position correspondences, and another factor was composed of correspondences whose features have no linguistic similarities, such as pitch-brightness and pitch-shape correspondences. These results confirm that there are at least two types of crossmodal correspondences that are likely to differ in terms of language mediation.