Colour/shape-taste correspondences across three languages in ChatGPT

Abstract

Crossmodal correspondences, the tendency for a sensory feature / attribute in one sensory modality (either physically present or merely imagined), to be associated with a sensory feature in another sensory modality, have been studied extensively, revealing consistent patterns, such as sweet tastes being associated with pink colours and round shapes across languages. The present research explores whether such correspondences are captured by ChatGPT, a large language model developed by OpenAI. Across twelve studies, this research investigates colour/shapes-taste crossmodal correspondences in ChatGPT-3.5 and -4o, focusing on associations between shapes/colours and the five basic tastes across three languages (English, Japanese, and Spanish). Studies 1A-F examined taste-shape associations, using prompts in three languages to assess ChatGPT's association of round and angular shapes with the five basic tastes. The results indicated significant, consistent, associations between shape and taste, with, for example, round shapes strongly associated with sweet/umami tastes and angular shapes with bitter/salty/sour tastes. The magnitude of shape-taste matching appears to be greater in ChatGPT-4o than in ChatGPT-3.5, and ChatGPT prompted in English and Spanish than ChatGPT prompted in Japanese. Studies 2A-F focused on colour-taste correspondences, using ChatGPT to assess associations between eleven colours and the five basic tastes. The results indicated that ChatGPT-4o, but not ChatGPT-3.5, generally replicates the patterns of colour-taste correspondences that have previously been observed in human participants. Specifically, ChatGPT-4o associates sweet tastes with pink, sour with yellow, salty with white/blue, bitter with black, and umami with red across languages. However, the magnitude/similarity of shape/colour-taste matching observed in ChatGPT-4o appears to be more pronounced (i.e., having little variance, large mean difference), which does not adequately reflect the subtle nuances typically seen in human shape/colour-taste correspondences. These findings suggest that ChatGPT captures colour/shapes-taste correspondences, with language- and GPT version-specific variations, albeit with some differences when compared to previous studies involving human participants. These findings contribute valuable knowledge to the field of crossmodal correspondences, explore the possibility of generative AI that resembles human perceptual systems and cognition across languages, and provide insight into the development and evolution of generative AI systems that capture human crossmodal correspondences.