'Play the fragrance': Designing musical soundscapes to match fragrances based on olfactory-auditory crossmodal correspondences.

This article investigated whether crossmodal correspondence, as a sensory translation phenomenon, can mediate crossmodal transfer from visual to auditory stimuli in category learning and whether multimodal category learning can influence the crossmodal correspondence between auditory and visual stimuli. Experiment 1 showed that the category knowledge acquired from elevation stimuli affected the categorization of pitch stimuli when there were robust crossmodal correspondence effects between elevation and size, indicating that crossmodal transfer occurred between elevation and pitch stimuli. Experiments 2 and 3 revealed that the size category knowledge could not be transferred to the categorization of pitches, but interestingly, size and pitch category learning determined the direction of the pitch-size correspondence, suggesting that the pitch-size correspondence was not stable and could be determined using multimodal category learning. Experiment 4 provided further evidence that there was no crossmodal transfer between size and pitch, due to the absence of a robust pitch-size correspondence. These results demonstrated that crossmodal transfer can occur between audio-visual stimuli with crossmodal correspondence, and multisensory category learning can change the corresponding relationship between audio-visual stimuli. These findings suggest that crossmodal transfer and crossmodal correspondence share similar abstract representations, which can be mediated by semantic content such as category labels.

Odour-sound correspondences provide some of the most fascinating and intriguing examples of crossmodal associations, in part, because it is unclear from where exactly they originate. Although frequently used as similes, or figures of speech, in both literature and poetry, such smell-sound correspondences have recently started to attract the attention of experimental researchers too. To date, the findings clearly demonstrate that the majority of non-synaesthetic individuals associate orthonasally-presented odours with various different sound properties, e.g., pitch, instrument type, and timbre, in a non-random manner. What is more, these auditory-olfactory associations exhibit a number of features that are common to other crossmodal associations, such as their consistency over time, and their bidirectionality. However, the psychological mechanism(s) underpinning these associations in the general population remain(s) unclear, with a number of distinct hypotheses having been put forward over the years. In this chapter, we consider auditory-olfactory associations in art and science, focusing on poetry, music composition, and performance. First, we provide examples of the use of auditory-olfactory synaesthetic metaphors in poetry, from William Shakespeare through to Romanticism, illustrating how crossmodal associations have appeared in literature for centuries. Then, we move on to focus on music composition and performance, describing a number of examples where auditory stimuli have been purposely matched with crossmodally corresponding olfactory and/or visual stimuli. Considering the scientific study of smell-sound correspondences, we review the key psychophysical studies demonstrating that, beyond the artistic context, robust and non-random crossmodal associations are also triggered in the majority of the general population. We discuss a number of hypotheses that have been put forward over the years to account for such associations, focusing on the idea that certain correspondences between olfactory and auditory stimuli are mediated by the emotional character of the component stimuli.KeywordsSensory perceptionSynaesthesiaOrthonasal olfactionAuditionCrossmodal correspondences

The aim of this study was to investigate the physiological effects on brain activity and autonomic nervous activity of forest-related visual, olfactory, and combined visual and olfactory stimuli. Twenty-one female Japanese university students (age, 21.1 ± 1.0 years) participated. In a soundproofed chamber with an artificial climate, each participant was presented for 90 s with the following conditions: an image of a forest landscape of Hinoki cypress trees with no odor (visual stimulus), a gray image with Hinoki cypress leaf essential oil (olfactory stimulus), an image of a forest landscape of Hinoki cypress trees with Hinoki cypress leaf essential oil (combined stimulus), and a gray image with no odor (control). As an indicator of brain activity, oxyhemoglobin concentrations were measured in the left and right prefrontal cortices using near-infrared time-resolved spectroscopy. Heart rate variability and heart rate were used as indicators of autonomic nervous activity. The high-frequency component of heart rate variability, which reflects parasympathetic nervous activity, and the low-frequency/high-frequency ratio, which reflects sympathetic nervous activity, were evaluated. The following results were obtained in comparison with the control conditions: (1) the combined stimuli resulted in significantly decreased oxyhemoglobin concentrations in the left and right prefrontal cortices; (2) the olfactory stimulus resulted in significantly decreased oxyhemoglobin concentration in the right prefrontal cortex; and (3) the visual stimulus resulted in significantly decreased sympathetic nervous activity related to arousal or situations of stress. Results of a questionnaire indicated that these forest-related stimuli significantly increased the participants’ feelings of “comfortable” and “relaxed,” with the visual and combined stimuli significantly increasing feelings related to the terms “natural” and “realistic.” In conclusion, forest-related visual, olfactory, and combined visual and olfactory stimuli induced physiological and psychological relaxation effects, and the combined visual and olfactory stimuli exhibited an additive effect.

Cross-modal correspondences refer to associations between apparently unrelated stimulus features in different senses. For example, high and low auditory pitches are associated with high and low visual elevations, respectively. Here we examined how this crossmodal correspondence between visual elevation and auditory pitch relates to auditory elevation. We used audiovisual combinations of highor low-frequency bursts of white noise and a visual stimulus comprising a white circle. Auditory and visual stimuli could each occur at high or low elevations. These multisensory stimuli could be congruent or incongruent for three correspondence types: cross-modal featural (auditory pitch/visual elevation), within-modal featural (auditory pitch/auditory elevation) and cross-modal spatial (auditory and visual elevation). Participants performed a 2AFC speeded classification (high or low) task while attending to auditory pitch, auditory elevation, or visual elevation. We tested for modulatory interactions between the three correspondence types. Modulatory interactions were absent when discriminating visual elevation. However, the within-modal featural correspondence affected the cross-modal featural correspondence during discrimination of auditory elevation and pitch, while the reverse modulation was observed only during discrimination of auditory pitch. The cross-modal spatial correspondence modulated the other two correspondences only when auditory elevation was being attended, was modulated by the cross-modal featural correspondence only during attention to auditory pitch, and was modulated by the within-modal featural correspondence while performing discrimination of either auditory elevation or pitch.We conclude that the cross-modal correspondence between auditory pitch and visual elevation interacts strongly with auditory elevation.

Electrophysiological evidence of crossmodal correspondence between auditory pitch and visual elevation affecting inhibition of return

Crossmodal correspondences have been widely demonstrated, although mechanisms that stand behind the phenomenon have not been fully established yet. According to the Evaluative similarity hypothesis crossmodal correspondences are influenced by evaluative (affective) similarity of stimuli from different sensory modalities (Jankovic, 2010, Journal of Vision 10(7), 859). From this view, detection of similar evaluative information in stimulation from different sensory modalities facilitates crossmodal correspondences and multisensory integration. The aim of this study was to explore the evaluative similarity hypothesis of crossmodal correspondences in children. In Experiment 1 two groups of participants (nine- and thirteen-year-olds) were asked to make explicit matches between presented auditory stimuli (1 s long sound clips) and abstract visual patterns. In Experiment 2 the same participants judged abstract visual patterns and auditory stimuli on the set of evaluative attributes measuring affective valence and arousal. The results showed that crossmodal correspondences are mostly influenced by evaluative similarity of visual and auditory stimuli in both age groups. The most frequently matched were visual and auditory stimuli congruent in both valence and arousal, followed by stimuli congruent in valence, and finally stimuli congruent in arousal. Evaluatively incongruent stimuli demonstrated low crossmodal associations especially in older group.

Past research on crossmodal correspondences as they relate to tactile perception has largely been restricted to solid substances. We investigated the role of haptically explored liquid viscosity in crossmodal correspondences with visually presented luminance, saturation, roundedness, size, number and visual elevation, as well as pure-tone pitch and kiki–bouba-type letter strings. In Experiment 1, we presented two tactile and two visual or auditory stimuli simultaneously, and found significant inter-participant agreement () when pairing viscosity with luminance, saturation, roundedness, size, pitch and letter string type. To assess whether these crossmodal correspondences were relative or absolute, another 32 participants were presented, in Experiment 2, with two tactile stimuli but only one visual/auditory stimulus per trial. In this second experiment, we found that high viscosity was paired with low luminance, roundness, low saturation, and the bouba-type letter string, while low viscosity was paired with high pitch. However, the inverse associations (e.g. low viscosity with high luminance, high viscosity with low pitch) were not significant. These findings indicate that viscosity can be added to the list of dimensions that invoke crossmodal correspondences, and that the majority of crossmodal correspondences involving viscosity are absolute rather than relative, since they appear without explicit comparisons along the visual/auditory dimensions we measured.

This research investigates crossmodal correspondences between auditory stimuli, specifically musical modes, and olfactory mental imagery, represented by fragrance families. Building on the emerging literature on crossmodal correspondences, this research explores different mechanisms that might help to explain these crossmodal correspondences such as their shared connotative meaning and identity-based meaning. The first study evaluated the fragrance families and subfamilies and musical modes and assessed potential mechanisms behind these associations. The second study examined the associations between the musical modes and fragrance families and subfamilies through a matching task. The results revealed consistent matches between different musical modes and corresponding fragrance families and subfamilies, indicating a crossmodal association between auditory and olfactory mental imagery. What is more, major modes were perceived as brighter and less intense, and were more liked than minor modes, with floral and fresh fragrances similarly rated as brighter and more liked than oriental and woody fragrances. These results suggest that crossmodal correspondences between auditory and olfactory stimuli are influenced by brightness, intensity, and hedonic factors. Understanding such crossmodal associations can potentially benefit various fields, including marketing, product design, and those interested in creating multisensory experiences.

Crossmodal correspondences: Assessing shape symbolism for cheese

As a core environmental setting in students’ daily lives and studies, campus green spaces have a direct impact on cognitive performance and psychological well-being. Grounded in Attention Restoration Theory, this study investigates the interactive effects of olfactory–visual stimuli on college students’ perceptual evaluation of campus green spaces and their restorative experiences. The experimental site of this study is Nanjing Institute of Technology in East China, where we employ systematic measurement methods to analyze the interactive psychological impact of olfactory and visual stimuli. Using rosemary and camphor leaves as representative aromatic plant substances, a comprehensive assessment of students’ emotional states and environmental perceptions is conducted through eye tracking, blood pressure monitoring, and attention tests. Finally, based on the empirical findings, corresponding strategies to improve multisensory landscape design in campus green spaces are proposed. The results indicate that the visual system is primarily responsible for spatial structure recognition, while the olfactory system is more involved in emotional regulation and atmosphere creation. Plant aromas directly influence physiological and emotional states, and modulate visual landscape evaluations through cross-modal mechanisms, thereby enhancing restorative effects compared to visual-only designs. This study validates the value of integrating environmental perception into campus landscape design, transcending the limitations of traditional single-sensory evaluation models. The findings contribute to the development of campus environment systems oriented toward physical and mental well-being.

We report two experiments designed to investigate cross-modal correspondences between a range of seven olfactory stimuli and both the pitch and instrument class of sounds as well as the angularity of visually presented shapes. The results revealed that odors were preferentially matched to musical features: For example, the odors of candied orange and iris flower were matched to significantly higher pitches than the odors of musk and roasted coffee. Meanwhile, the odor of creme brulee was associated with a more rounded shape than the musk odor. Moreover, by simultaneously testing cross-modal correspondences between olfactory stimuli and matches in two other modalities, we were able to compare the ratings associated with each correspondence. Stimuli judged as happier, more pleasant, and sweeter tended to be associated to both higher pitch and a more rounded shape, whereas other ratings seemed to be more specifically correlated with the choice of either pitch or shape. Odors rated as more arousing tended to be associated with the angular shape, but not with a particular pitch; odors judged as brighter were associated with higher pitch and, to a lesser extent, rounder shapes. In a follow-up experiment, we investigated whether people could match specific pieces of music (composed to represent odors) to three of the odors (candied orange, creme brulee, and ginger cookies). In one case (candied orange), a majority of the participants matched the odor to the intended piece of music. In another case (ginger cookies), another piece of music (than the one intended) was preferred. Finally, in the third case (creme brulee), people showed no preference in matching the odor to the pieces of music. Both theoretical and practical implications of these results are discussed.

Individuals in preclinical and clinical stages of Huntington's disease (HD) demonstrate impairments in olfactory functioning. In addition, HD patients are impaired in source memory for verbal stimuli. A task combining both source and odor memory may be particularly sensitive to early changes in HD. The present study examined source and item memory for olfactory and visual stimuli in 10 individuals with HD, 10 asymptomatic HD gene carriers, 8 nongene carriers who had a parent with HD, and 20 normal controls. During the study phase, a male and a female experimenter (sources) presented odors and objects to the participant in an alternating sequence. To assess item memory, a stimulus from the study phase (target) and a novel stimulus (distractor) were presented, and the participant was asked to choose the target. To assess source memory, the experimenter presented a stimulus and asked whether the male or female experimenter had previously presented the stimulus. Results indicate that source memory for both visual and olfactory stimuli was impaired in HD patients compared to normal controls. In asymptomatic gene carriers, however, source memory for olfactory stimuli, but not visual stimuli, was more impaired than in nongene carriers and normal controls. Furthermore, gene carriers and HD patients showed a similar degree of impairment in source memory for olfactory stimuli. The only significant impairment found in item memory was for olfactory stimuli in HD patients. These results suggest that source memory for olfactory stimuli may be particularly sensitive to neuropathological changes in preclinical stages of HD.

Temporal ventriloquism is the shift in perceived timing of a visual stimulus that occurs when an auditory stimulus is presented close in time. This study investigated whether crossmodal correspondence between auditory pitch and visual elevation modulates temporal ventriloquism. Participants were presented two visual stimuli (above and below fixation) across a range of stimulus onset asynchronies and were asked to judge the order of the events. A task-irrelevant auditory click was presented shortly before the first and another shortly after the second visual stimulus. There were two pitches used (low and high) and the congruency between the auditory and visual stimuli was manipulated. The results show that incongruent pairings between pitch and elevation abolish temporal ventriloquism. In contrast, the crossmodal correspondence effect was absent when the direction of the pitch change was fixed within sessions, reducing the saliency of the pitch change. The results support previous studies suggesting that in addition to spatial and temporal factors, crossmodal correspondences can influence binding of information across the senses, although these effects are likely to be dependent on the saliency of the crossmodal mapping.

Memory for olfactory stimuli may be particularly affected by age-related brain changes in humans and may be an early indicator of cognitive impairment and Alzheimer's disease. Studies involving rats have offered insights into impaired cognition in aged animals, but few have examined odor memory. Therefore, it is unclear whether aged rats are a good model for possible age-related changes in odor memory in humans. Young (6-month-old) and old (24-month-old) rats were tested on associative learning tasks involving visual and olfactory stimuli. The first task examined age-related differences in discrimination and reversal learning for olfactory and visual stimuli; the second task utilized an associative contextual learning task involving olfactory and visual cues. Although old rats were able to perform the olfactory and visual discrimination tasks as well as young rats, old rats displayed significant age-related impairment on the reversal learning and contextual learning tasks. The results suggest that aging may have a similar deleterious effect on odor memory in rats and in humans. The findings may have important implications for the selection of memory paradigms for future research studies on aging. In addition, the use of an animal model to investigate the effects of aging on odor memory will allow researchers the ability to investigate how age-related neuroanatomical and neurochemical changes may result in impaired odor memory.

Effects of sensory stimuli on the behavioural phase state of the desert locust, Schistocerca gregaria