Grapheme-color Associations Research Articles

Using immersive virtual reality to recreate the synaesthetic experience.

Synaesthesia is a condition where people experience unusual sensory or cognitive sensations in response to apparently unrelated stimuli. This paper presents two experiments which aimed to examine whether Virtual Reality (VR) technology can be used to recreate the synaesthetic experience. There is a lack of research in this area, with most studies focussing primarily on synaesthetic colors. Experiment 1 aimed to build on previous research by using not only a traditional color-picker but also VR to capture a more nuanced picture of synaesthetic perception. A multiple case study design was used to examine the experiences of six participants in detail. Data gathering took place via Zoom. During the initial data-gathering session, participants used a color-picker to provide grapheme-color associations. After this session, some of the participants' synaesthetic experiences were recreated using a VR-by-proxy approach. Results indicated that VR is capable of capturing elements of synaesthetic perception that other methods have been unable to, such as texture, small degrees of movement, and 3D structure. Experiment 2 expanded upon these findings by moving beyond the VR-by-proxy approach and asking three participants to recreate their own audiovisual synaesthetic associations in the VR environment. Inductive Thematic Analysis was used to analyze the results of this experiment. The potential of expanding this technique to other forms of perceptual diversity is discussed.

Perceptual Dissimilarity Analysis Distinguishes Grapheme-Color Synesthetes from Nonsynesthetes.

Synesthetes can be distinguished from nonsynesthetes on a variety of experimental tasks because their concurrent synesthetic experiences can affect task performance if these experiences match or conflict with some aspect of the stimulus. Here, we tested grapheme-color synesthetes and nonsynesthetic control participants using a novel perceptual similarity task to assess whether synesthetes' concurrent color experiences influence perceived grapheme similarity. Participants iteratively arranged graphemes and, separately, their associated synesthetic colors in a display, such that similar items were placed close together and dissimilar items further apart. The resulting relative inter-item distances were used to calculate the pair-wise (dis)similarity between items in the set, and thence to create separate perceptual representational dissimilarity matrices (RDMs) for graphemes and colors, on an individual basis. On the assumption that synesthetes' similarity judgments for graphemes would be influenced by their concurrent color experiences, we predicted that grapheme and color RDMs would be more strongly correlated for synesthetes than nonsynesthetes. We found that the mean grapheme-color RDM correlation was indeed significantly higher in synesthetes than nonsynesthetes; in addition, synesthetes' grapheme-color RDM correlations were more likely to be individually statistically significant, even after correction for multiple tests, than those of nonsynesthetes. Importantly, synesthetes' grapheme-color RDM correlations were scaled with the consistency of their grapheme-color associations as measured by their Synesthesia Battery (SB) scores. By contrast, the relationship between SB scores and grapheme-color RDM correlations for nonsynesthetes was not significant. Thus, dissimilarity analysis quantitatively distinguished synesthetes from nonsynesthetes, in a way that meaningfully reflects a key aspect of synesthetic experience.

Perceptual processing links autism and synesthesia: A co-twin control study

Synesthesia occurs more commonly in individuals fulfilling criteria for an autism spectrum diagnosis than in the general population. It is associated with autistic traits and autism-related perceptual processing characteristics, including a more detail-focused attentional style and altered sensory sensitivity. In addition, these characteristics correlate with the degree of grapheme-color synesthesia (consistency of grapheme-color associations) in non-synesthetes.We investigated a predominantly non-synesthetic twin sample, including individuals fulfilling criteria for an autism spectrum diagnosis or other neurodevelopmental disorders (n = 65, 14–34 years, 60% female). We modelled linear relationships between the degree of grapheme-color synesthesia and autistic traits, sensory sensitivity, and visual perception, both within-twin pairs (22 pairs) where all factors shared by twins are implicitly controlled (including 50–100% genetics), and across the entire cohort.We found that the degree of grapheme-color synesthesia was associated with autistic traits within the domain of Attention to Details and with sensory hyper-, but not hypo-sensitivity. These associations were stronger within-twin pairs than across the sample. Further, twins with a higher degree of grapheme-color synesthesia were better than their co-twins at identifying fragmented images (Fragmented Pictures Test).This is the first twin study on the association between synesthesia and autism-related perceptual features and traits. The results suggest that investigating these associations within-twin pairs, implicitly adjusting for potential confounding factors shared by twins, is more sensitive than doing so in non-related individuals. Consistent with previous findings, the results suggest an association between the degree of grapheme-color synesthesia and autism-related perceptual features, while utilizing a different measure for sensory sensitivity. The novel finding of enhanced fragmented picture integration in twins with a higher degree of grapheme-color synesthesia challenges the view of a generally more detail-focused attentional style in synesthesia and might be related to enhanced memory or mental imagery in more synesthetic individuals.

Consistency and strength of grapheme-color associations are separable aspects of synesthetic experience

Consistency of synesthetic associations over time is a widely used test of synesthesia. Since many studies suggest that consistency is not a completely reliable feature, we compared the consistency and strength of synesthetes’ grapheme-color associations. Consistency was measured by scores on the Synesthesia Battery and by the Euclidean distance in color space for the specific graphemes tested for each participant. Strength was measured by congruency magnitudes on the Implicit Association Test. The strength of associations was substantially greater for synesthetes than non-synesthetes, suggesting that this is a novel, objective marker of synesthesia. Although, intuitively, strong associations should also be consistent, consistency and strength were uncorrelated, indicating that they are likely independent, at least for grapheme-color synesthesia. These findings have implications for our understanding of synesthesia and for estimates of its prevalence since synesthetes who experience strong, but inconsistent, associations may not be identified by tests that focus solely on consistency.

Grapheme-color associations can transfer to novel graphemes when synesthetic colors function as grapheme "discriminating markers".

Synesthesia is a condition in which the perception of a stimulus in one modality automatically triggers a secondary sensation in another modality or processing stream. Our study focused on grapheme-color synesthesia, in which the visual perception of letters or numbers (graphemes) induces a specific color sensation (the synesthetic color). Grapheme-color synesthetes do not typically experience colors for novel graphemes. However, synesthetic colors associated with familiar graphemes can be transferred to graphemes learned later, even in adulthood. A previous study has shown that such a transfer can take place after only a 10-min writing exercise. In this study, we found that this immediate transfer occurs only when the synesthetic colors for familiar graphemes contribute to the discrimination of the graphemes to be learned. Synesthetes learned six novel graphemes, each of which was arbitrarily associated with one of six familiar graphemes. Half of the synesthetes were assigned to the heterogeneous condition, in which the synesthetic colors of one group of familiar graphemes were different from one another. The other half of the synesthetes were assigned to the homogeneous condition, in which the various colors of a whole group of familiar graphemes were categorically the same. After this learning session, less transfer of synesthetic colors to novel graphemes from the corresponding familiar graphemes occurred in the homogeneous condition than in the heterogeneous condition. These results support the view that synesthetic colors for graphemes may function as a grapheme acquisition aid.

Superior learning in synesthetes: Consistent grapheme-color associations facilitate statistical learning

In synesthesia activation in one sensory domain, such as smell or sound, triggers an involuntary and unusual secondary sensory or cognitive experience. In the present study, we ask whether the added sensory experience of synesthesia can aid statistical learning—the ability to track environmental regularities in order to segment continuous information. To investigate this, we measured statistical learning outcomes, using an aurally presented artificial language, in two groups of synesthetes alongside controls and simulated the multimodal experience of synesthesia in non-synesthetes. One group of synesthetes exclusively had grapheme-color (GC) synesthesia, in which the experience of color is automatically triggered by exposure to written or spoken graphemes. The other group had both grapheme-color and sound-color (SC+) synesthesia, in which the experience of color is also triggered by the waveform properties of a voice, such as pitch, timbre, and/or musical chords. Unlike GC-only synesthetes, the experience of color in the SC+ group is not perfectly consistent with the statistics that signal word boundaries. We showed that GC-only synesthetes outperformed both non-synesthetes and SC+ synesthetes, likely because the visual concurrents for GC-only synesthetes are highly consistent with the artificial language. We further observed that our simulations of GC synesthesia, but not SC+ synesthesia produced superior statistical learning, showing that synesthesia likely boosts learning outcomes by providing a consistent secondary cue. Findings are discussed with regard to how multimodal experience can improve learning, with the present data indicating that this boost is more likely to occur through explicit, as opposed to implicit, learning systems.

Why is the synesthete's “A” red? Using a five-language dataset to disentangle the effects of shape, sound, semantics, and ordinality on inducer–concurrent relationships in grapheme-color synesthesia

Grapheme-color synesthesia is a neurological phenomenon in which viewing a grapheme elicits an additional, automatic, and consistent sensation of color. Color-to-letter associations in synesthesia are interesting in their own right, but also offer an opportunity to examine relationships between visual, acoustic, and semantic aspects of language. Research using large populations of synesthetes has indeed found that grapheme-color pairings can be influenced by numerous properties of graphemes, but the contributions made by each of these explanatory factors are often confounded in a monolingual dataset (i.e., only English-speaking synesthetes). Here, we report the first demonstration of how a multilingual dataset can reveal potentially-universal influences on synesthetic associations, and disentangle previously-confounded hypotheses about the relationship between properties of synesthetic color and properties of the grapheme that induces it. Numerous studies have reported that for English-speaking synesthetes, “A” tends to be colored red more often than predicted by chance, and several explanatory factors have been proposed that could explain this association. Using a five-language dataset (native English, Dutch, Spanish, Japanese, and Korean speakers), we compare the predictions made by each explanatory factor, and show that only an ordinal explanation makes consistent predictions across all five languages, suggesting that the English “A” is red because the first grapheme of a synesthete's alphabet or syllabary tends to be associated with red. We propose that the relationship between the first grapheme and the color red is an association between an unusually-distinct ordinal position (“first”) and an unusually-distinct color (red). We test the predictions made by this theory, and demonstrate that the first grapheme is unusually distinct (has a color that is distant in color space from the other letters' colors). Our results demonstrate the importance of considering cross-linguistic similarities and differences in synesthesia, and suggest that some influences on grapheme-color associations in synesthesia might be universal.

Grapheme-color associations can immediately transfer to novel graphemes, but such synesthetic colors are unstable.

Grapheme-color associations can immediately transfer to novel graphemes, but such synesthetic colors are unstable.

Database of synesthetic color associations for Japanese kanji.

Synesthesia is a neurological phenomenon in which certain types of stimuli elicit involuntary perceptions in an unrelated pathway. A common type of synesthesia is grapheme-color synesthesia, in which the visual perception of letters and numbers stimulates the perception of a specific color. Previous studies have often collected relatively small numbers of grapheme-color associations per synesthete, but the accumulation of a large quantity of data has greater promise for uncovering the mechanisms underlying synesthetic association. In this study, we therefore collected large samples of data from a total of eight synesthetes. All told, we obtained over 1000 synesthetic colors associated with Japanese kanji characters from each of two synesthetes, over 100 synesthetic colors form each of three synesthetes, and about 80 synesthetic colors associated with Japanese hiragana, Latin letters, and Arabic numerals from each of three synesthetes. We then compiled the data into a database, called the KANJI-Synesthetic Colors Database (K-SCD), which has a total of 5122 colors for 483, 46, and 46 Japanese kanji, hiragana, and katakana characters, respectively, as well as for 26 Latin letters and ten Arabic numerals. In addition to introducing the K-SCD, this article demonstrates the database's merits by using two examples, in which two new rules for synesthetic association, "shape similarity" and "synesthetic color clustering," were found. The K-SCD is publicly accessible ( www.cv.jinkan.kyoto-u.ac.jp/site/uploads/K-SCD.xlsm ) and will be a valuable resource for those who wish to conduct statistical analyses using a rich dataset in order to uncover the rules governing synesthetic association and to understand its mechanisms.

Nonrandom Associations of Graphemes with Colors in Arabic.

Numerous studies demonstrate people associate colors with letters and numbers in systematic ways. But most of these studies rely on speakers of English, or closely related languages. This makes it difficult to know how generalizable these findings are, or what factors might underlie these associations. We investigated letter-color and number-color associations in Arabic speakers, who have a different writing system and unusual word structure compared to Standard Average European languages. We also aimed to identify grapheme-color synaesthetes (people who have conscious color experiences with letters and numbers). Participants associated colors with 28 basic Arabic letters and ten digits by typing color names that best fit each grapheme. We found language-specific principles determining grapheme-color associations. For example, the word formation process in Arabic was relevant for color associations. In addition, psycholinguistic variables, such as letter frequency and the intrinsic order of graphemes influenced associations. Contrary to previous studies we found no evidence for sounds playing a role in letter-color associations for Arabic, and only a very limited role for shape influencing color associations. These findings highlight the importance of linguistic and psycholinguistic features in cross-modal correspondences, and illustrate why it is important to play close attention to each language on its own terms in order to disentangle language-specific from universal effects.

Relation between synesthetic grapheme-color associations and the sub-types of synesthesia

Relation between synesthetic grapheme-color associations and the sub-types of synesthesia

Components of Attention in Grapheme-Color Synesthesia: A Modeling Approach.

Grapheme-color synesthesia is a condition where the perception of graphemes consistently and automatically evokes an experience of non-physical color. Many have studied how synesthesia affects the processing of achromatic graphemes, but less is known about the synesthetic processing of physically colored graphemes. Here, we investigated how the visual processing of colored letters is affected by the congruence or incongruence of synesthetic grapheme-color associations. We briefly presented graphemes (10–150 ms) to 9 grapheme-color synesthetes and to 9 control observers. Their task was to report as many letters (targets) as possible, while ignoring digit (distractors). Graphemes were either congruently or incongruently colored with the synesthetes’ reported grapheme-color association. A mathematical model, based on Bundesen’s (1990) Theory of Visual Attention (TVA), was fitted to each observer’s data, allowing us to estimate discrete components of visual attention. The models suggested that the synesthetes processed congruent letters faster than incongruent ones, and that they were able to retain more congruent letters in visual short-term memory, while the control group’s model parameters were not significantly affected by congruence. The increase in processing speed, when synesthetes process congruent letters, suggests that synesthesia affects the processing of letters at a perceptual level. To account for the benefit in processing speed, we propose that synesthetic associations become integrated into the categories of graphemes, and that letter colors are considered as evidence for making certain perceptual categorizations in the visual system. We also propose that enhanced visual short-term memory capacity for congruently colored graphemes can be explained by the synesthetes’ expertise regarding their specific grapheme-color associations.

Assessment of the hemispheric lateralization of grapheme-color synesthesia with Stroop-type tests.

Grapheme-color synesthesia, the idiosyncratic, arbitrary association of colors to letters or numbers, develops in childhood once reading is mastered. Because language processing is strongly left-lateralized in most individuals, we hypothesized that grapheme-color synesthesia could be left-lateralized as well. We used synesthetic versions of the Stroop test with colored letters and numbers presented either in the right or the left visual field of thirty-four synesthetes. Interference by synesthetic colors was stronger for stimuli in the right hemifield (first experiment, color naming task). Synesthetes were also faster in the right hemifield when naming the synesthetic color of graphemes (second experiment). Overall, the lateralization effect was 7 ms (the 95% confidence interval was [1.5 12] ms), a delay compatible with an additional callosal transfer for stimuli presented in the left hemifield. Though weak, this effect suggests that the association of synesthetic colors to graphemes may be preferentially processed in the left hemisphere. We speculate that this left-lateralization could be a landmark of synesthetic grapheme-color associations, if not found for color associations learnt by non-synesthete adults.

Developmental aspects of synaesthesia across the adult lifespan

In synaesthesia, stimuli such as sounds, words or letters trigger experiences of colors, shapes or tastes and the consistency of these experiences is a hallmark of this condition. In this study we investigate for the first time whether there are age-related changes in the consistency of synaesthetic experiences. We tested a sample of more than 400 grapheme-color synaesthetes who have color experiences when they see letters and/or digits with a well-established test of consistency. Our results showed a decline in the number of consistent grapheme-color associations across the adult lifespan. We also assessed age-related changes in the breadth of the color spectrum. The results showed that the appearance of primary colors (i.e., red, blue, and green) was mainly age-invariant. However, there was a decline in the occurrence of lurid colors while brown and achromatic tones occurred more often as concurrents in older age. These shifts in the color spectrum suggest that synaesthesia does not simply fade, but rather undergoes more comprehensive changes. We propose that these changes are the result of a combination of both age-related perceptual and memory processing shifts.

Acquiring synaesthesia: insights from training studies

Synaesthesia denotes a condition of remarkable individual differences in experience characterized by specific additional experiences in response to normal sensory input. Synaesthesia seems to (i) run in families which suggests a genetic component, (ii) is associated with marked structural and functional neural differences, and (iii) is usually reported to exist from early childhood. Hence, synaesthesia is generally regarded as a congenital phenomenon. However, most synaesthetic experiences are triggered by cultural artifacts (e.g., letters, musical sounds). Evidence exists to suggest that synaesthetic experiences are triggered by the conceptual representation of their inducer stimuli. Cases were identified for which the specific synaesthetic associations are related to prior experiences and large scale studies show that grapheme-color associations in synaesthesia are not completely random. Hence, a learning component is inherently involved in the development of specific synaesthetic associations. Researchers have hypothesized that associative learning is the critical mechanism. Recently, it has become of scientific and public interest if synaesthetic experiences may be acquired by means of associative training procedures and whether the gains of these trainings are associated with similar cognitive benefits as genuine synaesthetic experiences. In order to shed light on these issues and inform synaesthesia researchers and the general interested public alike, we provide a comprehensive literature review on developmental aspects of synaesthesia and specific training procedures in non-synaesthetes. Under the light of a clear working definition of synaesthesia, we come to the conclusion that synaesthesia can potentially be learned by the appropriate training.

Grapheme learning and grapheme-color synesthesia: toward a comprehensive model of grapheme-color association

Recent progress in grapheme-color synesthesia research has revealed that certain regularities, as well as individual differences, figure into grapheme-color associations. Although several factors are known to regulate grapheme-color associations, the impact of factors, including their interrelationships, on synesthesia remains unclear. We investigated determinants of synesthetic color for graphemes (characters, letters) of Hiragana, a phonetic script in the Japanese language, and the English alphabet. Results revealed that grapheme ordinality was the strongest predictor of synesthetic colors for Hiragana characters, followed by character sound, and visual shape. Ordinality and visual shapes also significantly predicted synesthetic colors for English alphabet letters, however, sounds did not. The relative impact of grapheme properties on grapheme-color associations and the differences between these two writing systems are accounted for by considering the way graphemes are processed in the brain and introduced during an individual's development. A new model is proposed which takes into account the developmental process of grapheme learning. The model provides comprehensive explanation of synesthetic grapheme-color association determination processes, including the differences across writing systems.

Impaired acquisition of novel grapheme-color correspondences in synesthesia

Grapheme-color synesthesia is a neurological phenomenon in which letters and numbers (graphemes) consistently evoke particular colors (e.g., A may be experienced as red). These sensations are thought to arise through the cross-activation of grapheme processing regions in the fusiform gyrus and color area V4, supported by anatomical and functional imaging. However, the developmental onset of grapheme-color synesthesia remains elusive as research in this area has largely relied on self-report of these experiences in children. One possible account suggests that synesthesia is present at or near birth and initially binds basic shapes and forms to colors, which are later refined to grapheme-color associations through experience. Consistent with this view, studies show that similarly shaped letters and numbers tend to elicit similar colors in synesthesia and that some synesthetes consciously associate basic shapes with colors; research additionally suggests that synesthetic colors can emerge for newly learned characters with repeated presentation. This model further predicts that the initial shape-color correspondences in synesthesia may persist as implicit associations, driving the acquisition of colors for novel characters. To examine the presence of latent color associations for novel characters, synesthetes and controls were trained on pre-defined associations between colors and complex shapes, on the assumption that the prescribed shape-color correspondences would on average differ from implicit synesthetic associations. Results revealed synesthetes were less accurate than controls to learn novel shape-color associations, consistent with our suggestion that implicit form-color associations conflicted with the learned pairings.

A longitudinal study of grapheme-color synesthesia in childhood: 6/7 years to 10/11 years

Grapheme-color synesthesia is a condition characterized by enduring and consistent associations between letter/digits and colors. This study is the continuation of longitudinal research begun by Simner et al. (2009) which aimed to explore the development of this condition in real time within a childhood population. In that earlier study we randomly sampled over 600 children and tested them aged 6/7 and 7/8 years. We identified the child synesthetes within that cohort and measured their development over 1 year, in comparison to a group of non-synesthetic children with both average and superior memories. We were able to show the beginnings of a developmental progression in which synesthetic associations (e.g., A = red) mature over time from relatively chaotic pairings into a system of fixed consistent associations. In the current study we return to this same population three years later when participants are now 10/11 years. We used the same paired-association memory task to determine the synesthetic status of our participants and to also establish synesthetes' inventories of grapheme-color associations. We compared their inventories to those from age 6/7 and 7/8 years to examine how synesthesia matures over time. Together with earlier findings, our study shows that grapheme-color synesthesia emerges with a protracted trajectory, with 34% of letters/digits fixed at age 6/7 years, 48% fixed at 7/8 years and 71% fixed at 10/11 years. We also show several cases where synesthesia is not developing in the same time-frame as peers, either because it has died out at an older age, or because it was slower to develop than other cases. Our study paints the first picture of the emergence of synesthesia in real-time over four years within a randomly sampled population of child synesthetes.

Individuals with Grapheme-Color Associations Exhibit Creativity

Synesthesia occurs when a single stimulus triggers two or more sensations simultaneously in different modalities. Grapheme-color association is a frequently reported type of synesthesia. Two types of grapheme-color associations have been identified: associators and projectors. Associators experience the colors conceptually, while projectors report actually perceiving the colors. Here, we evaluated the frequency of both types of grapheme-color association among a college sample. Our findings indicate that 8% of the sample demonstrated grapheme-color associator tendencies and 3% demonstrated grapheme-color projector tendencies. No participants demonstrated both associator and projector synesthetic tendencies. Additionally, as the relation between creativity and synesthesia has been the subject of interest of many experts in the field, participants' creative abilities were assessed through use of the Alternate Uses Test and self-report. The results support the link between synesthesia and creativity in projectors, but not associators.

Immediate transfer of synesthesia to a novel inducer

In synesthesia, a certain stimulus (e.g. grapheme) is associated automatically and consistently with a stable perceptual-like experience (e.g. color). These associations are acquired in early childhood and remain robust throughout the lifetime. Synesthetic associations can transfer to novel inducers in adulthood as one learns a second language that uses another writing system. However, it is not known how long this transfer takes. We found that grapheme-color associations can transfer to novel graphemes after only a 10-minute writing exercise. Most subjects experienced synesthetic associations immediately after learning a new Glagolitic grapheme. Using a Stroop task, we provide objective evidence for the creation of novel associations between the newly learned graphemes and synesthetic colors. Also, these associations generalized to graphemes handwritten by another person. The fast learning process and the generalization suggest that synesthesia begins at the semantic level of representation with the activation of a certain concept (the inducer), which then, uniquely for the synesthetes, activates representations at the perceptual level (the concurrent). Thus, the results imply that synesthesia is a much more flexible and plastic phenomenon than has been believed until now.