Similarity Judgments Research Articles

A tutorial review on methods for collecting similarity judgments from human observers.

Similarity is a central concept in the study of cognition, having been identified as an explanatory factor in the dynamics of myriad psychological phenomena. The collection of similarity judgments, however, can be a difficult, laborious, and time-consuming process. There is presently a vast and diverse array of methodologies applied throughout the psychological sciences from which to gather judgments of similarity perceptions, and each carries its own relative advantages and disadvantages. Each method may be suitable for a specific set of contexts and stimuli but be inappropriate for others. This tutorial review is meant to serve as a guided tour of common similarity judgment-gathering methods currently utilized in the psychological sciences, and to provide an overview of how and when researchers should leverage them.

Comparing color qualia structures through a similarity task in young children versus adults

Examination of the subjective qualitative aspects of an experience, or "qualia" in short, is a fundamental and core aspect of consciousness research. How can we characterize the particular quality of redness, i.e. a red quale? Based on a recent proposal of the structural characterization of qualia, which did not rely on verbal descriptions, we developed a task that obtained pairwise similarity judgments at four graded levels, with easy and intuitive visual interfaces designed to engage young children. We examined color qualia structures in children (3 to 12-y-old in Japan and 6 to 8-y-old in China) and compared these with those of Japanese adults. Approximately half of the assessed 3-y-old children completed the experiment via our touch panel device version of the task and had reliable responses. Despite known developmental and/or cultural effects of color term usage, we found that color qualia structures were quite similar across the age groups and cultures. Our finding supports the view that color qualia structures emerge early. We also observed age-related differences in the evaluations of some color pairs, which implied subtle changes in the structures behind color experience.

Evidence from odor similarity judgments suggests a widespread ability to imagine odors.

A persistent belief holds that humans can imagine visual content but not odors. While visual imagery is regarded as recreating a perceptual representation, it is unknown whether olfactory mental imagery shares a perceptual format. Visual imagery studies have demonstrated this perceptual formatting using distance and shape similarity judgments, whereas olfactory studies often use single-odor vividness ratings, complicating the establishment of perceptual formatting for odors. Using odor pair similarity scores from two experiments (odor-based: 8,880 ratings from 37 participants, including 20 women; label-based: 129,472 ratings from 2,023 participants, including 1,164 women), we observed a strong correlation (r = .71) between odor-based and label-based odor pairs. The correlation was unaffected by gender and age and was present in a wide range of self-perceived olfactory functions. Pleasantness similarity was the main determinant of overall similarity for both odor-based (r = -.63) and label-based (r = -.45) odor pairs. We then used a large language model to derive semantic similarity scores for the labels of all odor pairs. Semantic similarity only mediated a small part of the observed correlation, further supporting our conclusions that odor imagery shares a perceptual formatting with vision, that odor percepts may be elicited from verbal labels alone, and that odor pair pleasantness may be a dominant and accessible feature in this regard. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Core dimensions of human material perception

Visually categorizing and comparing materials is crucial for everyday behavior, but what organizational principles underlie our mental representation of materials? Here, we used a large-scale data-driven approach to uncover core latent dimensions of material representations from behavior. First, we created an image dataset of 200 systematically sampled materials and 600 photographs (STUFF dataset, https://osf.io/myutc/). Using these images, we next collected 1.87 million triplet similarity judgments and used a computational model to derive a set of sparse, positive dimensions underlying these judgments. The resulting multidimensional embedding space predicted independent material similarity judgments and the similarity matrix of all images close to the human intersubject consistency. We found that representations of individual images were captured by a combination of 36 material dimensions that were highly reproducible and interpretable, comprising perceptual (e.g., grainy, blue) as well as conceptual (e.g., mineral, viscous) dimensions. These results provide the foundation for a comprehensive understanding of how humans make sense of materials.

Supplemental Material for Evidence From Odor Similarity Judgments Suggests a Widespread Ability to Imagine Odors

Supplemental Material for Evidence From Odor Similarity Judgments Suggests a Widespread Ability to Imagine Odors

Is my "red" your "red"?: Evaluating structural correspondences between color similarity judgments using unsupervised alignment.

Whether one person's subjective experience of the "redness" of red is equivalent to another's is a fundamental question in consciousness studies. Intersubjective comparison of the relational structures of sensory experiences, termed "qualia structures", can constrain the question. We propose an unsupervised alignment method, based on optimal transport, to find the optimal mapping between the similarity structures of sensory experiences without presupposing correspondences (such as "red-to-red"). After collecting subjective similarity judgments for 93 colors, we showed that the similarity structures derived from color-neurotypical participants can be "correctly" aligned at the group level. In contrast, those of color-blind participants could not be aligned with color-neurotypical participants. Our results provide quantitative evidence for interindividual structural equivalence or difference of color qualia, implying that color-neurotypical people's "red" is relationally equivalent to other color-neurotypical's "red", but not to color-blind people's "red". This method is applicable across modalities, enabling general structural exploration of subjective experiences.

Automatic engagement of limbic and prefrontal networks in response to food images reflects distinct information about food hedonics and inhibitory control

Adaptive regulation of food consumption involves both identifying food as well as evaluating whether it should be eaten, a process that requires assessing relevant properties such as healthfulness and hedonic value. In order to identify how these fine-grained food properties are represented in the brain, we analyzed functional Magnetic Resonance Imaging data from 43 participants who viewed images of 36 different foods. A data-driven clustering approach based on Representational Similarity Analysis partitioned food-responsive brain regions into two sub-networks based on their multivariate response to food pictures: a Prefrontal network composed of fronto-parietal brain regions and a Limbic network composed of cortico-limbic and sub-cortical brain regions. Further analysis, using similarity judgments of those foods from a large online sample, revealed that the Prefrontal network predominantly represented information related to food healthfulness or processing, the key factor underlying food similarity. In another imaging task, we found that responses in the Prefrontal network were strongly influenced by judgments of food-related self-control, while the Limbic network responses were more affected by hedonic food judgments. These results suggest that, upon viewing food images, behaviorally relevant information is automatically retrieved from distinct brain networks that act as opponent processes in guiding food consumption.

From Form to Meaning: The Effects of Chinese Characters’ Iconicity on Children’s Perceptual and Semantic Processing Across 3.5 to 6.5 Years

ABSTRACT Research Findings: While Pictographic characters are often used to support Chinese children’s character learning, little is known about how this facilitates learning. Following the integrated model of symbol semantic categorization, this study examined the effects of Chinese characters’ iconicity on perceptual similarity and semantic consistency judgments among children aged 3.5–6.5 (N = 126, 64 boys). Results showed that perceptual similarity was highest for characters of Dongba script (DBS), followed by characters of Oracle bone script (OBS), and lowest for characters of simplified Chinese (SC). Semantic consistency accuracy was highest for DBS. Meanwhile, children under 58.57 months showed similar accuracy of OBS and SC, while older children performed better with SC. Semantic consistency speed was comparable across all character types for children under 55 months but faster for DBS and SC in older children. These findings suggest that higher iconicity facilitates perceptual similarity and semantic judgments, making characters of SC easier to judge as children age. Practices and Policy: These results indicate that Chinese characters with high iconicity can scaffold young children’s character learning. Educators should introduce characters with varying levels of iconicity based on children’s ages, starting with highly iconic characters and gradually transitioning to less iconic ones.

Semantic alignment: A measure to quantify the degree of semantic equivalence for English-Chinese translation equivalents based on distributional semantics.

The degree of semantic equivalence of translation pairs is typically measured by asking bilinguals to rate the semantic similarity of them or comparing the number and meaning of dictionary entries. Such measures are subjective, labor-intensive, and unable to capture the fine-grained variation in the degree of semantic equivalence. Thompson et al. (in Nature Human Behaviour, 4(10), 1029-1038, 2020) propose a computational method to quantify the extent to which translation equivalents are semantically aligned by measuring the contextual use across languages. Here, we refine this method to quantify semantic alignment of English-Chinese translation equivalents using word2vec based on the proposal that the degree of similarity between the contexts associated with a word and those of its multiple translations vary continuously. We validate our measure using semantic alignment from GloVe and fastText, and data from two behavioral datasets. The consistency of semantic alignment induced across different models confirms the robustness of our method. We demonstrate that semantic alignment not only reflects human semantic similarity judgment of translation equivalents but also captures bilinguals' usage frequency of translations. We also show that our method is more cognitively plausible than Thompson et al.'s method. Furthermore, the correlations between semantic alignment and key psycholinguistic factors mirror those between human-rated semantic similarity and these variables, indicating that computed semantic alignment reflects the degree of semantic overlap of translation equivalents in the bilingual mental lexicon. We further provide the largest English-Chinese translation equivalent dataset to date, encompassing 50,088 translation pairs for 15,734 English words, their dominant Chinese translation equivalents, and their semantic alignment Rc values.

View-symmetric representations of faces in human and artificial neural networks.

View-symmetric representations of faces in human and artificial neural networks.

Emotional distress and affective knowledge representation one year after the COVID-19 outbreak.

This study examines whether the detrimental effects of the COVID-19 pandemic on the affectivity of the population extend one year after the outbreak. In an online-mobile session, participants completed surveys (i.e., demographic characteristics, positive-negative affectivity, interoceptive awareness) and a similarity judgment task of triplets of emotional concepts, from which we derived 2D maps of their affective knowledge representation. Compared with pre-pandemic data derived from a comparable population, we report three main findings. First, we observed enhanced negative affectivity during the pandemic, but no changes in positive affectivity levels. Second, increased self-reported interoceptive awareness compared to pre-pandemic data, with greater attention to bodily sensations and adaptive aspects of interoceptive sensitivity. Furthermore, female participants reported higher scores than males on the questionnaire subscales of Emotional Awareness and Attention Regulation. Third, the effect of pandemic-related conditions is also apparent in the mental organization of emotional concepts, especially for female participants (i.e., reduced coherence in the organization of the concepts along the arousal dimension and more misclassification of concepts based on arousal) and participants who did not perform physical activity (a collapse of the arousal dimension). Some of the effects of the pandemic, thus, persist about a year after the outbreak. These results advise providing programs of psychological and emotional assistance throughout the pandemic beyond the outbreak, and that age-dependent gender differences should be accounted for to define tailored interventions. Physical activity might relieve pandemic-related stressors, so it should be promoted during particularly stressful periods for the population.

The influence of three-gendered grammatical systems on simultaneous bilingual cognition: The case of Ukrainian-Russian bilinguals

Abstract This paper examines the linguistic relativity principle (Whorf, 1956) by investigating the impact of grammatical gender on cognition in simultaneous bilinguals of three-gendered Ukrainian and Russian. It examines whether speakers of three-gendered languages show grammatical gender effects on categorisation, empirically addressing claims that such effects are insignificant due to the presence of the neuter gender (Sera et al., 2002). We conducted two experiments using a similarity judgement paradigm while manipulating the presence of neuter gender stimuli (Phillips & Boroditsky, 2003). Experiment 1, including neuter gender, revealed no significant effects, compatible with earlier studies on three-gendered languages. Conversely, Experiment 2, excluding neuter gender stimuli, showed significant language effects. Bilingual participants rated pairs as more similar when grammatical genders in both languages were congruent with the biological sex of a character. Significant effects were also found for pairs with mismatching grammatical genders in Ukrainian and Russian. Participants with higher proficiency in Ukrainian rated pairs as more similar when the grammatical gender of a noun in Ukrainian was congruent with the character’s biological sex, and incongruent in Russian. Our findings thus provide the first empirical demonstration that the exclusion of neuter gender online induces grammatical gender effects in speakers of three-gendered languages.

Word Vector Representation of Latin Cuengh Based on Root Feature Enhancement

The Latin Cuengh is a kind of language used in China’s minority areas. Due to its complex pronunciation and semantic system, it is difficult to spread widely. To deal with and protect this language further, this paper considers using the current word vector representation technology to study it. Word vector representation is the basic method and an important foundation of current research on natural language processing. It relies on a large number of data resources and is obtained through the paradigm of pre-training and feature learning. Due to the extreme lack of Latin Cuengh corpus resources, it is very difficult to obtain word vectors by relying on a large amount of data training. In this study, we propose a word vector representation method that combines the root features of Latin Cuengh words. Specifically, while training and learning the Latin Cuengh language corpus, this method uses the special word roots in the Latin Cuengh language to modify the training process, which can enhance the expression ability of the root features. The method uses the mask method based on BERT to mask the word roots after word segmentation and predict the masked word roots in the output layer of the model to obtain a better vector representation of Latin Cuengh words. The experimental results show that the word vector representation method proposed in this paper is effective and has the ability to express Latin Cuengh semantics. The accuracy rate of words semantic is nearly 2% points higher than that of BERT representation, and the judgment of the semantic similarity of words is more accurate.

The Neural Specificity of Interference Resolution in Phonological, Semantic, and Visual Domains at Different Ages.

The question of whether cognitive control is specific to certain domains or domain-general remains an extensively debated question at both cognitive and neural levels. This study examined the neural substrates associated with resistance to interference (RI) in phonological, semantic, and visual domains by using strictly matched tasks and determining the domain-general or domain-specific manner in which aging affects the neural substrates associated with RI. In an fMRI experiment, young and older participants performed a similarity judgment task with phonological, semantic, or visual interference buildup. For both age groups, domain-specific RI effects were observed at the univariate level, with increased involvement in the phonological domain of the right angular gyrus and the right lingual gyrus, in the semantic domain of the bilateral inferior frontal gyrus, the bilateral superior parietal and angular gyri and the left middle temporal gyrus, and in the visual domain of the middle/superior frontal gyri and occipital gyri. At the multivariate level, although RI effects could be decoded from neural patterns in the bilateral inferior frontal gyrus for all domains and age groups, between-domain prediction of RI conditions was associated with Bayesian evidence for the null hypothesis. This study supports the domain specificity of neural substrates associated with RI while stressing its age independency.

Cheaper Spaces

Similarity spaces are standardly constructed by collecting pairwise similarity judgments and subjecting those to a dimension-reduction technique such as multidimensional scaling or principal component analysis. While this approach can be effective, it has some known downsides, most notably, it tends to be costly and has limited generalizability. Recently, a number of authors have attempted to mitigate these issues through machine learning techniques. For instance, neural networks have been trained on human similarity judgments to infer the spatial representation of unseen stimuli. However, these newer methods are still costly and fail to generalize widely beyond their initial training sets. This paper proposes leveraging prebuilt semantic vector spaces as a cheap alternative to collecting similarity judgments. Our results suggest that some of those spaces can be used to approximate human similarity judgments at low cost and high speed.

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision.

We effortlessly extract behaviorally relevant information from dynamic visual input in order to understand the actions of others. In the current study, we develop and test a number of models to better understand the neural representational geometries supporting action understanding. Using fMRI, we measured brain activity as participants viewed a diverse set of 90 different video clips depicting social and nonsocial actions in real-world contexts. We developed five behavioral models using arrangement tasks: two models reflecting behavioral judgments of the purpose (transitivity) and the social content (sociality) of the actions depicted in the video stimuli; and three models reflecting behavioral judgments of the visual content (people, objects, and scene) depicted in still frames of the stimuli. We evaluated how well these models predict neural representational geometry and tested them against semantic models based on verb and nonverb embeddings and visual models based on gaze and motion energy. Our results revealed that behavioral judgments of similarity better reflect neural representational geometry than semantic or visual models throughout much of cortex. The sociality and transitivity models in particular captured a large portion of unique variance throughout the action observation network, extending into regions not typically associated with action perception, like ventral temporal cortex. Overall, our findings expand the action observation network and indicate that the social content and purpose of observed actions are predominant in cortical representation.

Reconciling category exceptions through representational shifts.

Real-world categories often contain exceptions that disobey the perceptual regularities followed by other members. Prominent psychological and neurobiological theories indicate that exception learning relies on the flexible modulation of object representations, but the specific representational shifts key to learning remain poorly understood. Here, we leveraged behavioral and computational approaches to elucidate the representational dynamics during the acquisition of exceptions that violate established regularity knowledge. In our study, participants (n = 42) learned novel categories in which regular and exceptional items were introduced successively; we then fitted a computational model to individuals' categorization performance to infer latent stimulus representations before and after exception learning. We found that in the representational space, exception learning not only drove confusable exceptions to be differentiated from regular items, but also led exceptions within the same category to be integrated based on shared characteristics. These shifts resulted in distinct representational clusters of regular items and exceptions that constituted hierarchically structured category representations, and the distinct clustering of exceptions from regular items was associated with a high ability to generalize and reconcile knowledge of regularities and exceptions. Moreover, by having a second group of participants (n = 42) to judge stimuli's similarity before and after exception learning, we revealed misalignment between representational similarity and behavioral similarity judgments, which further highlights the hierarchical layouts of categories with regularities and exceptions. Altogether, our findings elucidate the representational dynamics giving rise to generalizable category structures that reconcile perceptually inconsistent category members, thereby advancing the understanding of knowledge formation.

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision.

We effortlessly extract behaviorally relevant information from dynamic visual input in order to understand the actions of others. In the current study, we develop and test a number of models to better understand the neural representational geometries supporting action understanding. Using fMRI, we measured brain activity as participants viewed a diverse set of 90 different video clips depicting social and nonsocial actions in real-world contexts. We developed five behavioral models using arrangement tasks: two models reflecting behavioral judgments of the purpose (transitivity) and the social content (sociality) of the actions depicted in the video stimuli; and three models reflecting behavioral judgments of the visual content (people, objects, and scene) depicted in still frames of the stimuli. We evaluated how well these models predict neural representational geometry and tested them against semantic models based on verb and nonverb embeddings and visual models based on gaze and motion energy. Our results revealed that behavioral judgments of similarity better reflect neural representational geometry than semantic or visual models throughout much of cortex. The sociality and transitivity models in particular captured a large portion of unique variance throughout the action observation network, extending into regions not typically associated with action perception, like ventral temporal cortex. Overall, our findings expand the action observation network and indicate that the social content and purpose of observed actions are predominant in cortical representation.

A data-driven analysis of the perceptual and neural responses to natural objects reveals organising principles of human visual cognition.

A key challenge in understanding the functional organisation of visual cortex stems from the fact that only a small proportion of the objects experienced during natural viewing can be presented in a typical experiment. This constraint often leads to experimental designs that compare responses to objects from experimenter-defined stimulus conditions, potentially limiting the interpretation of the data. To overcome this issue, we used images from the THINGS initiative, which provides a systematic sampling of natural objects. A data-driven analysis was then applied to reveal the functional organisation of the visual brain, incorporating both perceptual and neural responses to these objects. Perceptual properties of the objects were taken from an analysis of similarity judgements, and neural properties were taken from whole brain fMRI responses to the same objects. Partial least squares regression (PLSR) was then used to predict neural responses across the brain from the perceptual properties while simultaneously applying dimensionality reduction. The PLSR model accurately predicted neural responses across visual cortex using only a small number of components. These components revealed smooth, graded neural topographies, which were similar in both hemispheres, and captured a variety of object properties including animacy, real-world size, and object category. However, they did not accord in any simple way with previous theoretical perspectives on object perception. Instead, our findings suggest that visual cortex encodes information in a statistically efficient manner, reflecting natural variability among objects.Significance statement The ability to recognise objects is fundamental to how we interact with our environment, yet the organising principles underlying neural representations of visual objects remain contentious. In this study, we sought to address this question by analysing perceptual and neural responses to a large, unbiased sample of objects. Using a data-driven approach, we leveraged perceptual properties of objects to predict neural responses using a small number of components. This model predicted neural responses with a high degree of accuracy across visual cortex. The components did not directly align with previous explanations of object perception. Instead, our findings suggest the organisation of the visual brain is based on the statistical properties of objects in the natural world.

Compositionality and Sentence Meaning: Comparing Semantic Parsing and Transformers on a Challenging Sentence Similarity Dataset

Abstract One of the major outstanding questions in computational semantics is how humans integrate the meaning of individual words into a sentence in a way that enables understanding of complex and novel combinations of words, a phenomenon known as compositionality. Many approaches to modeling the process of compositionality can be classified as either “vector-based” models, in which the meaning of a sentence is represented as a vector of numbers, or “syntax-based” models, in which the meaning of a sentence is represented as a structured tree of labeled components. A major barrier in assessing and comparing these contrasting approaches is the lack of large, relevant datasets for model comparison. This article aims to address this gap by introducing a new dataset, STS3k, which consists of 2,800 pairs of sentences rated for semantic similarity by human participants. The sentence pairs have been selected to systematically vary different combinations of words, providing a rigorous test and enabling a clearer picture of the comparative strengths and weaknesses of vector-based and syntax-based methods. Our results show that when tested on the new STS3k dataset, state-of-the-art transformers poorly capture the pattern of human semantic similarity judgments, while even simple methods for combining syntax- and vector-based components into a novel hybrid model yield substantial improvements. We further show that this improvement is due to the ability of the hybrid model to replicate human sensitivity to specific changes in sentence structure. Our findings provide evidence for the value of integrating multiple methods to better reflect the way in which humans mentally represent compositional meaning.