Nonverbal Categorization Research Articles

How language influences spatial thinking, categorization of motion events, and gaze behavior: a cross-linguistic comparison

Abstract According to Talmy, in verb-framed languages (e.g., French), the core schema of an event (Path) is lexicalized, leaving the co-event (Manner) in the periphery of the sentence or optional; in satellite-framed languages (e.g., English), the core schema is jointly expressed with the co-event in construals that lexicalize Manner and express Path peripherally. Some studies suggest that such differences are only surface differences that cannot influence the cognitive processing of events, while others support that they can constrain both verbal and non-verbal processing. This study investigates whether such typological differences, together with other factors, influence visual processing and decision-making. English and French participants were tested in three eye-tracking tasks involving varied Manner–Path configurations and language to different degrees. Participants had to process a target motion event and choose the variant that looked most like the target (non-verbal categorization), then describe the events (production), and perform a similarity judgment after hearing a target sentence (verbal categorization). The results show massive cross-linguistic differences in production and additional partial language effects in visualization and similarity judgment patterns – highly dependent on the salience and nature of events and the degree of language involvement. The findings support a non-modular approach to language–thought relations and a fine-grained vision of the classic lexicalization/conflation theory.

A context-sensitive and non-linguistic approach to abstract concepts.

Despite the recent upsurge in research on abstract concepts, there remain puzzles at the foundation of their empirical study. These are most evident when we consider what is required to assess a person's abstract conceptual abilities without using language as a prompt or requiring it as a response-as in classic non-verbal categorization tasks, which are standardly considered tests of conceptual understanding. After distinguishing two divergent strands in the most common conception of what it is for a concept to be abstract, we argue that neither reliably captures the kind of abstraction required to successfully categorize in non-verbal tasks. We then present a new conception of concept abstractness-termed 'trial concreteness'-that is keyed to individual categorization trials. It has advantages in capturing the context-relativity of the degree of abstraction required for the application of a concept and fittingly correlates with participant success in recent experiments. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Flexing Gender Perception: Brain Potentials Reveal the Cognitive Permeability of Grammatical Information.

A growing body of recent research suggests that verbal categories, particularly labels, impact categorization and perception. These findings are commonly interpreted as demonstrating the involvement of language on cognition; however, whether these assumptions hold true for grammatical structures has yet to be investigated. In the present study, we investigated the extent to which linguistic information, namely, grammatical gender categories, structures cognition to subsequently influence categorical judgments and perception. In a nonverbal categorization task, French-English bilinguals and monolingual English speakers made gender-associated judgments about a set of image pairs while event-related potentials were recorded. The image sets were composed of an object paired with either a female or male face, wherein the object was manipulated for their conceptual gender relatedness and grammatical gender congruency to the sex of the following target face. The results showed that grammatical gender modulated the N1 and P2/VPP, as well as the N300 exclusively for the French-English bilinguals, indicating the inclusion of language in the mechanisms associated with attentional bias and categorization. In contrast, conceptual gender information impacted the monolingual English speakers in the later N300 time window given the absence of a comparable grammatical feature. Such effects of grammatical categories in the early perceptual stream have not been found before, and further provide grounds to suggest that language shapes perception.

How do Korean–English bilinguals speak and think about motion events? Evidence from verbal and non-verbal tasks

AbstractThe present study compared both linguistic and non-linguistic representations of motion events in Korean–English sequential bilinguals sampled at varying proficiency levels (N = 80) against each other and against those of Korean and English monolinguals (N = 15 each). The bilinguals' L2 descriptions of motion events showed that their encoding patterns were influenced by both the first language (L1) and second language (L2) and also displayed unique behaviors that were not found in either monolingual norm. The non-verbal results on a triads-matching task demonstrated that bilinguals' categorization patterns followed L1-based patterns rather than L2-based patterns. The extent to which these bilinguals employed L2 encoding patterns in their motion event descriptions was largely modulated by L2 proficiency, whereas length of immersion experience in an L2-speaking country emerged as the only predictor of their non-verbal categorization patterns. These findings suggest that the bilinguals' verbal behavior seems more susceptible to change than their non-verbal behavior.

Human Non-linguistic Vocal Repertoire: Call Types and Their Meaning

Recent research on human nonverbal vocalizations has led to considerable progress in our understanding of vocal communication of emotion. However, in contrast to studies of animal vocalizations, this research has focused mainly on the emotional interpretation of such signals. The repertoire of human nonverbal vocalizations as acoustic types, and the mapping between acoustic and emotional categories, thus remain underexplored. In a cross-linguistic naming task (Experiment 1), verbal categorization of 132 authentic (non-acted) human vocalizations by English-, Swedish- and Russian-speaking participants revealed the same major acoustic types: laugh, cry, scream, moan, and possibly roar and sigh. The association between call type and perceived emotion was systematic but non-redundant: listeners associated every call type with a limited, but in some cases relatively wide, range of emotions. The speed and consistency of naming the call type predicted the speed and consistency of inferring the caller’s emotion, suggesting that acoustic and emotional categorizations are closely related. However, participants preferred to name the call type before naming the emotion. Furthermore, nonverbal categorization of the same stimuli in a triad classification task (Experiment 2) was more compatible with classification by call type than by emotion, indicating the former’s greater perceptual salience. These results suggest that acoustic categorization may precede attribution of emotion, highlighting the need to distinguish between the overt form of nonverbal signals and their interpretation by the perceiver. Both within- and between-call acoustic variation can then be modeled explicitly, bringing research on human nonverbal vocalizations more in line with the work on animal communication.

Satellite- vs. Verb-Framing Underpredicts Nonverbal Motion Categorization: Insights from a Large Language Sample and Simulations

Is motion cognition influenced by the large-scale typological patterns proposed in Talmy’s (2000) two-way distinction between verb-framed (V) and satellite-framed (S) languages? Previous studies investigating this question have been limited to comparing two or three languages at a time and have come to conflicting results. We present the largest cross-linguistic study on this question to date, drawing on data from nineteen genealogically diverse languages, all investigated in the same behavioral paradigm and using the same stimuli. After controlling for the different dependencies in the data by means of multilevel regression models, we find no evidence that S- vs. V-framing affects nonverbal categorization of motion events. At the same time, statistical simulations suggest that our study and previous work within the same behavioral paradigm suffer from insufficient statistical power. We discuss these findings in the light of the great variability between participants, which suggests flexibility in motion representation. Furthermore, we discuss the importance of accounting for language variability, something which can only be achieved with large cross-linguistic samples.

Differential effect of visual masking in perceptual categorization.

This article explores the visual information used to categorize stimuli drawn from a common stimulus space into verbal and nonverbal categories using 2 experiments. Experiment 1 explores the effect of target duration on verbal and nonverbal categorization using backward masking to interrupt visual processing. With categories equated for difficulty for long and short target durations, intermediate target duration shows an advantage for verbal categorization over nonverbal categorization. Experiment 2 tests whether the results of Experiment 1 can be explained by shorter target duration resulting in a smaller signal-to-noise ratio of the categorization stimulus. To test for this possibility, Experiment 2 used integration masking with the same stimuli, categories, and masks as Experiment 1 with a varying level of mask opacity. As predicted, low mask opacity yielded similar results to long target duration while high mask opacity yielded similar results to short target duration. Importantly, intermediate mask opacity produced an advantage for verbal categorization over nonverbal categorization, similar to intermediate target duration. These results suggest that verbal and nonverbal categorization are affected differently by manipulations affecting the signal-to-noise ratio of the stimulus, consistent with multiple-system theories of categorizations. The results further suggest that verbal categorization may be more digital (and more robust to low signal-to-noise ratio) while the information used in nonverbal categorization may be more analog (and less robust to lower signal-to-noise ratio). This article concludes with a discussion of how these new results affect the use of masking in perceptual categorization and multiple-system theories of perceptual category learning.

Learning to Think in a Second Language: Effects of Proficiency and Length of Exposure in English Learners of German

ABSTRACTThe aim of the current study is to investigate motion event cognition in second language learners in a higher education context. Based on recent findings that speakers of grammatical aspect languages like English attend less to the endpoint (goal) of events than do speakers of nonaspect languages like Swedish in a nonverbal categorization task involving working memory (Athanasopoulos & Bylund, 2013; Bylund & Athanasopoulos, 2015), the current study asks whether native speakers of an aspect language start paying more attention to event endpoints when learning a nonaspect language. Native English and German (a nonaspect language) speakers, and English learners of L2 German, who were pursuing studies in German language and literature at an English university, were asked to match a target scene with intermediate degree of endpoint orientation with two alternate scenes with low and high degree of endpoint orientation, respectively. Results showed that, compared to the native English speakers, the learners of German were more prone to base their similarity judgements on endpoint saliency, rather than ongoingness, primarily as a function of increasing L2 proficiency and year of university study. Further analyses revealed a nonlinear relationship between length of L2 exposure and categorization patterns, subserved by a progressive strengthening of the relationship between L2 proficiency and categorization as length of exposure increased. These findings present evidence that cognitive restructuring may occur through increasing experience with an L2, but also suggest that this relationship may be complex and unfold over a long period of time.

Continuous executive function disruption interferes with application of an information integration categorization strategy

Category learning is often characterized as being supported by two separate learning systems. A verbal system learns rule-defined (RD) categories that can be described using a verbal rule and relies on executive functions (EFs) to learn via hypothesis testing. A nonverbal system learns non-rule-defined (NRD) categories that cannot be described by a verbal rule and uses automatic, procedural learning. The verbal system is dominant in that adults tend to use it during initial learning but may switch to the nonverbal system when the verbal system is unsuccessful. The nonverbal system has traditionally been thought to operate independently of EFs, but recent studies suggest that EFs may play a role in the nonverbal system-specifically, to facilitate the transition away from the verbal system. Accordingly, continuously interfering with EFs during the categorization process, so that EFs are never fully available to facilitate the transition, may be more detrimental to the nonverbal system than is temporary EF interference. Participants learned an NRD or an RD category while EFs were untaxed, taxed temporarily, or taxed continuously. When EFs were continuously taxed during NRD categorization, participants were less likely to use a nonverbal categorization strategy than when EFs were temporarily taxed, suggesting that when EFs were unavailable, the transition to the nonverbal system was hindered. For the verbal system, temporary and continuous interference had similar effects on categorization performance and on strategy use, illustrating that EFs play an important but different role in each of the category-learning systems.

Relative Contribution of Perception/Cognition and Language on Spatial Categorization

This study investigated the relative contribution of perception/cognition and language-specific semantics in nonverbal categorization of spatial relations. English and Korean speakers completed a video-based similarity judgment task involving containment, support, tight fit, and loose fit. Both perception/cognition and language served as resources for categorization, and allocation between the two depended on the target relation and the features contrasted in the choices. Whereas perceptual/cognitive salience for containment and tight-fit features guided categorization in many contexts, language-specific semantics influenced categorization where the two features competed for similarity judgment and when the target relation was tight support, a domain where spatial relations are perceptually diverse. In the latter contexts, each group categorized more in line with semantics of their language, that is, containment/support for English and tight/loose fit for Korean. We conclude that language guides spatial categorization when perception/cognition alone is not sufficient. In this way, language is an integral part of our cognitive domain of space.

Conscious intention to speak proactively facilitates lexical access during overt object naming

The present study explored when and how the top-down intention to speak influences the language production process. We did so by comparing the brain’s electrical response for a variable known to affect lexical access, namely word frequency, during overt object naming and non-verbal object categorization. We found that during naming, the event-related brain potentials elicited for objects with low frequency names started to diverge from those with high frequency names as early as 152 ms after stimulus onset, while during non-verbal categorization the same frequency comparison appeared 200 ms later eliciting a qualitatively different brain response. Thus, only when participants had the conscious intention to name an object the brain rapidly engaged in lexical access. The data offer evidence that top-down intention to speak proactively facilitates the activation of words related to perceived objects.

Concept formation in non-verbal categorization tasks in brain-damaged patients with and without aphasia.

The study of thinking in patients with aphasic language impairments has long since been considered as means for learning about neuropsychological processes. In the present investigation, a non-verbal categorization task was used, requiring the subjects to sort several common objects according to different principles. The experimental group consisted of aphasic patients with motor and sensory aphasia. Two control groups were used, one consisting of normal subjects and the other of non-aphasic patients with injuries in the right hemisphere of the brain. The sorting behaviour was analysed according to several aspects. The final product of the categorization was assessed, as well as the behavioural processes observed during the sorting. The results were expressed in terms of different conceptual levels achieved. The results of the investigation showed that the aphasics were significantly inferior to normal controls both quantitatively and qualitatively in all sorting aspects. Possible mechanisms underlying the deficiencies in the sorting behaviour of the two aphasic groups were discussed.

Nonverbal categorization in fluent and nonfluent anomic aphasics

This study aimed to demonstrate that the naming difficulties of a particular group of aphasics, namely, fluent aphasics, are related to an underlying inability to organize feature set information. In order to test this hypothesis, the performance of fluent aphasics, nonfluent aphasics, and a nonaphasic brain-injured control group, was examined on a nonverbal categorization task, which was carefully structured in terms of instance typicality. Scores of visuoperceptual and naming tests were correlated with categorization task errors. As predicted, fluent aphasics showed a significant deficit in performance on the categorization task in comparison with other subjects. Differences in the nature of the errors the fluent aphasics made suggested that their problems were related to difficulties in abstracting the prototype for each category and in sorting category members with reference to these prototypes. For fluent aphasics, but not other subjects, a significant correlation was found between categorization task performance and naming ability.