Categorization Behavior Research Articles

Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks

Understanding the dynamical transformation of neural activity to behavior requires new capabilities to nonlinearly model, dissociate and prioritize behaviorally relevant neural dynamics and test hypotheses about the origin of nonlinearity. We present dissociative prioritized analysis of dynamics (DPAD), a nonlinear dynamical modeling approach that enables these capabilities with a multisection neural network architecture and training approach. Analyzing cortical spiking and local field potential activity across four movement tasks, we demonstrate five use-cases. DPAD enabled more accurate neural–behavioral prediction. It identified nonlinear dynamical transformations of local field potentials that were more behavior predictive than traditional power features. Further, DPAD achieved behavior-predictive nonlinear neural dimensionality reduction. It enabled hypothesis testing regarding nonlinearities in neural–behavioral transformation, revealing that, in our datasets, nonlinearities could largely be isolated to the mapping from latent cortical dynamics to behavior. Finally, DPAD extended across continuous, intermittently sampled and categorical behaviors. DPAD provides a powerful tool for nonlinear dynamical modeling and investigation of neural–behavioral data.

Pre-frontal cortex guides dimension-reducing transformations in the occipito-ventral pathway for categorization behaviors

To interpret our surroundings, the brain uses a visual categorization process. Current theories and models suggest that this process comprises a hierarchy of different computations that transforms complex, high-dimensional inputs into lower-dimensional representations (i.e., manifolds) in support of multiple categorization behaviors. Here, we tested this hypothesis by analyzing these transformations reflected in dynamic MEG source activity while individual participants actively categorized the same stimuli according to different tasks: face expression, face gender, pedestrian gender, and vehicle type. Results reveal three transformation stages guided by the pre-frontal cortex. At stage 1 (high-dimensional, 50-120ms), occipital sources represent both task-relevant and task-irrelevant stimulus features; task-relevant features advance into higher ventral/dorsal regions, whereas task-irrelevant features halt at the occipital-temporal junction. At stage 2 (121-150ms), stimulus feature representations reduce to lower-dimensional manifolds, which then transform into the task-relevant features underlying categorization behavior over stage 3 (161-350ms). Our findings shed light on how the brain's network mechanisms transform high-dimensional inputs into specific feature manifolds that support multiple categorization behaviors.

The Information-Processing Perspective on Categorization.

Categorization behavior can be fruitfully analyzed in terms of the trade-off between as high as possible faithfulness in the transmission of information about samples of the classes to be categorized, and as low as possible transmission costs for that same information. The kinds of categorization behaviors we associate with conceptual atoms, prototypes, and exemplars emerge naturally as a result of this trade-off, in the presence of certain natural constraints on the probabilistic distribution of samples, and the ways in which we measurefaithfulness. Beyond the general structure of categorization in these circumstances, the same information-centered perspective can shed light on other, more concrete properties of human categorization performance, such as the results of certain prominent experiments on supervised categorization.

Variability in cross-language and cross-dialect perception. How Irish and Chinese migrants process Australian English vowels.

We investigate how three adult groups- experienced L2 English listeners; experienced D2 (second dialect) listeners; and native L1/D1 listeners- categorise Australian English (AusE) lax front vowels /ɪ e æ/ in /hVt/, /hVl/ and /mVl/ environments in a forced-choice categorisation task of synthesised continua. In study 1, AusE listeners show predictable categorisations, with an effect of coarticulation raising the vowel in perception for nasal onset stimuli, and a following lateral lowering the vowel in perception. In study 2, Irish (D2) and Chinese listeners (L2) have different categorisations than AusE listeners, likely guided by their D1/L1. Coarticulation influences the D1/D2 groups in similar ways, but results in more difficulty and less agreement for the Chinese. We also investigate the role of extralinguistic factors. For the Chinese listeners, higher proficiency in English does not correlate with more Australian-like categorisation behaviour. However, having fewer Chinese in their social network results in more Australian-like categorisation for some stimuli. These findings lend partial support to the role of experience and exposure in L2/D2 contexts, whereby categorisation is likely still driven by native categories, with increased exposure leading to better mapping, but not to a restructuring of underlying phonetic categories.

Access to meaning from visual input: Object and word frequency effects in categorization behavior.

Object and word recognition are both cognitive processes that transform visual input into meaning. When reading words, the frequency of their occurrence ("word frequency," WF) strongly modulates access to their meaning, as seen in recognition performance. Does the frequency of objects in our world also affect access to their meaning? With object labels available in real-world image datasets, one can now estimate the frequency of occurrence of objects in scenes ("object frequency," OF). We explored frequency effects in word and object recognition behavior by employing a natural versus man-made categorization task (Experiment 1) and a matching-mismatching priming task (Experiments 2-3). In Experiment 1, we found a WF effect for both words and objects but no OF effect. In Experiment 2, we replicated the WF effect for both stimulus types during cross-modal priming but not uni-modal priming. Moreover, in cross-modal priming, we found an OF effect for both objects and words, but with faster responses when objects occur less frequently in image datasets. We replicated this counterintuitive OF effect in Experiment 3 and suggest that better recognition of rare objects might interact with the structure of object categories: while access to the meaning of objects and words is faster when their meaning often occurs in our language, the homogeneity of object categories seems to also impact recognition, mainly when semantic processing happens in the context of previously presented information. These findings have major implications for studies attempting to include frequency measures in investigations of access to meaning from visual inputs. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The pigeon as a machine: Complex category structures can be acquired by a simple associative model

Never known for its smarts, the pigeon has proven to be a prodigious classifier of complex visual stimuli. What explains its surprising success? Does it possess elaborate executive functions akin to those deployed by humans? Or does it effectively deploy an unheralded, but powerful associative learning mechanism? In a series of experiments, we first confirm that pigeons can learn a variety of category structures- some devised to foil the use of advanced cognitive processes. We then contrive a simple associative learning model to see how effectively the model learns the same tasks given to pigeons. The close fit of the associative model to pigeons' categorization behavior provides unprecedented support for associative learning as a viable mechanism for mastering complex category structures and for the pigeon's using this mechanism to adapt to a rich visual world. This model will help guide future neuroscientific research into the biological substrates of visual cognition.

The effects of sensorimotor and linguistic information on the basic-level advantage

The basic-level advantage is one of the best-known effects in human categorisation. Traditional accounts argue that basic-level categories present a maximally informative or entry level into a taxonomic organisation of concepts in semantic memory. However, these explanations are not fully compatible with most recent views on the structure of the conceptual system such as linguistic-simulation accounts, which emphasise the dual role of sensorimotor (i.e., perception-action experience of the world) and linguistic distributional information (i.e., statistical distribution of words in language) in conceptual processing. In four preregistered word→picture categorisation studies, we examined whether novel measures of sensorimotor and linguistic distance contribute to the basic level-advantage in categorical decision-making. Results showed that overlap in sensorimotor experience between category concept and member concept (e.g., animal→dog) predicted RT and accuracy at least as well as a traditional division into discrete subordinate, basic, and superordinate taxonomic levels. Furthermore, linguistic distributional information contributed to capturing effects of graded category structure where typicality ratings did not. Finally, when image label production frequency was taken into account (i.e., how often people actually produced specific labels for images), linguistic distributional information predicted RT and accuracy above and beyond sensorimotor information. These findings add to our understanding of how sensorimotor-linguistic theories of the conceptual system can explain categorisation behaviour.

The Social Route to Abstraction: Interaction and Diversity Enhance Performance and Transfer in a Rule-Based Categorization Task.

Capacities for abstract thinking and problem-solving are central to human cognition. Processes of abstraction allow the transfer of experiences and knowledge between contexts helping us make informed decisions in new or changing contexts. While we are often inclined to relate such reasoning capacities to individual minds and brains, they may in fact be contingent on human-specific modes of collaboration, dialogue, and shared attention. In an experimental study, we test the hypothesis that social interaction enhances cognitive processes of rule-induction, which in turn improves problem-solving performance. Through three sessions of increasing complexity, individuals and groups were presented with a problem-solving task requiring them to categorize a set of visual stimuli. To assess the character of participants' problem representations, after each training session, they were presented with a transfer task involving stimuli that differed in appearance, but shared relations among features with the training set. Besides, we compared participants' categorization behaviors to simulated agents relying on exemplar learning. We found that groups performed superior to individuals and agents in the training sessions and were more likely to correctly generalize their observations in the transfer phase, especially in the high complexity session, suggesting that groups more effectively induced underlying categorization rules from the stimuli than individuals and agents. Crucially, variation in performance among groups was predicted by semantic diversity in members' dialogical contributions, suggesting a link between social interaction, cognitive diversity, and abstraction.

Scene representations underlying categorization behaviour emerge 100 to 200 ms after stimulus onset

Scene representations underlying categorization behaviour emerge 100 to 200 ms after stimulus onset

Network Communications Flexibly Predict Visual Contents That Enhance Representations for Faster Visual Categorization.

Models of visual cognition generally assume that brain networks predict the contents of a stimulus to facilitate its subsequent categorization. However, understanding prediction and categorization at a network level has remained challenging, partly because we need to reverse engineer their information processing mechanisms from the dynamic neural signals. Here, we used connectivity measures that can isolate the communications of a specific content to reconstruct these network mechanisms in each individual participant (N = 11, both sexes). Each was cued to the spatial location (left vs right) and contents [low spatial frequency (LSF) vs high spatial frequency (HSF)] of a predicted Gabor stimulus that they then categorized. Using each participant's concurrently measured MEG, we reconstructed networks that predict and categorize LSF versus HSF contents for behavior. We found that predicted contents flexibly propagate top down from temporal to lateralized occipital cortex, depending on task demands, under supervisory control of prefrontal cortex. When they reach lateralized occipital cortex, predictions enhance the bottom-up LSF versus HSF representations of the stimulus, all the way from occipital-ventral-parietal to premotor cortex, in turn producing faster categorization behavior. Importantly, content communications are subsets (i.e., 55-75%) of the signal-to-signal communications typically measured between brain regions. Hence, our study isolates functional networks that process the information of cognitive functions.SIGNIFICANCE STATEMENT An enduring cognitive hypothesis states that our perception is partly influenced by the bottom-up sensory input but also by top-down expectations. However, cognitive explanations of the dynamic brain networks mechanisms that flexibly predict and categorize the visual input according to task-demands remain elusive. We addressed them in a predictive experimental design by isolating the network communications of cognitive contents from all other communications. Our methods revealed a Prediction Network that flexibly communicates contents from temporal to lateralized occipital cortex, with explicit frontal control, and an occipital-ventral-parietal-frontal Categorization Network that represents more sharply the predicted contents from the shown stimulus, leading to faster behavior. Our framework and results therefore shed a new light of cognitive information processing on dynamic brain activity.

Disentangling diagnostic object properties for human scene categorization

It usually only takes a single glance to categorize our environment into different scene categories (e.g. a kitchen or a highway). Object information has been suggested to play a crucial role in this process, and some proposals even claim that the recognition of a single object can be sufficient to categorize the scene around it. Here, we tested this claim in four behavioural experiments by having participants categorize real-world scene photographs that were reduced to a single, cut-out object. We show that single objects can indeed be sufficient for correct scene categorization and that scene category information can be extracted within 50 ms of object presentation. Furthermore, we identified object frequency and specificity for the target scene category as the most important object properties for human scene categorization. Interestingly, despite the statistical definition of specificity and frequency, human ratings of these properties were better predictors of scene categorization behaviour than more objective statistics derived from databases of labelled real-world images. Taken together, our findings support a central role of object information during human scene categorization, showing that single objects can be indicative of a scene category if they are assumed to frequently and exclusively occur in a certain environment.

Comparing pre-linguistic normalization models against US English listeners’ perception of natural and resynthesized vowels

Talkers vary in their vowel pronunciation. One hypothesis holds that listeners achieve robust speech perception through pre-linguistic normalization. In recent work (also submitted to ASA), we modeled listeners’ perception of naturally produced /h-VOWEL-d/ words. The best-performing normalization models accounted for ∼90% of the explainable variance in listeners’ responses. Here, we investigate whether the remaining 10% follow from (1) other mechanisms or whether (2) they reflect listeners’ ability to use more cues than available to models. We constructed a new set of *synthesized* /h-VOWEL-d/ stimuli that varied only in F1 and F2. Unsurprisingly, listeners (N = 24) performed worse on these synthesized stimuli than on the natural stimuli (estimated as inter-listener agreement in categorization). Critically though, we find (1) that the same normalization accounts that best explained listeners’ responses to natural stimuli also perform best explaining responses to synthesized stimuli; (2) the best performing model again accounted for ∼90% of explainable variance. This suggests that the ‘failure’ of normalization accounts to fully explain listeners’ categorization behavior is *not* due to restrictions in the ability to feed our models all available cues. Rather, normalization alone—while critical to perception—seems insufficient to fully explain listeners’ ability to adapt based on recent input.

Supplemental Material for Access to Meaning From Visual Input: Object and Word Frequency Effects in Categorization Behavior

Supplemental Material for Access to Meaning From Visual Input: Object and Word Frequency Effects in Categorization Behavior

Vocalization categorization behavior explained by a feature-based auditory categorization model.

Vocal animals produce multiple categories of calls with high between- and within-subject variability, over which listeners must generalize to accomplish call categorization. The behavioral strategies and neural mechanisms that support this ability to generalize are largely unexplored. We previously proposed a theoretical model that accomplished call categorization by detecting features of intermediate complexity that best contrasted each call category from all other categories. We further demonstrated that some neural responses in the primary auditory cortex were consistent with such a model. Here, we asked whether a feature-based model could predict call categorization behavior. We trained both the model and guinea pigs (GPs) on call categorization tasks using natural calls. We then tested categorization by the model and GPs using temporally and spectrally altered calls. Both the model and GPs were surprisingly resilient to temporal manipulations, but sensitive to moderate frequency shifts. Critically, the model predicted about 50% of the variance in GP behavior. By adopting different model training strategies and examining features that contributed to solving specific tasks, we could gain insight into possible strategies used by animals to categorize calls. Our results validate a model that uses the detection of intermediate-complexity contrastive features to accomplish call categorization.

Abstract Neural Representations of Category Membership beyond Information Coding Stimulus or Response.

For decades, researchers have debated whether mental representations are symbolic or grounded in sensory inputs and motor programs. Certainly, aspects of mental representations are grounded. However, does the brain also contain abstract concept representations that mediate between perception and action in a flexible manner not tied to the details of sensory inputs and motor programs? Such conceptual pointers would be useful when concepts remain constant despite changes in appearance and associated actions. We evaluated whether human participants acquire such representations using fMRI. Participants completed a probabilistic concept learning task in which sensory, motor, and category variables were not perfectly coupled or entirely independent, making it possible to observe evidence for abstract representations or purely grounded representations. To assess how the learned concept structure is represented in the brain, we examined brain regions implicated in flexible cognition (e.g., pFC and parietal cortex) that are most likely to encode an abstract representation removed from sensory-motor details. We also examined sensory-motor regions that might encode grounded sensory-motor-based representations tuned for categorization. Using a cognitive model to estimate participants' category rule and multivariate pattern analysis of fMRI data, we found the left pFC and human middle temporal visual area (MT)/V5 coded for category in the absence of information coding for stimulus or response. Because category was based on the stimulus, finding an abstract representation of category was not inevitable. Our results suggest that certain brain areas support categorization behavior by constructing concept representations in a format akin to a symbol that differs from stimulus-motor codes.

Classification of three-dimensional integral stimuli: Accounting for a replication and extension of Nosofsky and Palmeri (1996) with a dual discrimination invariance model.

The quest for determining the degree of learning difficulty associated with different types of categories has been instrumental in our understanding of human categorization behavior and, more broadly, human generalization. For instance, we now know that the topological nature of the dimensions (e.g., whether these are integral or separable) that define the family of categories generated with three binary dimensions yield two different learning difficulty orderings. In our study, for the first time, we replicated one such classic ordering involving integral dimensions and explored the impact that the choice of dimensional values, along with the nature of the stimulus presentation, had on learning difficulty during the learning phase of our four experiments. Two standard orderings were observed in our investigation which were consistent with the hypothesis that multilevel discrimination plays a key role in concept learning. We accounted for our empirical results by making this role explicit at three levels, the feature, object, and category structure levels, using a dual discrimination invariance model (DDIM) derived from the core invariance law of generalized invariance structure theory (GIST). The model involves a "discrimination switch" via what is referred to in GIST as the "tau discrimination threshold" where two distinct extreme degrees of discrimination yield two fundamental learning difficulty orderings able to account for results from our experiments without free parameters. We then show how the DDIM is not only a more accurate and general predictor than the best alternatives but also provides a plausible and tenable cognitive mechanism for understanding these behaviors. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Recurrent Connections in the Primate Ventral Visual Stream Mediate a Trade-Off Between Task Performance and Network Size During Core Object Recognition.

The computational role of the abundant feedback connections in the ventral visual stream is unclear, enabling humans and nonhuman primates to effortlessly recognize objects across a multitude of viewing conditions. Prior studies have augmented feedforward convolutional neural networks (CNNs) with recurrent connections to study their role in visual processing; however, often these recurrent networks are optimized directly on neural data or the comparative metrics used are undefined for standard feedforward networks that lack these connections. In this work, we develop task-optimized convolutional recurrent (ConvRNN) network models that more correctly mimic the timing and gross neuroanatomy of the ventral pathway. Properly chosen intermediate-depth ConvRNN circuit architectures, which incorporate mechanisms of feedforward bypassing and recurrent gating, can achieve high performance on a core recognition task, comparable to that of much deeper feedforward networks. We then develop methods that allow us to compare both CNNs and ConvRNNs to finely grained measurements of primate categorization behavior and neural response trajectories across thousands of stimuli. We find that high-performing ConvRNNs provide a better match to these data than feedforward networks of any depth, predicting the precise timings at which each stimulus is behaviorally decoded from neural activation patterns. Moreover, these ConvRNN circuits consistently produce quantitatively accurate predictions of neural dynamics from V4 and IT across the entire stimulus presentation. In fact, we find that the highest-performing ConvRNNs, which best match neural and behavioral data, also achieve a strong Pareto trade-off between task performance and overall network size. Taken together, our results suggest the functional purpose of recurrence in the ventral pathway is to fit a high-performing network in cortex, attaining computational power through temporal rather than spatial complexity.

Long-term priors constrain category learning in the context of short-term statistical regularities

Cognitive systems face a constant tension of maintaining existing representations that have been fine-tuned to long-term input regularities and adapting representations to meet the needs of short-term input that may deviate from long-term norms. Systems must balance the stability of long-term representations with plasticity to accommodate novel contexts. We investigated the interaction between perceptual biases or priors acquired across the long-term and sensitivity to statistical regularities introduced in the short-term. Participants were first passively exposed to short-term acoustic regularities and then learned categories in a supervised training task that either conflicted or aligned with long-term perceptual priors. We found that the long-term priors had robust and pervasive impact on categorization behavior. In contrast, behavior was not influenced by the nature of the short-term passive exposure. These results demonstrate that perceptual priors place strong constraints on the course of learning and that short-term passive exposure to acoustic regularities has limited impact on directing subsequent category learning.

REFRESH: A new approach to modeling dimensional biases in perceptual similarity and categorization.

Much categorization behavior can be explained by family resemblance: New items are classified by comparison with previously learned exemplars. However, categorization behavior also shows a variety of dimensional biases, where the underlying space has so-called “separable” dimensions: Ease of learning categories depends on how the stimuli align with the separable dimensions of the space. For example, if a set of objects of various sizes and colors can be accurately categorized using a single separable dimension (e.g., size), then category learning will be fast, while if the category is determined by both dimensions, learning will be slow. To capture these dimensional biases, almost all models of categorization supplement family resemblance with either rule-based systems or selective attention to separable dimensions. But these models do not explain how separable dimensions initially arise; they are presumed to be unexplained psychological primitives. We develop, instead, a pure family resemblance version of the Rational Model of Categorization (RMC), which we term the Rational Exclusively Family RESemblance Hierarchy (REFRESH), which does not presuppose any separable dimensions in the space of stimuli. REFRESH infers how the stimuli are clustered and uses a hierarchical prior to learn expectations about the variability of clusters across categories. We first demonstrate the dimensional alignment of natural-category features and then show how through a lifetime of categorization experience REFRESH will learn prior expectations that clusters of stimuli will align with separable dimensions. REFRESH captures the key dimensional biases and also explains their stimulus-dependence and how they are learned and develop.

Grounding deep neural network predictions of human categorization behavior in understandable functional features: The case of face identity

SummaryDeep neural networks (DNNs) can resolve real-world categorization tasks with apparent human-level performance. However, true equivalence of behavioral performance between humans and their DNN models requires that their internal mechanisms process equivalent features of the stimulus. To develop such feature equivalence, our methodology leveraged an interpretable and experimentally controlled generative model of the stimuli (realistic three-dimensional textured faces). Humans rated the similarity of randomly generated faces to four familiar identities. We predicted these similarity ratings from the activations of five DNNs trained with different optimization objectives. Using information theoretic redundancy, reverse correlation, and the testing of generalization gradients, we show that DNN predictions of human behavior improve because their shape and texture features overlap with those that subsume human behavior. Thus, we must equate the functional features that subsume the behavioral performances of the brain and its models before comparing where, when, and how these features are processed.