Category-level Information Research Articles

UniParser: Multi-Human Parsing With Unified Correlation Representation Learning.

Multi-human parsing is an image segmentation task necessitating both instance-level and fine-grained category-level information. However, prior research has typically processed these two types of information through distinct branch types and output formats, leading to inefficient and redundant frameworks. This paper introduces UniParser, which integrates instance-level and category-level representations in three key aspects: 1) we propose a unified correlation representation learning approach, allowing our network to learn instance and category features within the cosine space; 2) we unify the form of outputs of each modules as pixel-level results while supervising instance and category features using a homogeneous label accompanied by an auxiliary loss; and 3) we design a joint optimization procedure to fuse instance and category representations. By unifying instance-level and category-level output, UniParser circumvents manually designed post-processing techniques and surpasses state-of-the-art methods, achieving 49.3% AP on MHPv2.0 and 60.4% AP on CIHP. We have released our source code, pretrained models, and demos to facilitate future studies on https://github.com/cjm-sfw/Uniparser.

Duplex perception reveals brainstem auditory representations are modulated by listeners' ongoing percept for speech.

So-called duplex speech stimuli with perceptually ambiguous spectral cues to one ear and isolated low- versus high-frequency third formant "chirp" to the opposite ear yield a coherent percept supporting their phonetic categorization. Critically, such dichotic sounds are only perceived categorically upon binaural integration. Here, we used frequency-following responses (FFRs), scalp-recorded potentials reflecting phase-locked subcortical activity, to investigate brainstem responses to fused speech percepts and to determine whether FFRs reflect binaurally integrated category-level representations. We recorded FFRs to diotic and dichotic stop-consonants (/da/, /ga/) that either did or did not require binaural fusion to properly label along with perceptually ambiguous sounds without clear phonetic identity. Behaviorally, listeners showed clear categorization of dichotic speech tokens confirming they were heard with a fused, phonetic percept. Neurally, we found FFRs were stronger for categorically perceived speech relative to category-ambiguous tokens but also differentiated phonetic categories for both diotically and dichotically presented speech sounds. Correlations between neural and behavioral data further showed FFR latency predicted the degree to which listeners labeled tokens as "da" versus "ga." The presence of binaurally integrated, category-level information in FFRs suggests human brainstem processing reflects a surprisingly abstract level of the speech code typically circumscribed to much later cortical processing.

A computational account of item-based directed forgetting for nonwords: Incorporating orthographic representations in MINERVA 2.

Recent research on item-method directed forgetting demonstrates that forget instructions not only decrease recognition for targets, but also decrease false recognition for foils from the same semantic categories as targets instructed to be forgotten. According to the selective rehearsal account of directed forgetting, this finding suggests that remember instructions may engage elaborative rehearsal of the category-level information of items. In contrast to this explanation, Reid and Jamieson (Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 76(2), 75-86, 2022) proposed that the differential rates of false recognition may emerge at retrieval when foils from "remember" and "forget" categories are compared to traces in memory. Using MINERVA S, an instance model of memory based on MINERVA 2 that incorporates structured semantic representations, Reid and Jamieson successfully simulated lower false recognition for foils from "forget" categories without assuming rehearsal of category-level information. In this study, we extend the directed forgetting paradigm to categories consisting of orthographically related nonwords. Presumably participants would have difficulty rehearsing category-level information for these items because they would have no pre-experimental knowledge of these categories. To simulate the findings in MINERVA S, we imported structured orthographic representations rather than semantic representations. The model not only predicted differential rates of false recognition for foils from "remember" and "forget" categories, but also predicted higher rates of false recognition overall than what was observed for semantic categories. The empirical data closely matched these predictions. These data suggest that differential rates of false recognition due to remember and forget instructions emerge at retrieval when participants compare recognition probes to traces stored in memory.

Extraction and generalisation of category-level information during visual statistical learning in autistic people.

We examined whether information extracted during a visual statistical learning task could be generalised from specific exemplars to semantically similar ones. We then looked at whether performance in autistic people differed to non-autistic people during a visual statistical learning task and specifically examined whether differences in performance between groups occurred when sequential information was presented at a semantic level. We did this by assessing recall performance using a two-alternative forced choice paradigm after presenting participants with a sequence of naturalistic scene images. 125 adult participants (61 participants with an autism diagnosis and 64 non-autistic controls) were presented with a fast serial presentation sequence of images and given a cover task to avoid attention being explicitly drawn to patterns in the underlying sequences. This was followed by a two-alternative forced choice task to assess participants' implicit recall. Participants were presented with 1 of 3 unique versions of the task, in which the presentation and assessment of statistical regularities was done at either a low feature-based level or a high semantic-based level. Participants were able to generalise statistical information from specific exemplars to semantically similar ones. There was an overall significant reduction in visual statistical learning in the autistic group but we were unable to determine whether group differences occurred specifically in conditions where the learning of semantic information was required. These results provide evidence that participants are able to extract statistical information that is presented at the level of specific exemplars and generalise it to semantically similar contexts. We also showed a modest but statistically significant reduction in recall performance in the autistic participants relative to the non-autistic participants.

Attention Constraints and Learning in Categories

Many decision makers are thought to economize on attention by processing information at the simpler level of a category. We directly test whether such category focus reflects an adaptive response to attention constraints, in five preregistered experiments using an information sampling paradigm with mouse tracking. Consistent with rational principles, participants focus more on category-level information when individual differences are small, when the category contains more members, and when time constraints are more severe. Participants are sensitive to the statistical structure of the category even when it must be learned from experience, and they respond to a latent shift in this structure. Beliefs about category members tend to cluster together more when category focus is high—a key element of rational inattention. However, this is counteracted by greater weight placed on salient and idiosyncratic information when the category is large. Our results broadly substantiate influential theories of categorical thinking, giving us a clearer view on the drivers and consequences of inattention. This paper was accepted by Marie Claire Villeval, behavioral economics and decision analysis. Funding: This work was supported by the Office of Naval Research (N00014-21-1-2170), the Pershing Square Fund for Research on the Foundations of Human Behavior, and an NWO Vidi grant (VI.Vidi.201.059). Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2023.4803 .

Multi-scale Triplet Hashing for Medical Image Retrieval

For medical image retrieval task, deep hashing algorithms are widely applied in large-scale datasets for auxiliary diagnosis due to the retrieval efficiency advantage of hash codes. Most of which focus on features learning, whilst neglecting the discriminate area of medical images and hierarchical similarity for deep features and hash codes. In this paper, we tackle these dilemmas with a new Multi-scale Triplet Hashing (MTH) algorithm, which can leverage multi-scale information, convolutional self-attention and hierarchical similarity to learn effective hash codes simultaneously. The MTH algorithm first designs multi-scale DenseBlock module to learn multi-scale information of medical images. Meanwhile, a convolutional self-attention mechanism is developed to perform information interaction of the channel domain, which can capture the discriminate area of medical images effectively. On top of the two paths, a novel loss function is proposed to not only conserve the category-level information of deep features and the semantic information of hash codes in the learning process, but also capture the hierarchical similarity for deep features and hash codes. Extensive experiments on the Curated X-ray Dataset, Skin Cancer MNIST Dataset and COVID-19 Radiography Dataset illustrate that the MTH algorithm can further enhance the effect of medical retrieval compared to other state-of-the-art medical image retrieval algorithms.

Perceptual warping exposes categorical representations for speech in human brainstem responses

The brain transforms continuous acoustic events into discrete category representations to downsample the speech signal for our perceptual-cognitive systems. Such phonetic categories are highly malleable, and their percepts can change depending on surrounding stimulus context. Previous work suggests these acoustic-phonetic mapping and perceptual warping of speech emerge in the brain no earlier than auditory cortex. Here, we examined whether these auditory-category phenomena inherent to speech perception occur even earlier in the human brain, at the level of auditory brainstem. We recorded speech-evoked frequency following responses (FFRs) during a task designed to induce more/less warping of listeners’ perceptual categories depending on stimulus presentation order of a speech continuum (random, forward, backward directions). We used a novel clustered stimulus paradigm to rapidly record the high trial counts needed for FFRs concurrent with active behavioral tasks. We found serial stimulus order caused perceptual shifts (hysteresis) near listeners’ category boundary confirming identical speech tokens are perceived differentially depending on stimulus context. Critically, we further show neural FFRs during active (but not passive) listening are enhanced for prototypical vs. category-ambiguous tokens and are biased in the direction of listeners’ phonetic label even for acoustically-identical speech stimuli. These findings were not observed in the stimulus acoustics nor model FFR responses generated via a computational model of cochlear and auditory nerve transduction, confirming a central origin to the effects. Our data reveal FFRs carry category-level information and suggest top-down processing actively shapes the neural encoding and categorization of speech at subcortical levels. These findings suggest the acoustic-phonetic mapping and perceptual warping in speech perception occur surprisingly early along the auditory neuroaxis, which might aid understanding by reducing ambiguity inherent to the speech signal.

Category-Level Assignment for Cross-Domain Semantic Segmentation in Remote Sensing Images

Deep learning-based semantic segmentation has made great progress in understanding very-high-resolution (VHR) remote sensing images (RSIs). However, large-scale applications are still limited. The main reason is that diverse imaging modes and geographical differences make it difficult to transfer a model trained in the source domain to the target domain. To solve this problem, unsupervised domain adaptation (UDA) for VHR RSIs has received some attention, but the accuracy of cross-domain semantic segmentation still needs to be improved. Currently, one reasonable proposal for improving accuracy is to take a close look at the category-level information. In this paper, we reveal an integer programming mechanism for modeling the category-level relationship between the source and target domains. The mechanism is based on the solution of the assignment problem, and thus, the proposed method is called category-level assignment for UDA (ClA-UDA). In ClA-UDA, a category-level assignment problem with additional constraints is defined for UDA tasks, and the solution is provided. Based on the solution, an assignment-based image-to-image transferring algorithm (AIT) is first proposed to transfer the source-domain images based on the style of the target-domain images. AIT minimizes a weighted discrepancy, and provides an analytical solution for the transfer. Two assignment-based alignment losses are then introduced to align the source and target domains based on the category-level relationship in a concise way. To validate the performance of ClA-UDA, three VHR remote sensing image datasets are employed, and six UDA tasks are designed. Extensive experiments are conducted, and the results demonstrate the superiority of ClA-UDA compared to the existing methods.

How Consumers Process Unexpected Online Advertisements

The distinctiveness of neural information representation is crucial for successful memory performance but declines with advancing age. Computational models implicate age-related neural dedifferentiation on the level of item representations, but previous studies mostly focused on age differences of categorical information representation in higher-order visual regions. In an age-comparative fMRI study, we combined univariate analyses and whole-brain searchlight pattern similarity analyses to elucidate age differences in neural distinctiveness at both category and item levels and their relation to memory. Thirty-five younger (18-27 years old) and 32 older (67-75 years old) women and men incidentally encoded images of faces and houses, followed by an old/new recognition memory task. During encoding, age-related neural dedifferentiation was shown as reduced category-selective processing in ventral visual cortex and impoverished item specificity in occipital regions. Importantly, successful subsequent memory performance built on high item stability, that is, high representational similarity between initial and repeated presentation of an item, which was greater in younger than older adults. Overall, we found that differences in representational distinctiveness coexist across representational levels and contribute to interindividual and intraindividual variability in memory success, with item specificity being the strongest contributor. Our results close an important gap in the literature, showing that older adults9 neural representation of item-specific information in addition to categorical information is reduced compared with younger adults. <b>SIGNIFICANCE STATEMENT</b> A long-standing hypothesis links age-related cognitive decline to a loss of neural specificity. While previous evidence supports the notion of age-related neural dedifferentiation of category-level information in ventral visual cortex, whether or not age differences exist at the item level was a matter of debate. Here, we observed age group differences at both levels as well as associations between both categorical distinctiveness and item specificity to memory performance, with item specificity being the strongest contributor. Importantly, age differences in occipital item specificity were largely due to reduced item stability across repetitions in older adults. Our results suggest that age differences in neural representations can be observed across the entire cortical hierarchy and are not limited to category-level information.

A computational model of item-based directed forgetting.

Montagliani and Hockley (2019) presented evidence that item-method directed forgetting not only leads to worse recognition of forget-cued targets than remember-cued targets but also better rejection of foils associated with forget-cued targets than remember-cued targets. Based on that result, they proposed that participants elaboratively encode more category-level information about R-cued targets. We present a retrieval-based explanation of the result within an instance-based memory model. The model imports word representations from two distributional semantic models, latent semantic analysis (LSA) and random permutation model (RPM), into an instance-based model of memory, MINERVA 2. The model reproduced Montagliani and Hockley's results without requiring assumptions about elaborated encoding of category-level information at study. The simulations demonstrate that whereas Montagliani and Hockley's findings are consistent with an account grounded in elaborated encoding of words at study, the results do not force that conclusion. Instead, better encoding of remember-cued targets at study establishes the conditions for retrieval-time effects at test to produce a corresponding influence on false recognition for category-related foils. Our model can be used as a formal tool to think about and study the incidental consequences of item directed forgetting in recognition memory. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Age-related neural dedifferentiation for individual stimuli: an across-participant pattern similarity analysis

ABSTRACT Age-related neural dedifferentiation – reductions in the regional specificity and precision of neural representations – is proposed to compromise the ability of older adults to form sufficiently distinct neural representations to support episodic memory encoding. The computational model that spurred investigations of age-related neural dedifferentiation initially characterized this phenomenon as a reduction in the specificity of neural patterns for individual items or stimuli. Most investigations have focused on reductions in neural differentiation for patterns of neural activity associated with category-level information, such as reduced neural selectivity between categories of visual stimuli (e.g., scenes, objects, and faces). Here, I report a novel across-participant pattern similarity analysis method to measure neural distinctiveness for individual stimuli that were presented to participants on a single occasion. Measures of item-level pattern similarity during encoding showed a graded positive subsequent memory effect in younger, with no significant subsequent memory effect in older adults. These results suggest that age-related reductions in the distinctiveness of neural patterns for individual stimuli during age differences in memory encoding. Moreover, a measure of category-level similarity demonstrated a significant subsequent memory effect associated with item recognition (regardless of an object source memory detail), whereas the effect in older was associated with source memory. These results converge with predictions of computational models of dedifferentiation showing age-related reductions in the distinctiveness of neural patterns across multiple levels of representation.

A Two-Way alignment approach for unsupervised multi-Source domain adaptation

Domain adaptation aims at transferring knowledge from labeled source domain to unlabeled target domain. Current advances primarily concern single source domain and neglect the setting of multiple source domains. Previous unsupervised multi-source domain adaptation (MDA) algorithms only consider domain-level alignment, while neglecting the category-level information among multiple domains and the instance variations inside each domain. This paper introduces a Two-Way alignment framework for MDA (TWMDA), which considers both domain-level and category-level alignments, and addresses the instance variations. We first align the target and multiple sources on the domain-level by an adversarial learning process. To circumvent the drawbacks of adversarial learning, we further reduce the domain gap on the category-level by minimizing the distance between the category prototypes and unlabeled target instances. To address the instance variations, we design an instance weighting strategy for diverse source instances. The effectiveness of TWMDA is demonstrated on three benchmark datasets for image classification.

Tracking Age Differences in Neural Distinctiveness across Representational Levels.

The distinctiveness of neural information representation is crucial for successful memory performance but declines with advancing age. Computational models implicate age-related neural dedifferentiation on the level of item representations, but previous studies mostly focused on age differences of categorical information representation in higher-order visual regions. In an age-comparative fMRI study, we combined univariate analyses and whole-brain searchlight pattern similarity analyses to elucidate age differences in neural distinctiveness at both category and item levels and their relation to memory. Thirty-five younger (18-27 years old) and 32 older (67-75 years old) women and men incidentally encoded images of faces and houses, followed by an old/new recognition memory task. During encoding, age-related neural dedifferentiation was shown as reduced category-selective processing in ventral visual cortex and impoverished item specificity in occipital regions. Importantly, successful subsequent memory performance built on high item stability, that is, high representational similarity between initial and repeated presentation of an item, which was greater in younger than older adults. Overall, we found that differences in representational distinctiveness coexist across representational levels and contribute to interindividual and intraindividual variability in memory success, with item specificity being the strongest contributor. Our results close an important gap in the literature, showing that older adults' neural representation of item-specific information in addition to categorical information is reduced compared with younger adults.SIGNIFICANCE STATEMENT A long-standing hypothesis links age-related cognitive decline to a loss of neural specificity. While previous evidence supports the notion of age-related neural dedifferentiation of category-level information in ventral visual cortex, whether or not age differences exist at the item level was a matter of debate. Here, we observed age group differences at both levels as well as associations between both categorical distinctiveness and item specificity to memory performance, with item specificity being the strongest contributor. Importantly, age differences in occipital item specificity were largely due to reduced item stability across repetitions in older adults. Our results suggest that age differences in neural representations can be observed across the entire cortical hierarchy and are not limited to category-level information.

Structural neural correlates of individual differences in categorical perception

Listeners perceive speech sounds categorically. While group-level differences in categorical perception have been observed in children or individuals with reading disorders, recent findings suggest that typical adults vary in how categorically they perceive sounds. The current study investigated neural sources of individual variability in categorical perception of speech. Fifty-seven participants rated phonetic tokens on a visual analogue scale; categoricity and response consistency were measured and related to measures of brain structure from MRI. Increased surface area of the right middle frontal gyrus predicted more categorical perception of a fricative continuum. This finding supports the idea that frontal regions are sensitive to phonetic category-level information and extends it to make behavioral predictions at the individual level. Additionally, more gyrification in bilateral transverse temporal gyri predicted less consistent responses on the task, perhaps reflecting subtle variation in language ability across the population.

Efficient Anonymous Temporal-Spatial Joint Estimation at Category Level Over Multiple Tag Sets With Unreliable Channels

Radio-frequency identification (RFID) technologies have been widely used in inventory control, object tracking and supply chain management. One of the fundamental system functions is called cardinality estimation, which is to estimate the number of tags in a covered area. In this paper, we extend the research of this function in two directions. First, we perform joint cardinality estimation among tags that appear at different geographical locations and at different times. Moreover, we target at category-level information, which is more significant in practical scenarios where we need to monitor the tagged objects of many different categories. Second, we enforce anonymity in the process of information gathering in order to preserve the privacy of the tagged objects. These capabilities will enable new applications such as tracking how products of different categories are transferred in a large, distributed supply chain. We propose and implement a novel protocol to meet the requirements of anonymous category-level joint estimation over multiple tag sets. We formally analyze the performance of our estimator and determine the optimal system parameters. Moreover, we extend our protocol to unreliable channels and consider two channel error models. Extensive simulations show that the proposed protocol can efficiently and accurately estimate joint information over multiple tag sets at category level, while preserving tags' anonymity.

Modeling the Effect of Speed Emphasis in Probabilistic Category Learning

For many decisions in life, there is uncertainty about the outcome even if the stimulus is a familiar one. Under these circumstances, people often rely upon previous experience to make a choice. In general, more accurate responding is achieved when one considers decision-relevant information for longer. There are practical limits, however, to how long one can deliberate, and so decision-makers must balance decision speed against accuracy. Traditionally, the overall speed with which people respond has been understood in terms of changes in the evidence threshold used for decision-making. However, more recent findings suggest that speed pressure may also affect the quality of information people recruit for making decisions. The current study investigated the effects of emphasizing either response speed or response accuracy on decision-making in a probabilistic category learning task. Strikingly, conclusions from two model-based analyses of performance depended on whether data were analyzed at the level of individuals or the group average. For the group-level analyses, speed pressure selectively influenced evidence threshold. For the individual-level analyses, speed pressure affected both evidence threshold as well as the quality of information driving decision-making for four out of six participants. Our findings are consistent with recent results that identify common brain areas associated with setting evidence thresholds decision-making and representing category-level information in trial-by-trial associative learning.

Discrete Deep Hashing With Ranking Optimization for Image Retrieval.

For large-scale image retrieval task, a hashing technique has attracted extensive attention due to its efficient computing and applying. By using the hashing technique in image retrieval, it is crucial to generate discrete hash codes and preserve the neighborhood ranking information simultaneously. However, both related steps are treated independently in most of the existing deep hashing methods, which lead to the loss of key category-level information in the discretization process and the decrease in discriminative ranking relationship. In order to generate discrete hash codes with notable discriminative information, we integrate the discretization process and the ranking process into one architecture. Motivated by this idea, a novel ranking optimization discrete hashing (RODH) method is proposed, which directly generates discrete hash codes (e.g., +1/-1) from raw images by balancing the effective category-level information of discretization and the discrimination of ranking information. The proposed method integrates convolutional neural network, discrete hash function learning, and ranking function optimizing into a unified framework. Meanwhile, a novel loss function based on label information and mean average precision (MAP) is proposed to preserve the label consistency and optimize the ranking information of hash codes simultaneously. Experimental results on four benchmark data sets demonstrate that RODH can achieve superior performance over the state-of-the-art hashing methods.

Plasticity in auditory categorization is supported by differential engagement of the auditory-linguistic network

To construct our perceptual world, the brain categorizes variable sensory cues into behaviorally-relevant groupings. Categorical representations are apparent within a distributed fronto-temporo-parietal brain network but how this neural circuitry is shaped by experience remains undefined. Here, we asked whether speech and music categories might be formed within different auditory-linguistic brain regions depending on listeners’ auditory expertise. We recorded EEG in highly skilled (musicians) vs. less experienced (nonmusicians) perceivers as they rapidly categorized speech and musical sounds. Musicians showed perceptual enhancements across domains, yet source EEG data revealed a double dissociation in the neurobiological mechanisms supporting categorization between groups. Whereas musicians coded categories in primary auditory cortex (PAC), nonmusicians recruited non-auditory regions (e.g., inferior frontal gyrus, IFG) to generate category-level information. Functional connectivity confirmed nonmusicians’ increased left IFG involvement reflects stronger routing of signal from PAC directed to IFG, presumably because sensory coding is insufficient to construct categories in less experienced listeners. Our findings establish auditory experience modulates specific engagement and inter-regional communication in the auditory-linguistic network supporting categorical perception. Whereas early canonical PAC representations are sufficient to generate categories in highly trained ears, less experienced perceivers broadcast information downstream to higher-order linguistic brain areas (IFG) to construct abstract sound labels.

Alpha-band oscillations track the retrieval of precise spatial representations from long-term memory.

A hallmark of episodic memory is the phenomenon of mentally reexperiencing the details of past events, and a well-established concept is that the neuronal activity that mediates encoding is reinstated at retrieval. Evidence for reinstatement has come from multiple modalities, including functional magnetic resonance imaging and electroencephalography (EEG). These EEG studies have shed light on the time course of reinstatement but have been limited to distinguishing between a few categories. The goal of this work was to use recently developed experimental and technical approaches, namely continuous report tasks and inverted encoding models, to determine which frequencies of oscillatory brain activity support the retrieval of precise spatial memories. In experiment 1, we establish that an inverted encoding model applied to multivariate alpha topography tracks the retrieval of precise spatial memories. In experiment 2, we demonstrate that the frequencies and patterns of multivariate activity at study are similar to the frequencies and patterns observed during retrieval. These findings highlight the broad potential for using encoding models to characterize long-term memory retrieval.NEW & NOTEWORTHY Previous EEG work has shown that category-level information observed during encoding is recapitulated during memory retrieval, but studies with this time-resolved method have not demonstrated the reinstatement of feature-specific patterns of neural activity during retrieval. Here we show that EEG alpha-band activity tracks the retrieval of spatial representations from long-term memory. Moreover, we find considerable overlap between the frequencies and patterns of activity that track spatial memories during initial study and at retrieval.

Odor-evoked category reactivation in human ventromedial prefrontal cortex during sleep promotes memory consolidation.

Slow-wave sleep is an optimal opportunity for memory consolidation: when encoding occurs in the presence of a sensory cue, delivery of that cue during sleep enhances retrieval of associated memories. Recent studies suggest that cues might promote consolidation by inducing neural reinstatement of cue-associated content during sleep, but direct evidence for such mechanisms is scant, and the relevant brain areas supporting these processes are poorly understood. Here, we address these gaps by combining a novel olfactory cueing paradigm with an object-location memory task and simultaneous EEG-fMRI recording in human subjects. Using pattern analysis of fMRI ensemble activity, we find that presentation of odor cues during sleep promotes reactivation of category-level information in ventromedial prefrontal cortex that significantly correlates with post-sleep memory performance. In identifying the potential mechanisms by which odor cues selectively modulate memory in the sleeping brain, these findings bring unique insights into elucidating how and what we remember.