Classes Of Stimuli Research Articles

Using multivariate pattern analysis to increase effect sizes for event-related potential analyses.

Multivariate pattern analysis (MVPA) approaches can be applied to the topographic distribution of event-related potential (ERP) signals to "decode" subtly different stimulus classes, such as different faces or different orientations. These approaches are extremely sensitive, and it seems possible that they could also be used to increase effect sizes and statistical power in traditional paradigms that ask whether an ERP component differs in amplitude across conditions. To assess this possibility, we leveraged the open-source ERP CORE data set and compared the effect sizes resulting from conventional univariate analyses of mean amplitude with two MVPA approaches (support vector machine decoding and the cross-validated Mahalanobis distance, both of which are easy to compute using open-source software). We assessed these approaches across seven widely studied ERP components (N170, N400, N2pc, P3b, lateral readiness potential, error related negativity, and mismatch negativity). Across all components, we found that multivariate approaches yielded effect sizes that were as large or larger than the effect sizes produced by univariate approaches. These results indicate that researchers could obtain larger effect sizes, and therefore greater statistical power, by using multivariate analysis of topographic voltage patterns instead of traditional univariate analyses in many ERP studies.

Gender effect in affective processing: Alpha EEG source analysis on emotional slides and film-clips.

Past research on gender-related brain asymmetries in emotions was limited and not univocal. The present study analyzed EEG alpha activity (indexing cortical de-activation) from 64 scalp sites in 20 women and 20 men during a counterbalanced block presentation of emotional slides and short video-clips. Stimuli consisted of 45 brief clips of 13 s, divided into 15 erotic (pleasant), 15 neutral and 15 fear (unpleasant) contents. Slides consisted in 45 photo shots (presented for 13 s each) extracted from the videos. As expected, women perceived fear stimuli as more arousing and more unpleasant compared to men. Alpha EEG source analysis revealed gender effects depending on stimulus. Emotional film-clips elicited in both groups a pattern of greater right than left occipital activation. Instead, emotional pictures activated opposite occipital regions, as women showed greater activation in the left, men in the right hemisphere. Men also showed greater activation to Erotic compared to Fear stimuli (i.e., pictures/clips) in the posterior parietal complex. Results point to the relevance of emotional stimulus type to reveal gender effects: clips are ecological, dynamic and engaging, and forced a unified pattern of emotional responses that reset individual differences. Emotional pictures, less engaging, allowed individual differences to emerge and interact with the stimulus category.

A stimulus exposure of 50 ms elicits the uncanny valley effect

The uncanny valley (UV) effect captures the observation that artificial entities with near-human appearances tend to create feelings of eeriness. Researchers have proposed many hypotheses to explain the UV effect, but the visual processing mechanisms of the UV have yet to be fully understood. In the present study, we examined if the UV effect is as accessible in brief stimulus exposures compared to long stimulus exposures (Experiment 1). Forty-one participants, aged 21–31, rated each human-robot face presented for either a brief (50 ms) or long duration (3 s) in terms of attractiveness, eeriness, and humanness (UV indices) in a 7-point Likert scale. We found that brief and long exposures to stimuli generated a similar UV effect. This suggests that the UV effect is accessible at early visual processing. We then examined the effect of exposure duration on the categorisation of visual stimuli in Experiment 2. Thirty-three participants, aged 21–31, categorised faces as either human or robot in a two-alternative forced choice task. Their response accuracy and variance were recorded. We found that brief stimulus exposures generated significantly higher response variation and errors than the long exposure condition. This indicated that participants were more uncertain in categorising faces in the brief exposure condition due to insufficient time. Further comparisons between Experiment 1 and 2 revealed that the eeriest faces were not the hardest to categorise. Overall, these findings indicate (1) that both the UV effect and categorical uncertainty can be elicited through brief stimulus exposure, but (2) that categorical uncertainty is unlikely to cause the UV effect. These findings provide insights towards the perception of robotic faces and implications for the design of robots, androids, avatars, and artificial intelligence agents.

Assessing Spontaneous Categorical Processing of Visual Shapes via Frequency-Tagging EEG.

Categorization is an essential cognitive and perceptual process, which happens spontaneously. However, earlier research often neglected the spontaneous nature of this process by mainly adopting explicit tasks in behavioral or neuroimaging paradigms. Here, we use frequency-tagging (FT) during electroencephalography (EEG) in 22 healthy human participants (both male and female) as a direct approach to pinpoint spontaneous visual categorical processing. Starting from schematic natural visual stimuli, we created morph sequences comprising 11 equal steps. Mirroring a behavioral categorical perception discrimination paradigm, we administered a FT-EEG oddball paradigm, assessing neural sensitivity for equally sized differences within and between stimulus categories. Likewise, mirroring a behavioral category classification paradigm, we administered a sweep FT-EEG oddball paradigm, sweeping from one end of the morph sequence to the other, thereby allowing us to objectively pinpoint the neural category boundary. We found that FT-EEG can implicitly measure categorical processing and discrimination. More specifically, we could derive an objective neural index of the required level to differentiate between the two categories, and this neural index showed the typical marker of categorical perception (i.e., stronger discrimination across as compared with within categories). The neural findings of the implicit paradigms were also validated using an explicit behavioral task. These results provide evidence that FT-EEG can be used as an objective tool to measure discrimination and categorization and that the human brain inherently and spontaneously (without any conscious or decisional processes) uses higher-level meaningful categorization information to interpret ambiguous (morph) shapes.

Linguistic and attentional factors – Not statistical regularities – Contribute to word-selective neural responses with FPVS-oddball paradigms

Studies using frequency-tagging in electroencephalography (EEG) have dramatically increased in the past 10 years, in a variety of domains and populations. Here we used Fast Periodic Visual Stimulation (FPVS) combined with an oddball design to explore visual word recognition. Given the paradigm's high sensitivity, it is crucial for future basic research and clinical application to prove its robustness across variations of designs, stimulus types and tasks. This paradigm uses periodicity of brain responses to measure discrimination between two experimentally defined categories of stimuli presented periodically. EEG was recorded in 22 adults who viewed words inserted every 5 stimuli (at 2 Hz) within base stimuli presented at 10 Hz. Using two discrimination levels (deviant words among nonwords or pseudowords), we assessed the impact of relative frequency of item repetition (set size or item repetition controlled for deviant versus base stimuli), and of the orthogonal task (focused or deployed spatial attention). Word-selective occipito-temporal responses were robust at the individual level (significant in 95% of participants), left-lateralized, larger for the prelexical (nonwords) than lexical (pseudowords) contrast, and stronger with a deployed spatial attention task as compared to the typically used focused task. Importantly, amplitudes were not affected by item repetition. These results help understanding the factors influencing word-selective EEG responses and support the validity of FPVS-EEG oddball paradigms, as they confirm that word-selective responses are linguistic. Second, they show its robustness against design-related factors that could induce statistical (ir)regularities in item rate. They also confirm its high individual sensitivity and demonstrate how it can be optimized, using a deployed rather than focused attention task, to measure implicit word recognition processes in typical and atypical populations.

The effects of a training package to teach note taking on the formation of equivalence classes.

Effective note taking may enhance learning outcomes for students and serve as a directly observable form of mediation within a test context. Frampton et al. (2023) used stimulus fading to teach note taking in the form of a graphic organizer (GO) during matching-to-sample baseline relations training (MTS-BRT). Moderately high yields were observed with young adults despite the use of linear series training, abstract stimuli, and five-member classes. The present study taught the same note taking strategy using an intervention package including video illustration, voice-over instructions, and feedback to eight college students. Participants were taught to construct the GO during MTS-BRT with three three-member classes of familiar stimuli. Then the effects of MTS-BRT alone with three five-member classes of abstract stimuli was evaluated. Participants efficiently completed training with familiar stimuli and passed the posttest on the first attempt. With the abstract stimuli, participants engaged in GO construction during MTS-BRT and the six participants that demonstrated high levels of fidelity to the trained note taking strategy passed the posttest on the first attempt. These results replicate findings from Frampton et al. while using a more efficient intervention package. Benefits of teaching overt mediation responses are discussed as well as future directions for translation to applied contexts.

Developing a hippocampal neural prosthetic to facilitate human memory encoding and recall of stimulus features and categories.

Here, we demonstrate the first successful use of static neural stimulation patterns for specific information content. These static patterns were derived by a model that was applied to a subject's own hippocampal spatiotemporal neural codes for memory. We constructed a new model of processes by which the hippocampus encodes specific memory items via spatiotemporal firing of neural ensembles that underlie the successful encoding of targeted content into short-term memory. A memory decoding model (MDM) of hippocampal CA3 and CA1 neural firing was computed which derives a stimulation pattern for CA1 and CA3 neurons to be applied during the encoding (sample) phase of a delayed match-to-sample (DMS) human short-term memory task. MDM electrical stimulation delivered to the CA1 and CA3 locations in the hippocampus during the sample phase of DMS trials facilitated memory of images from the DMS task during a delayed recognition (DR) task that also included control images that were not from the DMS task. Across all subjects, the stimulated trials exhibited significant changes in performance in 22.4% of patient and category combinations. Changes in performance were a combination of both increased memory performance and decreased memory performance, with increases in performance occurring at almost 2 to 1 relative to decreases in performance. Across patients with impaired memory that received bilateral stimulation, significant changes in over 37.9% of patient and category combinations was seen with the changes in memory performance show a ratio of increased to decreased performance of over 4 to 1. Modification of memory performance was dependent on whether memory function was intact or impaired, and if stimulation was applied bilaterally or unilaterally, with nearly all increase in performance seen in subjects with impaired memory receiving bilateral stimulation. These results demonstrate that memory encoding in patients with impaired memory function can be facilitated for specific memory content, which offers a stimulation method for a future implantable neural prosthetic to improve human memory.

No evidence of real-world equivalence in chickens (Gallus gallus domesticus) categorizing visually diverse images of natural stimuli presented on LCD monitors.

Category learning is often tested with similar images that have no significance outside of the experiment for the subjects. By contrast, in nature animals often need to generalize a behavioral response like "eat" across visually distinct stimuli, such as spiders and seeds. Forming functional categories like "food" and "predator" may require conceptual rather than purely perceptual generalization. We trained free-range chickens to classify images assigned to one of four categories based on putative functional significance: inanimate objects, predators, food, and non-competing vertebrates. Images were visually diverse within each category, discouraging classification by perceptual similarity alone. In Experiment 1, chickens classified 80 images into four categories. Chickens then generalized to 80 new exemplars in each of three successive generalization tests. In Experiment 2, chickens saw new types of images to test whether their generalization was perceptual or functional. For example, chickens saw images of skunks for the predator category after training with images of hawks and snakes. Chickens used the "predator" response with these new images for both predators and non-threatening vertebrates, but not for objects or food, and did not successfully generalize any category other than predator. In Experiment 3, chickens categorized fractals as "food," and three of four chickens categorized a range of vertebrates they had not previously encountered as "predators," suggesting that chickens did not see the images as representing real world objects and animals. These results highlight constraints on the use of computer-generated images to assess categorization of natural stimuli in chickens.

Evidence for a Classical Dissociation between Face and Object Recognition in Developmental Prosopagnosia.

It is still a matter of debate whether developmental prosopagnosia is a disorder selective to faces or whether object recognition is also affected. In a previous study, based on a small sample of developmental prosopagnosics (DPs; N = 10), we found impairments in both domains although the difficulties were most pronounced for faces. Importantly, impairments with faces and objects were systematically related. We suggested that that the seemingly disproportional impairment for faces in DP was likely to reflect differences between stimulus categories in visual similarity. Here, we aimed to replicate these findings in a larger, independent sample of DPs (N = 21) using the same experimental paradigms. Contrary to our previous results, we found no disproportional effect of visual similarity on performance with faces or objects in the new DP group when compared to controls (N = 21). The new DP group performed within the control range, and significantly better than the old DP-group, on sensitive and demanding object recognition tasks, and we can demonstrate a classical dissociation between face and object recognition at the group level. These findings are perhaps the strongest evidence yet presented for a face-specific deficit in developmental prosopagnosia.

Neural correlates tracking different aspects of the emerging representation of novel visual categories.

Current studies investigating electroencephalogram correlates associated with categorization of sensory stimuli (P300 event-related potential, alpha event-related desynchronization, theta event-related synchronization) typically use an oddball paradigm with few, familiar, highly distinct stimuli providing limited insight about the aspects of categorization (e.g. difficulty, membership, uncertainty) that the correlates are linked to. Using a more complex task, we investigated whether such more specific links could be established between correlates and learning and how these links change during the emergence of new categories. In our study, participants learned to categorize novel stimuli varying continuously on multiple integral feature dimensions, while electroencephalogram was recorded from the beginning of the learning process. While there was no significant P300 event-related potential modulation, both alpha event-related desynchronization and theta event-related synchronization followed a characteristic trajectory in proportion with the gradual acquisition of the two categories. Moreover, the two correlates were modulated by different aspects of categorization, alpha event-related desynchronization by the difficulty of the task, whereas the magnitude of theta -related synchronization by the identity and possibly the strength of category membership. Thus, neural signals commonly related to categorization are appropriate for tracking both the dynamic emergence of internal representation of categories, and different meaningful aspects of the categorization process.

High-performing neural network models of visual cortex benefit from high latent dimensionality.

Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles of computational models in neuroscience. Here we examined the geometry of DNN models of visual cortex by quantifying the latent dimensionality of their natural image representations. A popular view holds that optimal DNNs compress their representations onto low-dimensional subspaces to achieve invariance and robustness, which suggests that better models of visual cortex should have lower dimensional geometries. Surprisingly, we found a strong trend in the opposite direction-neural networks with high-dimensional image subspaces tended to have better generalization performance when predicting cortical responses to held-out stimuli in both monkey electrophysiology and human fMRI data. Moreover, we found that high dimensionality was associated with better performance when learning new categories of stimuli, suggesting that higher dimensional representations are better suited to generalize beyond their training domains. These findings suggest a general principle whereby high-dimensional geometry confers computational benefits to DNN models of visual cortex.

Response-Category Conflict and Control Mode Determine Memory Performance for Distractors in a Flanker Paradigm.

Cognitive conflicts can lead to better memory for task-relevant information. However, little is known about memory performance for task-irrelevant information. The present study investigated memory performance of task-irrelevant distractors using a Flanker paradigm. In two experiments (N = 699), participants first completed a study phase with 56 flanker trials. The stimuli consisted of trial-unique pictures. A congruent trial consisted of a target flanked by two identical pictures from the same category (e.g., all mammals). An incongruent trial consisted of a target and flankers from different stimulus categories (e.g., a mammal flanked by two identical birds), which results in a response-category conflict. To explore the impact of different control modes, congruent and incongruent trials were presented in pure blocks (allowing a proactive control mode) or in mixed blocks (requiring a reactive control mode). Afterwards, recognition memory was tested in a surprise memory test. In general, the results showed better memory for congruent than incongruent flankers in pure blocks. In contrast in mixed blocks, the results showed better memory for incongruent than for congruent flankers. Thus, memory performance for distractors varies systematically with response-category conflict and control mode.

Subcortical responses to music and speech are alike while cortical responses diverge

Music and speech are encountered daily and are unique to human beings. Both are transformed by the auditory pathway from an initial acoustical encoding to higher level cognition. Studies of cortex have revealed distinct brain responses to music and speech, but differences may emerge in the cortex or may be inherited from different subcortical encoding. In the first part of this study, we derived the human auditory brainstem response (ABR), a measure of subcortical encoding, to recorded music and speech using two analysis methods. The first method, described previously and acoustically based, yielded very different ABRs between the two sound classes. The second method, however, developed here and based on a physiological model of the auditory periphery, gave highly correlated responses to music and speech. We determined the superiority of the second method through several metrics, suggesting there is no appreciable impact of stimulus class (i.e., music vs speech) on the way stimulus acoustics are encoded subcortically. In this study’s second part, we considered the cortex. Our new analysis method resulted in cortical music and speech responses becoming more similar but with remaining differences. The subcortical and cortical results taken together suggest that there is evidence for stimulus-class dependent processing of music and speech at the cortical but not subcortical level.

Scaling models of visual working memory to natural images

Over the last few decades, psychologists have developed precise quantitative models of human recall performance in visual working memory (VWM) tasks. However, these models are tailored to a particular class of artificial stimulus displays and simple feature reports from participants (e.g., the color or orientation of a simple object). Our work has two aims. The first is to build models that explain people’s memory errors in continuous report tasks with natural images. Here, we use image generation algorithms to generate continuously varying response alternatives that differ from the stimulus image in natural and complex ways, in order to capture the richness of people’s stored representations. The second aim is to determine whether models that do a good job of explaining memory errors with natural images also explain errors in the more heavily studied domain of artificial displays with simple items. We find that: (i) features taken from state-of-the-art deep encoders predict trial-level difficulty in natural images better than several reasonable baselines; and (ii) the same visual encoders can reproduce set-size effects and response bias curves in the artificial stimulus domains of orientation and color. Moving forward, our approach offers a scalable way to build a more generalized understanding of VWM representations by combining recent advances in both AI and cognitive modeling.

Toward a Fast and Robust MI-BCI: Online Adaptation of Stimulus Paradigm and Classification Model

Toward a Fast and Robust MI-BCI: Online Adaptation of Stimulus Paradigm and Classification Model

S-TBN: A New Neural Decoding Model to Identify Stimulus Categories From Brain Activity Patterns.

Neural decoding is still a challenging and a hot topic in neurocomputing science. Recently, many studies have shown that brain network patterns containing rich spatiotemporal structural information represent the brain's activation information under external stimuli. In the traditional method, brain network features are directly obtained using the standard machine learning method and provide to a classifier, subsequently decoding external stimuli. However, this method cannot effectively extract the multidimensional structural information hidden in the brain network. Furthermore, studies on tensors have show that the tensor decomposition model can fully mine unique spatiotemporal structural characteristics of a spatiotemporal structure in data with a multidimensional structure. This research proposed a stimulus-constrained Tensor Brain Network (s-TBN) model that involves the tensor decomposition and stimulus category-constraint information. The model was verified on real neuroimaging data obtained via magnetoencephalograph and functional mangetic resonance imaging). Experimental results show that the s-TBN model achieve accuracy matrices of greater than 11.06% and 18.46% on the accuracy matrix compared with other methods on two modal datasets. These results prove the superiority of extracting discriminative characteristics using the STN model, especially for decoding object stimuli with semantic information.

Normalization by orientation-tuned surround in human V1-V3.

An influential account of neuronal responses in primary visual cortex is the normalized energy model. This model is often implemented as a multi-stage computation. The first stage is linear filtering. The second stage is the extraction of contrast energy, whereby a complex cell computes the squared and summed outputs of a pair of the linear filters in quadrature phase. The third stage is normalization, in which a local population of complex cells mutually inhibit one another. Because the population includes cells tuned to a range of orientations and spatial frequencies, the result is that the responses are effectively normalized by the local stimulus contrast. Here, using evidence from human functional MRI, we show that the classical model fails to account for the relative responses to two classes of stimuli: straight, parallel, band-passed contours (gratings), and curved, band-passed contours (snakes). The snakes elicit fMRI responses that are about twice as large as the gratings, yet a traditional divisive normalization model predicts responses that are about the same. Motivated by these observations and others from the literature, we implement a divisive normalization model in which cells matched in orientation tuning ("tuned normalization") preferentially inhibit each other. We first show that this model accounts for differential responses to these two classes of stimuli. We then show that the model successfully generalizes to other band-pass textures, both in V1 and in extrastriate cortex (V2 and V3). We conclude that even in primary visual cortex, complex features of images such as the degree of heterogeneity, can have large effects on neural responses.

Generalization of beneficial exposure effects to untreated stimuli from another fear category

Previous research has shown that fear associated with one stimulus often spreads to other stimuli with similar perceptual features as well as across different stimulus categories. Exposure is considered as the most effective intervention to attenuate exaggerated fear. The extent to which exposure treatment effects can generalize to fears not targeted during treatment remains elusive. Previous studies on possible generalization of beneficial effects of exposure used stimuli sharing the same stimulus category and/or stimuli having high perceptual similarity. The current study examined whether exposure treatment generalization can be achieved for untreated stimuli which do not share any perceptual resemblance and belong to a different fear category. An analogue sample of fifty participants with fear of spiders (animal-related fears) and heights (natural environment-related fears) was tested. Participants have been randomly assigned to either an exposure treatment (n = 24) or a control condition (n = 26). Exposure treatment was designed to only target participants’ fear of spiders, leaving their fear of heights untreated. Results demonstrated that the effects of exposure treatment generalized to fear of heights, as indicated by a reduction in behavioral avoidance, as well as self-reported acrophobia symptoms. The present study confutes the assumption that generalization of exposure effects to untreated fears is based on perceptual similarity. Clearly, further research is required to determine the decisive factors, in order to expand the generalization effect permanently to any given type of fear.

Respiration modulates sleep oscillations and memory reactivation in humans

The beneficial effect of sleep on memory consolidation relies on the precise interplay of slow oscillations and spindles. However, whether these rhythms are orchestrated by an underlying pacemaker has remained elusive. Here, we tested the relationship between respiration, which has been shown to impact brain rhythms and cognition during wake, sleep-related oscillations and memory reactivation in humans. We re-analysed an existing dataset, where scalp electroencephalography and respiration were recorded throughout an experiment in which participants (N = 20) acquired associative memories before taking a nap. Our results reveal that respiration modulates the emergence of sleep oscillations. Specifically, slow oscillations, spindles as well as their interplay (i.e., slow-oscillation_spindle complexes) systematically increase towards inhalation peaks. Moreover, the strength of respiration - slow-oscillation_spindle coupling is linked to the extent of memory reactivation (i.e., classifier evidence in favour of the previously learned stimulus category) during slow-oscillation_spindles. Our results identify a clear association between respiration and memory consolidation in humans and highlight the role of brain-body interactions during sleep.

Temporal differences and commonalities between hand and tool neural processing

Object recognition is a complex cognitive process that relies on how the brain organizes object-related information. While spatial principles have been extensively studied, less studied temporal dynamics may also offer valuable insights into this process, particularly when neural processing overlaps for different categories, as it is the case of the categories of hands and tools. Here we focus on the differences and/or similarities between the time-courses of hand and tool processing under electroencephalography (EEG). Using multivariate pattern analysis, we compared, for different time points, classification accuracy for images of hands or tools when compared to images of animals. We show that for particular time intervals (~ 136–156 ms and ~ 252–328 ms), classification accuracy for hands and for tools differs. Furthermore, we show that classifiers trained to differentiate between tools and animals generalize their learning to classification of hand stimuli between ~ 260–320 ms and ~ 376–500 ms after stimulus onset. Classifiers trained to distinguish between hands and animals, on the other hand, were able to extend their learning to the classification of tools at ~ 150 ms. These findings suggest variations in semantic features and domain-specific differences between the two categories, with later-stage similarities potentially related to shared action processing for hands and tools.