Early Sensory Cortices Research Articles

Convergent and invariant object representations for sight, sound, and touch.

We continuously perceive objects in the world through multiple sensory channels. In this study, we investigated the convergence of information from different sensory streams within the cerebral cortex. We presented volunteers with three common objects via three different modalities-sight, sound, and touch-and used multivariate pattern analysis of functional magnetic resonance imaging data to map the cortical regions containing information about the identity of the objects. We could reliably predict which of the three stimuli a subject had seen, heard, or touched from the pattern of neural activity in the corresponding early sensory cortices. Intramodal classification was also successful in large portions of the cerebral cortex beyond the primary areas, with multiple regions showing convergence of information from two or all three modalities. Using crossmodal classification, we also searched for brain regions that would represent objects in a similar fashion across different modalities of presentation. We trained a classifier to distinguish objects presented in one modality and then tested it on the same objects presented in a different modality. We detected audiovisual invariance in the right temporo-occipital junction, audiotactile invariance in the left postcentral gyrus and parietal operculum, and visuotactile invariance in the right postcentral and supramarginal gyri. Our maps of multisensory convergence and crossmodal generalization reveal the underlying organization of the association cortices, and may be related to the neural basis for mental concepts.

The role of dendritic signaling in the anesthetic suppression of consciousness.

Despite considerable progress in the identification of the molecular targets of general anesthetics, it remains unclear how these drugs affect the brain at the systems level to suppress consciousness. According to recent proposals, anesthetics may achieve this feat by interfering with corticocortical top-down processes, that is, by interrupting information flow from association to early sensory cortices. Such a view entails two immediate questions. First, at which anatomical site, and by virtue of which physiological mechanism, do anesthetics interfere with top-down signals? Second, why does a breakdown of top-down signaling cause unconsciousness? While an answer to the first question can be gleaned from emerging neurophysiological evidence on dendritic signaling in cortical pyramidal neurons, a response to the second is offered by increasingly popular theoretical frameworks that place the element of prediction at the heart of conscious perception.

Altered functional magnetic resonance imaging responses to nonpainful sensory stimulation in fibromyalgia patients.

Fibromyalgia (FM) is a disorder characterized by chronic pain and enhanced responses to acute noxious events. However, the sensory systems affected in FM may extend beyond pain itself, as FM patients show reduced tolerance to non-nociceptive sensory stimulation. Characterizing the neural substrates of multisensory hypersensitivity in FM may thus provide important clues about the underlying pathophysiology of the disorder. The aim of this study was to characterize brain responses to non-nociceptive sensory stimulation in FM patients and their relationship to subjective sensory sensitivity and clinical pain severity. Functional magnetic resonance imaging (MRI) was used to assess brain response to auditory, visual, and tactile motor stimulation in 35 women with FM and 25 matched controls. Correlation and mediation analyses were performed to establish the relationship between brain responses and 3 types of outcomes: subjective hypersensitivity to daily sensory stimulation, spontaneous pain, and functional disability. Patients reported increased subjective sensitivity (increased unpleasantness) in response to multisensory stimulation in daily life. Functional MRI revealed that patients showed reduced task-evoked activation in primary/secondary visual and auditory areas and augmented responses in the insula and anterior lingual gyrus. Reduced responses in visual and auditory areas were correlated with subjective sensory hypersensitivity and clinical severity measures. FM patients showed strong attenuation of brain responses to nonpainful events in early sensory cortices, accompanied by an amplified response at later stages of sensory integration in the insula. These abnormalities are associated with core FM symptoms, suggesting that they may be part of the pathophysiology of the disease.

Attentional modulation of the inner ear: a combined otoacoustic emission and EEG study.

Attending to a single stimulus in a complex multisensory environment requires the ability to select relevant information while ignoring distracting input. The underlying mechanism and involved neuronal levels of this attentional gain control are still a matter of debate. Here, we investigated the influence of intermodal attention on different levels of auditory processing in humans. It is known that the activity of the cochlear amplifier can be modulated by efferent neurons of the medial olivocochlear complex. We used distortion product otoacoustic emission (DPOAE) measurements to monitor cochlear activity during an intermodal cueing paradigm. Simultaneously, central auditory processing was assessed by electroencephalography (EEG) with a steady-state paradigm targeting early cortical responses and analysis of alpha oscillations reflecting higher cognitive control of attentional modulation. We found effects of selective attention at all measured levels of the auditory processing: DPOAE levels differed significantly between periods of visual and auditory attention, showing a reduction during visual attention, but no change during auditory attention. Primary auditory cortex activity, as measured by the auditory steady-state response (ASSR), differed between conditions, with higher ASSRs during auditory than visual attention. Furthermore, the analysis of cortical oscillatory activity revealed increased alpha power over occipitoparietal and frontal regions during auditory compared with visual attention, putatively reflecting suppression of visual processing. In conclusion, this study showed both enhanced processing of attended acoustic stimuli in early sensory cortex and reduced processing of distracting input, both at higher cortical levels and at the most peripheral level of the hearing system, the cochlea.

Reinstatement of associative memories in early visual cortex is signaled by the hippocampus.

The cortical reinstatement hypothesis of memory retrieval posits that content-specific cortical activity at encoding is reinstated at retrieval. Evidence for cortical reinstatement was found in higher-order sensory regions, reflecting reactivation of complex object-based information. However, it remains unclear whether the same detailed sensory, feature-based information perceived during encoding is subsequently reinstated in early sensory cortex and what the role of the hippocampus is in this process. In this study, we used a combination of visual psychophysics, functional neuroimaging, multivoxel pattern analysis, and a well controlled cued recall paradigm to address this issue. We found that the visual information human participants were retrieving could be predicted by the activation patterns in early visual cortex. Importantly, this reinstatement resembled the neural pattern elicited when participants viewed the visual stimuli passively, indicating shared representations between stimulus-driven activity and memory. Furthermore, hippocampal activity covaried with the strength of stimulus-specific cortical reinstatement on a trial-by-trial level during cued recall. These findings provide evidence for reinstatement of unique associative memories in early visual cortex and suggest that the hippocampus modulates the mnemonic strength of this reinstatement.

Electrical neuroimaging reveals content-specific effects of threat in primary visual cortex and fronto-parietal attentional networks

Whereas effects of anticipatory anxiety on attention are usually assumed to remain largely undifferentiated, discrepant findings in the literature suggest that, depending on its content and causes, different modulatory effects on attention control and early sensory processing may arise. Using electrical neuroimaging and psychophysiology in a cross-over design, we tested the hypothesis that different types of anticipatory anxiety (bodily vs. psychological), transiently induced in healthy participants, had dissociable effects on brain systems regulating attention control. Attention control corresponded to the ability to maintain efficient goal-directed processing (indexed by the P300 ERP component and by activations in the attentional networks), as well as the ability to filter out irrelevant stimuli in early sensory cortex (C1 component, indexing attentional gating in V1). Results showed that while psychosocial threat, very much like perceptual load, primarily led to a stronger gating in V1, bodily threat resulted in impaired goal-directed processing within the fronto-parietal attentional network, as well as decreased filtering in V1. These results suggest that anticipatory anxiety is multifaceted, exerting different effects on attention control and early visual processing depending on its sub-type.

Microcircuitry of Agranular Frontal Cortex: Testing the Generality of the Canonical Cortical Microcircuit

We investigated whether a frontal area that lacks granular layer IV, supplementary eye field, exhibits features of laminar circuitry similar to those observed in primary sensory areas. We report, for the first time, visually evoked local field potentials (LFPs) and spiking activity recorded simultaneously across all layers of agranular frontal cortex using linear electrode arrays. We calculated current source density from the LFPs and compared the laminar organization of evolving sinks to those reported in sensory areas. Simultaneous, transient synaptic current sinks appeared first in layers III and V followed by more prolonged current sinks in layers I/II and VI. We also found no variation of single- or multi-unit visual response latency across layers, and putative pyramidal neurons and interneurons displayed similar response latencies. Many units exhibited pronounced discharge suppression that was strongest in superficial relative to deep layers. Maximum discharge suppression also occurred later in superficial than in deep layers. These results are discussed in the context of the canonical cortical microcircuit model originally formulated to describe early sensory cortex. The data indicate that agranular cortex resembles sensory areas in certain respects, but the cortical microcircuit is modified in nontrivial ways.

Relational associative learning induces cross-modal plasticity in early visual cortex.

Neurobiological theories of memory posit that the neocortex is a storage site of declarative memories, a hallmark of which is the association of two arbitrary neutral stimuli. Early sensory cortices, once assumed uninvolved in memory storage, recently have been implicated in associations between neutral stimuli and reward or punishment. We asked whether links between neutral stimuli also could be formed in early visual or auditory cortices. Rats were presented with a tone paired with a light using a sensory preconditioning paradigm that enabled later evaluation of successful association. Subjects that acquired this association developed enhanced sound evoked potentials in their primary and secondary visual cortices. Laminar recordings localized this potential to cortical Layers 5 and 6. A similar pattern of activation was elicited by microstimulation of primary auditory cortex in the same subjects, consistent with a cortico-cortical substrate of association. Thus, early sensory cortex has the capability to form neutral stimulus associations. This plasticity may constitute a declarative memory trace between sensory cortices.

Neural convergence and divergence in the mammalian cerebral cortex: From experimental neuroanatomy to functional neuroimaging

A development essential for understanding the neural basis of complex behavior and cognition is the description, during the last quarter of the twentieth century, of detailed patterns of neuronal circuitry in the mammalian cerebral cortex. This effort established that sensory pathways exhibit successive levels of convergence, from the early sensory cortices to sensory-specific and multisensory association cortices, culminating in maximally integrative regions. It was also established that this convergence is reciprocated by successive levels of divergence, from the maximally integrative areas all the way back to the early sensory cortices. This article first provides a brief historical review of these neuroanatomical findings, which were relevant to the study of brain and mind-behavior relationships and to the proposal of heuristic anatomofunctional frameworks. In a second part, the article reviews new evidence that has accumulated from studies of functional neuroimaging, employing both univariate and multivariate analyses, as well as electrophysiology, in humans and other mammals, that the integration of information across the auditory, visual, and somatosensory-motor modalities proceeds in a content-rich manner. Behaviorally and cognitively relevant information is extracted from and conserved across the different modalities, both in higher order association cortices and in early sensory cortices. Such stimulus-specific information is plausibly relayed along the neuroanatomical pathways alluded to above. The evidence reviewed here suggests the need for further in-depth exploration of the intricate connectivity of the mammalian cerebral cortex in experimental neuroanatomical studies.

Prior Expectations Bias Sensory Representations in Visual Cortex

Perception is strongly influenced by expectations. Accordingly, perception has sometimes been cast as a process of inference, whereby sensory inputs are combined with prior knowledge. However, despite a wealth of behavioral literature supporting an account of perception as probabilistic inference, the neural mechanisms underlying this process remain largely unknown. One important question is whether top-down expectation biases stimulus representations in early sensory cortex, i.e., whether the integration of prior knowledge and bottom-up inputs is already observable at the earliest levels of sensory processing. Alternatively, early sensory processing may be unaffected by top-down expectations, and integration of prior knowledge and bottom-up input may take place in downstream association areas that are proposed to be involved in perceptual decision-making. Here, we implicitly manipulated human subjects' prior expectations about visual motion stimuli, and probed the effects on both perception and sensory representations in visual cortex. To this end, we measured neural activity noninvasively using functional magnetic resonance imaging, and applied a forward modeling approach to reconstruct the motion direction of the perceived stimuli from the signal in visual cortex. Our results show that top-down expectations bias representations in visual cortex, demonstrating that the integration of prior information and sensory input is reflected at the earliest stages of sensory processing.

Influence of different types of downscaling on a cortical microcircuit model

Neural network models are routinely downscaled in terms of numbers of neurons or synapses because of a lack of computational resources or the limited capacity of a given neuromorphic hardware. Unfortunately the downscaling is often performed without explicit mention of the limitations this entails [1]. This is relevant since downscaling can substantially affect the dynamics. For instance, reducing the number of neurons N while preserving in-degrees K increases shared inputs and hence correlations. Theoretical results on scaling are derived using simplifying assumptions. Therefore we use simulations to systematically investigate the effects of downscaling on the dynamics of a layered microcircuit model of early sensory cortex [2]. The model consists of eight excitatory and inhibitory populations of leaky integrate-and-fire point neurons with current-based synapses and homogeneous Poisson input. The full-scale network, comprising approximately 80,000 neurons and 0.3 billion synapses, is in the balanced state [3,4], displaying asynchronous irregular activity. For networks of binary or integrate-and-fire neurons in this state, the total current-based synaptic input to a given neuron is well approximated by a Gaussian noise, of which the mean and variance determine the firing rate. One way of preserving this mean and variance has been touched upon: reducing N but keeping K constant. For reduced K, firing rates can be preserved by an appropriate choice of synaptic weights, the ratio between excitatory and inhibitory weights, and mean external input [3,4]. However, these scaling methods are based on single-neuron dynamics and ignore network effects. For instance, scaling synaptic weights J and mean external input to preserve rates implies constant J2 K [3], which changes the feedback on the population level and hence qualitatively changes the network state [5]. An alternative that takes into account both single-neuron and network effects is to choose J so that J K is constant and adjusting the external drive to maintain the input variance. This preserves the shape of cross-correlation functions in balanced E-I networks [5]. However, the magnitude of the correlations increases as K decreases, and a restriction to positive external input variance limits the compensation for increased intrinsic variance. We consider the effects of the above scaling options on correlations, firing rates, synchrony, and irregularity of spiking activity. Although each method achieves near-constancy of one or more dynamical features, none preserves all of them. These results can be used to downscale networks of neurons with current-based synapses in the asynchronous irregular regime in a controlled manner. However, since it is not yet possible to analytically predict all the effects of scaling, it remains essential to compare down-scaled simulations with their full-scale counterparts.

Spatiotemporal dynamics of neuronal population response in the primary visual cortex

One of the fundamental questions in system neuroscience is how the brain encodes external stimuli in the early sensory cortex. It has been found in experiments that even some simple sensory stimuli can activate large populations of neurons. It is believed that information can be encoded in the spatiotemporal profile of these collective neuronal responses. Here, we use a large-scale computational model of the primary visual cortex (V1) to study the population responses in V1 as observed in experiments in which monkeys performed visual detection tasks. We show that our model can capture very well spatiotemporal activities measured by voltage-sensitive-dye-based optical imaging in V1 of the awake state. In our model, the properties of horizontal long-range connections with NMDA conductance play an important role in the correlated population responses and have strong implications for spatiotemporal coding of neuronal populations. Our computational modeling approach allows us to reveal intrinsic cortical dynamics, separating them from those statistical effects arising from averaging procedures in experiment. For example, in experiments, it was shown that there was a spatially antagonistic center-surround structure in optimal weights in signal detection theory, which was believed to underlie the efficiency of population coding. However, our study shows that this feature is an artifact of data processing.

Sex-specific gray matter volume differences in females with developmental dyslexia

Developmental dyslexia, characterized by unexpected reading difficulty, is associated with anomalous brain anatomy and function. Previous structural neuroimaging studies have converged in reports of less gray matter volume (GMV) in dyslexics within left hemisphere regions known to subserve language. Due to the higher prevalence of dyslexia in males, these studies are heavily weighted towards males, raising the question whether studies of dyslexia in females only and using the same techniques, would generate the same findings. In a replication study of men, we obtained the same findings of less GMV in dyslexics in left middle/inferior temporal gyri and right postcentral/supramarginal gyri as reported in the literature. However, comparisons in women with and without dyslexia did not yield left hemisphere differences, and instead, we found less GMV in right precuneus and paracentral lobule/medial frontal gyrus. In boys, we found less GMV in left inferior parietal cortex (supramarginal/angular gyri), again consistent with previous work, while in girls differences were within right central sulcus, spanning adjacent gyri, and left primary visual cortex. Our investigation into anatomical variants in dyslexia replicates existing studies in males, but at the same time shows that dyslexia in females is not characterized by involvement of left hemisphere language regions but rather early sensory and motor cortices (i.e., motor and premotor cortex, primary visual cortex). Our findings suggest that models on the brain basis of dyslexia, primarily developed through the study of males, may not be appropriate for females and suggest a need for more sex-specific investigations into dyslexia.

Auditory Cortex Represents Both Pitch Judgments and the Corresponding Acoustic Cues

SummaryThe neural processing of sensory stimuli involves a transformation of physical stimulus parameters into perceptual features, and elucidating where and how this transformation occurs is one of the ultimate aims of sensory neurophysiology. Recent studies have shown that the firing of neurons in early sensory cortex can be modulated by multisensory interactions [1–5], motor behavior [1, 3, 6, 7], and reward feedback [1, 8, 9], but it remains unclear whether neural activity is more closely tied to perception, as indicated by behavioral choice, or to the physical properties of the stimulus. We investigated which of these properties are predominantly represented in auditory cortex by recording local field potentials (LFPs) and multiunit spiking activity in ferrets while they discriminated the pitch of artificial vowels. We found that auditory cortical activity is informative both about the fundamental frequency (F0) of a target sound and also about the pitch that the animals appear to perceive given their behavioral responses. Surprisingly, although the stimulus F0 was well represented at the onset of the target sound, neural activity throughout auditory cortex frequently predicted the reported pitch better than the target F0.

Crossmodal interactions of haptic and visual texture information in early sensory cortex

Both visual and haptic information add to the perception of surface texture. While prior studies have reported crossmodal interactions of both sensory modalities at the behavioral level, neuroimaging studies primarily investigated texture perception in separate visual and haptic paradigms. These experimental designs, however, only allowed to identify overlap in both sensory processing streams but no interaction of visual and haptic texture processing. By varying texture characteristics in a bimodal task, the current study investigated how these crossmodal interactions are reflected at the cortical level. We used fMRI to compare cortical activation in response to matching versus non-matching visual-haptic texture information. We expected that passive simultaneous presentation of matching visual-haptic input would be sufficient to induce BOLD responses graded with varying texture characteristics. Since no cognitive evaluation of the stimuli was required, we expected to find changes primarily at a rather early processing stage. Our results confirmed our assumptions by showing crossmodal interactions of visual-haptic texture information in early somatosensory and visual cortex. However, the nature of the crossmodal effects was slightly different in both sensory cortices. In early visual cortex, matching visual-haptic information increased the average activation level and induced parametric BOLD signal variations with varying texture characteristics. In early somatosensory cortex only the latter was true. These results challenge the notion that visual and haptic texture information is processed independently and indicate a crossmodal interaction of sensory information already at an early cortical processing stage.

Eccentricity dependent auditory enhancement of visual stimulus detection but not discrimination

Sensory perception is enhanced by the complementary information provided by our different sensory modalities and even apparently task irrelevant stimuli in one modality can facilitate performance in another. While perception in general comprises both, the detection of sensory objects as well as their discrimination and recognition, most studies on audio–visual interactions have focused on either of these aspects. However, previous evidence, neuroanatomical projections between early sensory cortices and computational mechanisms suggest that sounds might differentially affect visual detection and discrimination and differentially at central and peripheral retinal locations. We performed an experiment to directly test this by probing the enhancement of visual detection and discrimination by auxiliary sounds at different visual eccentricities and within the same subjects. Specifically, we quantified the enhancement provided by sounds that reduce the overall uncertainty about the visual stimulus beyond basic multisensory co-stimulation. This revealed a general trend for stronger enhancement at peripheral locations in both tasks, but a statistically significant effect only for detection and only at peripheral locations. Overall this suggests that there are topographic differences in the auditory facilitation of basic visual processes and that these may differentially affect basic aspects of visual recognition.

Why do we like what we like? When information flow matters

In a recent issue of Science, Salimpoor et al. (2013) reported a study in which they explored the neural correlates of aesthetic reward by measuring brain activity while people listened to a novel piece of music. Their results showed that the degree to which a song is found desirable is well predicted both by the level of activity in the nucleus accumbens and the degree of its functional connectivity with other areas, including the orbitofrontal cortex and the auditory cortices. Interestingly, in a previous study, Zeki and Stutters (2012) demonstrated that subjects' preference for kinetic stimuli correlates not just with the activity of the orbitofrontal cortex, but also with activity in a specific part of the visual cortex, namely area V5. Taken together, these two studies are important in highlighting the role of early sensory cortices in subjective preference, even if indirectly shown in the study by Salimpoor et al. (2013) where there was not a direct relationship between auditory activity and desirability. However, in terms of the synergistic relationship between early sensory cortices and reward regions like the nucleus accumbens, two possibilities remain. One is that greater local processing of stimuli with preferred configurations leads to greater connectivity with emotion areas, while another is that greater feedback from emotion areas to early sensory areas takes place during the processing of favored stimuli. In the absence of a definitive answer to this question in the literature, we propose that the latter option is the more plausible. Indeed, there is considerable evidence of feedback influences originating in distant emotion brain structures, such as the amygdala, on early sensory processing (Vuilleumier and Driver, 2007; Scharpf et al., 2010). Nonetheless, there is also support for the alternative view: indeed, low-level statistical regularities of biological significance may influence perceptual judgments and preference ratings (for a review see Graham and Redies, 2010). For instance, in the aesthetic domain, observers dislike images of abstract art that present unnatural statistics (Fernandez and Wilkins, 2008). We suggest that future investigations that consider the dynamics of information flow in response to aesthetic stimuli will provide insights into how their desirability arises. Such efforts will significantly contribute to characterizing the feedback and feedforward mechanisms involved in aesthetic judgments.

Odor mental imagery in non-experts in odors: a paradox?

In agreement with the theoretical framework stipulating that mental images arise from neural activity in early sensory cortices, the primary olfactory cortex [i.e., the piriform cortex (PC)] is activated when non-olfactory-experts try to generate odor mental images. This finding strongly contrasts with the allegation that it is typically impossible to mentally imagine odors. However, other neurophysiological or cognitive processes engaged in the endeavor of odor mental imagery such as sniffing, attention, expectation, and cross-modal interactions involve the PC and could explain this paradox. To unambiguously study the odor mental imagery, we first argued the need to investigate odor experts who have learned to specifically reactivate olfactory percepts. We then assert the necessity to explore the network dedicated to this function by considering variations in both the activity level and the connection strength of the areas belonging to this network as a function of the level of expertise of the odor experts.

FEF-microstimulation causes task-dependent modulation of occipital fMRI activity

Electrical microstimulation of FEF (FEF-EM) modulates neuronal activity in area V4 (Moore and Armstrong, 2003) and elicits functional magnetic resonance imaging (fMRI) activations in visual cortex in a bottom-up dependent manner (Ekstrom et al., 2008). Here we test the hypothesis that FEF-EM-induced modulations of fMRI activity are also function of task demands, which would suggest top-down dependent gating of FEF signals in early visual cortex. We scanned two monkeys performing a visually guided saccade task; a passive fixation task with a very similar visual display; and a passive fixation task without peripheral dots. We found increased effects of FEF-EM on fMRI-activity in visual cortex during saccades compared to fixation, indicating that the FEF-EM induced modulation is task-dependent. Finally, the effect of FEF-EM is mainly present in voxels which were less activated by visual stimuli in the absence of electrical stimulation. Our results show that the FEF-EM-induced pattern of activation in early visual cortex is topographically specific and more pronounced during increased task demands. These results fit with models suggesting that FEF is an important source modulating activity in early sensory cortex and that these influences can be enhanced by coincident bottom-up or top-down signals.

Early sensory cortex is activated in the absence of explicit input during crossmodal item retrieval: Evidence from MEG

Crossmodal associations form a fundamental aspect of our daily lives. In this study we investigated the neural correlates of crossmodal association in early sensory cortices using magnetoencephalography (MEG). We used a paired associate recognition paradigm in which subjects were tested after multiple training sessions over a span of four weeks. Subjects had to learn 12 abstract, nonlinguistic, pairs of auditory and visual objects that consisted of crossmodal (visual–auditory, VA; auditory–visual, AV) and unimodal (visual–visual, VV; auditory–auditory, AA) paired items. Visual objects included abstract, non-nameable, fractal-like images, and auditory objects included abstract tone sequences. During scanning, subjects were shown the first item of a pair (S1), followed by a delay, then the simultaneous presentation of a visual and auditory stimulus (S2). Subjects were instructed to indicate whether either of the S2 stimuli contained the correct paired associate of S1. Synthetic aperture magnetometry (SAMspm), a minimum variance beamformer, was then used to assess source power differences between the crossmodal conditions and their corresponding unimodal conditions (i.e., AV–AA and VA–VV) in the beta (15–30Hz) and low gamma frequencies (31–54Hz) during the S1 period. We found greater power during S1 in the corresponding modality-specific association areas for crossmodal compared with unimodal stimuli. Thus, even in the absence of explicit sensory input, the retrieval of well-learned, crossmodal pairs activate sensory areas associated with the corresponding modality. These findings support theories which posit that modality-specific regions of cortex are involved in the storage and retrieval of sensory-specific items from long-term memory.