Ventral Stream Research Articles

A focus on the multiple interfaces between action and perception and their neural correlates.

Successful behaviour relies on the appropriate interplay between action and perception. The well-established dorsal and ventral stream theories depicted two distinct functional pathways for the processes of action and perception, respectively. In physiological conditions, the two pathways closely cooperate in order to produce successful adaptive behaviour.As the coupling between perception and action exists, this requires an interface that is responsible for a common reading of the two functions. Several studies have proposed different types of perception and action interfaces, suggesting their role in the creation of the shared interaction channel. In the present review, we describe three possible perception and action interfaces: i) the motor code, including common coding approaches, ii) attention, and iii) object affordance; we highlight their potential neural correlates. From this overview, a recurrent neural substrate that underlies all these interface functions appears to be crucial: the parieto-frontal circuit. This network is involved in the mirror mechanism which underlies the perception and action interfaces identified as common coding and motor code theories. The same network is also involved in the spotlight of attention and in the encoding of potential action towards objects; these are manifested in the perception and action interfaces for common attention and object affordance, respectively. Within this framework, most studies were dedicated to the description of the role of the inferior parietal lobule; growing evidence, however, suggests that the superior parietal lobule also plays a crucial role in the interplay between action and perception. The present review proposes a novel model that is inclusive of the superior parietal regions and their relative contribution to the different action and perception interfaces.

Hierarchical organization of social action features along the lateral visual pathway

Recent theoretical work has argued that in addition to the classical ventral (what) and dorsal (where/how) visual streams, there is a third visual stream on the lateral surface of the brain specialized for processing social information. Like visual representations in the ventral and dorsal streams, representations in the lateral stream are thought to be hierarchically organized. However, no prior studies have comprehensively investigated the organization of naturalistic, social visual content in the lateral stream. To address this question, we curated a naturalistic stimulus set of 250 3-s videos of two people engaged in everyday actions. Each clip was richly annotated for its low-level visual features, mid-level scene and object properties, visual social primitives (including the distance between people and the extent to which they were facing), and high-level information about social interactions and affective content. Using a condition-rich fMRI experiment and a within-subject encoding model approach, we found that low-level visual features are represented in early visual cortex (EVC) and middle temporal (MT) area, mid-level visual social features in extrastriate body area (EBA) and lateral occipital complex (LOC), and high-level social interaction information along the superior temporal sulcus (STS). Communicative interactions, in particular, explained unique variance in regions of the STS after accounting for variance explained by all other labeled features. Taken together, these results provide support for representation of increasingly abstract social visual content-consistent with hierarchical organization-along the lateral visual stream and suggest that recognizing communicative actions may be a key computational goal of the lateral visual pathway.

39 Does Pseudoneglect Modifies The Visuospatial After-Effects Of Vertical Prism Adaptation?

Objective:The prism adaptation (PA) with rightward shifting lenses is a promising rehabilitation technique for left hemispatial neglect. The PA has also been applied in healthy individuals to investigate cognitive mechanism(s) underlining such adaptation. Importantly, studies have suggested that PA may primarily impact the functions of the dorsal or the ventral attentional stream, and we have previously reported that PA to the upward and downward shifting lenses leads to a significant aftereffect in vertical line bisection task. However, this post-adaptation effect, similarly to that seen in the horizontal plane, might have been modified by the presence of the vertical pseudoneglect healthy participants often experience prior to PA. Thus, the aim of this study was to test this hypothesis.Participants and Methods:30 right-handed healthy adults (age M=22,4) performed a computerized line bisection (LB) in vertical and horizontal condition. The bisections were performed twice: before and after PA procedure. Participants took experimental procedure three times, each in at least 24 h of break, each time in one of three conditions of shifting lenses; down, up, control. Both LB tasks (vertical and horizontal) consisted of 24 lines, each centered on 23" touch screen. The participants were asked to find the middle of the line. Throughout the experiment, participants were comfortably seated with their head positioned on a chinrest. Participants were fitted with prismatic goggles that deviated their visual field by 10 degrees. For the adaptation we used the Peg-the-mole procedure consisting of 120 pointing movements.Results:To assess the effect of the vertical PA on landmark judgments we performed a repeated measures ANOVA with direction of PA (upward/downward), the condition of LB (vertical/ horizontal) and pre- vs post adaptation as a between-subjects factor. This analysis revealed a main effect of the direction of PA (p< 0.001) and a main effect of condition (p< 0.001). Overall, however, only adaptation in up-shifting lenses led to significant aftereffects (p<0.05). Further, when we excluded participants who did not exhibited horizontal pseudoneglect in preadaptation LB, the effect of PA in downshifting PA emerged in vertical LB (p<0.05). Further, this group also exhibited the aftereffect of PA in up-shifting lenses for the horizontal (p<0.01) and the vertical LB (p<0.05). Additionally, these participants exhibited a congruent tendency after upward and downward PA, and tended to allocate their attention more upward and rightward.Conclusions:The results of this study confirm that the vertical PA evokes a visuo-spatial bias. Moreover, the PA aftereffect seems to be modified by the presence of the pre-adaptation pseudoneglect. Whereas the mechanism inducing this bias is not fully known, it might be explained in light of the interhemispheric activation-inhibition balance. Both the upward and downward PA may primarily lead to activation of the posterior regions of the right hemisphere, and this activation may result with the upward and rightward bias in the LB task. However, future research with neuroimaging techniques is needed to test this hypothesis.

Altered stability of dynamic brain functional architecture in primary open-angle glaucoma: a surface-based resting-state fMRI study.

Delineating the neuropathological characteristics of primary open-angle glaucoma (POAG) is critical for understanding its pathophysiology. While temporal stability represents a crucial aspect of the brain's functional architecture, the specific patterns underlying its contribution to POAG remain unclear. This study aims to analyze the brain functional abnormalities in POAG using functional stability, a dynamic functional connectivity (DFC) approach based on resting-state functional magnetic resonance imaging (rs-fMRI). Seventy patients with POAG and forty-five healthy controls underwent rs-fMRI and ophthalmological examinations. The stability of DFC was calculated as the concordance of DFC over time using a sliding-window approach, and the differences in stability between the two groups were compared. Subsequently, Spearman's correlation analyses were conducted to examine the relationship between functional stability and clinical indicators. Compared with healthy controls, patients with POAG exhibited significantly decreased functional stability in the visual network, including the early visual center, ventral and dorsal stream visual cortex in both hemispheres. Conversely, stability values increased in the bilateral inferior parietal gyrus and right inferior frontal cortex. In POAG patients, the dynamic stability of the left early visual cortex and ventral stream visual cortex correlated with the mean deviation of visual field defects (r = 0.251, p = 0.037). The evidence from this study suggests that functional stability may provide a new understanding of brain alterations in the progression of POAG.

Distinct Neural Components of Visually Guided Grasping during Planning and Execution.

Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actor's body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here, we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants of either sex viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors, that is, grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral stream areas during grasp planning then in premotor regions during grasp execution. Object mass was encoded in ventral stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects.SIGNIFICANCE STATEMENT Grasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and, surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

Simple and complex cells revisited: toward a selectivity-invariance model of object recognition.

This paper presents a theoretical perspective on modeling ventral stream processing by revisiting the computational abstraction of simple and complex cells. In parallel to David Marr's vision theory, we organize the new perspective into three levels. At the computational level, we abstract simple and complex cells into space partitioning and composition in a topological space based on the redundancy exploitation hypothesis of Horace Barlow. At the algorithmic level, we present a hierarchical extension of sparse coding by exploiting the manifold constraint in high-dimensional space (i.e., the blessing of dimensionality). The resulting over-parameterized models for object recognition differ from existing hierarchical models by disentangling the objectives of selectivity and invariance computation. It is possible to interpret our hierarchical construction as a computational implementation of cortically local subspace untangling for object recognition and face representation, which are closely related to exemplar-based and axis-based coding in the medial temporal lobe. At the implementation level, we briefly discuss two possible implementations based on asymmetric sparse autoencoders and divergent spiking neural networks.

Neurovascular coupling dysfunction of visual network organization in Parkinson's disease

Parkinson's disease (PD) has been showed perfusion and neural activity alterations in specific regions, such as the motor and visual networks; however, the clinical significance of coupling changes is still unknown. To identify how neurovascular coupling changes during the pathophysiology of PD, patients and healthy controls underwent multiparametric magnetic resonance imaging to measure neural activity organization of segregation and integration using amplitude of low-frequency fluctuation (ALFF) and functional connectivity strength (FCS), and measure vascular responses using cerebral blood flow (CBF). Neurovascular coupling was calculated as the global CBF-ALFF and CBF-FCS coupling and the regional CBF/ALFF and CBF/FCS ratio. Correlations and dynamic causal modeling was then used to evaluate relationships with disease-alterations to clinical variables and information flow. Neurovascular coupling was impaired in PD with decreased global CBF-ALFF and CBF-FCS coupling, as well as decreased CBF/ALFF in the parieto-occipital cortex (dorsal visual stream) and CBF/FCS in the temporo-occipital cortex (ventral visual stream); these decouplings were associated with motor and non-motor impairments. The distinctive patterns of neurovascular coupling alterations within the dorsal and ventral visual streams of the visual system could potentially provide additional understanding into the pathophysiological mechanisms of PD.

Hands off, brain off? A meta-analysis of neuroimaging data during active and passive driving.

Car driving is more and more automated, to such an extent that driving without active steering control is becoming a reality. Although active driving requires the use of visual information to guide actions (i.e., steering the vehicle), passive driving only requires looking at the driving scene without any need to act (i.e., the human is passively driven). After a careful search of the scientific literature, 11 different studies, providing 17 contrasts, were used to run a comprehensive meta-analysis contrasting active driving with passive driving. Two brain regions were recruited more consistently for active driving compared to passive driving, the left precentral gyrus (BA3 and BA4) and the left postcentral gyrus (BA4 and BA3/40), whereas a set of brain regions was recruited more consistently in passive driving compared to active driving: the left middle frontal gyrus (BA6), the right anterior lobe and the left posterior lobe of the cerebellum, the right sub-lobar thalamus, the right anterior prefrontal cortex (BA10), the right inferior occipital gyrus (BA17/18/19), the right inferior temporal gyrus (BA37), and the left cuneus (BA17). From a theoretical perspective, these findings support the idea that the output requirement of the visual scanning process engaged for the same activity can trigger different cerebral pathways, associated with different cognitive processes. A dorsal stream dominance was found during active driving, whereas a ventral stream dominance was obtained during passive driving. From a practical perspective, and contrary to the dominant position in the Human Factors community, our findings support the idea that a transition from passive to active driving would remain challenging as passive and active driving engage distinct neural networks.

Pictorial depth cues always influence reaching distance

We report five experiments to test the influence of pictorial depth on reaching. Our core method is to project a wide-field background of linear perspective and/or texture gradient onto a tabletop, and to measure the amplitude of reaches made to targets within it. In 63 healthy participants performing immediate open-loop reaches across Experiments 1–4, we observed a clear effect of pictorial depth. This effect was driven specifically by the convergence of the background pattern at the target position: for each additional degree of pictorial convergence, reaching distance increased by half a millimetre. In the individual experiments, we applied manipulations that might be expected to modify the influence of pictorial depth. We found no evidence that the effect was modified with monocular viewing, or when participants responded with the left hand, or if a memory delay was inserted before the response. Nor did participants become less susceptible to pictorial depth when visual feedback of terminal reaching errors was provided, although visual feedback during the reach did mitigate the influence of pictorial depth. Finally, the visual form agnosic patient DF showed an entirely normal effect of pictorial depth cues, which leads us to question the idea that this effect emanates from visual analyses of size and shape in the ventral stream, rather than from the dorsal stream, or from earlier stages of visual processing.

The Neural Correlates of Developmental Prosopagnosia: Twenty-Five Years on.

Faces play a crucial role in social interactions. Developmental prosopagnosia (DP) refers to the lifelong difficulty in recognizing faces despite the absence of obvious signs of brain lesions. In recent decades, the neural substrate of this condition has been extensively investigated. While early neuroimaging studies did not reveal significant functional and structural abnormalities in the brains of individuals with developmental prosopagnosia (DPs), recent evidence identifies abnormalities at multiple levels within DPs' face-processing networks. The current work aims to provide an overview of the convergent and contrasting findings by examining twenty-five years of neuroimaging literature on the anatomo-functional correlates of DP. We included 55 original papers, including 63 studies that compared the brain structure (MRI) and activity (fMRI, EEG, MEG) of healthy control participants and DPs. Despite variations in methods, procedures, outcomes, sample selection, and study design, this scoping review suggests that morphological, functional, and electrophysiological features characterize DPs' brains, primarily within the ventral visual stream. Particularly, the functional and anatomical connectivity between the Fusiform Face Area and the other face-sensitive regions seems strongly impaired. The cognitive and clinical implications as well as the limitations of these findings are discussed in light of the available knowledge and challenges in the context of DP.

Neural segregation and integration of sensory, decision, and action processes during object categorization

Neural and computational evidence suggests that perceptual decisions depend on an evidence accumulation process. The gradual reveal fMRI method, which prolongs a decision to match the slow temporal resolution of fMRI measurements, has classified dorsal visual stream regions as “Action” (alternatively, “Moment of Recognition” or “Commitment”) and ventral visual stream regions as “Accumulator.” Previous gradual reveal fMRI studies, however, only tested actions that were in response to decisions and, thus, related to evidence accumulation. To fully dissociate the contribution of sensory, decision, and motor components to Action and Accumulator regions in the dorsal and ventral visual streams, we extended the gradual reveal paradigm to also include responses made to cues where no decision was necessary. We found that the lateral occipital cortex in the ventral visual stream showed a highly selective Accumulator profile, whereas regions in the fusiform gyrus were influenced by action generation. Dorsal visual stream regions showed strikingly similar profiles as classical motor regions and also as regions of the salience network. These results suggest that the dorsal and ventral visual streams may appear highly segregated because they include a small number of regions that are highly selective for Accumulator or Action. However, the streams may be more integrated than previously thought and this integration may be accomplished by regions with graded responses that are less selective (i.e., more distributed).

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception.

The ventral visual processing hierarchy of the cortex needs to fulfill at least two key functions: perceived objects must be mapped to high-level representations invariantly of the precise viewing conditions, and a generative model must be learned that allows, for instance, to fill in occluded information guided by visual experience. Here, we show how a multilayered predictive coding network can learn to recognize objects from the bottom up and to generate specific representations via a top-down pathway through a single learning rule: the local minimization of prediction errors. Trained on sequences of continuously transformed objects, neurons in the highest network area become tuned to object identity invariant of precise position, comparable to inferotemporal neurons in macaques. Drawing on this, the dynamic properties of invariant object representations reproduce experimentally observed hierarchies of timescales from low to high levels of the ventral processing stream. The predicted faster decorrelation of error-neuron activity compared to representation neurons is of relevance for the experimental search for neural correlates of prediction errors. Lastly, the generative capacity of the network is confirmed by reconstructing specific object images, robust to partial occlusion of the inputs. By learning invariance from temporal continuity within a generative model, the approach generalizes the predictive coding framework to dynamic inputs in a more biologically plausible way than self-supervised networks with non-local error-backpropagation. This was achieved simply by shifting the training paradigm to dynamic inputs, with little change in architecture and learning rule from static input-reconstructing Hebbian predictive coding networks.

Altered neural intrinsic oscillations in patients with multiple sclerosis: effects of cortical thickness.

To investigate the effects of cortical thickness on the identification accuracy of fractional amplitude of low-frequency fluctuation (fALFF) in patients with multiple sclerosis (MS). Resting-state functional magnetic resonance imaging data were collected from 31 remitting MS, 20 acute MS, and 42 healthy controls (HCs). After preprocessing, we first calculated two-dimensional fALFF (2d-fALFF) maps using the DPABISurf toolkit, and 2d-fALFF per unit thickness was obtained by dividing 2d-fALFF by cortical thickness. Then, between-group comparison, clinical correlation, and classification analyses were performed in 2d-fALFF and 2d-fALFF per unit thickness maps. Finally, we also examined whether the effect of cortical thickness on 2d-fALFF maps was affected by the subfrequency band. In contrast with 2d-fALFF, more changed regions in 2d-fALFF per unit thickness maps were detected in MS patients, such as increased region of the right inferior frontal cortex and faded regions of the right paracentral lobule, middle cingulate cortex, and right medial temporal cortex. There was a significant positive correlation between the disease duration and the 2d-fALFF values in the left early visual cortex in remitting MS patients (r = 0.517, Bonferroni-corrected, p = 0.008 × 4 < 0.05). In contrast with 2d-fALFF, we detected a positive correlation between the 2d-fALFF per unit thickness of the right ventral stream visual cortex and the modified Fatigue Impact Scale (MFIS) scores (r = 0.555, Bonferroni-corrected, p = 0.017 × 4 > 0.05). For detecting MS patients, 2d-fALFF and 2d- fALFF per unit thickness both performed remarkably well in support vector machine (SVM) analysis, especially in the remitting phase (AUC = 86, 83%). Compared with 2d-fALFF, the SVM model of 2d-fALFF per unit thickness had significantly higher classification performance in distinguishing between remitting and acute MS. More changed regions and more clinically relevant 2d-fALFF per unit thickness maps in the subfrequency band were also detected in MS patients. By dividing the functional value by the cortical thickness, the identification accuracy of fALFF in MS patients was detected to be potentially influenced by cortical thickness. Additionally, 2d-fALFF per unit thickness is a potential diagnostic marker that can be utilized to distinguish between acute and remitting MS patients. Notably, we observed similar variations in the subfrequency band.

Rapid plasticity in the ventral visual stream elicited by a newly learnt auditory script in congenitally blind adults

Accumulating evidence in the last decades has given rise to a new theory of brain organization, positing that cortical regions are recruited for specific tasks irrespective of the sensory modality via which information is channeled. For instance, the visual reading network has been shown to be recruited for reading via the tactile Braille code in congenitally blind adults. Yet, how rapidly non-typical sensory input modulates activity in typically visual regions is yet to be explored. To this aim, we developed a novel reading orthography, termed OVAL, enabling congenitally blind adults to quickly acquire reading via the auditory modality. OVAL uses the EyeMusic, a visual-to-auditory sensory-substitution-device (SSD) to transform visually presented letters optimized for auditory transformation into sound. Using fMRI, we show modulation in the right ventral visual stream following 2-h of same-day training. Crucially, following more extensive training (i.e., ∼12 h) we show that OVAL reading recruits the left ventral visual stream including the location of the Visual Word Form Area, a key graphene-responsive region within the visual reading network. Our results show that while after 2 h of SSD training we can already observe the recruitment of the deprived ventral visual stream by auditory stimuli, computation-selective cross-modal recruitment requires longer training to establish.

Dyslexia Due to Visual Impairments.

Reading involves many different abilities that are necessary or sufficient conditions for fluent and flawless reading. The absence of one necessary or of all sufficient conditions is a cause of dyslexia. The present study investigates whether too short fixation times and an impaired ability to recognize a string of letters simultaneously are causes of dyslexia. The frequency and types of reading mistakes were investigated in a tachistoscopic pseudoword experiment with 100 children with dyslexia to test the impact of too short fixation times and the attempts of children with dyslexia to recognize more letters simultaneously than they can when reading pseudowords. The experiment demonstrates that all types of reading mistakes disappear when the fixation time increases and/or the number of letters that the children try to recognize simultaneously is reduced. The results cannot be interpreted as being due to altered visual crowding, impaired attention, or impaired phonological awareness, but can be regarded as an effect of impaired temporal summation and a dysfunction in the ventral stream of the visual system.

Perception and Memory in the Ventral Visual Stream and Medial Temporal Lobe.

Perception and memory are traditionally thought of as separate cognitive functions, supported by distinct brain regions. The canonical perspective is that perceptual processing of visual information is supported by the ventral visual stream, whereas long-term declarative memory is supported by the medial temporal lobe. However, this modular framework cannot account for the increasingly large body of evidence that reveals a role for early visual areas in long-term recognition memory and a role for medial temporal lobe structures in high-level perceptual processing. In this article, we review relevant research conducted in humans, nonhuman primates, and rodents. We conclude that the evidence is largely inconsistent with theoretical proposals that draw sharp functional boundaries between perceptual and memory systems in the brain. Instead, the weight of the empirical findings is best captured by a representational-hierarchical model that emphasizes differences in content, rather than in cognitive processes within the ventral visual stream and medial temporal lobe.

Contribution and functional connectivity between cerebrum and cerebellum on sub-lexical and lexical-semantic processing of verbs.

Language comprehension involves both sub-lexical (e.g., phonological) and lexical-semantic processing. We conducted a task using functional magnetic resonance imaging (fMRI) to compare the processing of verbs in these two domains. Additionally, we examined the representation of concrete-motor and abstract-non-motor concepts by including two semantic categories of verbs: motor and mental. The findings indicate that sub-lexical processing during the reading of pseudo-verbs primarily involves the left dorsal stream of the perisylvian network, while lexical-semantic representation during the reading of verbs predominantly engages the ventral stream. According to the embodied or grounded cognition approach, modality-specific mechanisms (such as sensory-motor systems) and the well-established multimodal left perisylvian network contribute to the semantic representation of both concrete and abstract verbs. Our study identified the visual system as a preferential modality-specific system for abstract-mental verbs, which exhibited functional connectivity with the right crus I/lobule VI of the cerebellum. Taken together, these results confirm the dissociation between sub-lexical and lexical-semantic processing and provide neurobiological evidence of functional coupling between specific visual modality regions and the right cerebellum, forming a network that supports the semantic representation of abstract concepts. Further, the results shed light on the underlying mechanisms of semantic processing and contribute to our understanding of how the brain processes abstract concepts.

Large-scale network dynamics underlying the first few hundred milliseconds after stimulus presentation: An investigation of visual recognition memory using iEEG.

Recognition memory is the ability to recognize previously encountered objects. Even this relatively simple, yet extremely fast, ability requires the coordinated activity of large-scale brain networks. However, little is known about the sub-second dynamics of these networks. The majority of current studies into large-scale network dynamics is primarily based on imaging techniques suffering from either poor temporal or spatial resolution. We investigated the dynamics of large-scale functional brain networks underlying recognition memory at the millisecond scale. Specifically, we analyzed dynamic effective connectivity from intracranial electroencephalography while epileptic subjects (n = 18) performed a fast visual recognition memory task. Our data-driven investigation using Granger causality and the analysis of communities with the Louvain algorithm spotlighted a dynamic interplay of two large-scale networks associated with successful recognition. The first network involved the right visual ventral stream and bilateral frontal regions. It was characterized by early, predominantly bottom-up information flow peaking at 115 ms. It was followed by the involvement of another network with predominantly top-down connectivity peaking at 220 ms, mainly in the left anterior hemisphere. The transition between these two networks was associated with changes in network topology, evolving from a more segregated to a more integrated state. These results highlight that distinct large-scale brain networks involved in visual recognition memory unfold early and quickly, within the first 300 ms after stimulus onset. Our study extends the current understanding of the rapid network changes during rapid cognitive processes.

Hierarchical organization of the human ventral visual streams revealed with magnetoencephalography.

The hierarchical organization between 25 ventral stream visual cortical regions and 180 cortical regions was measured with magnetoencephalography using the Human Connectome Project Multimodal Parcellation atlas in 83 Human Connectome Project participants performing a visual memory task. The aim was to reveal the hierarchical organization using a whole-brain model based on generative effective connectivity with this fast neuroimaging method. V1-V4 formed a first group of interconnected regions. Especially V4 had connectivity to a ventrolateral visual stream: V8, the fusiform face cortex, and posterior inferior temporal cortex PIT. These regions in turn had effectivity connectivity to inferior temporal cortex visual regions TE2p and TE1p. TE2p and TE1p then have connectivity to anterior temporal lobe regions TE1a, TE1m, TE2a, and TGv, which are multimodal. In a ventromedial visual stream, V1-V4 connect to ventromedial regions VMV1-3 and VVC. VMV1-3 and VVC connect to the medial parahippocampal gyrus PHA1-3, which, with the VMV regions, include the parahippocampal scene area. The medial parahippocampal PHA1-3 regions have connectivity to the hippocampal system regions the perirhinal cortex, entorhinal cortex, and hippocampus. These effective connectivities of two ventral visual cortical streams measured with magnetoencephalography provide support to the hierarchical organization of brain systems measured with fMRI, and new evidence on directionality.

Predicting the temporal-dynamic trajectories of cortical neuronal responses in non-human primates based on deep spiking neural network.

Deep convolutional neural networks (CNNs) are commonly used as computational models for the primate ventral stream, while deep spiking neural networks (SNNs) incorporated with both the temporal and spatial spiking information still lack investigation. We compared performances of SNN and CNN in prediction of visual responses to the naturalistic stimuli in area V4, inferior temporal (IT), and orbitofrontal cortex (OFC). The accuracies based on SNN were significantly higher than that of CNN in prediction of temporal-dynamic trajectory and averaged firing rate of visual response in V4 and IT. The temporal dynamics were captured by SNN for neurons with diverse temporal profiles and category selectivities, and most sensitively captured around the time of peak responses for each brain region. Consistently, SNN activities showed significantly stronger correlations with IT, V4 and OFC responses. In SNN, correlations with neural activities were stronger for later time-step features than early time-step features. The temporal-dynamic prediction was also significantly improved by considering preceding neural activities during the prediction. Thus, our study demonstrated SNN as a powerful temporal-dynamic model for cortical responses to complex naturalistic stimuli.