Visual Neuroscience Research Articles

Embracing deepfakes and AI-generated images in neuroscience research.

The rise of deepfakes and AI-generated images has raised concerns regarding their potential misuse. However, this commentary highlights the valuable opportunities these technologies offer for neuroscience research. Deepfakes deliver accessible, realistic and customisable dynamic face stimuli, while generative adversarial networks (GANs) can generate and modify diverse and high-quality static content. These advancements can enhance the variability and ecological validity of research methods and enable the creation of previously unattainable stimuli. When AI-generated images are informed by brain responses, they provide unique insights into the structure and function of visual systems. The authors argue that experimental psychologists and cognitive neuroscientists stay informed about these emerging tools and embrace their potential to advance the field of visual neuroscience.

Characteristics and predictive factors of visual function improvements after monocular perceptual learning in amblyopia

Monocular perceptual learning has shown promising performance in restoring visual function in amblyopes beyond the critical period in the laboratory. However, the treatment outcome is variable and indeterminate in actual clinical and neuroscientific practice. We aimed to explore the efficacy of monocular perceptual learning in the clinical setting. By combining continuous monitoring of perceptual learning and clinical measurements, we evaluated the efficacy and characteristics of visual acuity and contrast sensitivity function improvement and further explored the individualized effect after perceptual learning. Amblyopes (average age:17 ± 7 years old) were trained in a monocular two-alternative forced choice identification task at the 50% contrast threshold of the amblyopic eye for 10–15 days. We found that monocular perceptual learning improves both visual acuity and contrast sensitivity function in amblyopia. The broader activation of spatial contrast sensitivity, with a significant improvement in lower spatial frequencies, contributed to improving visual acuity. Visual acuity changes in the early stage can predict the endpoint treatment outcomes. Our results confirm the efficacy of monocular perceptual learning and suggest potential predictors of training outcomes to assist in the future management of clinical intervention and vision neuroscience research in amblyopia beyond the critical period of visual plasticity.

Visual processing in the fly, from photoreceptors to behavior.

Originally a genetic model organism, the experimental use of Drosophila melanogaster has grown to include quantitative behavioral analyses, sophisticated perturbations of neuronal function, and detailed sensory physiology. A highlight of these developments can be seen in the context of vision, where pioneering studies have uncovered fundamental and generalizable principles of sensory processing. Here we begin with an overview of vision-guided behaviors and common methods for probing visual circuits. We then outline the anatomy and physiology of brain regions involved in visual processing, beginning at the sensory periphery and ending with descending motor control. Areas of focus include contrast and motion detection in the optic lobe, circuits for visual feature selectivity, computations in support of spatial navigation, and contextual associative learning. Finally, we look to the future of fly visual neuroscience and discuss promising topics for further study.

Stephen Massey: a career in visual neuroscience.

This review is a memoir by Dr. Stephen C. Massey's longtime collaborator, Dr. Stephen L. Mills, and written, for the most part, in the first person. It also serves as an introduction to the virtual festschrift to celebrate Dr. Massey's retirement. and. The references cited here are only a few of the highlights of Dr. Massey's distinguished career. A complete list is found here: https://pubmed.ncbi.nlm.nih.gov/?term=massey+sc+%28retina+or+photoreceptors%29&sort=date.

Neural correlates of face perception modeled with a convolutional recurrent neural network

Objective. Event-related potential (ERP) sensitivity to faces is predominantly characterized by an N170 peak that has greater amplitude and shorter latency when elicited by human faces than images of other objects. We aimed to develop a computational model of visual ERP generation to study this phenomenon which consisted of a three-dimensional convolutional neural network (CNN) connected to a recurrent neural network (RNN). Approach. The CNN provided image representation learning, complimenting sequence learning of the RNN for modeling visually-evoked potentials. We used open-access data from ERP Compendium of Open Resources and Experiments (40 subjects) to develop the model, generated synthetic images for simulating experiments with a generative adversarial network, then collected additional data (16 subjects) to validate predictions of these simulations. For modeling, visual stimuli presented during ERP experiments were represented as sequences of images (time x pixels). These were provided as inputs to the model. By filtering and pooling over spatial dimensions, the CNN transformed these inputs into sequences of vectors that were passed to the RNN. The ERP waveforms evoked by visual stimuli were provided to the RNN as labels for supervised learning. The whole model was trained end-to-end using data from the open-access dataset to reproduce ERP waveforms evoked by visual events. Main results. Cross-validation model outputs strongly correlated with open-access (r = 0.98) and validation study data (r = 0.78). Open-access and validation study data correlated similarly (r = 0.81). Some aspects of model behavior were consistent with neural recordings while others were not, suggesting promising albeit limited capacity for modeling the neurophysiology of face-sensitive ERP generation. Significance. The approach developed in this work is potentially of significant value for visual neuroscience research, where it may be adapted for multiple contexts to study computational relationships between visual stimuli and evoked neural activity.

The psychosis human connectome project: Design and rationale for studies of visual neurophysiology

Visual perception is abnormal in psychotic disorders such as schizophrenia. In addition to hallucinations, laboratory tests show differences in fundamental visual processes including contrast sensitivity, center-surround interactions, and perceptual organization. A number of hypotheses have been proposed to explain visual dysfunction in psychotic disorders, including an imbalance between excitation and inhibition. However, the precise neural basis of abnormal visual perception in people with psychotic psychopathology (PwPP) remains unknown. Here, we describe the behavioral and 7 tesla MRI methods we used to interrogate visual neurophysiology in PwPP as part of the Psychosis Human Connectome Project (HCP). In addition to PwPP (n = 66) and healthy controls (n = 43), we also recruited first-degree biological relatives (n = 44) in order to examine the role of genetic liability for psychosis in visual perception. Our visual tasks were designed to assess fundamental visual processes in PwPP, whereas MR spectroscopy enabled us to examine neurochemistry, including excitatory and inhibitory markers. We show that it is feasible to collect high-quality data across multiple psychophysical, functional MRI, and MR spectroscopy experiments with a sizable number of participants at a single research site. These data, in addition to those from our previously described 3 tesla experiments, will be made publicly available in order to facilitate further investigations by other research groups. By combining visual neuroscience techniques and HCP brain imaging methods, our experiments offer new opportunities to investigate the neural basis of abnormal visual perception in PwPP.

Eye movement characteristics in a mental rotation task presented in virtual reality.

Eye-tracking technology provides a reliable and cost-effective approach to characterize mental representation according to specific patterns. Mental rotation tasks, referring to the mental representation and transformation of visual information, have been widely used to examine visuospatial ability. In these tasks, participants visually perceive three-dimensional (3D) objects and mentally rotate them until they identify whether the paired objects are identical or mirrored. In most studies, 3D objects are presented using two-dimensional (2D) images on a computer screen. Currently, visual neuroscience tends to investigate visual behavior responding to naturalistic stimuli rather than image stimuli. Virtual reality (VR) is an emerging technology used to provide naturalistic stimuli, allowing the investigation of behavioral features in an immersive environment similar to the real world. However, mental rotation tasks using 3D objects in immersive VR have been rarely reported. Here, we designed a VR mental rotation task using 3D stimuli presented in a head-mounted display (HMD). An eye tracker incorporated into the HMD was used to examine eye movement characteristics during the task synchronically. The stimuli were virtual paired objects oriented at specific angular disparities (0, 60, 120, and 180°). We recruited thirty-three participants who were required to determine whether the paired 3D objects were identical or mirrored. Behavioral results demonstrated that the response times when comparing mirrored objects were longer than identical objects. Eye-movement results showed that the percent fixation time, the number of within-object fixations, and the number of saccades for the mirrored objects were significantly lower than that for the identical objects, providing further explanations for the behavioral results. In the present work, we examined behavioral and eye movement characteristics during a VR mental rotation task using 3D stimuli. Significant differences were observed in response times and eye movement metrics between identical and mirrored objects. The eye movement data provided further explanation for the behavioral results in the VR mental rotation task.

Comparison of Graft Outcomes Reusing Original Intermediate-Term Cold Storage Solution for Entire Corneal Donor Storage Period With Exchanged Fresh Storage Solution After Donor Preparation in the Cornea Preservation Time Study.

Department of Ophthalmology and Visual Neuroscience, University of Minnesota, Minneapolis, MN Financial disclosures/conflicts of interest: None reported.

Face distortions in prosopometamorphopsia provide new insights into the organization of face perception

Prosopometamorphopsia (PMO) is a striking condition of visual perception in which facial features appear distorted, for example drooping, swelling, or twisting. Although numerous cases have been reported, few of those investigations have carried out formal testing motivated by theories of face perception. However, because PMO involves conscious visual distortions to faces which participants can report, it can be used to probe fundamental questions about face representations. Here we review cases of PMO that address theoretical questions in visual neuroscience including face specificity, inverted face processing, the importance of the vertical midline, dissociable representations for each half of the face, hemispheric specialization, the relationship between face recognition and conscious face perception, and the reference frames that face representations are embedded within. Finally, we list and touch upon eighteen open questions that make clear how much is left to learn about PMO and the potential it has to provide important advances in face perception.

Marmoset core visual object recognition behavior is comparable to that of macaques and humans

Among the smallest simian primates, the common marmoset offers promise as an experimentally tractable primate model for neuroscience with translational potential to humans. However, given its exceedingly small brain and body, the gap in perceptual and cognitive abilities between marmosets and humans requires study. Here, we performed a comparison of marmoset behavior to that of three other species in the domain of high-level vision. We first found that marmosets outperformed rats- a marmoset-sized rodent- on a simple recognition task, with marmosets robustly recognizing objects across views. On a more challenging invariant object recognition task used previously in humans, marmosets also achieved high performance. Notably, across hundreds of images, marmosets' image-by-image behavior was highly similar to that of humans- nearly as human-like as macaque behavior. Thus, core aspects of visual perception are conserved across monkeys and humans, and marmosets present salient behavioral advantages over other small model organisms for visual neuroscience.

Encoding fidelity of binocular receptive fields with internal noise in the presence of external variability from natural scenes

Characterizing the properties of receptive fields that are involved in encoding behaviorally-relevant latent variables (binocular disparity, motion) is a focus of visual neuroscience research. The factors that determine which receptive field properties are most useful for encoding a particular latent variable in natural scenes are not well-understood. Here, using a stereo-image database with groundtruth disparity information at each pixel and a nonlinear-linear-nonlinear subunit modeling framework, we examine how external variability and internal noise jointly determine the binocular receptive field properties that are most useful for encoding binocular disparity in natural scenes. We compute the encoding fidelity, as indexed by Fisher information, of binocular receptive field pairs across preferred spatial frequency (SF) and differences in preferred phase disparity (binocular phase shift). Two major findings emerge. First, stimulus disparity and external variability (image and depth variability) determine the usefulness of different encoding subspaces, as defined by the preferred SFs of the binocular receptive field pairs. The most useful preferred SF decreases with stimulus disparity but increases with local depth variability. Second, internal noise determines the usefulness within the encoding subspaces of different basis elements, as defined by the binocular phase shifts of the receptive fields. In particular, with internal noise, encoding fidelity varies with the particular receptive field pairs within the subspace. Non-orthogonal receptive fields are most useful, a result that deviates from the quadrature-pairs posited by the disparity energy model. Most subunit modeling frameworks do not inject noise into the individual subunit responses. This modeling choice renders all subunit receptive field pairs spanning a subspace computationally equivalent. Incorporating subunit response noise makes more specific predictions regarding the particular receptive fields that are most useful for encoding a particular latent variable. The current modeling efforts bring us closer to understanding the principles governing the design of real neural systems in natural scenes.

A large and rich EEG dataset for modeling human visual object recognition

The human brain achieves visual object recognition through multiple stages of linear and nonlinear transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a large and rich dataset of high temporal resolution EEG responses to images of objects on a natural background. This dataset includes 10 participants, each with 82,160 trials spanning 16,740 image conditions. Through computational modeling we established the quality of this dataset in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the high number of conditions as well as the trial repetitions of the EEG dataset contribute to the trained models’ prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output EEG responses for arbitrary input images. We release this dataset as a tool to foster research in visual neuroscience and computer vision.

Tuning landscapes of the ventral stream

A goal in visual neuroscience is to explain how neurons respond to natural scenes. However, neurons are generally tested using simpler stimuli, often because they can be transformed smoothly, allowing the measurement of tuning functions (i.e., response peaks and slopes). Here, we test the idea that all classic tuning curves can be viewed as slices of a higher-dimensional tuning landscape. We use activation-maximizing stimuli ("prototypes") as landmarks in a generative image space and map tuning functions around these peaks. We find that neurons show smooth bell-shaped tuning consistent with radial basis functions, spanning a vast image transformation range, with systematic differences in landscape geometry from V1 to inferotemporal cortex. By modeling these trends, we infer that neurons in the higher visual cortex have higher intrinsic feature dimensionality. Overall, these results suggest that visual neurons are better viewed as signaling distances to prototypes on an image manifold.

The policy of testing hypotheses in Chilean science. The role of a hypothesis-driven research funding programme in the installation of a hypothesis-driven experimental system in visual neuroscience

O'Malley et al. (2009) and Haufe (2013) suggest that the philosophical idea of science as hypothesis testing generates a pernicious bias towards hypothesis-driven research and against exploratory research in the review process of research proposals and the allocation of resources. This paper addresses a conceptual objection to the argument by O'Malley et al. (2009) and Haufe (2013). We argue that the funding agencies' concepts of good science do not belong to epistemological or philosophical contexts but to political and institutional contexts. This means that correcting (potential) biases in research funding does not entail correcting funding agencies' (supposed) philosophies of science. To illustrate this point, we provide an in-depth historical case study: the granting of funds to neuroscientist Pedro Maldonado by the Chilean funding programme FONDECYT. This is a relevant comparison as FONDECYT's guidelines explicitly promote hypothesis-driven research and endorse a view of “good science” as hypothesis testing. However, we will see that the overall influence of the philosophical idea of science as hypothesis testing over this funding programme, the research project, and the actual practice of hypothesis testing is somewhat limited. The concept of science as hypothesis testing seems to play a crucial institutional or political (not philosophical) role in allowing the conceptual articulation of social expectations and researchers' expectations.

Adaptive optics in the Arctic? A commentary on Fosbury and Jeffery.

Adaptive optics in the Arctic? A commentary on Fosbury and Jeffery.

Insect opsins and evo-devo: what have we learned in 25 years?

The visual pigments known as opsins are the primary molecular basis for colour vision in animals. Insects are among the most diverse of animal groups and their visual systems reflect a variety of life histories. The study of insect opsins in the fruit fly Drosophila melanogaster has led to major advances in the fields of neuroscience, development and evolution. In the last 25 years, research in D. melanogaster has improved our understanding of opsin genotype-phenotype relationships while comparative work in other insects has expanded our understanding of the evolution of insect eyes via gene duplication, coexpression and homologue switching. Even so, until recently, technology and sampling have limited our understanding of the fundamental mechanisms that evolution uses to shape the diversity of insect eyes. With the advent of genome editing and in vitro expression assays, the study of insect opsins is poised to reveal new frontiers in evolutionary biology, visual neuroscience, and animal behaviour. This article is part of the theme issue 'Understanding colour vision: molecular, physiological, neuronal and behavioural studies in arthropods'.

Predicting brain activation maps for arbitrary tasks with cognitive encoding models

A deep understanding of the neural architecture of mental function should enable the accurate prediction of a specific pattern of brain activity for any psychological task, based only on the cognitive functions known to be engaged by that task. Encoding models (EMs), which predict neural responses from known features (e.g., stimulus properties), have succeeded in circumscribed domains (e.g., visual neuroscience), but implementing domain-general EMs that predict brain-wide activity for arbitrary tasks has been limited mainly by availability of datasets that 1) sufficiently span a large space of psychological functions, and 2) are sufficiently annotated with such functions to allow robust EM specification. We examine the use of EMs based on a formal specification of psychological function, to predict cortical activation patterns across a broad range of tasks. We utilized the Multi-Domain Task Battery, a dataset in which 24 subjects completed 32 ten-minute fMRI scans, switching tasks every 35s and engaging in 44 total conditions of diverse psychological manipulations. Conditions were annotated by a group of experts using the Cognitive Atlas ontology to identify putatively engaged functions, and region-wise cognitive EMs (CEMs) were fit, for individual subjects, on neocortical responses. We found that CEMs predicted cortical activation maps of held-out tasks with high accuracy, outperforming a permutation-based null model while approaching the noise ceiling of the data, without being driven solely by either cognitive or perceptual-motor features. Hierarchical clustering on the similarity structure of CEM generalization errors revealed relationships amongst psychological functions. Spatial distributions of feature importances systematically overlapped with large-scale resting-state functional networks (RSNs), supporting the hypothesis of functional specialization within RSNs while grounding their function in an interpretable data-driven manner. Our implementation and validation of CEMs provides a proof of principle for the utility of formal ontologies in cognitive neuroscience and motivates the use of CEMs in the further testing of cognitive theories.

Cortical mechanisms of visual brightness.

SUMMARYThe primary visual cortex signals the onset of light and dark stimuli with ON and OFF cortical pathways. Here, we demonstrate that both pathways generate similar response increments to large homogeneous surfaces and their response average increases with surface brightness. We show that, in cat visual cortex, response dominance from ON or OFF pathways is bimodally distributed when stimuli are smaller than one receptive field center but unimodally distributed when they are larger. Moreover, whereas small bright stimuli drive opposite responses from ON and OFF pathways (increased versus suppressed activity), large bright surfaces drive similar response increments. We show that this size-brightness relation emerges because strong illumination increases the size of light surfaces in nature and both ON and OFF cortical neurons receive input from ON thalamic pathways. We conclude that visual scenes are perceived as brighter when the average response increments from ON and OFF cortical pathways become stronger.

Neuroscience of Sight Informs Visual Language

Visual literacy is both an awareness of visual input and output, encoding and decoding, processing via the eyes and output through visual creation. When informed by the neuroscience of sight and memory, artists and educators can objectively understand their own visual work as well as nurture the visual language of their students (Kasdorf, 2020). This arts-based and practice-based research was part of the author’s thesis during her master’s program in Museum and Gallery Practice at UCL Qatar (Kasdorf, 2020). The analysis further informs the author’s dialogue in creative workshops and classes in Qatar, with children and adults. In the case of workshops with illustration students, participants engage with the aesthetics of archival regional picture books to develop a personal voice. Their outcomes resist mainstream styles and embrace childlike exploration, a perspective that has been exercised in noteworthy Arab publishing history (). Observing Ephemeral Moments An entry point into understanding visual neuroscience is pinpointing ephemeral moments; to acknowledge the “spark” of aesthetic preference. Masland (2017) describes that raw visual data is amalgamated for efficiency by the retina, thereby curated by the personal circuitry. Neurons are generally latent with consistent light and activated with changes in light and direction (Masland, 2017). More determinant of individuality is the heightened activity of neurons in the working memory in relation to a preferred visual; even when the visual is out of direct view, neuronic activity persists (Self & Roelfsema, 2017). By repeatedly recognizing these ephemeral moments, subconscious activity can be harnessed to build visual style and language. Children and Visual Literacy A child’s visual communication is noteworthy and telling of themselves (Khamis, 1962). With knowledge of the neuroscience of sight, an educator’s visual observation and objective language can nurture a child’s uninhibited and automated drawing and creating, as opposed to simply providing external validation. In turn, the child learns to value their own work objectively, without resorting to binary terms such as “good” or “bad”. A Wider Spectrum of “Normal” Visual literacy that incorporates knowledge of the neuroscience of sight supports more possibilities in drawing and creating, and thereby a broader sense of “normal”. Additionally, if drawing can allow for more diverse exploration, the same is enabled for people; visually literate people contribute to a broader normative knowledge system in society. Drawing mirrors the relationships of people; if there is less of a “right” way to draw, there is less of a “right” way for people to be, and less of a “right” way for children (and adults) to understand themselves.

Visual neuroscience: A shrewd look at perceptual learning

A new study provides insight into the neuronal mechanisms that underlie visual learning in the tree shrew, revealing how improved coding for trained stimuli in visual cortex can negatively affect the perception of other stimuli.