Natural Visual Input Research Articles

A chromatic feature detector in the retina signals visual context changes.

The retina transforms patterns of light into visual feature representations supporting behaviour. These representations are distributed across various types of retinal ganglion cells (RGCs), whose spatial and temporal tuning properties have been studied extensively in many model organisms, including the mouse. However, it has been difficult to link the potentially nonlinear retinal transformations of natural visual inputs to specific ethological purposes. Here, we discover a nonlinear selectivity to chromatic contrast in an RGC type that allows the detection of changes in visual context. We trained a convolutional neural network (CNN) model on large-scale functional recordings of RGC responses to natural mouse movies, and then used this model to search in silico for stimuli that maximally excite distinct types of RGCs. This procedure predicted centre colour opponency in transient suppressed-by-contrast (tSbC) RGCs, a cell type whose function is being debated. We confirmed experimentally that these cells indeed responded very selectively to Green-OFF, UV-ON contrasts. This type of chromatic contrast was characteristic of transitions from ground to sky in the visual scene, as might be elicited by head or eye movements across the horizon. Because tSbC cells performed best among all RGC types at reliably detecting these transitions, we suggest a role for this RGC type in providing contextual information (i.e. sky or ground) necessary for the selection of appropriate behavioural responses to other stimuli, such as looming objects. Our work showcases how a combination of experiments with natural stimuli and computational modelling allows discovering novel types of stimulus selectivity and identifying their potential ethological relevance.

Unsupervised changes in core object recognition behavior are predicted by neural plasticity in inferior temporal cortex.

Temporal continuity of object identity is a feature of natural visual input and is potentially exploited - in an unsupervised manner - by the ventral visual stream to build the neural representation in inferior temporal (IT) cortex. Here, we investigated whether plasticity of individual IT neurons underlies human core object recognition behavioral changes induced with unsupervised visual experience. We built a single-neuron plasticity model combined with a previously established IT population-to-recognition-behavior-linking model to predict human learning effects. We found that our model, after constrained by neurophysiological data, largely predicted the mean direction, magnitude, and time course of human performance changes. We also found a previously unreported dependency of the observed human performance change on the initial task difficulty. This result adds support to the hypothesis that tolerant core object recognition in human and non-human primates is instructed - at least in part - by naturally occurring unsupervised temporal contiguity experience.

Resilience of temporal processing to early and extended visual deprivation

Early visual deprivation is known to have profound consequences on the subsequent development of spatial visual processing. However, its impact on temporal processing is not well characterized. We have examined spatial and temporal contrast sensitivity functions following treatment for early and extended bilateral visual deprivation in fifteen children born with congenital cataracts in rural India. The results reveal a marked difference in post-treatment spatial and temporal sensitivities. Whereas spatial processing in newly sighted children is significantly impaired relative to age-matched controls, temporal processing exhibits remarkable resilience and is comparable to that in the control group. This difference in spatial and temporal outcomes is especially surprising given our computational analyses of video sequences which indicate a strong linkage between the spatial and temporal spectral content of natural visual inputs. We consider possible explanations for this discrepancy.

Rapid and active stabilization of visual cortical firing rates across light–dark transitions

The dynamics of neuronal firing during natural vision are poorly understood. Surprisingly, mean firing rates of neurons in primary visual cortex (V1) of freely behaving rodents are similar during prolonged periods of light and darkness, but it is unknown whether this reflects a slow adaptation to changes in natural visual input or insensitivity to rapid changes in visual drive. Here, we use chronic electrophysiology in freely behaving rats to follow individual V1 neurons across many dark-light (D-L) and light-dark (L-D) transitions. We show that, even on rapid timescales (1 s to 10 min), neuronal activity was only weakly modulated by transitions that coincided with the expected 12-/12-h L-D cycle. In contrast, a larger subset of V1 neurons consistently responded to unexpected L-D and D-L transitions, and disruption of the regular L-D cycle with 60 h of complete darkness induced a robust increase in V1 firing on reintroduction of visual input. Thus, V1 neurons fire at similar rates in the presence or absence of natural stimuli, and significant changes in activity arise only transiently in response to unexpected changes in the visual environment. Furthermore, although mean rates were similar in light and darkness, pairwise correlations were significantly stronger during natural vision, suggesting that information about natural scenes in V1 may be more strongly reflected in correlations than individual firing rates. Together, our findings show that V1 firing rates are rapidly and actively stabilized during expected changes in visual input and are remarkably stable at both short and long timescales.

Simulated Saccadic Stimuli Suppress ON-Type Direction-Selective Retinal Ganglion Cells via Glycinergic Inhibition.

Two types of mammalian direction-selective ganglion cells (DSGCs), ON and ONOFF, operate over different speed ranges. The directional axes of the ON-DSGCs are thought to align with the axes of the vestibular system and provide sensitivity at rotational velocities that are too slow to activate the semicircular canals. ONOFF-DSGCs respond to faster image velocities. Using natural images that simulate the natural visual inputs to freely moving animals, we show that simulated visual saccades suppress responses in ON-DSGCs but not ONOFF-DSGCs recorded in retinas of domestic rabbits of either gender. Analysis of the synaptic inputs shows that this saccadic suppression results from glycinergic inputs that are specific to ON-DSGCs and are absent in ONOFF-DSGCs. When this glycinergic input is blocked, both cell types respond similarly to visual saccades and display essentially identical speed tuning. The results demonstrate that glycinergic circuits within the retina can produce saccadic suppression of retinal ganglion cell activity. The cell-type-specific targeting of the glycinergic circuits further supports the proposed physiological roles of ON-DSGCs in retinal-image stabilization and of ONOFF-DSGCs in detecting local object motion and signaling optical flow.SIGNIFICANCE STATEMENT In the mammalian retina, ON direction-selective ganglion cells (DSGCs) respond preferentially to slow image motion, whereas ONOFF-DSGCs respond better to rapid motion. The mechanisms producing this different speed tuning remain unclear. Here we show that simulated visual saccades suppress ON-DSGCs, but not ONOFF-DSGCs. This selective saccadic suppression is because of the selective targeting of glycinergic inhibitory synaptic inputs to ON-DSGCs. The different saccadic suppression in the two cell types points to different physiological roles, consistent with their projections to distinct areas within the brain. ON-DSGCs may be critical for providing the visual feedback signals that contribute to stabilizing the image on the retina, whereas ONOFF-DSGCs may be important for detecting the onset of saccades or for signaling optical flow.

Stimulus- and goal-oriented frameworks for understanding natural vision.

Our knowledge of sensory processing has advanced dramatically in the last few decades, but this understanding remains far from complete, especially for stimuli with the large dynamic range and strong temporal and spatial correlations characteristic of natural visual inputs. Here we describe some of the issues that make understanding the encoding of natural images a challenge. We highlight two broad strategies for approaching this problem: a stimulus-oriented framework and a goal-oriented one. Different contexts can call for one framework or the other. Looking forward, recent advances, particularly those based in machine learning, show promise in borrowing key strengths of both frameworks and by doing so illuminating a path to a more comprehensive understanding of the encoding of natural stimuli.

Active Vision: Dynamic Reformatting of Visual Information by the Saccade-Drift Cycle

Visual processing depends on rapid parsing of global features followed by analysis of fine detail. A new study suggests that this transformation is enabled by a cycle of saccades and fixational drifts, which reformat visual input to match the spatiotemporal sensitivity of fast and slow neuronal pathways.

Contrast sensitivity, V1 neural activity, and natural vision.

Visual sensitivity and activity in brain area V1 were studied in a paradigm that included saccadic eye movements and natural visual input. V1 responses and contrast sensitivity were significantly reduced compared with results in common laboratory paradigms. The parallel neural and perceptual effects of eye movements and stimulus complexity appear to be due to a form of rapid adaptation that carries across saccades.

Specificity and timescales of cortical adaptation as inferences about natural movie statistics.

Adaptation is a phenomenological umbrella term under which a variety of temporal contextual effects are grouped. Previous models have shown that some aspects of visual adaptation reflect optimal processing of dynamic visual inputs, suggesting that adaptation should be tuned to the properties of natural visual inputs. However, the link between natural dynamic inputs and adaptation is poorly understood. Here, we extend a previously developed Bayesian modeling framework for spatial contextual effects to the temporal domain. The model learns temporal statistical regularities of natural movies and links these statistics to adaptation in primary visual cortex via divisive normalization, a ubiquitous neural computation. In particular, the model divisively normalizes the present visual input by the past visual inputs only to the degree that these are inferred to be statistically dependent. We show that this flexible form of normalization reproduces classical findings on how brief adaptation affects neuronal selectivity. Furthermore, prior knowledge acquired by the Bayesian model from natural movies can be modified by prolonged exposure to novel visual stimuli. We show that this updating can explain classical results on contrast adaptation. We also simulate the recent finding that adaptation maintains population homeostasis, namely, a balanced level of activity across a population of neurons with different orientation preferences. Consistent with previous disparate observations, our work further clarifies the influence of stimulus-specific and neuronal-specific normalization signals in adaptation.

Enhanced learning of natural visual sequences in newborn chicks.

To what extent are newborn brains designed to operate over natural visual input? To address this question, we used a high-throughput controlled-rearing method to examine whether newborn chicks (Gallus gallus) show enhanced learning of natural visual sequences at the onset of vision. We took the same set of images and grouped them into either natural sequences (i.e., sequences showing different viewpoints of the same real-world object) or unnatural sequences (i.e., sequences showing different images of different real-world objects). When raised in virtual worlds containing natural sequences, newborn chicks developed the ability to recognize familiar images of objects. Conversely, when raised in virtual worlds containing unnatural sequences, newborn chicks' object recognition abilities were severely impaired. In fact, the majority of the chicks raised with the unnatural sequences failed to recognize familiar images of objects despite acquiring over 100h of visual experience with those images. Thus, newborn chicks show enhanced learning of natural visual sequences at the onset of vision. These results indicate that newborn brains are designed to operate over natural visual input.

Temporal differentiation by photoreceptors reduces spatiotemporal correlation of natural visual input on the retina

Temporal differentiation by photoreceptors reduces spatiotemporal correlation of natural visual input on the retina

Slow feature analysis on retinal waves leads to V1 complex cells.

The developing visual system of many mammalian species is partially structured and organized even before the onset of vision. Spontaneous neural activity, which spreads in waves across the retina, has been suggested to play a major role in these prenatal structuring processes. Recently, it has been shown that when employing an efficient coding strategy, such as sparse coding, these retinal activity patterns lead to basis functions that resemble optimal stimuli of simple cells in primary visual cortex (V1). Here we present the results of applying a coding strategy that optimizes for temporal slowness, namely Slow Feature Analysis (SFA), to a biologically plausible model of retinal waves. Previously, SFA has been successfully applied to model parts of the visual system, most notably in reproducing a rich set of complex-cell features by training SFA with quasi-natural image sequences. In the present work, we obtain SFA units that share a number of properties with cortical complex-cells by training on simulated retinal waves. The emergence of two distinct properties of the SFA units (phase invariance and orientation tuning) is thoroughly investigated via control experiments and mathematical analysis of the input-output functions found by SFA. The results support the idea that retinal waves share relevant temporal and spatial properties with natural visual input. Hence, retinal waves seem suitable training stimuli to learn invariances and thereby shape the developing early visual system such that it is best prepared for coding input from the natural world.

Population code in mouse V1 facilitates readout of natural scenes through increased sparseness.

The neural code is believed to have adapted to the statistical properties of the natural environment. However, the principles that govern the organization of ensemble activity in the visual cortex during natural visual input are unknown. We recorded populations of up to 500 neurons in the mouse primary visual cortex and characterized the structure of their activity, comparing responses to natural movies with those to control stimuli. We found that higher-order correlations in natural scenes induce a sparser code, in which information is encoded by reliable activation of a smaller set of neurons and can be read-out more easily. This computationally advantageous encoding for natural scenes was state-dependent and apparent only in anesthetized and active awake animals, but not during quiet wakefulness. Our results argue for a functional benefit of sparsification that could be a general principle governing the structure of the population activity throughout cortical microcircuits.

Natural image sequences constrain dynamic receptive fields and imply a sparse code

In their natural environment, animals experience a complex and dynamic visual scenery. Under such natural stimulus conditions, neurons in the visual cortex employ a spatially and temporally sparse code. For the input scenario of natural still images, previous work demonstrated that unsupervised feature learning combined with the constraint of sparse coding can predict physiologically measured receptive fields of simple cells in the primary visual cortex. This convincingly indicated that the mammalian visual system is adapted to the natural spatial input statistics. Here, we extend this approach to the time domain in order to predict dynamic receptive fields that can account for both spatial and temporal sparse activation in biological neurons. We rely on temporal restricted Boltzmann machines and suggest a novel temporal autoencoding training procedure. When tested on a dynamic multi-variate benchmark dataset this method outperformed existing models of this class. Learning features on a large dataset of natural movies allowed us to model spatio-temporal receptive fields for single neurons. They resemble temporally smooth transformations of previously obtained static receptive fields and are thus consistent with existing theories. A neuronal spike response model demonstrates how the dynamic receptive field facilitates temporal and population sparseness. We discuss the potential mechanisms and benefits of a spatially and temporally sparse representation of natural visual input.

Behaviourally relevant temporal dynamics of local statistics in natural images

Event Abstract Back to Event Behaviourally relevant temporal dynamics of local statistics in natural images Alexander Schwegmann1*, Jens P. Lindemann1 and Martin Egelhaaf1 1 Bielefeld University, Biology, Germany Evolution is assumed to have shaped the sensory systems of animals to process their typical input at the best according to a given computational task. Parallel to evolving optimized sensory systems animals acquired behavioural strategies that actively shape the input in order to facilitate extraction of the relevant information. For example, the saccadic flight in blowflies is assumed to separate translational and rotational components of retinal image movements already at the behavioural level. To understand the functioning of a visual system, it is important to analyse the temporal changes in statistical properties of the natural visual input of behaving animals. So far research has either focused on the global statistics of static natural scenes or analysed the time course of the input of single photo receptors. To assess to what extent the visual system has to deal with variations in local image statistics we recorded image sequences in a large variety of cluttered natural environments using a green-sensitive high dynamic range panorama camera. The camera was linearly displaced along 1 m to approximate the translatory motion during the intersaccadic intervals of freely flying flies. Saccadic rotations were approximated by image rotation of the panoramic images at the beginning and end of the movement trajectory. The distortions added by the panoramic mirror were computationally removed and the intensity remapped to represent the true intensity values linearly. The analysis was performed on differently sized image patches at different viewing directions and across different scenes taken at places with differing spatial layout. We analysed first order statistics like the time course of local luminance, the contrast, and second order statistics like the overall spatial power spectra and the spatial power along different directions. The distributions of the statistical parameters depend strongly on the scene type. Temporal changes during translational movements are much smaller than those observed during rotations (up to a factor of 20). Distinct changes during translations are only elicited by nearby objects. Fluctuations in the analysed statistical parameters are stronger within small image patches than in large ones. Furthermore the patch size in the tested range has no distinct influence on the mean luminance and spatial frequency spectrum, but on the r.m.s. contrast. Keywords: adaptation, motion vision, natural image statistics, temporal statistics Conference: Tenth International Congress of Neuroethology, College Park. Maryland USA, United States, 5 Aug - 10 Aug, 2012. Presentation Type: Poster (but consider for student poster award) Topic: Sensory: Vision Citation: Schwegmann A, Lindemann JP and Egelhaaf M (2012). Behaviourally relevant temporal dynamics of local statistics in natural images. Conference Abstract: Tenth International Congress of Neuroethology. doi: 10.3389/conf.fnbeh.2012.27.00091 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 19 Apr 2012; Published Online: 07 Jul 2012. * Correspondence: Mr. Alexander Schwegmann, Bielefeld University, Biology, Bielefeld, D-33501, Germany, alexander.schwegmann@uni-bielefeld.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Alexander Schwegmann Jens P Lindemann Martin Egelhaaf Google Alexander Schwegmann Jens P Lindemann Martin Egelhaaf Google Scholar Alexander Schwegmann Jens P Lindemann Martin Egelhaaf PubMed Alexander Schwegmann Jens P Lindemann Martin Egelhaaf Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Space design

Space stations, Moon bases and Mars bases are artificial habitats intended to support human life in extreme conditions. Their purpose is to pursue human progress and to gain knowledge and experience of the environment surrounding our planet. This research focuses on visual investigation in order to improve interface design in space habitat interiors. The subject of this article is why the visual interface (as created through color, light, and artistic and natural visual inputs) is to be considered as fundamental for user reliability in isolated space habitats. The aim is to improve the quality of living conditions in today's International Space Station (ISS) and in future long-term missions to Mars but also in more immediate prospects such as Space tourism. Taking into account experiences from an internship in Thales Alenia Space, ESA Space Habitat Workshops and PhD studies on Space Habitability, the authors' purpose is to enhance the development of concepts and projects on spacecraft visual interface with an ergonomic approach. The main topics in this paper are the following: (1) Visual needs in Space habitat interiors. (2) Sight modification in Space environment. (3) Design: Project Requirements for Outer Space. (4) Moon Base Design.

Auditory Cortex: Representation through Sparsification?

The recent discovery of combination-sensitive neurons in the primary auditory cortex of awake marmosets may reconcile previous, apparently contradictory, findings that cortical neurons produce strong, sustained responses, but also represent stimuli sparsely.

TOWARD A VISUAL COGNITIVE SYSTEM USING ACTIVE TOP-DOWN SACCADIC CONTROL

The saccadic selection of relevant visual input for preferential processing allows the efficient use of computational resources. Based on saccadic active human vision, we aim to develop a plausible saccade-based visual cognitive system for a humanoid robot. This paper presents two initial steps toward our objective by extending the saccade-based model of human memory called NIM1 to a plausible model of natural visual classification. NIM builds feature-vector representations from selected local image samples and uses these to make memory-based decisions. As a first step, we adapt NIM to a straightforward saccade-based model for the classification of natural visual input called NIM-CLASS and evaluate the model in a face-classification experiment. As a second step, we aim to approach the interactive nature of human vision by extending NIM-CLASS to NIM-CLASSTD by adding active top-down saccadic control. We then assess to what extent top-down control enhances classification performance. The results show that the incorporation of top-down saccadic control benefits classification performance compared to the purely bottom-up control, reducing the amount of visual input required for correct classification. We conclude that NIM-CLASSTD may provide a fruitful basis for an active visual cognitive system in a humanoid robot that enables efficient visual processing.

Human eye-head co-ordination in natural exploration

During natural behavior humans continuously adjust their gaze by moving head and eyes, yielding rich dynamics of the retinal input. Sensory coding models, however, typically assume visual input as smooth or a sequence of static images interleaved by volitional gaze shifts. Are these assumptions valid during free exploration behavior in natural environments? We used an innovative technique to simultaneously record gaze and head movements in humans, who freely explored various environments (forest, train station, apartment). Most movements occur along the cardinal axes, and the predominance of vertical or horizontal movements depends on the environment. Eye and head movements co-occur more frequently than their individual statistics predicts under an independence assumption. The majority of co-occurring movements point in opposite directions, consistent with a gaze-stabilizing role of eye movements. Nevertheless, a substantial fraction of eye movements point in the same direction as co-occurring head movements. Even under the very most conservative assumptions, saccadic eye movements alone cannot account for these synergistic movements. Hence nonsaccadic eye movements that interact synergistically with head movements to adjust gaze cannot be neglected in natural visual input. Natural retinal input is continuously dynamic, and cannot be faithfully modeled as a mere sequence of static frames with interleaved large saccades.

Modeling Recognition Memory Using the Similarity Structure of Natural Input

The natural input memory (NIM) model is a new model for recognition memory that operates on natural visual input. A biologically informed perceptual preprocessing method takes local samples (eye fixations) from a natural image and translates these into a feature-vector representation. During recognition, the model compares incoming preprocessed natural input to stored representations. By complementing the recognition memory process with a perceptual front end, the NIM model is able to make predictions about memorability based directly on individual natural stimuli. We demonstrate that the NIM model is able to simulate experimentally obtained similarity ratings and recognition memory for individual stimuli (i.e., face images).