Award Lecture: 2021 Tillyer Award Lecture: The Early Visual Encoding: Computations and Consequences.

Abstract

The visual system takes sensory measurements of the light incident at the eyes and uses these to make perceptual inferences about external world. The sensory measurements do not preserve all of the information available in the light signal. One approach to understanding the implications of the initial stages of visual processing is ideal observer analysis, which evaluates the information available to support psychophysical discriminations at various stages of the early visual representation. We are interested in extending this type of analysis to take into account the statistical structure of natural images. To do so, we developed an open-source computational model of the initial visual encoding, ISETBio (isetbio.org). ISETBio incorporates specification of visual displays, retinal image formation through the eye's optics, spatio-spectral sampling by the retinal cone mosaic, Poisson noise in the cone photopigment excitations, transduction of excitations to photocurrent, and fixational eye movements. In this talk, I will introduce ISETBio and illustrate a set of insights it enables about visual processing by reviewing a number of computational examples. These examples will include ways that combining ISETBio with Bayesian image-reconstruction methods helps us understand how the interaction of the visual encoding and the statistical structure of natural images shapes visual performance.