An Image-Based Model for Early Visual Processing

Abstract

Figure 2: Contrast discrimination. Increment threshold vs. pedestal contrast for gratigs of 8.37 cyc/deg and 1497ms presentation time. Early spatial vision was explored extensively over several decades in many psychophysical detection and discrimination experiments. Thus a large body of such data is available. Goris et al. (2013) integrated this psychophysical literature and proposed a model based on maximumlikelihood decoding of a neurophysiologically inspired population of model neurons. Their neural population model (NPM) is unique in its ability to predict several data sets simultaneously, using a single set of parameters. However, the NPM is only one-dimensional, operating on the activity of abstract spatial frequency channels. Thus it cannot be applied to arbitrary images as a generic front-end to explore the influence of early visual processing on midor high-level vision. Bradley et al. (2014), on the other hand, recently presented an early vision model operating on images. Their model is thus able to make predictions for arbitrary images. However, compared to the NPM, their model lacks in nonlinear processing, which is replaced by an effective masking contrast depending on the detection target. Thus while Bradley et al. fit a range of detection data they do not fit nonlinear aspects of early vision like the dipper function. Here we combine both approaches into a model which includes nonlinear processing and statistically efficient decoding and operates on images explicitly calculating channel activities over the image. The model applies optical degradation and retinal processing to the image before the image is passed to a spatial frequency and orientation decomposition followed by divisive inhibition. We tested the predictions of our model against a broad range of early psychophysical experiments and found it predicts some hallmarks of early visual processing like the contrast sensitivity function (Figure 1), the dipper function for contrast discrimination (Figure 2) and oblique masking data (Figure 3). The measured data we plot against stems from standard 2AFC contrast detection and discrimination experiments run for Wichmann’s DPhil thesis (1999). Early vision was extensively explored empirically and there are plenty of models which explain different parts of the empirical findings. These results provide detailed information about the first processing steps in visual perception thought to happen in the retina, LGN and V1, which stimuli can be differentiated or detected using their output and in what format the output is available to any higher visual processing. However, research on higher visual processing rarely employs the information available from early vision psychophysics. The main reasons may be that the models described in the early spatial vision literature typically do not predict behaviour on arbitrary images, because they often do not process images, but only predict the (putative) responses of the early visual system to Gabor patches or simple grating stimuli. We try to overcome these limitations by explaining many psychophysical observations within a model taking images as input.