The statistics of contour fragments in natural scenes

Abstract

AbstractSummary: Recent electrophysiology recordings in macaque V4/IT suggest that single neuron response to synthetic closed contours can be largely captured by models which only consider a small number of contour fragments (Brincat and Connor 2004). Motivated by this experimental work, we sought firstly to characterize the statistics of contour fragments in natural scenes, and secondly to generate synthetic images which reflect the measured contour-fragment statistics.To detect contour fragments, we defined a set of feature detectors which respond only in the presence of two edges co-occurring at a fixed relative angle – implemented as a logical ‘AND’ of two Gabor-like, laplacian-of-gaussian linear filters. We then determined the pairwise correlations of these contour fragments in a natural image ensemble. If efficient coding extends to higher cortical centers and processing in the ventral visual stream can be modeled as a sequence of logical operations on linear shape features, then the pairwise statistics we measure should be informative about neural shape coding.Using these statistics directly, it is possible to produce a generative model of simple images which contain the measured statistics. We implemented a modified Ising model and solved the inverse problem of determining the optimal model parameters which satisfy the measured correlations. The resulting Ising-like model of the pairwise statistics can generate the probability of any arrangement of contour fragments as measured in the natural image ensemble.As a complementary approach to producing images with naturalistic contour fragment statistics, it is possible to start with a natural scene and isolate the target features. This is achieved by applying our contour fragment detection processing to the single scene and then separately visualizing the fragments detected. This second procedure lends itself to parametric randomization of the generated image.Narrative Elaboration: The central question guiding our study is how shapes are represented in inferotemporal cortex. To that end, we have investigated natural images in order to motivate experiments capable of targeting the extent to which neural processing of shapes involves representing shapes as combinations of key contour features. To simplify, we are focusing on black-and-white images and prioritizing contour features. This project suggests it is possible to generate synthetic images containing only a select set of contour statistics. Our subsequent goals include conducting collaborative macaque electrophysiology experiments with our generated images as visual stimuli.