Natural scene statistics relate to perceptual salience of second-, third-, and fourth-order spatial correlations

Abstract

The statistical regularities of natural scenes are a starting point for understanding the characteristics of early visual processing, e.g. the center-surround architecture of retinal ganglion cell receptive fields. Can this matching between natural signal statistics and neural processing mechanisms be extended beyond the sensory periphery? Our recent work [1] revealed that human sensitivity to fourth-order correlations in synthetic textures, known to arise in cortex, is closely related to the structure of fourth-order spatial correlations in natural scenes. This leads us to propose a specific organizing principle: The perceptual salience of visual textures increases with the variance, or unpredictability, of the corresponding correlations over the ensemble of natural scenes. To test this principle, we examined the statistical regularities of binarized natural images as characterized by correlations between adjacent pixels within a 2 × 2 square. Local binary textures can be described by four types of second-order correlations between pixels arranged in vertical (βV), horizontal (βH) and diagonal (β and β/) configurations, four types of third-order correlations between pixel triplets (θ1,2,3,4), and one type of fourth-order correlation between pixel quadruplets (α). We measured the values of these correlations in a large ensemble of image patches, and we compared the results to psychophysical experiments that measure human sensitivity to synthetic visual textures. Both the ordering and magnitude of natural image variances was found to match perceptual sensitivities to synthetic textures generated with the corresponding correlations (Figure ​(Figure1a).1a). Furthermore, the principal components of the full 9D space of image statistics match the principal components of the corresponding space of sensitivities (Figure 1b-e). These results suggest that central neural mechanisms are efficiently tuned to the higher-order statistics of natural scenes. Figure 1 (a) Variances in image statistics match perceptual sensitivities. Principal components of image statistics match principal components of human sensitivities, as shown by the fractional contribution of (b) βHV, (c) β/\, (d) θ, and ...