Human Texture Vision as Multi-Order Spectral Analysis.

Kosuke Okada,Isamu Motoyoshi

doi:10.3389/fncom.2021.692334

Kosuke Okada, Isamu Motoyoshi

Open Access

https://doi.org/10.3389/fncom.2021.692334

Copy DOI

Abstract

Texture information plays a critical role in the rapid perception of scenes, objects, and materials. Here, we propose a novel model in which visual texture perception is essentially determined by the 1st-order (2D-luminance) and 2nd-order (4D-energy) spectra. This model is an extension of the dimensionality of the Filter-Rectify-Filter (FRF) model, and it also corresponds to the frequency representation of the Portilla-Simoncelli (PS) statistics. We show that preserving two spectra and randomizing phases of a natural texture image result in a perceptually similar texture, strongly supporting the model. Based on only two single spectral spaces, this model provides a simpler framework to describe and predict texture representations in the primate visual system. The idea of multi-order spectral analysis is consistent with the hierarchical processing principle of the visual cortex, which is approximated by a multi-layer convolutional network.

Highlights

The primate visual system rapidly analyzes texture information, or image statistics or ensemble, from complex natural images (Landy and Graham, 2004; Rosenholtz, 2014; Whitney and Yamanashi Leib, 2018), and uses it for the immediate perception and recognition of scenes, objects, and surface materials (Lowe, 1999; Oliva and Torralba, 2001, 2006; Motoyoshi et al, 2007; Rosenholtz et al, 2012)
We extended the dimensions of FRF processing and proposed a novel model that texture perception is based on the 1st-order (2D-luminance) and 2nd-order (4Denergy) amplitude spectra of the image
The model is represented within only two single spectral spaces (+pixel histogram), and it provides a simple framework to describe and predict texture representations in various visual tasks, including scene and material perception

Summary

INTRODUCTION

The primate visual system rapidly analyzes texture information, or image statistics or ensemble, from complex natural images (Landy and Graham, 2004; Rosenholtz, 2014; Whitney and Yamanashi Leib, 2018), and uses it for the immediate perception and recognition of scenes, objects, and surface materials (Lowe, 1999; Oliva and Torralba, 2001, 2006; Motoyoshi et al, 2007; Rosenholtz et al, 2012). The FRF model consists of two stages of image processing based on spatial filtering and energy computation. The FRF model can be extended and considered as a simple Fourier spectral analysis of the luminance data (1storder, 2D) and the subband energy data (2nd-order, 4D) On this basis, we propose a novel model that states visual texture processing is represented as 1st- and 2nd-order spectral analyses (Figure 3). We propose a novel model that states visual texture processing is represented as 1st- and 2nd-order spectral analyses (Figure 3) From this viewpoint, the 1st-order spectrum has a detailed frequency representation of the luminance image, including a wide range of periodic variations, and the 2nd-order spectrum has a 4D frequency representation of the subband energy data. We did not control stimulus duration, if we controlled it to a short time, the importance of the statistics (and the rank between synthesis conditions), might have changed due to temporal dynamics in the hierarchy of neural processing

DISCUSSION

Methods

Findings

ETHICS STATEMENT