Naturalistic Textures Research Articles

The representational geometry for naturalistic textures in macaque V1 and V2.

Our understanding of visual cortical processing has relied primarily on studying the selectivity of individual neurons in different areas. A complementary approach is to study how the representational geometry of neuronal populations differs across areas. Though the geometry is derived from individual neuronal selectivity, it can reveal encoding strategies difficult to infer from single neuron responses. In addition, recent theoretical work has begun to relate distinct functional objectives to different representational geometries. To understand how the representational geometry changes across stages of processing, we measured neuronal population responses in primary visual cortex (V1) and area V2 of macaque monkeys to an ensemble of synthetic, naturalistic textures. Responses were lower dimensional in V2 than V1, and there was a better alignment of V2 population responses to different textures. The representational geometry in V2 afforded better discriminability between out-of-sample textures. We performed complementary analyses of standard convolutional network models, which did not replicate the representational geometry of cortex. We conclude that there is a shift in the representational geometry between V1 and V2, with the V2 representation exhibiting features of a low-dimensional, systematic encoding of different textures and of different instantiations of each texture. Our results suggest that comparisons of representational geometry can reveal important transformations that occur across successive stages of visual processing.

Neuronal and Behavioral Responses to Naturalistic Texture Images in Macaque Monkeys.

The visual world is richly adorned with texture, which can serve to delineate important elements of natural scenes. In anesthetized macaque monkeys, selectivity for the statistical features of natural texture is weak in V1, but substantial in V2, suggesting that neuronal activity in V2 might directly support texture perception. To test this, we investigated the relation between single cell activity in macaque V1 and V2 and simultaneously measured behavioral judgments of texture. We generated stimuli along a continuum between naturalistic texture and phase-randomized noise and trained two macaque monkeys to judge whether a sample texture more closely resembled one or the other extreme. Analysis of responses revealed that individual V1 and V2 neurons carried much less information about texture naturalness than behavioral reports. However, the sensitivity of V2 neurons, especially those preferring naturalistic textures, was significantly closer to that of behavior compared with V1. The firing of both V1 and V2 neurons predicted perceptual choices in response to repeated presentations of the same ambiguous stimulus in one monkey, despite low individual neural sensitivity. However, neither population predicted choice in the second monkey. We conclude that neural responses supporting texture perception likely continue to develop downstream of V2. Further, combined with neural data recorded while the same two monkeys performed an orientation discrimination task, our results demonstrate that choice-correlated neural activity in early sensory cortex is unstable across observers and tasks, untethered from neuronal sensitivity, and therefore unlikely to directly reflect the formation of perceptual decisions.

A comparison of statistical learning of naturalistic textures between DCNNs and the human visual hierarchy

A comparison of statistical learning of naturalistic textures between DCNNs and the human visual hierarchy

Figure-ground segmentation based on motion in the archerfish.

Figure-ground segmentation is a fundamental process in visual perception that involves separating visual stimuli into distinct meaningful objects and their surrounding context, thus allowing the brain to interpret and understand complex visual scenes. Mammals exhibit varying figure-ground segmentation capabilities, ranging from primates that can perform well on figure-ground segmentation tasks to rodents that perform poorly. To explore figure-ground segmentation capabilities in teleost fish, we studied how the archerfish, an expert visual hunter, performs figure-ground segmentation. We trained archerfish to discriminate foreground objects from the background, where the figures were defined by motion as well as by discontinuities in intensity and texture. Specifically, the figures were defined by grating, naturalistic texture, and random noise moving in counterphase with the background. The archerfish performed the task well and could distinguish between all three types of figures and grounds. Their performance was comparable to that of primates and outperformed rodents. These findings suggest the existence of a complex visual process in the archerfish visual system that enables the delineation of figures as distinct from backgrounds, and provide insights into object recognition in this animal.

Visual statistical learning of naturalistic textures.

The visual system continuously adapts to the statistical properties of the environment. In this study, we demonstrated that training significantly enhanced subjects' perceptual sensitivity to co-occurrence statistics in naturalistic textures. The learning effect was specific to the statistical component and spatial location. By examining the time course of learning, we found that learning was accelerated at an untrained location. Our findings establish a link between statistical learning and visual perception, indicating multistage plasticity beyond V1 in the visual hierarchy. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Mice and primates use distinct strategies for visual segmentation.

The rodent visual system has attracted great interest in recent years due to its experimental tractability, but the fundamental mechanisms used by the mouse to represent the visual world remain unclear. In the primate, researchers have argued from both behavioral and neural evidence that a key step in visual representation is 'figure-ground segmentation', the delineation of figures as distinct from backgrounds. To determine if mice also show behavioral and neural signatures of figure-ground segmentation, we trained mice on a figure-ground segmentation task where figures were defined by gratings and naturalistic textures moving counterphase to the background. Unlike primates, mice were severely limited in their ability to segment figure from ground using the opponent motion cue, with segmentation behavior strongly dependent on the specific carrier pattern. Remarkably, when mice were forced to localize naturalistic patterns defined by opponent motion, they adopted a strategy of brute force memorization of texture patterns. In contrast, primates, including humans, macaques, and mouse lemurs, could readily segment figures independent of carrier pattern using the opponent motion cue. Consistent with mouse behavior, neural responses to the same stimuli recorded in mouse visual areas V1, RL, and LM also did not support texture-invariant segmentation of figures using opponent motion. Modeling revealed that the texture dependence of both the mouse's behavior and neural responses could be explained by a feedforward neural network lacking explicit segmentation capabilities. These findings reveal a fundamental limitation in the ability of mice to segment visual objects compared to primates.

Sensitivity to naturalistic texture relies primarily on high spatial frequencies.

Natural images contain information at multiple spatial scales. Though we understand how early visual mechanisms split multiscale images into distinct spatial frequency channels, we do not know how the outputs of these channels are processed further by mid-level visual mechanisms. We have recently developed a texture discrimination task that uses synthetic, multi-scale, "naturalistic" textures to isolate these mid-level mechanisms. Here, we use three experimental manipulations (image blur, image rescaling, and eccentric viewing) to show that perceptual sensitivity to naturalistic structure is strongly dependent on features at high object spatial frequencies (measured in cycles/image). As a result, sensitivity depends on a texture acuity limit, a property of the visual system that sets the highest retinal spatial frequency (measured in cycles/degree) at which observers can detect naturalistic features. Analysis of the texture images using a model observer analysis shows that naturalistic image features at high object spatial frequencies carry more task-relevant information than those at low object spatial frequencies. That is, the dependence of sensitivity on high object spatial frequencies is a property of the texture images, rather than a property of the visual system. Accordingly, we find human observers' ability to extract naturalistic information (their efficiency) is similar for all object spatial frequencies. We conclude that the mid-level mechanisms that underlie perceptual sensitivity effectively extract information from all image features below the texture acuity limit, regardless of their retinal and object spatial frequency.

Texture recognition based on multi-sensory integration of proprioceptive and tactile signals

The sense of touch plays a fundamental role in enabling us to interact with our surrounding environment. Indeed, the presence of tactile feedback in prostheses greatly assists amputees in doing daily tasks. In this line, the present study proposes an integration of artificial tactile and proprioception receptors for texture discrimination under varying scanning speeds. Here, we fabricated a soft biomimetic fingertip including an 8 × 8 array tactile sensor and a piezoelectric sensor to mimic Merkel, Meissner, and Pacinian mechanoreceptors in glabrous skin, respectively. A hydro-elastomer sensor was fabricated as an artificial proprioception sensor (muscle spindles) to assess the instantaneous speed of the biomimetic fingertip. In this study, we investigated the concept of the complex receptive field of RA-I and SA-I afferents for naturalistic textures. Next, to evaluate the synergy between the mechanoreceptors and muscle spindle afferents, ten naturalistic textures were manipulated by a soft biomimetic fingertip at six different speeds. The sensors’ outputs were converted into neuromorphic spike trains to mimic the firing pattern of biological mechanoreceptors. These spike responses are then analyzed using machine learning classifiers and neural coding paradigms to explore the multi-sensory integration in real experiments. This synergy between muscle spindle and mechanoreceptors in the proposed neuromorphic system represents a generalized texture discrimination scheme and interestingly irrespective of the scanning speed.

Perceptual Texture Dimensions Modulate Neuronal Response Dynamics in Visual Cortical Area V4.

Texture is an important visual attribute for surface pattern discrimination and therefore object segmentation, but the neural bases of texture perception are largely unknown. Previously, we demonstrated that the responses of V4 neurons to naturalistic texture patches are sensitive to four key features of human texture perception: coarseness, directionality, regularity, and contrast. To begin to understand how distinct texture perception emerges from the dynamics of neuronal responses, in 2 macaque monkeys (1 male, 1 female), we investigated the relative contribution of the four texture attributes to V4 responses in terms of the strength and timing of response modulation. We found that the different feature dimensions are associated with different temporal dynamics. Specifically, the response modulation associated with directionality and regularity was significantly delayed relative to that associated with coarseness and contrast, suggesting that the latter are fundamentally simpler feature dimensions. The population of texture-selective neurons could be grouped into multiple clusters based on the combination of feature dimensions encoded, and those subpopulations displayed distinct temporal dynamics characterized by the weighted combinations of multiple features. Finally, we applied a population decoding approach to demonstrate that texture category information can be obtained from short temporal windows across time. These results demonstrate that the representation of different perceptually relevant texture features emerge over time in the responses of V4 neurons. The observed temporal organization provides a framework to interpret how the processing of surface features unfolds in early and midlevel cortical stages, and could ultimately inform the interpretation of perceptual texture dynamics.SIGNIFICANCE STATEMENT To delineate how neuronal responses underlie our ability to perceive visual textures, we related four key perceptual dimensions (coarseness, directionality, regularity, and contrast) of naturalistic textures to the strength and timing of modulation of neuronal responses in area V4, an intermediate stage in the form-processing, ventral visual pathway. Our results provide the first characterization of V4 temporal dynamics for texture encoding along perceptually defined axes.

Spatial frequency dependence of naturalistic texture perception

Spatial frequency dependence of naturalistic texture perception

Spatio-Temporal Encoding Improves Neuromorphic Tactile Texture Classification

With the increase in interest in the deployment of robots in unstructured environments to work alongside humans, the development of human-like sense of touch for robots becomes important. In this work, we implement a multi-channel neuromorphic tactile system that encodes contact episodes as discrete spike events that mimic the behavior of slow adapting mechanoreceptors. We study the impact of information pooling across artificial mechanoreceptors on classification performance of spatially non-uniform naturalistic textures. We encoded the spatio-temporal activation patterns of mechanoreceptors through gray-level co-occurrence matrix computed from time-varying mean spiking rate-based tactile response volume. We found that this approach greatly improved texture classification in comparison to the use of individual mechanoreceptor response alone. In addition, the performance was also more robust to changes in sliding velocity. The importance of exploiting precise spatial and temporal correlations between sensory channels is evident from the fact that on either removal of precise temporal information or altering of spatial structure of response pattern, a significant performance drop was observed. This study thus demonstrates the superiority of population coding approaches that can exploit the precise spatio-temporal information encoded in activation patterns of mechanoreceptor populations. It, therefore, makes an advance in the direction of development of bio-inspired tactile systems required for realistic touch applications in robotics and prostheses.

Flexible contextual modulation of naturalistic texture perception in peripheral vision.

Peripheral vision comprises most of our visual field, and is essential in guiding visual behavior. Its characteristic capabilities and limitations, which distinguish it from foveal vision, have been explained by the most influential theory of peripheral vision as the product of representing the visual input using summary statistics. Despite its success, this account may provide a limited understanding of peripheral vision, because it neglects processes of perceptual grouping and segmentation. To test this hypothesis, we studied how contextual modulation, namely the modulation of the perception of a stimulus by its surrounds, interacts with segmentation in human peripheral vision. We used naturalistic textures, which are directly related to summary-statistics representations. We show that segmentation cues affect contextual modulation, and that this is not captured by our implementation of the summary-statistics model. We then characterize the effects of different texture statistics on contextual modulation, providing guidance for extending the model, as well as for probing neural mechanisms of peripheral vision.

Naturalistic texture perception relies preferentially on high spatial frequencies

Naturalistic texture perception relies preferentially on high spatial frequencies

Morphological Neural Computation Restores Discrimination of Naturalistic Textures in Trans-radial Amputees

Humans rely on their sense of touch to interact with the environment. Thus, restoring lost tactile sensory capabilities in amputees would advance their quality of life. In particular, texture discrimination is an important component for the interaction with the environment, but its restoration in amputees has been so far limited to simplified gratings. Here we show that naturalistic textures can be discriminated by trans-radial amputees using intraneural peripheral stimulation and tactile sensors located close to the outer layer of the artificial skin. These sensors exploit the morphological neural computation (MNC) approach, i.e., the embodiment of neural computational functions into the physical structure of the device, encoding normal and shear stress to guarantee a faithful neural temporal representation of stimulus spatial structure. Two trans-radial amputees successfully discriminated naturalistic textures via the MNC-based tactile feedback. The results also allowed to shed light on the relevance of spike temporal encoding in the mechanisms used to discriminate naturalistic textures. Our findings pave the way to the development of more natural bionic limbs.

Cuneate spiking neural network learning to classify naturalistic texture stimuli under varying sensing conditions

We implemented a functional neuronal network that was able to learn and discriminate haptic features from biomimetic tactile sensor inputs using a two-layer spiking neuron model and homeostatic synaptic learning mechanism. The first order neuron model was used to emulate biological tactile afferents and the second order neuron model was used to emulate biological cuneate neurons. We have evaluated 10 naturalistic textures using a passive touch protocol, under varying sensing conditions. Tactile sensor data acquired with five textures under five sensing conditions were used for a synaptic learning process, to tune the synaptic weights between tactile afferents and cuneate neurons. Using post-learning synaptic weights, we evaluated the individual and population cuneate neuron responses by decoding across 10 stimuli, under varying sensing conditions. This resulted in a high decoding performance. We further validated the decoding performance across stimuli, irrespective of sensing velocities using a set of 25 cuneate neuron responses. This resulted in a median decoding performance of 96% across the set of cuneate neurons. Being able to learn and perform generalized discrimination across tactile stimuli, makes this functional spiking tactile system effective and suitable for further robotic applications.

Laminar Differences in Responses to Naturalistic Texture in Macaque V1 and V2.

Most single units recorded from macaque secondary visual cortex (V2) respond with higher firing rates to synthetic texture images containing "naturalistic" higher-order statistics than to spectrally matched "noise" images lacking these statistics. In contrast, few single units in V1 show this property. We explored how the strength and dynamics of response vary across the different layers of visual cortex by recording multiunit (defined as high-frequency power in the local field potential) and gamma-band activity evoked by brief presentations of naturalistic and noise images in V1 and V2 of anesthetized macaque monkeys of both sexes. As previously reported, recordings in V2 showed consistently stronger responses to naturalistic texture than to spectrally matched noise. In contrast to single-unit recordings, V1 multiunit activity showed a preference for images with naturalistic statistics, and in gamma-band activity this preference was comparable across V1 and V2. Sensitivity to naturalistic image structure was strongest in the supragranular and infragranular layers of V1, but weak in granular layers, suggesting that it might reflect feedback from V2. Response timing was consistent with this idea. Visual responses appeared first in V1, followed by V2. Sensitivity to naturalistic texture emerged first in V2, followed by the supragranular and infragranular layers of V1, and finally in the granular layers of V1. Our results demonstrate laminar differences in the encoding of higher-order statistics of natural texture, and suggest that this sensitivity first arises in V2 and is fed back to modulate activity in V1.SIGNIFICANCE STATEMENT The circuit mechanisms responsible for visual representations of intermediate complexity are largely unknown. We used a well validated set of synthetic texture stimuli to probe the temporal and laminar profile of sensitivity to the higher-order statistical structure of natural images. We found that this sensitivity emerges first and most strongly in V2 but soon after in V1. However, sensitivity in V1 is higher in the laminae (extragranular) and recording modalities (local field potential) most likely affected by V2 connections, suggesting a feedback origin. Our results show how sensitivity to naturalistic image structure emerges across time and circuitry in the early visual cortex.

Fingerprint‐Enhanced Capacitive‐Piezoelectric Flexible Sensing Skin to Discriminate Static and Dynamic Tactile Stimuli

Inspired by the structure and functions of the human skin, a highly sensitive capacitive‐piezoelectric flexible sensing skin with fingerprint‐like patterns to detect and discriminate between spatiotemporal tactile stimuli including static and dynamic pressures and textures is presented. The capacitive‐piezoelectric tandem sensing structure is embedded in the phalange of a 3D‐printed robotic hand, and a tempotron classifier system is used for tactile exploration. The dynamic tactile sensor, interfaced with an extended gate configuration to a common source metal oxide semiconductor field effect transistor (MOSFET), exhibits a sensitivity of 2.28 kPa−1. The capacitive sensing structure has nonlinear characteristics with sensitivity varying from 0.25 kPa−1 in the low‐pressure range (<100 Pa) to 0.002 kPa−1 in high pressure (≈2.5 kPa). The output from the presented sensor under a closed‐loop tactile scan, carried out with an industrial robotic arm, is used as latency‐coded spike trains in a spiking neural network (SNN) tempotron classifier system. With the capability of performing a real‐time binary naturalistic texture classification with a maximum accuracy of 99.45%, the presented bioinspired skin finds applications in robotics, prosthesis, wearable sensors, and medical devices.

Speed uncertainty and motion perception with naturalistic random textures

Speed uncertainty and motion perception with naturalistic random textures

Development of sensitivity to naturalistic textures in macaque: psychophysics and physiology

Development of sensitivity to naturalistic textures in macaque: psychophysics and physiology

Texture Stationarization: Turning Photos into Tileable Textures

Texture synthesis has grown into a mature field in computer graphics, allowing the synthesis of naturalistic textures and images from photographic exemplars. Surprisingly little work, however, has been dedicated to synthesizing tileable textures, that is, textures that when laid out in a regular grid of tiles form a homogeneous appearance suitable for use in memory-sensitive real-time graphics applications. One of the key challenges in doing so is that most natural input exemplars exhibit uneven spatial variations that, when tiled, show as repetitive patterns. We propose an approach to synthesize tileable textures while enforcing stationarity properties that effectively mask repetitions while maintaining the unique characteristics of the exemplar. We explore a number of alternative measures for texture stationarity and show how each measure can be integrated into a standard texture synthesis method PatchMatch to enforce stationarity at user-controlled scales. We demonstrate the efficacy of our approach using a database of 118 exemplar images, both from publicly available sources as well as new ones captured under uncontrolled conditions, and we quantitatively analyze alternative stationarity measures for their robustness across many test runs using different random seeds. In conclusion, we suggest a novel synthesis approach that employs local histogram matching to reliably turn input photographs of natural surfaces into tiles well suited for artifact-free tiling.