Ventral Stream Of Visual Cortex Research Articles

Representation Learning in Sensory Cortex: A Theory

We review and apply a computational theory based on the hypothesis that the feedforward path of the ventral stream in visual cortex’s main function is the encoding of invariant representations of images. A key justification of the theory is provided by a result linking invariant representations to small sample complexity for image recognition - that is, invariant representations allow learning from very few labeled examples. The theory characterizes how an algorithm that can be implemented by a set of “simple” and “complex” cells - a “Hubel Wiesel module” – provides invariant and selective representations. The invariance can be learned in an unsupervised way from observed transformations. Our results show that an invariant representation implies several properties of the ventral stream organization, including the emergence of Gabor receptive filelds and specialized areas. The theory requires two stages of processing: the first, consisting of retinotopic visual areas such as V1, V2 and V4 with generic neuronal tuning, leads to representations that are invariant to translation and scaling; the second, consisting of modules in IT (Inferior Temporal cortex), with class- and object-specific tuning, provides a representation for recognition with approximate invariance to class specific transformations, such as pose (of a body, of a face) and expression. In summary, our theory is that the ventral stream’s main function is to implement the unsupervised learning of “good” representations that reduce the sample complexity of the final supervised learning stage.

Retinex-like computations in human lightness perception and their possible realization in visual cortex

Land and McCann’s original Retinex theory [1] described how the brain might achieve color constancy by spatially integrating the outputs of edge detector neurons in visual cortex (i.e., Hubel and Wiesel cells). Given a collection of reflective surfaces, separated by hard edges (a Mondrian stimulus) and viewed under uniform illumination, Retinex first computes luminance ratios at the borders between surfaces, then multiplies these ratios along image paths to compute the relative ratios of noncontiguous surfaces. This multiplication is equivalent to summing steps in log luminance. Here I review results from the human lightness literature supporting the key Retinex assumption that biological lightness computations involve a spatial integration of steps in log luminance. However, to explain perceptual data, the original Retinex algorithm must be supplemented with additional perceptual principles that together determine the weights given to particular image edges. These include: distance-dependent edge weighting, different weights for incremental and decremental luminance steps, contrast gain acting between edges, top-down control of edge weights, and computations in object-centered coordinates. I outline a theory, informed by recent findings from neurophysiology, of how these computations might be carried out by neural circuits in the ventral stream of visual cortex.

Unsupervised learning of invariant representations

The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n→∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n→1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a “good” representation for supervised learning, characterized by small sample complexity. We consider the case of visual object recognition, though the theory also applies to other domains like speech. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translation, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and selective signature can be computed for each image or image patch: the invariance can be exact in the case of group transformations and approximate under non-group transformations. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such signature. The theory offers novel unsupervised learning algorithms for “deep” architectures for image and speech recognition. We conjecture that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and selective for recognition—and show how this representation may be continuously learned in an unsupervised way during development and visual experience.

Edge integration in achromatic color perception and the lightness-darkness asymmetry

To maintain color constancy, the human visual system must distinguish surface reflectance-based variations in wavelength and luminance from variations due to illumination. Edge integration theory proposes that this is accomplished by spatially integrating steps in luminance and color contrast that likely result from reflectance changes. Thus, a neural representation of relative reflectance within the visual scene is constructed. An anchoring rule-the largest reflectance in the neural representation appears white-is then applied to map relative lightness onto an absolute lightness scale. A large body of data on human lightness judgments is here shown to be consistent with an edge integration model in which the visual system performs a weighted sum of steps in log luminance across space. Three hypotheses are proposed regarding how weights are applied to edges. First, weights decline with distance from the target surface whose lightness is being computed. Second, larger weights are given to edges whose dark sides point towards the target. Third, edge integration is carried out along a path leading from a common background field, or surround, to the target location. The theory accounts for simultaneous contrast; quantitative lightness judgments made with classical disk-annulus, Gilchrist dome, and Gelb displays; and perceptual filling-in lightness. A cortical theory of lightness in the ventral stream of visual cortex (areas V1 → V4) is proposed to instantiate the edge integration algorithm. The neural model is shown to be capable of unifying the quantitative laws of edge integration in lightness perception with the laws governing brightness, including Stevens' power law brightness model, and makes novel predictions about the quantitative laws governing induced darkness.

Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds

A system for the detection, segmentation and recognition of multi-class hand postures against complex natural backgrounds is presented. Visual attention, which is the cognitive process of selectively concentrating on a region of interest in the visual field, helps human to recognize objects in cluttered natural scenes. The proposed system utilizes a Bayesian model of visual attention to generate a saliency map, and to detect and identify the hand region. Feature based visual attention is implemented using a combination of high level (shape, texture) and low level (color) image features. The shape and texture features are extracted from a skin similarity map, using a computational model of the ventral stream of visual cortex. The skin similarity map, which represents the similarity of each pixel to the human skin color in HSI color space, enhanced the edges and shapes within the skin colored regions. The color features used are the discretized chrominance components in HSI, YCbCr color spaces, and the similarity to skin map. The hand postures are classified using the shape and texture features, with a support vector machines classifier. A new 10 class complex background hand posture dataset namely NUS hand posture dataset-II is developed for testing the proposed algorithm (40 subjects, different ethnicities, various hand sizes, 2750 hand postures and 2000 background images). The algorithm is tested for hand detection and hand posture recognition using 10 fold cross-validation. The experimental results show that the algorithm has a person independent performance, and is reliable against variations in hand sizes and complex backgrounds. The algorithm provided a recognition rate of 94.36 %. A comparison of the proposed algorithm with other existing methods evidences its better performance.

A spiking neural network based cortex-like mechanism and application to facial expression recognition.

In this paper, we present a quantitative, highly structured cortex-simulated model, which can be simply described as feedforward, hierarchical simulation of ventral stream of visual cortex using biologically plausible, computationally convenient spiking neural network system. The motivation comes directly from recent pioneering works on detailed functional decomposition analysis of the feedforward pathway of the ventral stream of visual cortex and developments on artificial spiking neural networks (SNNs). By combining the logical structure of the cortical hierarchy and computing power of the spiking neuron model, a practical framework has been presented. As a proof of principle, we demonstrate our system on several facial expression recognition tasks. The proposed cortical-like feedforward hierarchy framework has the merit of capability of dealing with complicated pattern recognition problems, suggesting that, by combining the cognitive models with modern neurocomputational approaches, the neurosystematic approach to the study of cortex-like mechanism has the potential to extend our knowledge of brain mechanisms underlying the cognitive analysis and to advance theoretical models of how we recognize face or, more specifically, perceive other people's facial expression in a rich, dynamic, and complex environment, providing a new starting point for improved models of visual cortex-like mechanism.

Pose Invariant Face Recognition Using Neuro-Biologically Inspired Features

A face recognition algorithm using biologically inspired C2 standard model features is proposed. The C2 feature extraction system, which is based on the standard model of the ventral stream of visual cortex, is modified for the extraction of facial features. The S2 units in the learning stage are tuned to frontal and profile views of the faces to provide pose invariant recognition. A new learning function is utilized to extract discriminative facial features. The spatial invariance is limited both in the local and global pooling stages. The reduced spatial invariance in the local pooling preserves the minute shape and texture details of the facial components, whereas that in the global pooling stage preserves the component location information. The faces are classified using a support vector machines classifier with linear kernel. The proposed algorithm is tested using two face datasets (MIT-CBCL, FEI) with wide variations in profiles, and compared with existing algorithms (the original C2 feature extraction algorithm and MPCALDA algorithm). The experimental results provide evidence of the better performance of the proposed approach for pose invariant face recognition, by at least 6.5%.

Enhanced Biologically Inspired Model for Object Recognition

The biologically inspired model (BIM) proposed by Serre presents a promising solution to object categorization. It emulates the process of object recognition in primates' visual cortex by constructing a set of scale- and position-tolerant features whose properties are similar to those of the cells along the ventral stream of visual cortex. However, BIM has potential to be further improved in two aspects: mismatch by dense input and randomly feature selection due to the feedforward framework. To solve or alleviate these limitations, we develop an enhanced BIM (EBIM) in terms of the following two aspects: 1) removing uninformative inputs by imposing sparsity constraints, 2) apply a feedback loop to middle level feature selection. Each aspect is motivated by relevant psychophysical research findings. To show the effectiveness of the EBIM, we apply it to object categorization and conduct empirical studies on four computer vision data sets. Experimental results demonstrate that the EBIM outperforms the BIM and is comparable to state-of-the-art approaches in terms of accuracy. Moreover, the new system is about 20 times faster than the BIM.

HAND POSTURE AND FACE RECOGNITION USING A FUZZY-ROUGH APPROACH

A novel algorithm based on fuzzy-rough sets is proposed for the recognition of hand postures and face. Features of the image are extracted using the computational model of the ventral stream of visual cortex. The recognition algorithm translates each quantitative value of the feature into fuzzy sets of linguistic terms using membership functions. The membership functions are formed by the fuzzy partitioning of the feature space into fuzzy equivalence classes, using the feature cluster centers generated by the subtractive clustering technique. A rule base generated from the lower and upper approximations of the fuzzy equivalence classes classifies the images through a voting process. Using genetic algorithm, the number of features required for classification is reduced by identifying the predictive image features. The margin of classification, which is a measure of the discriminative power of the classifier, is used to ensure the quality of classification process. The fitness function suggested assists in the feature selection process without compromising on the classification accuracy and margin. The proposed algorithm is tested using two hand posture and three face datasets. The algorithm provided good classification accuracy, at a less computational effort. The selection of relevant features further reduced the computational costs of both feature extraction and classification algorithms, which makes it suitable for real-time applications. The performance of the proposed algorithm is compared with that of support vector machines.