Space-variant Vision Research Articles

An overview of space-variant and active vision mechanisms for resource-constrained human inspired robotic vision

In order to explore and understand the surrounding environment in an efficient manner, humans have developed a set of space-variant vision mechanisms that allow them to actively attend different locations in the surrounding environment and compensate for memory, neuronal transmission bandwidth and computational limitations in the brain. Similarly, humanoid robots deployed in everyday environments have limited on-board resources, and are faced with increasingly complex tasks that require interaction with objects arranged in many possible spatial configurations. The main goal of this work is to describe and overview biologically inspired, space-variant human visual mechanism benefits, when combined with state-of-the-art algorithms for different visual tasks (e.g. object detection), ranging from low-level hardwired attention vision (i.e. foveal vision) to high-level visual attention mechanisms. We overview the state-of-the-art in biologically plausible space-variant resource-constrained vision architectures, namely for active recognition and localization tasks.

Implementations and Implications of Foveated Vision

Humans are equipped with space-variant vision, i.e. a concentration of photoreceptors, retinal ganglion cells and other visual resources at the central fovea, and a sparser coverage of other regions within a wide 180 degree field of view. If the entire visual field was equipped with foveal ganglion cell resolution, then the brain would have to cope with approximately 350 times more visual information. We will review the relatively small number of existing hardware implementations and patents involving space variant vision. Space-variant vision is challenging to implement, because it comes along with distorted image representations, complicating standard geometry-based processing. Recent learning algorithms for feature detection and transformations are more flexible and may cope with foveated images. Foveated vision requires an active vision system: ballistic eye-movements termed frequently move the fovea to points of interest in the visual field. The metric of saccades is adjustable, and the resolution increase at the fovea may play a role in supplying the feedback to the system. Furthermore, saccades are related to visual space perception and embodied vision.

Log-polar mapping template design: From task-level requirements to geometry parameters

The best parameters defining the geometry of a visual sensor generally depend on the particular visual task the sensor is intended to be used in. However, translating task requirements directly into low-level geometric parameters may be difficult, since deep knowledge of the sensor design is usually required, but end users of a sensor need not necessarily be its designers. A framework is suggested to facilitate this translation by including an intermediate layer, the design criteria, between task requirements and sensor parameters. The proposed framework is illustrated with a log-polar space-variant vision model. The motivation behind using this particular sensor is the observation that, in the literature, little attention has been paid to the proper choice of the sensor parameters, or the lack of justification of the chosen configuration. Sets of general-purpose design criteria and task specifications are provided and discussed. The process of finding the best geometric parameters for a given set of (many and/or mutually conflicting) design criteria is automated with a multi-objective genetic algorithm. Some examples are given demonstrating the feasibility and potential of the approach.

Inverse log polar transformation algorithm based on sub-pixel interpolation

Log polar transformation is an important algorithm of space-variant vision theory. It well depicts the retinocortical mapping. Due to the logarithmic and the arctangent operation in the transformation, the log polar coordinate of an image is mostly a decimal fraction and the range is quite narrow. Aiming at solving these problems, this paper puts forward an inverse log polar transformation based on sub-pixel interpolation. The acquired log polar image of the new algorithm has no mosaic phenomenon, meanwhile it is a better simulation of retina-cortical mapping.

Learning to track colored objects with log-polar vision

An approach bringing together space-variant vision through a simple color segmentation technique and learning is presented. The proposed approach is employed to control the movement of a 5 degree of freedom (d.o.f.) robotic head. Color information is used to determine the position of the object of interest in the image plane and, consequently, to track it during its motion. The distance of the target from the center of the image is used to feed both a closed-loop and an open-loop controller. Most important, the parameters of the controllers are learnt on-line in a self-supervised fashion. Experiments are presented to demonstrate empirically the feasibility of the approach and its application to a real world control problem.

Optical flow in log-mapped image plane - a new approach

Foveating vision sensors are important in both machine and biological vision. The term space-variant or foveating vision refers to sensor architectures based on smooth variation of resolution across the visual field, like that of the human visual system. Traditional image processing techniques do not hold when applied directly to such an image representation since the translation symmetry and the neighborhood structure in the spatial domain is broken by the space-variant properties of the sensor. Unfortunately, there has been little systematic development of image processing tools that are explicitly designed for foveated vision. The author proposes a novel approach to compute the optical flow directly on log-mapped images. We propose the use of a generalized dynamic image model (GDIM) based method for computing the optical flow as opposed to the brightness constancy model (BCM) based method. We introduce a new notion of "variable window" and use the space-variant form of gradient operator while computing the spatio-temporal gradient in log-mapped images for a better accuracy and to ensure that the local neighborhood is preserved. We emphasize that the proposed method must be numerically accurate, provide a consistent interpretation, and be capable of computing the peripheral motion. Experimental results on both the synthetic and real images have been presented to show the efficacy of the proposed method.

Real-time tracking of multiple objects in space-variant vision based on magnocellular visual pathway

In this paper, we propose a space-variant image representation model based on properties of magnocellular visual pathway, which perform motion analysis, in human retina. Then, we present an algorithm for the tracking of multiple objects in the proposed space-variant model. The proposed space-variant model has two effective image representations for object recognition and motion analysis, respectively. Each image representation is based on properties of two types of ganglion cell, which are the beginning of two basic visual pathways; one is parvocellular and the other is magnocellular. Through this model, we can get the efficient data reduction capability with no great loss of important information. And, the proposed multiple objects tracking method is restricted in space-variant image. Typically, an object-tracking algorithm consists of several processes such as detection, prediction, matching, and updating. In particular, the matching process plays an important role in multiple objects tracking. In traditional vision, the matching process is simple when the target objects are rigid. In space-variant vision, however, it is very complicated although the target is rigid, because there may be deformation of an object region in the space-variant coordinate system when the target moves to another position. Therefore, we propose a deformation formula in order to solve the matching problem in space-variant vision. By solving this problem, we can efficiently implement multiple objects tracking in space-variant vision.

Fourier Analysis and Cortical Architectures: The Exponential Chirp Transform

The use of visual representations in which pixel-size and local neighborhood topology are not constant is termed space-variant vision. This is the dominant visual architecture in all higher vertebrate visual systems, and is coming to play an important role in real-time active vision applications in the form of log-polar, foveating pyramid, and related approaches to machine vision.The breaking of translation symmetry that is unavoidably associated with space-variant vision presents a major algorithmic complication for image processing. In this paper we use a Lie group approach to derive a kernel which provides a generalization of the Fourier Transform that provides a quasi-shift invariantf2f2We use the term quasi-shift invariant to refer to the non-uniform sampling nature of space-variant maps like the log-polar mapping. Thus, for example, if an object is shifted across the aperture of a space-variant system, our methods will produce invariance up to, but not including, the application of a band-pass filter that is position dependent. This will be made clear in the text.template matching capability in the distorted (range) coordinates of the space-variant mapping. We work out the special case of the log-polar mapping, which is the principle space-variant mapping in use; in this case, we call the associated integral transform the ‘exponential chirp transform’ (ECT). The method is, however, general for other forms of mapping, or warp, function.Examples from the two-dimensional (image processing) log-polar transformation are presented along with the demonstration that the ECT preserves the foveating aspect of the space domain mapping, and therefore provides a quasi-shift invariant realization for the applications of matched filter and phase-only filter. This work provides, for the first time, a conceptual basis for combining global spatial frequency methods with space-variant mappings in a way which is consistent with the anatomical fact that human vision, at the cortical level, takes place in log-polar coordinates.

Space-variant Fourier analysis: the exponential chirp transform

Space-variant (or foveating) vision architectures are of importance in both machine and biological vision. In this paper, we focus on a particular space-variant map, the log-polar map, which approximates the primate visual map, and which has been applied in machine vision by a number of investigators during the past two decades. Associated with the log-polar map, we define a new linear integral transform, which we call the exponential chirp transform. This transform provides frequency domain image processing for space-variant image formats, while preserving the major aspects of the shift-invariant properties of the usual Fourier transform. We then show that a log-polar coordinate transform in frequency provides a fast exponential chirp transform. This provides size and rotation, in addition to shift, invariant properties in the transformed space. Finally, we demonstrate the use of the fast exponential chirp algorithm on a database of images in a template matching task, and also demonstrate its uses for spatial filtering.

Space-variant active vision: Definition, overview and examples

The term space-variant vision was introduced in the late 1980s to refer to sensor architectures based on a smooth variation of resolution across the workspace, like that of the human visual system. The use of such sensor architectures is rapidly becoming an important factor in machine vision in which the constraints of size weight, cost and performance must be jointly optimized. The structure of this paper consists of four parts. A review of the four generic architectures for vision will be presented, providing a context for the term “ active vision”, and a justification for the importance, and the connection between, space-variant architectures and active vision methods. A brief quantitative review of the specific space-variant properties of primate visual cortex topography will be provided, in the cortex of sensor design. The engineering and algorithmic problems that are associated with exploiting space-variant systems will be stated. Examples of several recently constructed miniature space-variant active vision systems will be briefly reviewed, along with a brief discussion of solutions to the basic problem areas in space-variant vision.

Dynamic aspects in active vision

The term active stresses the role of the motion or, generally speaking, the dynamic interaction of the observer with the environment. This concept emphasizes the relevance of determining scene properties from the temporal evolution of image features. Within the active vision approach, two different aspects are considered: the advantages of space-variant vision for dynamic visual processing and the qualitative analysis of optical flow. A space-variant sampling of the image plane has many good properties in relation to active vision. In particular, two examples are presented in which a log-polar representation is used for active vergence control and to estimate the time-to-impact during tracking egomotion. These are just two modules which could fit into a complete active vision system, but already highlight the advantages of space-variant sensing within the active vision paradigm. In the second part of the paper the extraction of qualitative properties from the optical flow is discussed. A new methodology is proposed in which optical flows are analyzed in terms of anomalies as unexpected velocity patterns or inconsistencies with respect to some predicted dynamic feature. Two kinds of knowledge are necessary: about the dynamic of the scene (for example, the approximate motion of the camera) and about the task to be accomplished, which is analogous to (qualitatively) knowing the kind of scene I would expect to see. The first simply implies the measurement of some motion parameters (for example, with inertial sensors) directly on the camera, or to put some constraints in the egomotion. The second requirement implies that the visual process is task-driven. Some examples are presented in which the method is successfully applied to robotic tasks.