Space-variant Sensors Research Articles

Modulation transfer function of spatially variant sampling retina-like sensor

Space-variant sensors are of great importance in both machine and biological vision, and they have superior imaging property than the rectilinear sensors during high-speed forward motion. We focus on the modulation transfer function (MTF) of a particular spatially variant sampling retina-like sensor, pixels of which are arranged in a log-polar fashion. The 3D MTF of the sensor has been built using Hankel Transformation (HT) based on sampling and imaging property of the sensor. Moreover, the output image is simulated in the light of MTF. This research helps to model the imaging process of an imaging system containing retina-like sensor.

EMorph: Towards Neuromorphic Robotic Vision

The eMorph project aims at introducing a new concept for vision in the field of humanoid robotics. The system that is currently being developed is inspired by the biology of mammalian visual systems, introducing concepts such as stimulus-driven signal acquisition and processing, together with space-variant sensor design coupled with active vision. This approach is leading to the realization of a system that goes beyond current thinking in robotic vision.

Sensor Geometry and Sampling Methods for Space-Variant Image Processing

: Space-variant imaging sensors have many advantages over conventional raster imaging sensors. They provide a large field of view for a given pixel count while maintaining a high resolution at the centre of the field of view and, in addition, produce a mapping that is scale and rotation invariant. The effectiveness of the sensor depends greatly upon the geometry used and the sampling methods employed. In this paper, we define a sensor geometry and introduce an ideal weighted sampling method, where the pixels in the image lying at the intersection of sensor cells, are subdivided into smaller sub-pixels, and an interpolation method using a variable width interpolation mask, whose size varies exponentially with the size and shape of the cells in the sensor array. We compare the computational requirements of these methods, and show that they are scale and rotation invariant, when the image is scaled or rotated about its centre, giving the sensor a functionality similar to that provided by the retinal mapping present in the mammalian retina. These results illustrate the advantages that can be obtained in real-time tracking applications in computer vision, where computational and memory requirements need to be kept to a minimum.

Time to collision from first-order spherical image motion

In the absence of contraints on either object motion or surface slant, a narrow field of view constraint has to be assumed to compute time to collision as a scaled depth from first-order image motion measurements. In this work, time to collision and scaled depth are regarded as different visual entities, and it is shown that a bound for time to collision can always be computed regardless of the field of view, thus extending the range of applicability of time to collision based techniques in areas such as mobile robotics and visual surveillance. The method relies on computing in closed form the spherical motion field and the associated parallax from image plane measurements obtained with either conventional cameras or space-variant sensors. An experimental validation of the main theoretical results highlights the difference between time to collision and scaled depth, and addresses a comparison of time to collision approaches using both dense and sparse motion estimates.

Binocular tracking: integrating perception and control

Presents an active binocular tracking system using log-polar images with contributions in both the perceptual and control aspects. The control part is based on the visual servoing framework, including kinematics and dynamics. We introduce a fixation constraint that simplifies the tracking problem by decoupling the visual kinematics and allowing us to express system dynamics in image coordinates. Simple dynamic controllers are designed for each degree of freedom directly from image features. In the perceptual part, we use a space variant sensor that emphasizes the center of the visual field (log-polar geometry). We present a disparity estimation algorithm for log-polar images and provide a theoretical analysis to illustrate the advantages of using space variant images. The overall system is implemented in the Medusa binocular head without any specific processing hardware. The use of log-polar images allows real-time performance (50 Hz). Tracking experiments are presented to illustrate system performance with different control strategies and objects of different shapes and motions.

Space-variant nonorthogonal structure CMOS image sensor design

A CMOS log-polar image sensor has been designed and fabricated. As a result, a systematic approach has been proposed to design space-variant sensors and layouts. The pixels in this sensor are distributed in a polar fashion; the image plane consists of concentric rings containing the elementary sensing cells. Such a structure, where polygons use any space orientation, does not match very well with current design tools and CMOS fabrication processes. An approach to design nonorthogonal repetitive structures using standard fabrication processes and computer-aided design (CAD) tools is presented. The result of this work is an image sensor, with log-polar structure, suitable for image processing since the log-polar mapping has interesting mathematical and image data reduction properties.

Resolution consideration in spatially variant sensors

Log polar transformations for space variant systems have been proposed and used in active vision research. The idea is to generate an image with a varying resolution over a wide angle field of view. The fovea is of high resolution and the periphery is of exponentially reduced resolution. The justifications for such a sensor are: (i) it provides high resolution and a wide viewing angle; (ii) feature invariance in the fovea simplifies foveation; (iii) it allows multi-resolution analysis; and (iv) it is cheaper and more efficient to build a variable resolution sensor over a particular field of view rather than a uniform high resolution sensor. The receptor density of the human retina is very high, i.e. of the order of 10 8 receptors at the fovea. The question is, what resolution should space variant active vision systems have? Real visual sensors have been implemented but is the resolution produced high enough? This paper investigates the resolution requirements of a space variant sensor by simulation for a tracking system using raytracing.

A miniaturized space-variant active vision system: Cortex-I

We have developed a prototype miniaturized active vision system whose sensor architecture is based on a logarithmically structured space-variant pixel geometry. A space-variant image's resolution changes across the image. Typically, the central part of the image has a very high resolution, and the resolution falls off gradually in the periphery. Our system integrates a miniature CCD-based camera, pan-tilt actuator, controller, general purpose processors and display. Due to the ability of space-variant sensors to cover large work-spaces, yet provide high acuity with an extremely small number of pixels, space-variant active vision system architectures provide the potential for radical reductions in system size and cost. We have realized this by creating an entire system that takes up less than a third of a cubic foot. In this thesis, we describe a prototype space-variant active vision system (Cortex-I) which performs such tasks as tracking moving objects and license plate reading, and functions as a video telephone. We report on the design and construction of the camera (which is 8 x 8 x 8mm), its readout, and a fast mapping algorithm to convert the uniform image to a space-variant image. We introduce a new miniature pan-tilt actuator, the Spherical Pointing Motor (SPM), which is 4 x 5 x 6cm. The basic idea behind the SPM is to orient a permanent magnet to the magnetic field induced by three orthogonal coils by applying the appropriate ratio of currents to the coils. Finally, we present results of integrating the system with several applications. Potential application domains for systems of this type include vision systems for mobile robots and robot manipulators, traffic monitoring systems, security and surveillance, telerobotics, and consumer video communications. The long-range goal of this project is to demonstrate that major new applications of robotics will become feasible when small low-cost machine vision systems can be mass-produced. This notion of "commodity robotics" is expected to parallel the impact of the personal computer, in the sense of opening up new application niches for what has until now been expensive and therefore limited technology.

Space variant image processing

This paper describes a graph-based approach to image processing, intended for use with images obtained from sensors having space variant sampling grids. The connectivity graph (CG) is presented as a fundamental framework for posing image operations in any kind of space variant sensor. Partially motivated by the observation that human vision is strongly space variant, a number of research groups have been experimenting with space variant sensors. Such systems cover wide solid angles yet maintain high acuity in their central regions. Implementation of space variant systems pose at least two outstanding problems. First, such a system must be active, in order to utilize its high acuity region; second, there are significant image processing problems introduced by the non-uniform pixel size, shape and connectivity. Familiar image processing operations such as connected components, convolution, template matching, and even image translation, take on new and different forms when defined on space variant images. The present paper provides a general method for space variant image processing, based on a connectivity graph which represents the neighbor-relations in an arbitrarily structured sensor. We illustrate this approach with the following applications: (1) Connected components is reduced to its graph theoretic counterpart. We illustrate this on a logmap sensor, which possesses a difficult topology due to the branch cut associated with the complex logarithm function. (2) We show how to write local image operators in the connectivity graph that are independent of the sensor geometry. (3) We relate the connectivity graph to pyramids over irregular tessalations, and implement a local binarization operator in a 2-level pyramid. (4) Finally, we expand the connectivity graph into a structure we call a transformation graph, which represents the effects of geometric transformations in space variant image sensors. Using the transformation graph, we define an efficient algorithm for matching in the logmap images and solve the template matching problem for space variant images. Because of the very small number of pixels typical of logarithmic structured space variant arrays, the connectivity graph approach to image processing is suitable for real-time implementation, and provides a generic solution to a wide range of image processing applications with space variant sensors.

Shape description with a space-variant sensor: algorithms for scan-path, fusion, and convergence over multiple scans

The authors discuss three algorithms related to the blending of a single scene from multiple frames acquired from a space-variant sensor. Given a series of space-variant contour-based scenes with different fixation points, they show how to fuse these into a single multiscan view, which incorporates the information present in the individual scans. They demonstrate an (attentional) algorithm which recursively examines the current knowledge of the scene in order best to choose the next fixation point in terms of focusing attention on regions of maximum boundary curvature. They discuss a simple metric for evaluating convergence over a scan path. This may be used to compare the performance of various attentional algorithms. They discuss their work in light of both machine and biological vision.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>