Omnidirectional Vision Research Articles

A Minimalistic Approach to Appearance-Based Visual SLAM

This paper presents a vision-based approach to simultaneous localization and mapping (SLAM) in indoor/outdoor environments with minimalistic sensing and computational requirements. The approach is based on a graph representation of robot poses, using a relaxation algorithm to obtain a globally consistent map. Each link corresponds to a relative measurement of the spatial relation between the two nodes it connects. The links describe the likelihood distribution of the relative pose as a Gaussian distribution. To estimate the covariance matrix for links obtained from an omnidirectional vision sensor, a novel method is introduced based on the relative similarity of neighboring images. This new method does not require the determination of distances to image features using multiple-view geometry, for example. Combined indoor and outdoor experiments demonstrate that the approach can handle different environments (without modification of the parameters), and it can cope with violations of the ldquoflat floor assumptionrdquo to some degree and scales well with increasing size of the environment, producing topologically correct and geometrically accurate maps at low computational cost. Further experiments demonstrate that the approach is also suitable for combining multiple overlapping maps, e.g., for solving the multirobot SLAM problem with unknown initial poses.

Omnidirectional vision and conformal geometric algebra for visual landmark identification

SUMMARYThe automatic landmark identification is very important in autonomous robot navigation tasks. In this paper, we use a monocular omnidirectional vision system to extract the image features and the conformal geometric algebra to compute the projective invariants from such features. We show how these features can be used to compute projective and permutationp2-invariantsfrom any kind of omnidirectional vision system. Thep2-invariantsrepresent scene sublandmarks, and a set of them characterize a landmark. The advantage of this representation is that the landmarks are more robust than the single cross-ratio.

Geometry-Based Distributed Scene Representation With Omnidirectional Vision Sensors

This paper addresses the problem of efficient representation of scenes captured by distributed omnidirectional vision sensors. We propose a novel geometric model to describe the correlation between different views of a 3-D scene. We first approximate the camera images by sparse expansions over a dictionary of geometric atoms. Since the most important visual features are likely to be equivalently dominant in images from multiple cameras, we model the correlation between corresponding features in different views by local geometric transforms. For the particular case of omnidirectional images, we define the multiview transforms between corresponding features based on shape and epipolar geometry constraints. We apply this geometric framework in the design of a distributed coding scheme with side information, which builds an efficient representation of the scene without communication between cameras. The Wyner-Ziv encoder partitions the dictionary into cosets of dissimilar atoms with respect to shape and position in the image. The joint decoder then determines pairwise correspondences between atoms in the reference image and atoms in the cosets of the Wyner-Ziv image in order to identify the most likely atoms to decode under epipolar geometry constraints. Experiments demonstrate that the proposed method leads to reliable estimation of the geometric transforms between views. In particular, the distributed coding scheme offers similar rate-distortion performance as joint encoding at low bit rate and outperforms methods based on independent decoding of the different images.

Homography-based method for calibrating an omnidirectional vision system

We present a homography-based method for calibrating an omnidirectional vision system with a parabolic mirror. Assuming that the intrinsic parameters of the camera are known a priori, we focus on finding the solution for the mirror parameter and its positions. We first estimate the homographic matrix partially using six or more point correspondences. Then the rotation matrix and two components of the translation vector can be estimated. Finally, the remaining parameters are computed. In this method, a closed-form solution for all the variables is obtained using the homographic matrix. Another advantage is the enhanced robustness in implementation via the use of two over-constrained linear systems. Numerical simulations and real data experiments are also performed to validate the proposed algorithm.

THE SPATIAL SEMANTIC HIERARCHY IMPLEMENTED WITH AN OMNIDIRECTIONAL VISION SYSTEM

In this article, we propose a new approach to the map building task: the implementation of the Spatial Semantic Hierarchy (SSH), proposed by B. Kuipers, on a real robot fitted with an omnidirectional camera. The original Kuiper's formulation of the SSH was slightly modified, in order to manage in a more efficient way the knowledge the real robot collects while moving in the environment. The sensory data experienced by the robot are transformed by the different levels of the SSH in order to obtain a compact representation of the environment. This knowledge is stored in the form of a topological map and, eventually, of a metrical map. The aim of this article is to show that a catadioptric omnidirectional camera is a good sensor for the SSH and nicely couples with several elements of the SSH. The panoramic view and rotational invariance of our omnidirectional camera makes the identification and labelling of places a simple matter. A deeper insight is that the tracking and identification of events on an omnidirectional image such as occlusions and alignments can be used for the segmentation of continuous sensory image data into the discrete topological and metric elements of a map. The proposed combination of the SSH and omnidirectional vision provides a powerful general framework for robot maping and offers new insights into the concept of “place.” Some preliminary experiments performed with a real robot in an unmodified office environment are presented.

능동 전방향 거리 측정 시스템을 이용한 이동로봇의 위치 추정

An active omni-directional raging system using an omni-directional vision with structured light has many advantages compared to the conventional ranging systems: robustness against external illumination noise because of the laser structured light and computational efficiency because of one shot image containing <TEX>$360^{\circ}$</TEX> environment information from the omni-directional vision. The omni-directional range data represents a local distance map at a certain position in the workspace. In this paper, we propose a matching algorithm for the local distance map with the given global map database, thereby to localize a mobile robot in the global workspace. Since the global map database consists of line segments representing edges of environment object in general, the matching algorithm is based on relative position and orientation of line segments in the local map and the global map. The effectiveness of the proposed omni-directional ranging system and the matching are verified through experiments.

Fuzzy edge detection for omnidirectional images

The use of omnidirectional vision has increased during these past years. It provides a very large field of view. Nevertheless, omnidirectional images contain significant radial distortions and conventional image processing is not adapted to these specific images. This paper presents an edge detector adapted to the image geometry. Fuzzy sets will be used to take into account all imprecisions introduced by the sampling process. The Prewitt filter applied to omnidirectional image will be studied to illustrate this paper.

Development of Mirror for Small Size ODV by Ultraprecision Cutting

ODV (OmniDirectional Vision) is an optical system which has the view angle more than 180° with little distortion. If this ODV can be made smaller than now, various applications for medical and/or industrial fields will be expected. The optical system of ODV for side view is composed of a primary mirror and a secondary mirror. Therefore, the machining method for making these small mirrors was investigated.At first, the occurrence of interference fringes and the decrease in resolving power due to tool marks were examined. The experimental analysis made it clear that it was necessary to reduce the tool marks per unit length in order to decrease the interference fringes, while the tool marks hardly affected the resolving power . The reflect side of the mirror has the thin part because of the mounting space for CCD camera. Therefore, the cutting test of thin part in simple model was carried out. It was found that the distortion didn't occur in the reflected image as long as the flatness was less than about 150nm. Then, the ring-shaped jig was used for holding the mirrors, since these structures were very small and flexible. It was confirmed that the accuracy of mirrors cut with using this jig was within the permissible range. As a result, the making method of high precision mirrors for small size ODV was established in this study.

The Basic Analysis for Object Zone Detection with Volume Intersection Method Using Omni-Directional Vision Sensors for Man-Machine Co-Existence Systems

The Basic Analysis for Object Zone Detection with Volume Intersection Method Using Omni-Directional Vision Sensors for Man-Machine Co-Existence Systems

全方位センサのベイジアンネットワーク表現に基づく高齢者非日常行動検出モニタリングシステム

This paper describes an unusual behavior detection system based on an omni-directional vision sensor as one of the important elements in realizing “Sensing and Robotic Support Room” for elderly people. Such support rooms are expected to be further developed in the future with the high performance to automatically recognize elderly people's actions and behavior patterns and detect the unusual patterns using some sensors and to support their daily motions using some robotic manipulator control systems. The proposed monitoring system using an omni-directional vision sensor automatically learns the daily behavior patterns and detects the unusual behavior patterns and actions using Bayesian Network approach. The Bayesian Network is constructed using image feature values such as the area and center of gravity values extracted from the captured image sequence and the respective behavior patterns are represented as the conditional probabilities. Unusual behavior patterns can be automatically detected based on the low generation probability values. Some experiments based on the investigation of elderly people's typical daily behavior patterns show the effectiveness of the proposed system.

Multispectral omnidirectional vision sensor: design, calibration, and utilization

The 3-D reconstruction of an environment with high accuracy and real textures represents an important challenge for applications related to multimedia and cinematography. The complete development, from electronic design to algorithmic restitution of the environment, of a multispectral omnidirectional stereoscopic sensor that tackles this problem is presented. Although panoramic sensors exist, the vision system presented here has four channels of acquisition coupled to a projector of structured light emitting in the infrared. This original process separates the acquisition of color images from 3-D reconstruction. The panoramic color images are corrected using the International Color Consortium profile applied to images coming from the two cameras. The reconstruction of the environment uses only the information available in the infrared channel. An active calibration technique is also presented. Experimental results demonstrate the feasibility of this new class of multispectral omnidirectional sensors.

Knowledge propagation in a distributed omnidirectional vision system

In this paper an omnidirectional Distributed Vision System (DVS) is presented. The presented DVS is able to learn to navigate a mobile robot in its working environment without any prior knowledge a...

Development of “Hibikino-Musashi” omni-directional mobile robot

“Hibikino-Musashi,” a middle-size league RoboCup soccer team, is a joint team of three organizations: Kyushu Institute of Technology, the University of Kitakyushu, and the Kitakyushu Foundation for the Advancement of Industry, Science and Technology (FAIS). We recently developed a new omni-directional mobile robot platform with an omni-directional vision system. In this paper, our robot platform is presented.

Catadioptric single-shot rangefinder for textured map building in robot navigation

The paper focuses on combining the advantages of both omnidirectional vision and structured light in order to obtain panoramic depth information for generating a complete 3D visual model of the surroundings of a mobile robot. The model is completed by mapping the imaged texture onto the objects through the central view point. This process is called single-shot scene reconstruction because a visual model of the scene is obtained from a single omnidirectional image. Several such reconstructions are then registered to build a 3D map of the scene while the robot is moving. The authors call this process range scan registration modelling. A mobile platform provided with a catadioptric camera coupled with an omnidirectional structured light projector was used for this purpose. Sensor calibration, image processing and experimental results are included.

Instantaneous robot self-localization and motion estimation with omnidirectional vision

This paper presents two related methods for autonomous visual guidance of robots: localization by trilateration, and interframe motion estimation. Both methods use coaxial omnidirectional stereopsis (omnistereo), which returns the range r to objects or guiding points detected in the images. The trilateration method achieves self-localization using r from the three nearest objects at known positions. The interframe motion estimation is more general, being able to use any features in an unknown environment. The guiding points are detected automatically on the basis of their perceptual significance and thus they need not have either special markings or be placed at known locations. The interframe motion estimation does not require previous motion history, making it well suited for detecting acceleration (in 20th of a second) and thus supporting dynamic models of robot’s motion which will gain in importance when autonomous robots achieve useful speeds. An initial estimate of the robot’s rotation ω (the visual compass) is obtained from the angular optic flow in an omnidirectional image. A new noniterative optic flow method has been developed for this purpose. Adding ω to all observed (robot relative) bearings θ gives true bearings towards objects (relative to a fixed coordinate frame). The rotation ω and the r , θ coordinates obtained at two frames for a single fixed point at unknown location are sufficient to estimate the translation of the robot. However, a large number of guiding points are typically detected and matched in most real images. Each such point provides a solution for the robot’s translation. The solutions are combined by a robust clustering algorithm Clumat that reduces rotation and translation errors. Simulator experiments are included for all the presented methods. Real images obtained from ScitosG5 autonomously moving robot were used to test the interframe rotation and to show that the presented vision methods are applicable to real images in real robotics scenarios.

Self-localization in non-stationary environments using omni-directional vision

This paper presents an image-based approach for localization in non-static environments using local feature descriptors, and its experimental evaluation in a large, dynamic, populated environment where the time interval between the collected data sets is up to two months. By using local features together with panoramic images, robustness and invariance to large changes in the environment can be handled. Results from global place recognition with no evidence accumulation and a Monte Carlo localization method are shown. To test the approach even further, experiments were conducted with up to 90% virtual occlusion in addition to the dynamic changes in the environment.

Multilayered Fuzzy Behavior Control for an Autonomous Mobile Robot with Multiple Omnidirectional Vision System: MOVIS

We propose a multiple omnidirectional vision system (MOVIS) with three omnidirectional cameras and calculation for measuring an object position and localization in an autonomous mobile robot. In identifying the robot’s location, we improved measurement accuracy by correcting the absolute location based on landmark measurement error in the origin of absolute coordinates. We propose omnidirectional behavior control for collision avoidance and object chasing using fuzzy reasoning in an autonomous mobile robot with MOVIS. We also report experimental results confirming the efficiency of our proposal using a RoboCup soccer robot in a dynamic environment.

Omnidirectional Vision Based Topological Navigation

In this work we present a novel system for autonomous mobile robot navigation. With only an omnidirectional camera as sensor, this system is able to build automatically and robustly accurate topologically organised environment maps of a complex, natural environment. It can localise itself using such a map at each moment, including both at startup (kidnapped robot) or using knowledge of former localisations. The topological nature of the map is similar to the intuitive maps humans use, is memory-efficient and enables fast and simple path planning towards a specified goal. We developed a real-time visual servoing technique to steer the system along the computed path. A key technology making this all possible is the novel fast wide baseline feature matching, which yields an efficient description of the scene, with a focus on man-made environments.

Robot navigation behaviors based on omnidirectional vision and information theory

In this work we present a reactive autonomous robot navigation system based only on omnidirectional vision. It does not rely on any prior knowledge about the environment apart from assuming a structured one, like indoor corridors or outdoor avenues. The direction of the corridor is estimated from the entropy analysis of a 1-D omnidirectional image. The 2-D omnidirectional image is analyzed for obstacle avoidance and for keeping a safety distance from the borders of the corridor. Both methods are non-metric and no 3-D information is needed. The system performs well with different resolutions and the catadioptric sensor needs no calibration. We present results from indoor and outdoor experiments.

衛星搭載全方位モニタカメラの開発

We have developed an OmniDirectional Vision system (ODV) using two reflecting mirrors to monitor a spacecraft on orbit. Recently, a spacecraft is loaded with many deployable structures. The ODV provides a panoramic image of 360 degree surroundings in real time and is possible to monitor two or more deployable structures at the same time. Moreover, the CCD and the lens are not exposed to cosmic rays, because these are covered with metallic mirrors. Therefore, a loss of the CCD element and a blur of the lens due to the cosmic rays are not occurred easily. We have designed the ODV for the Unmanned Space Experiment Recovery System (USERS) spacecraft which was launched on September 10, 2002. The optical characteristics of the ODV has been evaluated and the results show that the ODV can acquire a sharp image. Two ODVs were adopted as monitor cameras of the USERS. From the input images of the ODVs on orbit, the deployment of the paddles and the Capillary Pumped Deployable Radiator (CPDR) and the separation of the Re-entry Module (REM) were confirmed.