Omnidirectional Image Sensor Research Articles

Landing Control on a Mobile Platform for Multi-copters using an Omnidirectional Image Sensor

This paper proposes a vision-based target following and landing system for a quadrotor vehicle on a moving platform. The system employs a vision-based landing site detection and locating algorithm using an omnidirectional lens. The latest smartphone was attached to the UAV and served as an on-board image acquisition and process unit. Measurements from the omnidirectional camera are combined with a proper dynamic model in order to estimate position and velocity of the moving platform. An adaptive control scheme was implemented on the flight computer to deal with unknown disturbances in outdoor environments. The system was validated on a quadrotor UAV and the vehicle successfully landed on the moving platform during outdoor flight tests.

어안 렌즈와 레이저 스캐너를 이용한 3차원 전방향 영상 SLAM

This paper proposes a novel three-dimensional mapping algorithm in Omni-Directional Vision SLAM based on a fisheye image and laser scanner data. The performance of SLAM has been improved by various estimation methods, sensors with multiple functions, or sensor fusion. Conventional 3D SLAM approaches which mainly employed RGB-D cameras to obtain depth information are not suitable for mobile robot applications because RGB-D camera system with multiple cameras have a greater size and slow processing time for the calculation of the depth information for omni-directional images. In this paper, we used a fisheye camera installed facing downwards and a two-dimensional laser scanner separate from the camera at a constant distance. We calculated fusion points from the plane coordinates of obstacles obtained by the information of the two-dimensional laser scanner and the outline of obstacles obtained by the omni-directional image sensor that can acquire surround view at the same time. The effectiveness of the proposed method is confirmed through comparison between maps obtained using the proposed algorithm and real maps.

Multi-robot Mapping Using Omnidirectional-Vision SLAM Based on Fisheye Images

This paper proposes a global mapping algorithm for multiple robots from an omnidirectional-vision simultaneous localization and mapping (SLAM) approach based on an object extraction method using Lucas– Kanade optical flow motion detection and images obtained through fisheye lenses mounted on robots. The multi-robot mapping algorithm draws a global map by using map data obtained from all of the individual robots. Global mapping takes a long time to process because it exchanges map data from individual robots while searching all areas. An omnidirectional image sensor has many advantages for object detection and mapping because it can measure all information around a robot simultaneously. The process calculations of the correction algorithm are improved over existing methods by correcting only the object’s feature points. The proposed algorithm has two steps: first, a local map is created based on an omnidirectional-vision SLAM approach for individual robots. Second, a global map is generated by merging individual maps from multiple robots. The reliability of the proposed mapping algorithm is verified through a comparison of maps based on the proposed algorithm and real maps.

어안 워핑 이미지 기반의 Ego motion을 이용한 위치 인식 알고리즘

This paper proposes a novel localization algorithm based on ego-motion which used Lucas-Kanade Optical Flow and warping image obtained through fish-eye lenses mounted on the robots. The omnidirectional image sensor is a desirable sensor for real-time view-based recognition of a robot because the all information around the robot can be obtained simultaneously. The preprocessing (distortion correction, image merge, etc.) of the omnidirectional image which obtained by camera using reflect in mirror or by connection of multiple camera images is essential because it is difficult to obtain information from the original image. The core of the proposed algorithm may be summarized as follows: First, we capture instantaneous <TEX>$360^{\circ}$</TEX> panoramic images around a robot through fish-eye lenses which are mounted in the bottom direction. Second, we extract motion vectors using Lucas-Kanade Optical Flow in preprocessed image. Third, we estimate the robot position and angle using ego-motion method which used direction of vector and vanishing point obtained by RANSAC. We confirmed the reliability of localization algorithm using ego-motion based on fisheye warping image through comparison between results (position and angle) of the experiment obtained using the proposed algorithm and results of the experiment measured from Global Vision Localization System.

Self-Tuning RBFNs Mobile Robot Systems through Bacterial Foraging Particle Swarm Optimization Learning Algorithm

A radial basis function neural networks (RBFNs) mobile robot control system is automatically developed with the image processing and learned by the bacterial foraging particle swarm optimization (BFPSO) algorithm in this paper. The image-based architecture of robot model is self-generated to travel the routing path in the dynamical and complicated environments. The visible omni-directional image sensors capture the surrounding environment to represent the behavior model of the mobile robot system. Three parameterize RBFNs model with the centers and spreads of each radial basis function, and the connection weights to solve the mobile robot path traveling and routing problems. Several free parameters of radial basis functions can be automatically tuned by the direct of the specified fitness function. In additional, the proper number of radial basis functions of the constructed RBFNs can be chosen by the defined fitness function which takes this factor into account. The desired multiple objectives of the RBFNs control system are proposed to simultaneously approach the shorter path and avoid the unexpected obstacles. Evaluations of PSO and BFPSO show that the developed RBFNs robot systems skip the obstacles and efficiently achieve the desired targets as soon as possible.

魚眼レンズを用いた自船周りの映像表示技術

Recently the research and the development on sailing support system using video information have been advancing owing to the development of image processing technology. As for vessels, the sailing monitoring systems to watch all directions of the vessels by using omni directional image sensors have been proposed. But there are some problems on this system as follows: (1) It is difficult to confirm the distance between the vessels and the bridge when they leave or reach the shore. (2) It is too slow to do the real time processing. This is why we developed the system which can judge a positional relationship between the vessels and the bridges visually by proposing on a real time a pseudo image from the sky. We are making use of USB camera equipped with fish-eye lenses to get images. In this paper, we experimented by attaching the camera at the medium seized vessel, which have a total length of 40 meters. As a result, we found that it is possible to display a positional relationship between our ship and bridges when our ship leaves or reaches the shore, and also to display corrected images on a real time at the frame rate 10fps.

Video Surveillance System for Elderly Person Living Alone by Person Tracking and Fall Detection

The detection of accidents on elderly person living alone and the communication between the elderly person and his/her family are very important. This paper describes a new method for tracking and fall detection of elderly person using omni-directional image sensor, and the Itawari-kan communication system that supports their communications and gives alarms for detected accidents on the elderly person. This system tracks the person's head position in real-time by image processing on images captured by some omni-directional image sensor. Then, the system transmits the information of the detected head position to another site. The computer of recipient site generates the computer graphics animation of the tracked person and displays the animation on a monitor. When the system detects an accident from the head position, the system gives an alarm. This method reduces traffic on network and keeps the privacy for the tracked person. We made a prototype system of the Itawari-kan communication system. Experiments on the system showed good feasibility of the proposed system.

Iconic memory-based omnidirectional route panorama navigation

A route navigation method for a mobile robot with an omnidirectional image sensor is described. The route is memorized from a series of consecutive omnidirectional images of the horizon when the robot moves to its goal. While the robot is navigating to the goal point, input is matched against the memorized spatio-temporal route pattern by using dual active contour models and the exact robot position and orientation is estimated from the converged shape of the active contour models.

Real-Time Omnidirectional Image Sensors

Conventional T.V. cameras are limited in their field of view. A real-time omnidirectional camera which can acquire an omnidirectional (360 degrees) field of view at video rate and which could be applied in a variety of fields, such as autonomous navigation, telepresence, virtual reality and remote monitoring, is presented. We have developed three different types of omnidirectional image sensors, and two different types of multiple-image sensing systems which consist of an omnidirectional image sensor and binocular vision. In this paper, we describe the outlines and fundamental optics of our developed sensors and show examples of applications for robot navigation.

Superresolution modeling using an omnidirectional image sensor

Recently, many virtual reality and robotics applications have been called on to create virtual environments from real scenes. A catadioptric omnidirectional image sensor composed of a convex mirror can simultaneously observe a 360-degree field of view making it useful for modeling man-made environments such as rooms, corridors, and buildings, because any landmarks around the sensor can be taken in and tracked in its large field of view. However, the angular resolution of the omnidirectional image is low because of the large field of view captured. Hence, the resolution of surface texture patterns on the three-dimensional (3-D) scene model generated is not sufficient for monitoring details. To overcome this, we propose a high resolution scene texture generation method that combines an omnidirectional image sequence using image mosaic and superresolution techniques.

Memory‐based self‐localization using omnidirectional images

AbstractFor a robot, recognizing its location in a real environment is important in terms of visual navigation, etc. This paper proposes a memory‐based self‐localization method based on the use of omnidirectional images. In the proposed method, information that is constant relative to rotation around the axis of an omnidirectional image sensor is first extracted by generating autocorrelation images from omnidirectional images containing global information that is useful for self‐localization. Next, eigenspaces are formed from the generated autocorrelation images, and self‐localization is performed by searching for the learning images that are closest to the input image in the eigenspace. Additionally, performing self‐localization in two stages (general and local) results in robust self‐localization. Lastly, experiments were conducted using time‐series omnidirectional images that were actually captured both indoors and outdoors to verify the effectiveness of the proposed method. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(5): 56–68, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.1203

Neural Networks for Autonomous Path-Following with an Omnidirectional Image Sensor

This paper presents a path-following system implemented with two different types of neural networks, that enables an autonomous mobile robot to return along a previously learned path in a dynamic environment. The path-following is based on data provided by an omnidirectional conical visual system, derived from the COPIS sensor, but with different optical reflective properties. The system uses optical and software processing and a neural network to learn the path, described as a sequence of selected points. In the navigation phase it drives the robot along this learned path. Interesting results have been achieved using low cost equipment. Test and results are presented.

Estimating head pose by using an omnidirectional image sensor

Estimating head pose by using an omnidirectional image sensor

2A1-C11 全方位視覚センサを用いた広視野角テレオペレーション

2A1-C11 全方位視覚センサを用いた広視野角テレオペレーション

Environmental Map Generation and Egomotion Estimation Using an Omnidirectional Image Sensor

Generation of a stationary environmental map is one of the important tasks for computer vision. Under the assumption of known motion of a camera, environmental maps of a real scene can be successfully generated by monitoring azimuth changes in an image. Several researchers have used this property for robot navigation. However, it is difficult to observe the exact motion parameters of the camera because of encoder measurement error of the robot. Therefore, observation errors in the generated environmental map accumulate in long movements of the robot. To generate a large environmental map, it is desirable not to assume known camera motion. In this paper, under the assumption of unknown motions of the camera, we propose a method to generate a stationary environmental map and estimate the egomotion by using an omnidirectional image sensor.

全方位視覚センサを用いたロバストな環境マップ生成と自己位置推定

Generation of a stationary environmental map is one of important tasks for robot navigation. Under the assumption of a known motion of a robot, environmental maps of a real scene are successfully generated by monitoring azimuth changes in an image. However, it is difficult to observe the exact motion parameters of the robot because of measurement error of the encoder of the robot. Therefore, observational errors in the generated environmental map accumulated in long movements of the robot. Furthermore, the exact stationary map generation is difficult because of ambiguity of correspondence caused by occlusion.In this paper, we propose a method to generate a stationary environmental map and estimate the egomotion of a robot, recursively, by using an omnidirectional image sensor. And the method can find mismatching and bring to right correspondence by evaluating estimation error of each objects location.

全方位視覚センサによる動環境での静止環境地図および自己位置の同時推定

Generation of an environmental map is one of important tasks for robot navigation. We have proposed a method to generate a static environmental map and estimate the egomotion of a robot by using an omnidirectional image sensor, based on the assumption of unknown translational motions of the robot. However, since both robot and objects move in the environment, the map generation and the robot egomotion estimation by using a single camera are difficult because of ambiguity of correspondence caused by occlusion. In this paper, we improve our previous method for treating applicable to a dynamic environment. The method can detect a moving object and find occlusion and mismatching by evaluating estimation error of each objects location.

未知並進運動する移動ロボットにおける静止環境マップ生成

We have designed several omnidirectional image sensors such as COPIS (COnic Projection Image Sensor) with a conical mirror and HyperOmni Vision with a hyperboloidal mirror, for navigating the robot in an environment. Described here is a method for generating an environmental map by using the Omnidirectional Image Sensor. Based on the assumption of translational motions of the robot, the proposed method can generate an environmental map and estimate egomotion of the robot. In particular, the method is suitable for map generation during long robot movement because deadlocking error of the robot does not accumulate. The method has been evaluated on a prototype of HyperOmni Vision in actual environment.

ＨｙｐｅｒＯｍｎｉ Ｖｉｓｉｏｎと全方位Ｈｏｕｇｈ変換を用いた線分の三次元再構築

We have proposed an omnidirectional image sensor called HyperOmni Vision. This sensor can acquire omnidirectional and perspective images in real time. In this paper, we propose a new Hough transform method for an omnidirectional image by using a cubic Hough space. Furthremore, This report presents a method for reconstructing 3D line segments from an omnidirectional input image sequence obtained by HyperOmni Vision with known motion.

HyperOmni vision: Visual navigation with an omnidirectional image sensor

For robot navigation, we need sensor information as to the condition of the environment around the robot. In this paper, we describe an omnidirectional vision system (HyperOmni Vision) using a hyperboloidal mirror and describe its distinctive features, together with a visual navigation method using HyperOmni Vision. In existing omnidirectional vision systems, since the optical system does not use center projection, an independent method of acquiring the visual information is required. However, since the omnidirectional vision system using a hyperboloidal mirror has an omnidirectional visual field and the optical characteristic of center projections, its images can be transformed into images taken directly with an ordinary camera. For this reason, it can produce images suitable for processing, and not only independent techniques, but also conventional image processing technology can be utilized. We will report here the configuration, distinctive features, and characteristics of the optical system of HyperOmni Vision and describe a mobile robot using HyperOmni Vision and its techniques of movement estimation and obstacle detection by template matching. © 1997 Scripta Technica, Inc. Syst Comp Jpn, 28(4): 36–47, 1997