Catadioptric Omnidirectional Vision Research Articles

Gas Pipeline Inspection Using Autonomous Robots With Omni-Directional Cameras

Maintenance of pipelines is a major concern for oil and gas companies and millions of dollars are spent on preventive inspections every year. The type of equipment used in carrying out inspections is expensive and very sophisticated requiring very specialized manpower. Here, a novel approach to pipeline inspection is presented using a small mobile robot and a catadioptric omni-directional vision system. The system captures panoramic videos using conical and/or hyperbolic mirrors, which are transmitted to a monitoring station to examine the inner surface of gas pipelines. The videos are prepped, unwrapped, and un-warped to obtain a realistic flat image of the whole inner surface of the pipeline. The Simulated Annealing optimization method is implemented to optimize the system parameters using different mirror shapes and unwrapping and un-warping techniques. Simulations and experimental studies using a mobile robot equipped with a CCD camera and a conical mirror are presented here to demonstrate the viability of this inspection technique. The developed robotic system can be used to visually pre-scan pipelines before using much more expensive tools such as magnetic flux leakage (MFL) or ultrasound (UT) based inspection systems. These latter tools cost tens of thousands of dollars per hour and will be used only after suspicious areas are visually detected. This will reduce the inspection time and cost significantly.

A Riemannian approach for free-space extraction and path planning using catadioptric omnidirectional vision

This paper presents a Riemannian approach for free-space extraction and path planning using color catadioptric vision. The problem is formulated considering color catadioptric images as Riemannian manifolds and solved using the Riemannian Eikonal equation with an anisotropic fast marching numerical scheme. This formulation allows the integration of adapted color and spatial metrics in an incremental process. First, the traversable ground (namely free-space) is delimited using a color structure tensor built on the multi-dimensional components of the catadioptric image. Then, the Eikonal equation is solved in the image plane incorporating a generic metric tensor for central catadioptric systems. This built Riemannian metric copes with the geometric distortions in the catadioptric image plane introduced by the curved mirror in order to compute the geodesic distance map and the shortest path between image points. We present comparative results using Euclidean and Riemannian distance transforms and show the effectiveness of the Riemannian approach to produce safest path planning.

Sea-skyline-based image stabilization of a buoy-mounted catadioptric omnidirectional vision system

Marine monitoring systems have the requirements of a large field of view, low power consumption, real-time viewing, and economical and automatic functionality. This paper establishes an omnidirectional vision system used in marine buoys that meets these requirements. We present a framework for image stabilization, which is achieved by omnidirectional sea-skyline detection in a marine environment. We propose an optimal edge estimation method to calculate the sea-skyline ellipsis according to the sea-skyline characteristics in panoramic images. We construct a compact panoramic image stabilization model based on the sea-skyline and propose a reconstruction method for the invalid regions using the key frame. The experimental results and analysis show that the proposed approach is capable of acquiring stable video in real-time marine monitoring tasks and that the target detection is sufficiently effective, efficient, and accurate for a real-time ship target detection application.

Calibration of a catadioptric omnidirectional vision system with conic mirror

Omnidirectional vision systems that enable 360° imaging have been widely used in several research areas, including close-range photogrammetry, which allows the accurate 3D measurement of objects. To achieve accurate results in Photogrammetric applications, it is necessary to model and calibrate these systems. The major contribution of this paper relates to the rigorous geometric modeling and calibration of a catadioptric, omnidirectional vision system that is composed of a wide-angle lens camera and a conic mirror. The indirect orientation of the omnidirectional images can also be estimated using this rigorous mathematical model. When calibrating the system, which is composed of a wide-angle camera and a conic mirror, misalignment of the conical mirror axis with respect to the camera’s optical axis is a critical problem that must be considered in mathematical models. The interior calibration technique developed in this paper encompasses the following steps: wide-angle camera calibration; conic mirror modeling; and estimation of the transformation parameters between the camera and conic mirror reference systems. The main advantage of the developed technique is that it does not require accurate physical alignment between the camera and conic mirror axis. The exterior orientation is based on the properties of the conic mirror reflection. Experiments were conducted with images collected from a calibration field, and the results verified that the catadioptric omnidirectional system allows for the generation of ground coordinates with high geometric quality, provided that rigorous photogrammetric processes are applied.

Parametric distortion-adaptive neighborhood for omnidirectional camera.

Catadioptric omnidirectional images exhibit serious nonlinear distortion due to the involved quadratic mirror. Conventional pinhole model-based methods perform poorly when directly applied to the deformed omnidirectional images. This study constructs a catadioptric geometry system to analyze the variation of the neighborhood of an object in terms of the elevation and azimuth directions in a spherical coordinate system. To accurately represent the distorted visual information, a parametric neighborhood mapping model is proposed based on the catadioptric geometry. Unlike the conventional catadioptric models, the prior information of the system is effectively integrated into the neighborhood formulation framework. Then the distortion-adaptive neighborhood can be directly calculated based on its measurable image radial distance. This method can significantly improve the computational efficiency of algorithm since statistical neighborhood sampling is not used. On the basis of the proposed neighborhood model, a distortion-invariant Haar wavelet transform is presented to perform the robust human detection and tracking in catadioptric omnidirectional vision. The experimental results verify the effectiveness of the proposed neighborhood mapping model and prove that the distorted neighborhood in the omnidirectional image follows a nonlinear pattern.

A mobile platform with a catadioptric sensor

Abstract. The aim of this paper is to present a mobile catadioptric omnidirectional vision system composed of a camera and a cone-shaped mirror integrated with a direct georeferencing system. The relationship between image and object space is established with generic/empiric or physical models. The models were implemented and tested with real data and some results are presented. The results showed that accuracies around 5 cm in planimetry can be achieved, which is suitable for several applications, including the generation of control scenes that was the original motivation of this work.

Contour coding based rotating adaptive model for human detection and tracking in thermal catadioptric omnidirectional vision

In this paper, we introduce a novel surveillance system based on thermal catadioptric omnidirectional (TCO) vision. The conventional contour-based methods are difficult to be applied to the TCO sensor for detection or tracking purposes due to the distortion of TCO vision. To solve this problem, we propose a contour coding based rotating adaptive model (RAM) that can extract the contour feature from the TCO vision directly as it takes advantage of the relative angle based on the characteristics of TCO vision to change the sequence of sampling automatically. A series of experiments and quantitative analyses verify that the performance of the proposed RAM-based contour coding feature for human detection and tracking are satisfactory in TCO vision.

Pipeline Inspection using Catadioptric Omni-directional Vision Systems

Oil and gas companies spend millions of dollars on inspection of pipelines every year. The type of equipment used in carrying out the inspections is very sophisticated and requires very specialized manpower. In this article we present a novel approach to pipeline inspection using a small mobile robot and an omnidirectional vision system. It is a combination of a reflective convex mirror and a perspective camera. Panoramic videos captured by the camera from the mirror are either stored or transmitted to a monitoring station to examine the inner surface of the pipeline. The videos are prepped, unwrapped, and unwarped to get a realistic image of the whole inner surface area of the pipeline. Several mirror shapes and unwarping techchniques are used here to show the efficiency of this inspection technique. The image acquisition is instantaneous with a large field of view, which makes their use in dynamic environment suitable. The cost of such system is relatively low compared to other available inspections systems.

On improved calibration method for the catadioptric omnidirectional vision with a single viewpoint

The single viewpoint constraint is a principal optical characteristic for most catadioptric omnidirectional vision. Single viewpoint catadioptric omnidirectional vision is very useful because it allows the generation of geometrically correct perspective images from one omnidirectional image. Therefore precise calibration for single viewpoint constraint is needed during system assembling. However, in most image detection based calibration methods, the nonlinear optical distortion brought by lens is often neglected. Hence the calibration precision is poor. In this paper, a new calibration method of single viewpoint constraint for the catadioptric omni-directional vision is proposed. Firstly, an image correction algorithm is obtained by training a neural network. Then, according to characteristics of the space circular perspective projection, the corrected image of the mirror boundary is used to estimate its position and attitude relative to the camera to guide the calibration. Since the estimate is conducted based on actual imaging model rather than the simplified model, the estimate error is largely reduced, and the calibration accuracy is significantly improved. Experiments are conducted on simulated images and real images to show the accuracy and the effectiveness of the proposed method.

Design of Vertically Aligned Binocular Omnistereo Vision Sensor

Catadioptric omnidirectional vision sensor (ODVS) with a fixed single view point is a fast and reliable single panoramic visual information acquisition equipment. This paper presents a new type of binocular stereo ODVS which composes of two ODVS with the same parameters. The single view point of each ODVS is fixed on the same axis with face-to-face, back-to-back, and facetoback configuration; the single view point design is implemented by catadioptric technology such as the hyperboloid, constant angular resolution, and constant vertical resolution. The catadioptric mirror design uses the method of increasing the resolution of the view field and the scope of the image in the vertical direction. The binocular stereo ODVS arranged in vertical is designed spherical, cylindrical surfaces and rectangular plane coordinate system for 3D calculations. Using the collinearity of two view points, the binocular stereo ODVS is able to easily align the azimuth, while the camera calibration, feature points match, and other cumbersome steps have been simplified. The experiment results show that the proposed design of binocular stereo ODVS can solve the epipolar constraint problems effectively, match three-dimensional image feature points rapidly, and reduce the complexity of three-dimensional measurement considerably.

Household robots look and learn

The work illustrates that a catadioptric omnidirectional vision system can be successfully applied for basic mobile robot navigation tasks, such as localization and environment learning. In combination with other capabilities of such a sensor, such as the recognition and tracking of humans, and because the price of such systems can be made low, this system is particularly suited for systems that are expected to operate in offices and homes in the near future, such as robotic servant systems, entertainment robots, and help for the elderly and disabled.