Omnidirectional Camera System Research Articles

Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information

This paper reports on a mutual information (MI) based algorithm for automatic extrinsic calibration of a 3D laser scanner and optical camera system. By using MI as the registration criterion, our method is able to work in situ without the need for any specific calibration targets, which makes it practical for in-field calibration. The calibration parameters are estimated by maximizing the mutual information obtained between the sensor-measured surface intensities. We calculate the Cramer-Rao-Lower-Bound (CRLB) and show that the sample variance of the estimated parameters empirically approaches the CRLB for a sufficient number of views. Furthermore, we compare the calibration results to independent ground-truth and observe that the mean error also empirically approaches to zero as the number of views are increased. This indicates that the proposed algorithm, in the limiting case, calculates a minimum variance unbiased (MVUB) estimate of the calibration parameters. Experimental results are presented for data collected by a vehicle mounted with a 3D laser scanner and an omnidirectional camera system.

Scene flow for 3D laser scanner and camera system

Estimating the three-dimensional (3D) motion from sparse laser point clouds is a highly challenging endeavour facing computer and robotic vision engineers. In this study, a novel method is proposed for robustly estimating the scene flow from a laser scanner assisted by a camera. Conditional random field (CRF) is constructed by a spatial structure of point clouds, the energy of which is minimised by a synchronous calibrated image. With the high frame rate of a laser scanner, the authors' method allows for estimating the potential motion field as the CRF label. The authors ran an experiment on a public dataset to demonstrate that their method can accurately estimate rigid motion in outdoor scenes. They also tested the method on a laser scanner and omni-directional camera system to find that it also accurately estimates the rigid and semi-rigid motion of objects in a controlled indoor environment.

Depth perception based on omnidirectional ring structured light

A depth perception method based on omnidirectional image and ring structured light is proposed. Firstly, a new omnidirectional ring structured light pattern (ORSL) was presented. ORSL can simplify feature point extraction and locate the scene position more robust and fast. Secondly, a new sensor is proposed, which including an omnidirectional camera, laser projector and control circuit system. Thirdly, measurement accuracy was improved on error correction methods. Finally, a pose estimation method is designed to compute the location and direction of the imaging system. The experimental results demonstrated that the proposed method can acquire omnidirectional depth information about large-scale scene. The error analysis of multiple sets of depth data show the effectiveness of the method.

High‐Quality Panoramic Image Generation Using Multiple PAL Images

SUMMARYVideo surveillance using intelligent camera systems is widely used in many fields such as traffic control, driving assistance, and surgery. But image acquisition of a large field of view for video surveillance makes infeasible the use of multiple camera systems or mechanically controlled cameras. To solve this problem, an omnidirectional camera system using a panoramic annular lens may be used, with many advantages such as compactness and the lack of a need for calibration. Nevertheless, the low spatial resolution of the angle of elevation for the vertical direction of images captured using PAL may incur drawbacks such as quality degradation of images after panoramic expansion. To overcome these disadvantages, a high‐quality panoramic image generation method is required. Therefore, we propose a high‐quality panoramic image generation method using multiple panoramic annular lens images. By synthesizing multiple PAL images at different zoom rates, we can obtain fine quality panoramic images.

Estimating building inventory for rapid seismic vulnerability assessment: Towards an integrated approach based on multi-source imaging

We propose an integrated approach to estimating building inventory for seismic vulnerability assessment, which can be applied to different urban environments and be efficiently scaled depending on the desired level of detail. The approach employs a novel multi-source method for evaluating structural vulnerability-related building features based on satellite remote sensing and ground-based omnidirectional imaging. It aims to provide a comparatively cost- and time-efficient way of inventory data capturing over large areas. The latest image processing algorithms and computer vision techniques are used on multiple imaging sources within the framework of an integrated sampling scheme, where each imaging source and technique is used to infer specific, scale-dependent information. Globally available low-cost data sources are preferred and the tools are being developed on an open-source basis to allow for a high degree of transferability and usability. An easily deployable omnidirectional camera-system is introduced for ground-based data-capturing. After a general description of the approach and the developed tools and techniques, preliminary results from a first application to our study area, Bishkek, Kyrgyzstan, are presented.

Ford Campus vision and lidar data set

In this paper we describe a data set collected by an autonomous ground vehicle testbed, based upon a modified Ford F-250 pickup truck. The vehicle is outfitted with a professional (Applanix POS-LV) and consumer (Xsens MTi-G) inertial measurement unit, a Velodyne three-dimensional lidar scanner, two push-broom forward-looking Riegl lidars, and a Point Grey Ladybug3 omnidirectional camera system. Here we present the time-registered data from these sensors mounted on the vehicle, collected while driving the vehicle around the Ford Research Campus and downtown Dearborn, MI, during November–December 2009. The vehicle path trajectory in these data sets contains several large- and small-scale loop closures, which should be useful for testing various state-of-the-art computer vision and simultaneous localization and mapping algorithms.

A nonparametric learning approach to range sensing from omnidirectional vision

We present a novel approach to estimating depth from single omnidirectional camera images by learning the relationship between visual features and range measurements available during a training phase. Our model not only yields the most likely distance to obstacles in all directions, but also the predictive uncertainties for these estimates. This information can be utilized by a mobile robot to build an occupancy grid map of the environment or to avoid obstacles during exploration—tasks that typically require dedicated proximity sensors such as laser range finders or sonars. We show in this paper how an omnidirectional camera can be used as an alternative to such range sensors. As the learning engine, we apply Gaussian processes, a nonparametric approach to function regression, as well as a recently developed extension for dealing with input-dependent noise. In practical experiments carried out in different indoor environments with a mobile robot equipped with an omnidirectional camera system, we demonstrate that our system is able to estimate range with an accuracy comparable to that of dedicated sensors based on sonar or infrared light.

Extrinsic Calibration of a 3D Laser Scanner and an Omnidirectional Camera

We propose an approach for external calibration of a 3D laser scanner with an omnidirectional camera system. The utility of an accurate calibration is that it allows for precise co-registration between the camera imagery and the 3D point cloud. This association can be used to enhance various state of the art algorithms in computer vision and robotics. The extrinsic calibration technique used here is similar to the calibration of a 2D laser range finder and a single camera as proposed by Zhang (2004), but has been extended to the case where we have a 3D laser scanner and an omnidirectional camera system. The procedure requires a planar checkerboard pattern to be observed simultaneously from the laser scanner and the camera system from a minimum of 3 views. The normal of the planar surface and 3D points lying on the surface constrain the relative position and orientation of the laser scanner and the omnidirectional camera system. These constraints can be used to form a non-linear optimization problem that is solved for the extrinsic calibration parameters and the covariance associated with the estimated parameters. Results are presented for a real world data set collected by a vehicle mounted with a 3D laser scanner and an omnidirectional camera system.

Two-step calibration algorithm to estimate the extrinsic parameters of the omnidirectional camera

The omnidirectional camera system with its wide view angle is widely utilized for camera calibration and 3-D reconstruction due to its advantage that a large amount of environmental information can be obtained using a small number of images. Since the straight-line features of the environment are projected as contours by the omnidirectional camera model, the corresponding contours among views can be effectively used for camera calibration. This paper presents a two-step calibration algorithm to estimate the extrinsic parameters of the omnidirectional camera by using the corresponding contours. The first step involves obtaining camera parameters by minimizing an angular error function between the epipolar plane and back-projected vectors of end points on the corresponding contours. With this, the final parameters minimizing a distance error between the computed and the real contours in the third view are estimated. Our experiments on synthetic and real images demonstrate that the algorithm precisely estimates the extrinsic parameters of the camera.

Examination of a Wheelchair Active Safety System using a Stereo Omni-directional Camera System

Examination of a Wheelchair Active Safety System using a Stereo Omni-directional Camera System