Generic Camera Model Research Articles

Viewpoint Generation Using Geodesics and Associated Semi-Automated Coverage Path Planning of Panels for Inspection

The coverage of a surface using multiple viewpoints is a topic of great interest for robotic path planning in inspection applications. Two approaches for coverage path planning are broadly addressed in the literature—geometric methods and optimization methods. While the optimization methods may be the most flexible, they are frequently difficult to implement in practical applications due to their NP hard nature. We present here a geometric algorithm for the coverage path planning of panels used for aerospace applications using a generic camera model that can represent area inspection techniques like thermography and laser shearography. This algorithm relies on drawing a 2D grid on the 3D surface of the panel using geodesic lines on the surface. The coverage of the surface is performed by propagating geodesic lines from a starting point until the patch thus covered diverges too much from a flat surface, and after that, the coverage is continued from another point. The propagation of the geodesic lines is stopped when they begin to converge or diverge, and we define two criteria for the stoppage. We show that the proposed algorithm has good results for 3D virtual models and emphasize its speed, simplicity, and reliability for such applications.

Accurate and Flexible Calibration Method for a Class of Visual Sensor Networks

Accurate and flexible calibration is a prerequisite for visual sensor networks to retrieve metric information from image data. This paper presents the design and implementation of an accurate and flexible calibration method for a class of visual sensor networks intended for 3D measurement and tracking in large volumes. The proposed method employs a generic camera model that is applicable for wide-angle lens cameras as well as for conventional cameras. It does not require all the employed cameras to share a common field of view, and only pairwise overlap is needed. In the calibration process, the poses between stereo cameras are first initialized using essential matrix decomposition and then optimized by the Levenberg-Marquardt algorithm. A weighted vision graph is proposed to select optimal transformation paths among cameras by using Dijkstra's shortest path algorithms for multi-camera calibration. Then, the global coordinates are constructed using a four-marker calibration triangle. Finally, a Unity3D-based virtual platform, in which the total number and configurations of cameras, as well as the environment scene, can be arbitrarily edited, is designed to test the proposed calibration algorithms. Extensive experiments based on synthetic and real data are performed to demonstrate the effectiveness of the proposed multi-camera calibration method. Experimental results show that the multi-camera calibration method is accurate and easy-to-implement in the presence of noise.

A Robust Pose Estimation Method for Multicopters Using Off-Board Multiple Cameras

Pose estimation is an important issue for the visual servoing of multicopters. With a process model customized for multicopters and a generic camera model, an extended Kalman filter (EKF) method is proposed in this paper to estimate the pose of multicopters based on off-board multiple cameras. In consideration of EKF implementation, a general point correspondence method that can deal with any number of feature points and cameras is further designed. The proposed EKF method is applicable to not only a single fish-eye (or conventional) camera but also multiple fish-eye (or conventional) cameras. Observability analysis and experiment results show the proposed EKF method is more robust against noise and marker missing than existing filtering methods.

Absolute Pose Estimation of Central Cameras Using Planar Regions.

A novel method is proposed for the absolute pose estimation of a central 2D camera with respect to 3D depth data without the use of any dedicated calibration pattern or explicit point correspondences. The proposed method has no specific assumption about the data source: plain depth information is expected from the 3D sensing device and a central camera is used to capture the 2D images. Both the perspective and omnidirectional central cameras are handled within a single generic camera model. Pose estimation is formulated as a 2D-3D nonlinear shape registration task which is solved without point correspondences or complex similarity metrics. It relies on a set of corresponding planar regions, and the pose parameters are obtained by solving an overdetermined system of nonlinear equations. The efficiency and robustness of the proposed method were confirmed on both large scale synthetic data and on real data acquired from various types of sensors.

Modeling lens optics and rendering virtual views from fisheye imagery

Production of high-quality virtual reality content from real sensed data is a challenging task due to several factors such as calibration of multiple cameras and rendering of virtual views. In this paper, we present a pipeline that maximizes the performance of virtual view rendering from an imagery captured by a camera equipped with fisheye lens optics. While such optics offer a wide field-of-view, it also introduces specific distortions. These have to be taken into account while rendering virtual views for a target application (e.g., head-mounted displays). We integrate a generic camera model into a fast rendering pipeline where we can tune intrinsic and extrinsic camera parameters along with resolution to meet the device or user requirements. We specifically target CPU-based implementation and quality in par with GPU-based rendering approaches. Using the adopted generic camera model, we numerically tabulate the required backward projection mapping and store it in a look-up table. This approach offers a tradeoff between memory and computational complexity in terms of operations for calculating the mapping values. Finally, we complement our method with an interpolator, which handles occlusions efficiently. Experimental results demonstrate the viability, robustness and accuracy of the proposed pipeline.

Calibration and applications of visible imaging cameras on the C-2U advanced beam-driven field-reversed configuration device.

Two filtered, fast-imaging instruments, with radial and axial views, respectively, were used on the C-2U device to visualize line emission from impurities and hydrogenic neutrals. Novel calibration techniques needed to be developed for these instruments because the accelerated pace of C-2U operations precluded access to the interior of the vacuum vessel, targets used in typical calibration methods were not available, and in order to account for effects which have not been sufficiently addressed in the literature. Spatial calibration involved optimizing parameters in a generic camera model: ex situ using a checkerboard target and in situ using the vacuum vessel port geometry. Radiometric calibration was performed ex situ in three stages. First, the camera relative response function was mapped using an algorithm developed for high dynamic-range imaging. Second, the non-uniformity of the optical system was measured using a large LCD monitor with a characterized angular emission pattern. Finally, the absolute photon efficiency of each interference filter was determined using a calibrated uniform radiance source while also accounting for reduction in the filter transmission for off-normal rays. Periodically during the run campaign, line emission from neutral beams fired into a gas target was used as an in situ reference to check for degradation of viewport transmission. One application using calibrated camera data was tomographic reconstruction of passive impurity emission, which provided a sanity check to the excluded-flux radius inferred from wall-mounted magnetic sensors.

Robust Pose Estimation for Multirotor UAVs Using Off-Board Monocular Vision

This paper deals with the problem of pose estimation (or motion estimation) for multirotor unmanned aerial vehicles (UAVs) by using only an off-board camera. An extended Kalman filter (EKF) is often adopted to solve this problem. However, the accuracy and robustness of an EKF are limited partly by the usage of an existing linear constant-velocity process model applicable to many rigid objects. For such a reason, a nonlinear constant-velocity process model featured with the characteristics of multirotor UAVs is proposed in this paper, the superiority of which is explained from the perspective of observability. With the new process model and a generic camera model, a practical EKF method suitable for conventional cameras and fish-eye cameras is then proposed. By taking EKF implementation into account, a general correspondence method that could handle any number of feature points is further designed. Simulation and real experiments show that the proposed EKF method is more robust against noise and occlusion than currently employed filtering methods.

The concept of smooth generic camera calibration for optical metrology

Abstract Modern digital cameras with sophisticated optics enable accurate shape measurements via a plethora of metrological techniques. In order to take advantage of the data delivered by high-resolution cameras, one needs a sufficiently flexible model to describe the imaging geometry and an adequate calibration procedure to determine the model parameters and estimate their uncertainties. In this report, we suggest the concept of a smooth generic camera model that has several advantages compared to the state-of-the-art: the description is resolution-independent and takes advantage of the natural smoothness of the projection geometry; the global and the local symmetries inherent to the view ray-based parametrization are regularized in an explicitly unbiased manner. The calibration procedure uses dense datasets that can be obtained with the technique of active screens and consistently accounts for the uncertainty in the registration data that can be extracted during pre-processing. The concept is illustrated with synthetic examples based on physically-accurate rendering.

Calibration of multiple fish‐eye cameras using a wand

Fish‐eye cameras are becoming increasingly popular in computer vision, but their use for three‐dimensional measurement is limited partly because of the lack of an accurate, efficient and user‐friendly calibration procedure. For such a purpose, the authors propose a method to calibrate the intrinsic and extrinsic parameters (including radial distortion parameters) of two/multiple fish‐eye cameras simultaneously by using a wand under general motions. Thanks to the generic camera model used, the proposed calibration method is also suitable for two/multiple conventional cameras and mixed cameras (e.g. two conventional cameras and a fish‐eye camera). Simulation and real experiments demonstrate the effectiveness of the proposed method. Moreover, the authors develop the camera calibration toolbox, which is available online.

Calibration of Smooth Camera Models

Generic imaging models can be used to represent any camera. Current generic models are discrete and define a mapping between each pixel in the image and a straight line in 3D space. This paper presents a modification of the generic camera model that allows the simplification of the calibration procedure. The only requirement is that the coordinates of the 3D projecting lines are related by functions that vary smoothly across space. Such a model is obtained by modifying the general imaging model using radial basis functions (RBFs) to interpolate image coordinates and 3D lines, thereby allowing both an increase in resolution (due to their continuous nature) and a more compact representation. Using this variation of the general imaging model, we also develop a calibration procedure. This procedure only requires that a 3D point be matched to each pixel. In addition, not all the pixels need to be calibrated. As a result, the complexity of the procedure is significantly decreased. Normalization is applied to the coordinates of both image and 3D points, which increases the accuracy of the calibration. Results with both synthetic and real datasets show that the model and calibration procedure are easily applicable and provide accurate calibration results.

Generic camera model and its calibration for computational integral imaging and 3D reconstruction

Integral imaging (II) is an important 3D imaging technology. To reconstruct 3D information of the viewed objects, modeling and calibrating the optical pickup process of II are necessary. This work focuses on the modeling and calibration of an II system consisting of a lenslet array, an imaging lens, and a charge-coupled device camera. Most existing work on such systems assumes a pinhole array model (PAM). In this work, we explore a generic camera model that accommodates more generality. This model is an empirical model based on measurements, and we constructed a setup for its calibration. Experimental results show a significant difference between the generic camera model and the PAM. Images of planar patterns and 3D objects were computationally reconstructed with the generic camera model. Compared with the images reconstructed using the PAM, the images present higher fidelity and preserve more high spatial frequency components. To the best of our knowledge, this is the first attempt in applying a generic camera model to an II system.

Autonomous Navigation of Vehicles from a Visual Memory Using a Generic Camera Model

In this paper, we present a complete framework for autonomous vehicle navigation using a single camera and natural landmarks. When navigating in an unknown environment for the first time, usual behavior consists of memorizing some key views along the performed path to use these references as checkpoints for future navigation missions. The navigation framework for the wheeled vehicles presented in this paper is based on this assumption. During a human-guided learning step, the vehicle performs paths that are sampled and stored as a set of ordered key images, as acquired by an embedded camera. The visual paths are topologically organized, providing a visual memory of the environment. Given an image of the visual memory as a target, the vehicle navigation mission is defined as a concatenation of visual path subsets called visual routes. When autonomously running, the control guides the vehicle along the reference visual route without explicitly planning any trajectory. The control consists of a vision-based control law that is adapted to the nonholonomic constraint. Our navigation framework has been designed for a generic class of cameras (including conventional, catadioptric, and fisheye cameras). Experiments with an urban electric vehicle navigating in an outdoor environment have been carried out with a fisheye camera along a 750-m-long trajectory. Results validate our approach.

Generic and real-time structure from motion using local bundle adjustment

This paper describes a method for estimating the motion of a calibrated camera and the three-dimensional geometry of the filmed environment. The only data used is video input. Interest points are tracked and matched between frames at video rate. Robust estimates of the camera motion are computed in real-time, key frames are selected to enable 3D reconstruction of the features. We introduce a local bundle adjustment allowing 3D points and camera poses to be refined simultaneously through the sequence. This significantly reduces computational complexity when compared with global bundle adjustment. This method is applied initially to a perspective camera model, then extended to a generic camera model to describe most existing kinds of cameras. Experiments performed using real-world data provide evaluations of the speed and robustness of the method. Results are compared to the ground truth measured with a differential GPS. The generalized method is also evaluated experimentally, using three types of calibrated cameras: stereo rig, perspective and catadioptric.

A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses

Fish-eye lenses are convenient in such applications where a very wide angle of view is needed, but their use for measurement purposes has been limited by the lack of an accurate, generic, and easy-to-use calibration procedure. We hence propose a generic camera model, which is suitable for fish-eye lens cameras as well as for conventional and wide-angle lens cameras, and a calibration method for estimating the parameters of the model. The achieved level of calibration accuracy is comparable to the previously reported state-of-the-art.