Calibration Of Multiple Cameras Research Articles

Development of a method to use standard hospital gamma cameras as triage whole body monitors in UK emergencies.

This paper outlines the process by which a medical gamma camera can be utilised to support assessment of internal radionuclides for the public. While hospital based gamma cameras are able to detect photopeaks, they are often limited to an energy range of 40-540 keV. However, radionuclides with photopeak energies above 540 keV can still be detected as the partial collection of photon energy increases the count rate at lower energies. By combining extensive mathematical modelling with empirical calibration of multiple gamma cameras it is possible to develop a linear correlation between the efficiency of counting point sources and the overall counting efficiency for the camera. Once established, a simple protocol can be used to characterise any gamma camera, using optimal system settings, and hence generate a system efficiency with sufficient accuracy to allow the camera to be used in a triage process to committed effective doses of 2 mSv.&#xD.

Simultaneous Calibration of Multiple Cameras and Generation of Omnidirectional Images

Abstract. Omnidirectional images are increasingly being used in various areas, such as urban mapping, virtual reality, agriculture, and robotics. These images can be generated by different acquisition systems, including multi-camera systems, which can acquire higher-resolution images. Stitching techniques are often used and can be suitable for non-metric applications, but rigorous photogrammetric processing is recommended when having more accurate requirements. The main challenges related to this kind of product are the system calibration and the generation of the final omnidirectional images. When using multi-camera systems, the displacement of the cameras' perspective centres can affect the generation of the omnidirectional images and the resulting accuracy. A common approach to minimising the resulting parallax error is to establish a value for the projection cylinder radius as close as possible to the object's depth. This work proposes a highly accurate simultaneous calibration technique for multiple camera systems using self-calibrating bundle adjustment with constraints of stability of the relative orientation parameters. These parameters are later used to generate a projecting cylindrical surface, maintaining the original camera perspective centres and relative orientation angles. The experiments show that using constraints improved both the calibration results and the final omnidirectional images. Residual mismatches between points in overlapping areas are subpixel.

High-precision binocular camera calibration method based on a 3D calibration object.

A high-precision binocular camera calibration method is proposed to address the issues of poor calibration accuracy and large calibration errors in current practical applications. This method uses a triangular stereo sphere as the calibration object and employs steps, such as ellipse fitting, Cholesky decomposition, homography matrix solution, and nonlinear optimization, to compute the intrinsic and extrinsic parameters, distortion parameters, and relative pose of the binocular camera. Moreover, this method simplifies the correspondences between primitives, enabling simultaneous calibration of multiple viewpoint cameras. This method is also suitable for both binocular cameras consisting of two different structured monocular cameras and those composed of two image sensors within the same structure. Experimental results showed that this method outperforms traditional algorithms in terms of binocular camera calibration accuracy, calibration errors between left and right cameras, and robustness, resulting in a significant improvement in overall algorithm performance.

Wand-Based Calibration of Unsynchronized Multiple Cameras for 3D Localization.

Three-dimensional (3D) localization plays an important role in visual sensor networks. However, the frame rate and flexibility of the existing vision-based localization systems are limited by using synchronized multiple cameras. For such a purpose, this paper focuses on developing an indoor 3D localization system based on unsynchronized multiple cameras. First of all, we propose a calibration method for unsynchronized perspective/fish-eye cameras based on timestamp matching and pixel fitting by using a wand under general motions. With the multi-camera calibration result, we then designed a localization method for the unsynchronized multi-camera system based on the extended Kalman filter (EKF). Finally, extensive experiments were conducted to demonstrate the effectiveness of the established 3D localization system. The obtained results provided valuable insights into the camera calibration and 3D localization of unsynchronized multiple cameras in visual sensor networks.

Multiple Projector Camera Calibration by Fiducial Marker Detection

Multiple Projector Camera Calibration by Fiducial Marker Detection

Spatio-Temporal Calibration of Multiple Kinect Cameras Using 3D Human Pose.

RGB and depth cameras are extensively used for the 3D tracking of human pose and motion. Typically, these cameras calculate a set of 3D points representing the human body as a skeletal structure. The tracking capabilities of a single camera are often affected by noise and inaccuracies due to occluded body parts. Multiple-camera setups offer a solution to maximize coverage of the captured human body and to minimize occlusions. According to best practices, fusing information across multiple cameras typically requires spatio-temporal calibration. First, the cameras must synchronize their internal clocks. This is typically performed by physically connecting the cameras to each other using an external device or cable. Second, the pose of each camera relative to the other cameras must be calculated (Extrinsic Calibration). The state-of-the-art methods use specialized calibration session and devices such as a checkerboard to perform calibration. In this paper, we introduce an approach to the spatio-temporal calibration of multiple cameras which is designed to run on-the-fly without specialized devices or equipment requiring only the motion of the human body in the scene. As an example, the system is implemented and evaluated using Microsoft Azure Kinect. The study shows that the accuracy and robustness of this approach is on par with the state-of-the-art practices.

Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras Using Adaptive Voxelization

Determining the extrinsic parameter between multiple LiDARs and cameras is essential for autonomous robots, especially for solid-state LiDARs, where each LiDAR unit has a very small Field-of-View (FoV), and multiple units are often used collectively. The majority of extrinsic calibration methods are proposed for 360° mechanical spinning LiDARs where the FoV overlap with other LiDAR or camera sensors is assumed. Few research works have been focused on the calibration of small FoV LiDARs and cameras nor on the improvement of the calibration speed. In this work, we consider the problem of extrinsic calibration among small FoV LiDARs and cameras, with the aim to shorten the total calibration time and further improve the calibration precision. We first implement an adaptive voxelization technique in the extraction and matching of LiDAR feature points. Such a process could avoid the redundant creation of k-d trees in LiDAR extrinsic calibration and extract LiDAR feature points in a more reliable and fast manner than existing methods. We then formulate the multiple LiDAR extrinsic calibration into a LiDAR Bundle Adjustment (BA) problem. By deriving the cost function up to second-order, the solving time and precision of the non-linear least square problem are further boosted. Our proposed method has been verified on data collected in four targetless scenes and under two types of solid-state LiDARs with a completely different scanning pattern, density, and FoV. The robustness of our work has also been validated under eight initial setups, with each setup containing 100 independent trials. Compared with the state-of-the-art methods, our work has increased the calibration speed 15 times for LiDAR-LiDAR extrinsic calibration (averaged result from 100 independent trials) and 1.5 times for LiDAR-Camera extrinsic calibration (averaged result from 50 independent trials) while remaining accurate. To benefit the robotics community, we have also open-sourced our implementation code on GitHub.

Automatic calibration of multiple cameras and lidars for autonomous vehicles

Autonomous navigation of unmanned vehicles (UVs) is currently one of the most interesting scientific and technical problems, and this is even more true for UVs moving across rough terrain. As a rule, such vehicles are equipped with different sensors operating simultaneously, and sophisticated software is being developed to collect and analyze miscellaneous data. For the joint use of data from multiple cameras and lidars, all of them need to be mutually referenced in a single coordinate system. This problem is solved in a process of system calibration. The general idea is to place an object of special type in the field of view of the sensors, so that its characteristic points can be automatically detected from different points of view by all sensors. Then the repeated survey of the object allows you to create the required number of tie points for mutual alignment of the sensors. This work presents a technique for automatic calibration of a system of cameras and lidars using an original calibration object. The presented results of the experiments show that the calibration accuracy is sufficiently high.

Robust Extrinsic Calibration of Multiple RGB-D Cameras with Body Tracking and Feature Matching.

RGB-D cameras have been commercialized, and many applications using them have been proposed. In this paper, we propose a robust registration method of multiple RGB-D cameras. We use a human body tracking system provided by Azure Kinect SDK to estimate a coarse global registration between cameras. As this coarse global registration has some error, we refine it using feature matching. However, the matched feature pairs include mismatches, hindering good performance. Therefore, we propose a registration refinement procedure that removes these mismatches and uses the global registration. In an experiment, the ratio of inliers among the matched features is greater than 95% for all tested feature matchers. Thus, we experimentally confirm that mismatches can be eliminated via the proposed method even in difficult situations and that a more precise global registration of RGB-D cameras can be obtained.

Calibration of multiple cameras based on ChArUco board

Calibration of multiple cameras based on ChArUco board

Calibration of multiple cameras for large-scale experiments using a freely moving calibration target

Obtaining accurate experimental data from Lagrangian tracking and tomographic velocimetry requires an accurate camera calibration consistent over multiple views. Established calibration procedures are often challenging to implement when the length scale of the measurement volume exceeds that of a typical laboratory experiment. Here, we combine tools developed in computer vision and non-linear camera mappings used in experimental fluid mechanics, to successfully calibrate a four-camera setup that is imaging inside a large tank of dimensions sim 10 times 25 times 6 ; mathrm {m}^3. The calibration procedure uses a planar checkerboard that is arbitrarily positioned at unknown locations and orientations. The method can be applied to any number of cameras. The parameters of the calibration yields direct estimates of the positions and orientations of the four cameras as well as the focal lengths of the lenses. These parameters are used to assess the quality of the calibration. The calibration allows us to perform accurate and consistent linear ray-tracing, which we use to triangulate and track fish inside the large tank. An open-source implementation of the calibration in Matlab is available.Graphic abstract

Multi-camera calibration method based on a multi-plane stereo target.

Machine vision techniques, including camera calibration methods, are of great importance for the development of vision-based measurements. However, in multi-camera calibration methods, rapidly constructing accurate geometric relationships among different coordinates is very difficult. Herein, we present a multi-camera calibration method capable of calibrating the intrinsic and extrinsic parameters of four cameras using only a single captured image per camera. Unlike Zhang's method, which relies on multiple captured images to calibrate the cameras, the method uses a multi-plane stereo target containing multiple fixed planes to which coded patterns are attached. This target greatly reduces the time required for calibration and improves calibration robustness. The proposed method was experimentally compared with traditional camera calibration. The problem affecting the calibration accuracy in single calibration of multiple cameras is that the feature points on the captured images produce occlusion or different degrees of blurring; in the calibration of multiple cameras multiple times, the error accumulation caused by the calibration of two adjacent cameras is solved. This demonstration of a multi-camera calibration method improves camera calibration and provides a new design philosophy, to the best of our knowledge, for machine vision and vision-based measurement.

Interview

Ouk Choi Professor Ouk Choi from Incheon National University, Korea, talks to Electronics Letters about the paper ‘Robust alternating optimization for extrinsic calibration of RGB-D cameras’, page 992. My research field is computer vision, and I am currently interested in the acquisition, processing, and understanding of 3D data. During my Ph.D. studies, my research topic was to solve highly ambiguous correspondence problems such as matching windows in image pairs of buildings. After receiving my Ph.D. degree, I joined the Samsung Advanced Institute of Technology, where I conducted research projects on time-of-flight cameras. I was intrigued by the fact that time-of-flight cameras can acquire 3D data without solving difficult correspondence problems. Since then, my research has been directed toward 3D computer vision. Nowadays, I am building systems that can acquire multi-view depth images of subjects, so that the acquired datasets can be used for machine learning and other research purposes. Early time-of-flight cameras had many problems such as depth noise, low resolution, and phase wrapping. Many of these problems seem to have been solved in commercially available RGB-D cameras such as the Kinect v2. In addition, they are factory calibrated, so that we can access the intrinsic parameters without calibration. However, if we want to use multiple RGB-D cameras with their 3D measurements represented in the same reference coordinate system, finding the extrinsic parameters such as the rotation and translation between the cameras needs to be done manually. In my Letter, we describe a robust and efficient bundle adjustment algorithm that is indispensable in realising a fully automatic, easy-to-use, and online extrinsic calibration method. In a previous work referenced in my Letter, Su et al. proposed an efficient bundle adjustment algorithm, which alternates between solving for better extrinsic parameters and solving for better calibration target locations. In the presence of outliers, the algorithm tends to fail but this problem was not addressed in their work. On the other hand, in our previous work, Kwon et al. proposed a robust bundle adjustment algorithm, which works in the presence of outliers. However, the algorithm is highly inefficient. In my Letter, my colleagues and I have reported a bundle adjustment algorithm that is as efficient as Su et al.’s algorithm and as robust as Kwon et al.’s algorithm. The most attractive point of our work is that we can perform online bundle adjustment with the proposed algorithm. Although the execution time is not real-time with a single processor, the computational complexity of the algorithm is linear with the number of RGB-D cameras and the number of calibration images. In addition, the algorithm is highly parallelisable, which means that it can be implemented to work in real-time with sufficient computing resources. This will enable online extrinsic calibration of multiple RGB-D cameras. Once we compute an initial solution, we can add images of the spherical calibration target and apply our bundle adjustment algorithm frame-by-frame until a certain level of accuracy is attained. We plan to capture 3D data of human subjects as such data can be used to build a learning-based generative model of 3D human shape and pose as well as establish a parametric generative model. In addition, such a multi-view capture system can be used for analysing the interaction between a person and an object or for analysing social behaviour among people. The collected data and developed algorithms will be used to realise a virtual agent who seems to act human. In the future, it will be possible to generate a movie with only a scenario, using the virtual actors and their human-like behaviours. Traditionally, 3D computer vision was a combination of theories on multiple-view geometry and optimisation-based parameter estimation algorithms. Many research projects have contributed to building 3D models of our surrounding world, and it is now easy to access the online 3D maps provided by Google or Apple. Optimisation-based estimation algorithms require good initial solutions based on theories and feature correspondence across multiple views. Recently, machine learning algorithms such as random forests and deep neural networks are directly applied to raw data, successfully replacing the expert-knowledge-based methods for finding initial solutions. In addition, the research interest is being shifted from static scenes to dynamic objects. Deep learning-based methods are now applied to reconstructing and analysing human performance capture, from which we can obtain a generative model, which can change its shape, pose, and facial expression. As generated models get more accurate and realistic, we will be able to see virtual humanoids in the 3D maps of our surrounding world. I guess that it will take no more than ten years to see such human-like dynamic agents.

Calibration of the Relative Orientation between Multiple Depth Cameras Based on a Three-Dimensional Target.

Depth cameras play a vital role in three-dimensional (3D) shape reconstruction, machine vision, augmented/virtual reality and other visual information-related fields. However, a single depth camera cannot obtain complete information about an object by itself due to the limitation of the camera’s field of view. Multiple depth cameras can solve this problem by acquiring depth information from different viewpoints. In order to do so, they need to be calibrated to be able to accurately obtain the complete 3D information. However, traditional chessboard-based planar targets are not well suited for calibrating the relative orientations between multiple depth cameras, because the coordinates of different depth cameras need to be unified into a single coordinate system, and the multiple camera systems with a specific angle have a very small overlapping field of view. In this paper, we propose a 3D target-based multiple depth camera calibration method. Each plane of the 3D target is used to calibrate an independent depth camera. All planes of the 3D target are unified into a single coordinate system, which means the feature points on the calibration plane are also in one unified coordinate system. Using this 3D target, multiple depth cameras can be calibrated simultaneously. In this paper, a method of precise calibration using lidar is proposed. This method is not only applicable to the 3D target designed for the purposes of this paper, but it can also be applied to all 3D calibration objects consisting of planar chessboards. This method can significantly reduce the calibration error compared with traditional camera calibration methods. In addition, in order to reduce the influence of the infrared transmitter of the depth camera and improve its calibration accuracy, the calibration process of the depth camera is optimized. A series of calibration experiments were carried out, and the experimental results demonstrated the reliability and effectiveness of the proposed method.

Toward Efficient 3-D Colored Mapping in GPS-/GNSS-Denied Environments

Efficient 3-D mapping provides useful and detailed 3-D data for many applications. In this letter, we present a multisensor calibration and mapping method, to provide highly efficient and relatively accurate colored mapping for GPS-/global navigation satellite system-denied environments. The sensor data include 3-D laser scanning point clouds and camera images. A simultaneous localization and mapping (SLAM)-assisted calibration method is first proposed for multiple multibeam light detection and ranging (LiDAR) and multiple camera calibration. An improved SLAM method with loop closure is proposed for 3-D mapping. With the proposed calibration and mapping methods, centimeter-level colored point clouds can be obtained efficiently. The proposed method was tested with both backpacked and car-mounted systems on indoor and outdoor scenes. Experimental results show the effectiveness and efficiency of the proposed calibration and mapping methods.

Multi-scale 3D roughness quantification of concrete interfaces and pavement surfaces with a single-camera set-up

The quantification of the surface roughness of concrete pavements is important for upgrading and maintenance operations. This paper explores the application of stereoscopy in the morphology assessment of surfaces with exposed aggregates. The approach herein proposed is based on a single camera to avoid the need for multiple view points and calibration of multiple cameras. An application example is used to address the issues related to the point cloud reconstruction and filtering of the surface points acquired, whilst keeping computational costs reduced. The accuracy of the technique is evaluated by a detailed comparison with a scan using several roughness quantification statistical parameters. Kurtosis is shown to better compare surface profiles and overcome limitations found in standard parameters. A good match between techniques is achieved, with global mean errors of less than 3% in the surface roughness, and local mean errors below 5%. The proposed technique is low-cost and has the potential to be used for the automatic acquisition and characterisation of concrete surfaces.

Multi-Cue-Based Circle Detection and Its Application to Robust Extrinsic Calibration of RGB-D Cameras.

RGB-Depth (RGB-D) cameras are widely used in computer vision and robotics applications such as 3D modeling and human–computer interaction. To capture 3D information of an object from different viewpoints simultaneously, we need to use multiple RGB-D cameras. To minimize costs, the cameras are often sparsely distributed without shared scene features. Due to the advantage of being visible from different viewpoints, spherical objects have been used for extrinsic calibration of widely-separated cameras. Assuming that the projected shape of the spherical object is circular, this paper presents a multi-cue-based method for detecting circular regions in a single color image. Experimental comparisons with existing methods show that our proposed method accurately detects spherical objects with cluttered backgrounds under different illumination conditions. The circle detection method is then applied to extrinsic calibration of multiple RGB-D cameras, for which we propose to use robust cost functions to reduce errors due to misdetected sphere centers. Through experiments, we show that the proposed method provides accurate calibration results in the presence of outliers and performs better than a least-squares-based method.

A Novel Method for Extrinsic Calibration of Multiple RGB-D Cameras Using Descriptor-Based Patterns.

This paper presents a novel method to estimate the relative poses between RGB-D cameras with minimal overlapping fields of view. This calibration problem is relevant to applications such as indoor 3D mapping and robot navigation that can benefit from a wider field of view using multiple RGB-D cameras. The proposed approach relies on descriptor-based patterns to provide well-matched 2D keypoints in the case of a minimal overlapping field of view between cameras. Integrating the matched 2D keypoints with corresponding depth values, a set of 3D matched keypoints are constructed to calibrate multiple RGB-D cameras. Experiments validated the accuracy and efficiency of the proposed calibration approach.

Modeling lens optics and rendering virtual views from fisheye imagery

Production of high-quality virtual reality content from real sensed data is a challenging task due to several factors such as calibration of multiple cameras and rendering of virtual views. In this paper, we present a pipeline that maximizes the performance of virtual view rendering from an imagery captured by a camera equipped with fisheye lens optics. While such optics offer a wide field-of-view, it also introduces specific distortions. These have to be taken into account while rendering virtual views for a target application (e.g., head-mounted displays). We integrate a generic camera model into a fast rendering pipeline where we can tune intrinsic and extrinsic camera parameters along with resolution to meet the device or user requirements. We specifically target CPU-based implementation and quality in par with GPU-based rendering approaches. Using the adopted generic camera model, we numerically tabulate the required backward projection mapping and store it in a look-up table. This approach offers a tradeoff between memory and computational complexity in terms of operations for calculating the mapping values. Finally, we complement our method with an interpolator, which handles occlusions efficiently. Experimental results demonstrate the viability, robustness and accuracy of the proposed pipeline.

Automatic graph based spatiotemporal extrinsic calibration of multiple Kinect V2 ToF cameras

Abstract This paper presents a novel technique for the automatic calibration of motion capture systems composed of multiple Time of Flight cameras (ToF) or any equivalent device able to provide 3D point clouds together with chromatic information. The calibration procedure is designed to be simple, fast and unsupervised: a colored sphere moved freely by hand is used as calibration tool. Three elements in particular distinguish the method we propose from other state of the art solutions: it takes into account and propagates measurement uncertainties, it allows the identification and recovery of systematic time delays in camera synchronization, it optimize the extrinsic parameters by means of a graph based approach. A widely used ToF, Microsoft Kinect V2, was used for the comparison with state of the art multi-ToF calibration techniques. Experimental results demonstrated that the proposed method is the most accurate both in terms of shape reconstruction and spatial consistency. Furthermore, the experimental results underlined that the application of the global graph optimization over a more standard couple-based unweighed matching achieves a higher accuracy in the assessment of the extrinsic parameters. Thus enabling the reduction of both systematic errors and data dispersion in 3D point clouds of reference shapes. The comparison among acquisition systems composed of 3 and 6 devices underlined that graph optimization becomes relevant when the number of devices grows. Taking into account several 3D configurations calibrated with the proposed method, it was assessed that the uncertainty of the extrinsic parameters is generally lower than 2 mm and 1 0 − 2 radiants, 0.6°. The resulting dense 3D reconstruction of objects, both static and in motion, achieved experimental errors lower than to 10 mm, approximately half of other state of the art methods, and, in first approximation, homogeneous for the entire monitored area. The software developed can be found at the site of the MIRo lab (Measurement Instrumentation and Robotics lab) at link 1 under the Creative Commons Attribution conditions.