Camera Parameter Estimation Research Articles

Estimating camera parameters from starry night photographs

We propose an efficient, specific method for estimating camera parameters from a single starry night image. Such an image consists of a collection of disks representing stars, so traditional estimation methods for common pictures do not work. Our method uses a database, a star catalog, that stores the positions of stars on the celestial sphere. Our method computes magnitudes (i.e., brightnesses) of stars in the input image and uses them to find the corresponding stars in the star catalog. Camera parameters can then be estimated by a simple geometric calculation. Our method is over ten times faster and more accurate than a previous method.

High-speed real-time augmented reality tracking algorithm model of camera based on mixed feature points

At this stage of augmented reality, simple feature descriptions are mainly used in camera real-time motion tracking, but this is prone to the problem of unstable camera motion tracking. Aiming at the balance between real-time performance and stability, a new method model of real-time camera motion tracking based on mixed features was proposed. By comprehensively using feature points and feature lines as scene features, feature extraction, optimization, and fusion are used to construct hybrid features, and the hybrid features are unified for real-time camera parameter estimation. An image feature optimization method based on scene structure analysis is proposed to meet the computing constraints of mobile terminals. An iterative feature line-screening method is proposed to calculate a stable feature line set, and based on the scene feature composition and feature geometry, a hybrid feature is adaptively constructed to improve the tracking stability of the camera. Based on improved SIFT feature matching target detection and tracking algorithm, a hybrid feature point detection operator detection algorithm is used to achieve rapid feature point extraction, and the speed of descriptor generation is reduced by reducing the feature descriptor vector dimension. The experimental results prove that the proposed target detection and tracking algorithm has good real-time and robustness, and improves the success rate of target detection and tracking.

Calibration method for the electrically tunable lens based on shape-changing polymer.

In this paper, a calibration method for the camera system with electrically tunable lens (ETL) based on shape-changing polymer (SCP) is proposed to improve the accuracy, robustness and practicality of the system. The camera model of the ETL based on SCP is proposed based on the analyses of its optical properties. The calibration strategy, including initial estimation of camera parameters and bundle adjustment is presented. To eliminate the influence of temperature on ETL in machine vision applications, a real-time temperature compensation method is proposed. The proposed method makes use of the existing calibration hardware without adding new components to the system. Both simulations and experiments are conducted to evaluate the effectiveness and accuracy of the proposed camera model and calibration method. The measurement error with the proposed calibration method is below 20 microns at high magnification, whose measurement accuracy is improved by five times than the existing method at high magnification. With the proposed calibration method for the camera system with ETL based on SCP, the calibration workload is reduced and accurate calibration at high magnification is achieved. It also benefits the development of autofocusing 3D measurement technology.

Efficient development of vision-based dense three-dimensional displacement measurement algorithms using physics-based graphics models

This research investigates a framework for the efficient development of vision-based dense three-dimensional displacement measurement algorithms to support reliable structural health monitoring of civil structures. The framework exploits the use of a photo-realistic synthetic model, termed a physics-based graphics model, to simulate the entire process of vision-based measurement. At the same time, the synthetic environment is used to evaluate the performance of different post-processing algorithms quantitatively for a given measurement scenario, such as camera selection and camera placement. The effectiveness of the framework is demonstrated by optimizing the algorithms for the three-dimensional displacement measurement of a 14-bay laboratory truss structure. The vision-based dense three-dimensional displacement estimation algorithms optimized in this study consist of four steps: (1) camera parameter estimation, (2) camera motion estimation and compensation, (3) vision-based two-dimensional tracking, and (4) projection of two-dimensional tracking results to three-dimensional space. The algorithms use the knowledge from the finite element model to facilitate the implementation and maximize the measurement outcome, that is, model-informed approach. To test and evaluate the model-informed approach, synthetic videos are rendered for two measurement scenarios, that is, using a Digital Single Lens Reflex camera mounted on a tripod and using an Unmanned Aerial Vehicle camera. Then, the performance of the model-informed approach is evaluated by comparing the estimated displacement with the ground truth values. Based on the performance evaluation, an algorithm with the highest expected performance is selected for each of the two measurement scenarios. Finally, the selected algorithm is tested in a laboratory experiment. In contrast to the existing literature that investigates fixed individual measurement scenarios, the proposed framework can be used to test different measurement scenarios and estimate the outcome of each scenario before performing actual tests, facilitating the implementation of vision-based measurement for the structural health monitoring of civil structures.

Semi-Supervised 3D Human Pose Estimation by Jointly Considering Temporal and Multiview Information

Three-dimensional human pose estimation is usually conducted in a supervised manner. However, because collecting labeled 3D skeletons is expensive and time-consuming, semi-supervised methods that need much fewer amount of labeled 3D data are urgently demanded. Some semi-supervised learning methods propose to independently consider information from consecutive video frames, or frames simultaneously captured from multiple viewpoints. In this article, we propose to jointly consider temporal information and multiview information in a unified adversarial learning framework. Given a 2D skeleton, a pose generator network is developed to estimate the corresponding 3D skeleton, and a camera network is developed to estimate camera parameters. The estimated 3D skeleton is evaluated by a critic network to examine whether the estimated one is a plausible 3D human pose or not. Based on the estimated camera parameters, the estimated 3D skeleton can be re-projected into a 2D skeleton, which should be similar to the input 2D skeleton. The ideas of re-projection and adversarial learning enable the scheme of self supervision. We design network architectures of the aforementioned networks to take 2D skeletons from multiple viewpoints in temporally consecutive frames. By jointly considering two types of information, we verify that performance can be largely improved.

JOINT GEOMETRIC CALIBRATION OF COLOR AND THERMAL CAMERAS FOR SYNCHRONIZED MULTIMODAL DATASET CREATING

Abstract. With increasing performance and availability of thermal cameras the number of applications using them in various purposes grows noticeable. Nowadays thermal vision is widely used in industrial control and monitoring, thermal mapping of industrial areas, surveillance and robotics which output huge amount of thermal images. This circumstance creates the necessary basis for applying deep learning which demonstrates the state-of-the-art performance for the most complicated computer vision tasks. Using different modalities for scene analysis allows to outperform results of mono-modal processing, but in case of machine learning it requires synchronized annotated multimodal dataset. The prerequisite condition for such dataset creating is geometric calibration of sensors used for image acquisition. So the purpose of the performed study was to develop a technique for joint calibration of color and long wave infra-red cameras which are to be used for collecting multimodal dataset needed for the tasks of computer vision algorithms developing and evaluating.The paper presents the techniques for camera parameters estimation and experimental evaluation of interior orientation of color and long wave infra-red cameras for further exploiting in datasets collecting. Also the results of geometrically calibrated camera exploiting for 3D reconstruction and 3D model realistic texturing based on visible and thermal imagery are presented. They proved the effectivity of the developed techniques for collecting and augmenting synchronized multimodal imagery dataset for convolutional neural networks model training and evaluating.

Camera calibration using synthetic random speckle pattern and digital image correlation

Accurate camera calibration is of fundamental importance to various vision-based 3D metrological techniques. Despite camera calibration methods using regular planar calibration targets (e.g., checkerboard or circular pattern) have been widely adopted, their accuracy is less-than-desirable due to the limited number and the low registration accuracy of control points. This work presents an alternative calibration method that uses a calibration target based on a synthetic random speckle pattern. Specifically, a set of regularly distributed control points is first specified on the synthetic speckle pattern displayed on a monitor. These control points are then precisely matched to their counterparts on the captured calibration images using the state-of-the-art digital image correlation (DIC) algorithm. Compared with the existing camera calibration methods, the proposed method possesses the advantages of much more effective control points, higher control point match accuracy, which lead to more accurate and precise estimation of camera parameters. To evaluate the performance of the proposed calibration method, simulated calibration tests using images with varied noise levels and real calibration tests were performed on the planar calibration targets using speckle, circular and checkerboard patterns. The influences of various DIC calculation parameters (i.e. the number of control points and the subset size) on calibration results are also studied using noiseless simulated images. The reprojection errors on synthetic noiseless images and real images are computed as 0.004 and 0.038 pixels, respectively, confirming the high calibration accuracy delivered by the proposed method.

CALIBRATING PHOTOGRAMMETRIC AIRBORNE CAMERA SYSTEMS WITH DIFFRACTIVE OPTICAL ELEMENTS

Abstract. This paper presents a laboratory approach for geometric calibration of airborne camera systems. The setup uses an incoming laser beam, which is split by Diffractive Optical Elements (DOE) into a number of beams with precisely-known propagation directions. Each point of the diffraction pattern represents a point at infinity and is invariant against translation. A single image is sufficient to allow a complete camera calibration in accordance with classical camera calibration methods using the pinhole camera model and a distortion model. The presented method is time saving, since complex bundle adjustment procedures with several images are not necessary. It is well suited for the use with frame camera systems, but it works in principle also for pushbroom scanners. In order to prove the reliability, a conventional test field calibration is compared against the presented approach, showing that all estimated camera parameters are just insignificantly different. Furthermore a test flight over the Zeche Zollern reference target has been conducted. The aerotriangulation results shows that calibrating an airborne camera system with DOE is a feasible solution.

On-Barn Pig Weight Estimation Based on Body Measurements by Structure-from-Motion (SfM).

Information on the body shape of pigs is a key indicator to monitor their performance and health and to control or predict their market weight. Manual measurements are among the most common ways to obtain an indication of animal growth. However, this approach is laborious and difficult, and it may be stressful for both the pigs and the stockman. The present paper proposes the implementation of a Structure from Motion (SfM) photogrammetry approach as a new tool for on-barn animal reconstruction applications. This is possible also to new software tools allowing automatic estimation of camera parameters during the reconstruction process even without a preliminary calibration phase. An analysis on pig body 3D SfM characterization is here proposed, carried out under different conditions in terms of number of camera poses and animal movements. The work takes advantage of the total reconstructed surface as reference index to quantify the quality of the achieved 3D reconstruction, showing how as much as 80% of the total animal area can be characterized.

Procedural Modeling of a Building from a Single Image

AbstractCreating a virtual city is demanded for computer games, movies, and urban planning, but it takes a lot of time to create numerous 3D building models. Procedural modeling has become popular in recent years to overcome this issue, but creating a grammar to get a desired output is difficult and time consuming even for expert users. In this paper, we present an interactive tool that allows users to automatically generate such a grammar from a single image of a building. The user selects a photograph and highlights the silhouette of the target building as input to our method. Our pipeline automatically generates the building components, from large‐scale building mass to fine‐scale windows and doors geometry. Each stage of our pipeline combines convolutional neural networks (CNNs) and optimization to select and parameterize procedural grammars that reproduce the building elements of the picture. In the first stage, our method jointly estimates camera parameters and building mass shape. Once known, the building mass enables the rectification of the façades, which are given as input to the second stage that recovers the façade layout. This layout allows us to extract individual windows and doors that are subsequently fed to the last stage of the pipeline that selects procedural grammars for windows and doors. Finally, the grammars are combined to generate a complete procedural building as output. We devise a common methodology to make each stage of this pipeline tractable. This methodology consists in simplifying the input image to match the visual appearance of synthetic training data, and in using optimization to refine the parameters estimated by CNNs. We used our method to generate a variety of procedural models of buildings from existing photographs.

A Method for Determination of the Extrinsic Camera Parameters from a Pair of Images with the Use of Dual Quaternions

Real-world problems associated with the use of the moving vehicles present a problem for estimation of the unknown motion parameters on the basis of the data obtained from a static camera set on the surface of those vehicles. The initial data (in absence of information about the vehicle motion) are the images obtained from different viewpoints. The traditional approach to estimation of the rotation and translation parameters, which consists in determination of the so-called fundamental matrix and the subsequent calculation of the required matrices has several drawbacks. In particular, if the largest portion of the predefined points in the images is located on the same plane, the fundamental matrix estimation involves serious errors, which, in turn, lead to errors in estimation of the camera parameters. Unlike in the traditional approach, in this paper the corresponding points are used to estimate not the fundamental matrix, but the internal and external camera parameters directly. Besides, this paper presents a multiple view geometry model, based on three-dimensional images and camera parameters in the form of dual quaternions. The proposed approach to the problem is a new method of estimation of the unknown camera parameters, which is more accurate and reliable compared with the traditional one. This method was implemented as a program in C + +. Using the developed program, the authors carried out an experiment to establish a correlation between the errors in the input data (points of coordinates on the planes of the camera) and the errors in the estimated rotation and translation parameters. As the result, it was confirmed that the accuracy of the parameters' estimation in most cases surpasses the quality of the results obtained by using the fundamental matrix.

In-air versus underwater comparison of 3D reconstruction accuracy using action sport cameras

Action sport cameras (ASC) have achieved a large consensus for recreational purposes due to ongoing cost decrease, image resolution and frame rate increase, along with plug-and-play usability. Consequently, they have been recently considered for sport gesture studies and quantitative athletic performance evaluation. In this paper, we evaluated the potential of two ASCs (GoPro Hero3+) for in-air (laboratory) and underwater (swimming pool) three-dimensional (3D) motion analysis as a function of different camera setups involving the acquisition frequency, image resolution and field of view. This is motivated by the fact that in swimming, movement cycles are characterized by underwater and in-air phases what imposes the technical challenge of having a split volume configuration: an underwater measurement volume observed by underwater cameras and an in-air measurement volume observed by in-air cameras. The reconstruction of whole swimming cycles requires thus merging of simultaneous measurements acquired in both volumes. Characterizing and optimizing the instrumental errors of such a configuration makes mandatory the assessment of the instrumental errors of both volumes.In order to calibrate the camera stereo pair, black spherical markers placed on two calibration tools, used both in-air and underwater, and a two-step nonlinear optimization were exploited. The 3D reconstruction accuracy of testing markers and the repeatability of the estimated camera parameters accounted for system performance. For both environments, statistical tests were focused on the comparison of the different camera configurations. Then, each camera configuration was compared across the two environments. In all assessed resolutions, and in both environments, the reconstruction error (true distance between the two testing markers) was less than 3mm and the error related to the working volume diagonal was in the range of 1:2000 (3×1.3×1.5m3) to 1:7000 (4.5×2.2×1.5m3) in agreement with the literature. Statistically, the 3D accuracy obtained in the in-air environment was poorer (p<10−5) than the one in the underwater environment, across all the tested camera configurations. Related to the repeatability of the camera parameters, we found a very low variability in both environments (1.7% and 2.9%, in-air and underwater). This result encourage the use of ASC technology to perform quantitative reconstruction both in-air and underwater environments.

Visibility detection approach to road scene foggy images

A cause of vehicle accidents is the reduced visibility due to bad weather conditions such as fog. Therefore, an onboard vision system should take visibility detection into account. In this paper, we propose a simple and effective approach for measuring the visibility distance using a single camera placed onboard a moving vehicle. The proposed algorithm is controlled by a few parameters and mainly includes camera parameter estimation, region of interest (ROI) estimation and visibility computation. Thanks to the ROI extraction, the position of the inflection point may be measured in practice. Thus, combined with the estimated camera parameters, the visibility distance of the input foggy image can be computed with a single camera and just the presence of road and sky in the scene. To assess the accuracy of the proposed approach, a reference target based visibility detection method is also introduced. The comparative study and quantitative evaluation show that the proposed method can obtain good visibility detection results with relatively fast speed.

Method for calibrating intrinsic camera parameters using orthogonal vanishing points

To self-calibrate the intrinsic camera parameters, we designed a calibration pattern and proposed a method to estimate camera parameters involving distortion coefficients from orthogonal vanishing points. We first obtained four pairs of vanishing points from an image of the calibration pattern and selected the two pairs with the smallest error. Then, after analyzing the effect of the intrinsic parameter error from the noise of the vanishing points, we employed an optimization algorithm to obtain the best vanishing points. Using two images with different orientations, we could obtain the intrinsic parameters by using a linear method. Finally, according to the preliminary calibrated intrinsic parameters and a criterion that the airline is a static straight line in the pictorial plane, the arithmetic mean of the camera distortion, which is based on the common characteristic points, could be obtained. Simulated and experimental results indicated that the two-dimensional reconstruction error is 0.32 pixels. Although this method has the same error level as the traditional method, it has a better flexibility and can achieve real-time camera self-calibration.

A 12-camera body scanning system based on close-range photogrammetry for precise applications

ABSTRACTA precise 3D body scanning system, designed for applications in the medical and biomedical field, is described. The system consists of 12 digital cameras, mounted on a rigid frame which maintains the B/H ratio constant. An LED lighting system, integrated with the rigid frame, ensures a homogenous illumination of the subject. A modified firmware installed on each device enables the synchronised remote release driven by remote control. Data transmission to the computer is carried out wirelessly. A process for the estimation of the extrinsic camera parameters has been implemented with the aim of obtaining a 3D model in 1:1 scale. Analyses have been performed in order to state the repeatability of the process used to estimate the external camera calibration parameters as well as the precision of the 3D digital models of a body obtained for a mannequin and for a live human subject.

Efficient Light-Field Rendering Using Depth Maps for 100-Mpixel Multi-Projection 3D Display

In order to achieve an immersive, natural 3D experience on a large screen, a 100-Mpixel multi-projection 3D display was developed. Ninety-six projectors were used to increase the number of rays emanating from each pixel in the horizontal direction to 96. Conventional algorithms use a large number of cameras or input images to process a large number of light rays. This creates difficulties in the design of both the large acquiring system and substantial memory storage. In this paper, we propose an efficient light-field rendering algorithm that utilizes only a few input colors and depth images. Using a depth map and estimated camera parameters, synthesized light-field images are directly generated. This algorithm requires a much lighter memory load than conventional light-field rendering algorithms. It is also much simpler than the image-based rendering algorithm because it does not require the generation of so many multiview images.

Real-time camera tracking using hybrid features in mobile augmented reality

Often only simple features are employed in mobile augmented reality (AR) due to the limited computational capacity of mobile terminals, which often leads to unsteadiness of the camera tracking. In this paper, we propose a novel approach to real-time camera tracking in mobile AR using hybrid features to alleviate this problem. By integrating feature points and lines as scene features, hybrid features are generated through the process of point/line features extraction, optimization and fusion, and are used in the real-time estimation of camera parameters. A method for image feature optimization is proposed based on the scene structural analysis to meet the computational constraints of mobile terminals. In order to improve the stability of camera tracking, an iterative screening method is proposed to choose a set of stable feature lines, and hybrid features are adaptively constructed based on the composition and geometry of scene features. It is shown from the experimental results that the proposed method produces more stable and smoother camera trajectories in comparison with the method only using feature points, and a good balance is achieved between the stability and the real-time computation of the camera tracking on a mobile platform.

Local iterative DLT soft-computing vs. interval-valued stereo calibration and triangulation with uncertainty bounding in 3D reconstruction

The use of stereo vision for 3D data gathering is affected by constraints in the position of the cameras, the quality of the optical elements and the numerical algorithms for calibration and matching. Also, there is not a wide agreement on the best procedure for bounding the 3D errors within an uncertainty volume. In this work, this problem is solved by implementing the whole set of computations, including calibration and triangulation, with interval data. This is in contrast with previous works that rely on Direct Linear Transform (DLT) as a camera model. To keep better with real lens aberrations, a local iterative modification is proposed that provides an on-demand set of calibration parameters for each 3D point, comprising those nearest in 3D space. In this way, the estimated camera parameters are closely related with camera aberrations at the lens area through which that 3D point is imaged. To further reduce the triangulation uncertainty volume, a Soft Computing approach is proposed that represents each 3D point uncertainty as a cloud of crisp points compatible with interval-valued calibration data.Real data from previous works in related research areas is used to judge whether the new approach improves the precision and accuracy of other crisp and interval-valued estimations without degrading precision, and it is concluded that the new technique is able to significantly improve the uncertainty volumes.

Geometric error correction of convergent multi‐view images

A geometric error correction of convergent multi-view images is proposed. Shape factors and camera parameters of the captured multi-view image are used to estimate an ideal multi-camera arrangement and its camera parameters. A correction transform of each viewpoint to reduce the geometric error is calculated based on the original and estimated camera parameters. The error-corrected multi-view image is obtained by applying the transform to the captured images. The experimental results show that the proposed method efficiently reduces the geometric error of the convergent multi-view images.

Camera pose estimation under dynamic intrinsic parameter change for augmented reality

In this paper, we propose a method for estimating the camera pose for an environment in which the intrinsic camera parameters change dynamically. In video see-through augmented reality (AR) technology, image-based methods for estimating the camera pose are used to superimpose virtual objects onto the real environment. In general, video see-through-based AR cannot change the image magnification that results from a change in the camera׳s field-of-view because of the difficulty of dealing with changes in the intrinsic camera parameters. To remove this limitation, we propose a novel method for simultaneously estimating the intrinsic and extrinsic camera parameters based on an energy minimization framework. Our method is composed of both online and offline stages. An intrinsic camera parameter change depending on the zoom values is calibrated in the offline stage. Intrinsic and extrinsic camera parameters are then estimated based on the energy minimization framework in the online stage. In our method, two energy terms are added to the conventional marker-based method to estimate the camera parameters: reprojection errors based on the epipolar constraint and the constraint of the continuity of zoom values. By using a novel energy function, our method can accurately estimate intrinsic and extrinsic camera parameters. We confirmed experimentally that the proposed method can achieve accurate camera parameter estimation during camera zooming.