Camera Calibration Model Research Articles

Nonlinear optimization of optical camera multiparameter via triple integrated Gradient-based optimizer algorithm

Metaheuristic techniques are widely used to solve various complex real-world problems. Camera calibration is a critical task in machine vision, which aims to determine the optical camera parameters through 2D image features and corresponding 3D spatial features. To overcome the limitations of traditional methods, such as weak local optima avoidance and low calibration accuracy, a triple integrated Gradient-based optimizer (TIGBO_CC) is proposed for nonlinear optimization of optical camera parameters. Several improvements are made to the original Gradient-based optimizer (GBO). First, in the exploration phase, a fitness-distance balance is adopted to select the most contributing candidate to guide the update of the solution. Second, a random search technique is adopted in the exploitation phase. The switching between the original mode and the additional mode follows the probability. Finally, a neighborhood local search is also introduced. Small-step fine search is performed near the optimal solution to improve the quality of the final solution. TIGBO_CC is evaluated on 12 classical functions, the CEC2020 test suite, and the CEC2022 test suite. The experiments include sensitivity analysis, effectiveness analysis, scalability analysis, function value, convergence curve, box plot, Friedman test, Wilcoxon test, etc. Furthermore, a nonlinear camera calibration model is established. Taking the mean reprojection error as the objective function, TIGBO_CC solves the optimal optical camera parameters by minimizing this function. In different scenarios, the calibration method based on TIGBO_CC has high precision and strong reproducibility. Comprehensive analysis shows that TIGBO_CC has excellent optimization ability and practicality.

A non-coplanar high-precision calibration method for cameras based on an affine coordinate correction model

Traditional non-coplanar calibration methods such as Tsai’s method have many problems, such as insufficient calibration accuracy, inconvenient operation, inaccurate models, etc. This paper proposes a new high-precision non-coplanar calibration method that aims to solve these problems. Like Tsai’s method, the proposed calibration method utilizes a one-dimensional displacement stage and a two-dimensional plane target to generate a virtual 3D feature point sequence. As an improvement, an affine coordinate correction model is applied to ensure the accuracy and orthogonality of the obtained virtual 3D coordinates. A novel and accurate camera calibration model is further established. Compared with Tsai’s model, which uses a radial alignment constraint and ignores the orthonormal constraint of the rotation matrix, the proposed calibration model fully considers the degrees of freedom of the camera’s parameters to be calibrated, as well as the lens’s nonlinear distortion parameters. More accurate analytical solutions of intrinsic and extrinsic parameters can be obtained with the proposed calibration model. Finally, a novel high-precision non-coplanar calibration method is proposed based on the proposed calibration model. The reprojection experiment proves that the calibration accuracy of this calibration method is better than that of Tsai’s and Zhang’s calibration methods under the same calibration conditions. As a supplement, a novel binocular camera system extrinsic parameter calibration method with known intrinsic parameters is proposed. With accurate intrinsic and extrinsic parameters, the binocular camera system’s relative measurement accuracy could be within 1/10 000. Overall, this method can be used in experimental and industrial applications that require high-precision calibration parameters.

Experimental Tests and Simulations on Correction Models for the Rolling Shutter Effect in UAV Photogrammetry

Many unmanned aerial vehicles (UAV) host rolling shutter (RS) cameras, i.e., cameras where image rows are exposed at slightly different times. As the camera moves in the meantime, this causes inconsistencies in homologous ray intersections in the bundle adjustment, so correction models have been proposed to deal with the problem. This paper presents a series of test flights and simulations performed with different UAV platforms at varying speeds over terrain of various morphologies with the objective of investigating and possibly optimising how RS correction models perform under different conditions, in particular as far as block control is concerned. To this aim, three RS correction models have been applied in various combinations, decreasing the number of fixed ground control points (GCP) or exploiting GNSS-determined camera stations. From the experimental tests as well as from the simulations, four conclusions can be drawn: (a) RS affects primarily horizontal coordinates and varies notably from platform to platform; (b) if the ground control is dense enough, all correction models lead practically to the same mean error on checkpoints; however, some models may cause large errors in elevation if too few GCP are used; (c) in most cases, a specific correction model is not necessary since the affine deformation caused by RS can be adequately modelled by just applying the extended Fraser camera calibration model; (d) using GNSS-assisted block orientation, the number of necessary GCP is strongly reduced.

Effect of camera network configuration on the accuracy of digitization of architectural objects

Computer vision techniques enabled a significant boom in the use of photogrammetry in recent years, especially in the digitization of cultural heritage. Thanks to Structure from Motion, it is possible to automatically reconstruct the 3D geometry of even complicated objects. However, each technology has its strengths and weaknesses, so even SfM can often lead to unreliable results, which are usually caused by an inappropriate camera network configuration or an unsuitable camera calibration model. As part of this study, we analyzed the impact of different imaging configurations on the digitization of architectural objects, and in particular the possibility of automated photogrammetric linking of the exterior and interior through door and window openings. Photogrammetric point clouds were compared to scans from terrestrial laser scanning. The results indicate the possibility of reducing financial costs in the future if a laser scanner is not available.

Improvement of Image Stitching Using Binocular Camera Calibration Model

Image stitching is the process of stitching several images that overlap each other into a single, larger image. The traditional image stitching algorithm searches the feature points of the image, performs alignments, and constructs the projection transformation relationship. The traditional algorithm has a strong dependence on feature points; as such, if feature points are sparse or unevenly distributed in the scene, the stitching will be misaligned or even fail completely. In scenes with obvious parallaxes, the global homography projection transformation relationship cannot be used for image alignment. To address these problems, this paper proposes a method of image stitching based on fixed camera positions and a hierarchical projection method based on depth information. The method does not depend on the number and distribution of feature points, so it avoids the complexity of feature point detection. Additionally, the effect of parallax on stitching is eliminated to a certain extent. Our experiments showed that the proposed method based on the camera calibration model can achieve more robust stitching results when a scene has few feature points, uneven feature point distribution, or significant parallax.

Laser triangulation measurement system with Scheimpflug calibration based on the Monte Carlo optimization strategy.

We propose a linear laser triangulation measurement system using Scheimpflug calibration based on the Monte Carlo optimization strategy. A Scheimpflug inclination camera calibration model is introduced in the measurement system for improving the image definition in small-range measurements with a large depth-of-field. To address the nonlinear optimization problem between the instrument resolution and measurement range, the Monte Carlo method is adopted to determine the optimal optical parameters (scattering angle, Scheimpflug angle, and focus length) in a practical measurement system. Furthermore, we experimentally constructed the measurement system to demonstrate the measurement precision by measuring a standard step block (measurement range 15 mm). The performance parameters of the maximum measurement error, maximum standard deviation, and linearity are obtained as ±7 μm, 0.225 μm, and 0.046%, respectively. Finally, the proposed measurement system based on the Monte Carlo optimization strategy is promising for high-precision measurements in industrial applications and provides guidance for optimizing the design parameters of ranging measurement sensors.

High-precision calibration to zoom lens of optical measurement machine based on FNN.

We proposed a calibration method for high-precision zoom lenses of optical measurement machines based on Fully Connected Neural Network (FNN), using a 5-layer neural network instead of a camera calibration model, to achieve continuous calibration of zoom lenses at any zoom setting by calibrating typical zooms. From the experimental verification, the average calibration error of this method is 9.83×10-4mm and the average measurement error at any zoom setting is 0.01317mm. The overall calibration precision is better than that of Zhang's calibration method and can meet the application requirements of a high-precision optical measurement machine. The method proposed in this paper provided a new solution and a new idea for the calibration of zoom lenses, which can be widely used in the fields of precision parts inspection and machine-vision measurement.

Optimization and Estimation Algorithm of Vehicle Spatial Form Based on Monocular Traffic Camera

Accurate and rapid acquisition of vehicle spatial form is of great importance in the fields of intelligent transportation and autonomous driving. However, given the limitations of projective geometry, it is difficult to obtain the 3-D structure of vehicles using monocular cameras. The purpose of this paper is, therefore, to estimate the vehicle spatial form using monocular traffic cameras. Firstly, we establish the camera calibration model of the road scene, and jointly construct the geometric constraint model of the vehicle spatial form by vanishing points. Secondly, the contour and edge constraints of the vehicle are obtained based on Mask R-CNN. Then, based on these constraints, the error constraint function is constructed to calculate the projection error of the vehicle spatial form. Finally, a particle swarm optimization algorithm is used to iteratively optimize the parameters in the constraint space to obtain accurate vehicle spatial form information. Experiments are carried on the BrnoCompSpeed data set and the home-made data set. The experimental results show that the processing time of a single frame is less than 0.5 s and the average accuracy is higher than 94%. Moreover, the proposed algorithm has good robustness to the issue of vehicle occlusion and queuing in the scene, which outperforms existing methods.

A novel algorithm for space camera geometry parameters on-orbit calibration

The precision of geometry parameters is important for stereo mapping tasks. The current methods mainly by ground calibration field which exhibit disadvantages. Inspired by simplified method with laser reference beam we proposed in previous work, a novel on-orbit calibration method based on dual vector attitude determination algorithm is proposed, overcoming the limitation of previous methods. First, a single camera calibration model is established with the proposed method. Errors introduced by input parameters are then analyzed and expressed by simplified expressions. Furthermore, residual errors between the two algorithms are calculated. Finally, an experimental system with two-array cameras is set up in a vacuum chamber with well vibration isolation. The results show that the proposed method is well performance in real time with high accuracy. Moreover, the phenomenon of the periodic sinusoidal change of the angle between cameras in a circle is revealed, which is essential to stereo mapping tasks.

Three Birds, One Stone: Unified Laser-Based 3-D Reconstruction Across Different Media

The laser-based 3-D reconstruction is crucial for autonomous manipulation and resource exploration in both the air and underwater scenarios, due to its accurate precision and robustness to disturbances. However, most current laser-based 3-D reconstruction sensors cannot be applied across different media without recalibration directly, e.g., air, air$\rightarrow $ glass$\rightarrow $ air, and air$\rightarrow $ glass$\rightarrow $ water, which is called killing three birds with one stone. This is because the variation of the medium density could change the sensor calibration parameters and further make the sensor suffer from a systematic geometric bias. To address these challenges, a unified laser-based 3-D reconstruction method is proposed in order that the laser-based scanner could be used across different media without recalibration, where we first explicitly model the refraction of the underwater vision systems and transform it across different media into a unified sensor reference frame. More specifically, an underwater refractive camera calibration model is designed to calibrate the orientation and position of the refractive interface, which can improve the accuracy of underwater reconstruction for the laser-based scanner, and we then present a refractive pose estimation model with a unified sensor reference frame, which can help the sensor to be applied across different scenarios directly. For the experiments, we validate the performance of our method on our underwater 3-D scanner prototype. Several reconstruction results on different objects and scenarios demonstrate the effectiveness of our proposed method and the practicality of our designed sensor.

Three-dimensional mechanical parts reconstruction technology based on two-dimensional image

With the development of computer technology and three-dimensional reconstruction technology, three-dimensional reconstruction based on visual images has become one of the research hotspots in computer graphics. Three-dimensional reconstruction based on visual image can be divided into three-dimensional reconstruction based on single photo and video. As an indirect three-dimensional modeling technology, this method is widely used in the fields of film and television production, cultural relics restoration, mechanical manufacturing, and medical health. This article studies and designs a stereo vision system based on two-dimensional image modeling technology. The system can be divided into image processing, camera calibration, stereo matching, three-dimensional point reconstruction, and model reconstruction. In the part of image processing, common image processing methods, feature point extraction algorithm, and edge extraction algorithm are studied. On this basis, interactive local corner extraction algorithm and interactive local edge detection algorithm are proposed. It is found that the Harris algorithm can effectively remove the features of less information and easy to generate clustering phenomenon. At the same time, the method of limit constraints is used to match the feature points extracted from the image. This method has high matching accuracy and short time. The experimental research has achieved good matching results. Using the platform of binocular stereo vision system, each step in the process of three-dimensional reconstruction has achieved high accuracy, thus achieving the three-dimensional reconstruction of the target object. Finally, based on the research of three-dimensional reconstruction of mechanical parts and the designed binocular stereo vision system platform, the experimental results of edge detection, camera calibration, stereo matching, and three-dimensional model reconstruction in the process of three-dimensional reconstruction are obtained, and the full text is summarized, analyzed, and prospected.

Corrigendum to Comparison of pre- and self-calibrated camera calibration models for UAS-derived nadir imagery for a SfM application

Corrigendum to Comparison of pre- and self-calibrated camera calibration models for UAS-derived nadir imagery for a SfM application

Metode Pinhole Model untuk Menentukan Jarak Perpindahan Kendaraan dalam Video

Intelligent Transportation System is a solution to overcome the problem of transportation. The purpose of this study was to design a system to calculate the distance of vehicle movement in the video. One example of application of intelligent transport system is to calculate the speed of a vehicle that can be used in traffic engineering. Parameters used to obtain the speed of the vehicle is the distance parameter. The distance of vehicle movement in the video can be obtained by using a pinhole camera calibration model. An appropriate calibration of camera can provide the intended parameter. There are several stages to perform calculation of the distance of object movement. The first stage was a detection of vehicle within the frame. The detection was required to obtain the center point of vehicle object. The next stage was to detect an object within the frame. This was performed to understand the movement of object from one frame to another frame. The data used for the study were video data with mov format. The result of research showed that the method of pinhole model was applicable to calculate the distance of object movement in video. The application of this method can help in calculating the object distance without manual calculation.

SIMULATION AND ANALYSIS OF PHOTOGRAMMETRIC UAV IMAGE BLOCKS: INFLUENCE OF CAMERA CALIBRATION ERROR

Abstract. Unmanned aerial vehicles (UAV) are increasingly used for topographic mapping. The camera calibration for UAV image blocks can be performed a priori or during the bundle block adjustment (self-calibration). For an area of interest with flat, corridor configuration, the focal length of camera is highly correlated with the height of camera. Furthermore, systematic errors of camera calibration accumulate on the longer dimension and cause deformation. Therefore, special precautions must be taken when estimating camera calibration parameters. In this paper, a simulated, error-free aerial image block is generated. error is then added on camera calibration and given as initial solution to bundle block adjustment. Depending on the nature of the error and the investigation purpose, camera calibration parameters are either fixed or re-estimated during the bundle block adjustment. The objective is to investigate how certain errors in the camera calibration impact the accuracy of 3D measurement without the influence of other errors. All experiments are carried out with Fraser camera calibration model being employed. When adopting a proper flight configuration, an error on focal length for the initial camera calibration can be corrected almost entirely during bundle block adjustment. For the case where an erroneous focal length is given for pre-calibration and not re-estimated, the presence of oblique images limits the drift on camera height hence gives better camera pose estimation. Other than that, the error on focal length when neglecting its variation during the acquisition (e.g., due to camera temperature increase) is also investigated; a bowl effect is observed when one focal length is given in camera pre-calibration to the whole image block. At last, a local error is added in image space to simulate camera flaws; this type of error is more difficult to be corrected with the Fraser camera model and the accuracy of 3D measurement degrades substantially.

A Novel Vision-Based Pose Measurement Method Considering the Refraction of Light.

Nowadays, due to the advantages of non-contact and high-speed, vision-based pose measurements have been widely used for aircraft performance testing in a wind tunnel. However, usually glass ports are used to protect cameras against the high-speed airflow influence, which will lead to a big measurement error. In this paper, to further improve the vision-based pose measurement accuracy, an imaging model which considers the refraction light of the observation window was proposed. In this method, a nonlinear camera calibration model considering the refraction brought by the wind tunnel observation window, was established first. What’s more, a new method for the linear calibration of the normal vector of the glass observation window was presented. Then, combining with the proposed matching method based on coplanarity constraint, the six pose parameters of the falling target could be calculated. Finally, the experimental setup was established to conduct the pose measurement study in the laboratory, and the results satisfied the application requirements. Besides, experiments for verifying the vision measurement accuracy were also performed, and the results indicated that the displacement and angle measurement accuracy approximately increased by 57% and 33.6%, respectively, which showed the high accuracy of the proposed method.

Temperature Compensation Method for Digital Cameras in 2D and 3D Measurement Applications.

This paper presents the results of several studies concerning the effect of temperature on digital cameras. Experiments were performed using three different camera models. The presented results conclusively demonstrate that the typical camera design does not adequately take into account the effect of temperature variation on the device’s performance. In this regard, a modified camera design is proposed that exhibits a highly predictable behavior under varying ambient temperature and facilitates thermal compensation. A novel temperature compensation method is also proposed. This compensation model can be applied in almost every existing camera application, as it is compatible with every camera calibration model. A two-dimensional (2D) and three-dimensional (3D) application of the proposed compensation model is also described. The results of the application of the proposed compensation approach are presented herein.

Development of a faster classification system for metal parts using machine vision under different lighting environments

A machine vision system for the automatic classification process is developed under different lighting environments, and has been applied to the operation of a robot arm with 6 degrees of freedom (DOF). In order to obtain accurate positioning information, the overall image is captured by a CMOS camera which is mounted above the working platform. The effects of back-lighting and front-lighting environments to the proposed system were investigated. With the front-lighting environment, four different conditions were performed. For each condition, global and local contrast threshold operations were used to obtain good image quality. In this study, a quadratic transformation used to describe the relationship between the image coordinates and the world coordinates was proposed, which has been compared to linear transformation as well as the camera calibration model in MATLAB tool. Experimental results show that in a back-lighting environment, the image quality is improved, such that the positions of the centers of objects are more accurate than in a front-lighting environment. According to the calibration results, the quadratic transformation is more accurate than other methods. By calculating the calibration deviation using the quadratic transformation, the maximum positive deviation is 0.48 mm and 0.38 mm in the X and Y directions, respectively. The maximum negative deviation is − 0.34 mm and − 0.43 mm in X and Y directions, respectively. The proposed system is effective, robust, and can be valuable to industry, as it offers an automated robotic system with an improved flexibility for separating dissimilar items of nominal value, in order to reclaim material investments, and decrease the expense of purchasing the same items again.

Tree DBH measurement method based on smartphone and machine vision technology.

Aiming at the problems of high labor intensity and low efficiency, complicated operation and high cost of advanced equipment in the measurement of DBH in existing traditional forest resources, and proposing a method for measuring diameter at breast height (DBH) based on smartphone and machine vision technology, and combining with camera calibration, 3D reconstruction, machine vision, close-range photogrammetry, and other technologies. First, The original image is obtained by smartphone, and the visual saliency map of the preprocessed image is processing by using the lab color space model and 3×3 operator to convolving with image. Combining with the H component of the HSV (Hue, Saturation, Value) color model to enhance the color contrast part of the tree trunk in the image. The image segmentation algorithm is used to identify and obtain the target tree's contour in a natural environment. Then, an improved camera calibration model with nonlinear distortion terms is used to calibrate the camera's internal and external parameters. Finally, achieving reconstruction of a 3D world with camera parameters and two-dimensional image information is carried out to measure DBH at a height of 1.3 m. Experimental results indicated that within a certain range, the relative error of the DBH measurements was lower than 2.50%. This method has a high measurement accuracy, meeting the accuracy requirement of DBH measurement in forest resources investigation and could be applied in forestry resource surveys.

Research on intelligent parking system algorithm based on camera calibration model

Research on intelligent parking system algorithm based on camera calibration model

Method for Measuring Clearance of Automobile Body Panel with Double Structured Light System

According to the changes of the autobody measuring clearance often face the shooting angle problem caused by internal decline caused by the lack of data acquisition accuracy, a high precision and robust measurement of binocular gap line structured light and the system parameter calibration method of system integration. First the mathematical measurement model of the system is established according to the Zhang Zhengyou camera calibration model to calculate the camera parameters, the extrinsic parameters of the camera calibration of the light plane, while the introduction of auxiliary line structured light laser by solving the intersection of two camera coordinate system, finally gives the calculation model of the door gap and combined with the calibration data for the gap size. Compared with a three-dimensional scanning measuring device, the system can measure only two images at the same time. The experimental results show that when the shooting angle is less than 45 degrees at the working distance of 400mm and 50*50 field of view, the accuracy reaches 0.08mm, which meets the needs of fast and real-time measurement gap.